2024-09-15T00:00:23.558 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T00:00:23.573 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T00:00:23.660 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904819 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit} email: ceph-qa@ceph.com first_in_suite: false job_id: '7904819' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: rbd qos iops burst: 100 rbd qos iops limit: 50 rbd qos schedule tick min: 100 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 4271 sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 sleep_before_teardown: 0 subset: 97/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 targets: smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMtslxk6quUC5fU9BkATtHtvsvtS1wu46HhC7ZY9r9OWe1xR2JwhXHSmmiB9W7oT/lnIkMti9pbfICeXcQMUQBI= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHyccrCZFFuU+xC5jjUDWpy2Q+vd4VdSvu4eMPM91ukqCHVLMqDzfjZIe71D8rklHFi/SBAmHNvJ/hSBkvv+mHw= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-14_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T00:00:23.660 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa; will attempt to use it 2024-09-15T00:00:23.661 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa/tasks 2024-09-15T00:00:23.661 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T00:00:23.664 INFO:teuthology.task.internal:Checking packages... 2024-09-15T00:00:23.685 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ecab7be58963a5bd03910cd73eef66405d7fdd50' 2024-09-15T00:00:23.685 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T00:00:23.685 INFO:teuthology.packaging:ref: None 2024-09-15T00:00:23.685 INFO:teuthology.packaging:tag: None 2024-09-15T00:00:23.685 INFO:teuthology.packaging:branch: squid 2024-09-15T00:00:23.685 INFO:teuthology.packaging:sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-15T00:00:23.686 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-15T00:00:23.903 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-324.gecab7be5 2024-09-15T00:00:23.905 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T00:00:23.915 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T00:00:23.915 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T00:00:23.922 INFO:teuthology.task.internal:Saving configuration 2024-09-15T00:00:23.935 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T00:00:23.940 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T00:00:23.963 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904819', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 23:55:08.345700', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMtslxk6quUC5fU9BkATtHtvsvtS1wu46HhC7ZY9r9OWe1xR2JwhXHSmmiB9W7oT/lnIkMti9pbfICeXcQMUQBI='} 2024-09-15T00:00:23.987 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904819', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 23:55:08.344057', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHyccrCZFFuU+xC5jjUDWpy2Q+vd4VdSvu4eMPM91ukqCHVLMqDzfjZIe71D8rklHFi/SBAmHNvJ/hSBkvv+mHw='} 2024-09-15T00:00:23.987 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T00:00:23.996 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-15T00:00:23.996 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-15T00:00:23.997 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T00:00:24.056 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f372a5cf250>, signals=[15]) 2024-09-15T00:00:24.056 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T00:00:24.072 INFO:teuthology.task.internal:Opening connections... 2024-09-15T00:00:24.072 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-09-15T00:00:24.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:00:24.151 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2024-09-15T00:00:24.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:00:24.227 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T00:00:24.268 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-09-15T00:00:24.287 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-09-15T00:00:24.287 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-09-15T00:00:24.345 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2024-09-15T00:00:24.345 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="9" 2024-09-15T00:00:24.345 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2024-09-15T00:00:24.345 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2024-09-15T00:00:24.345 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="9" 2024-09-15T00:00:24.345 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el9" 2024-09-15T00:00:24.345 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T00:00:24.345 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2024-09-15T00:00:24.346 INFO:teuthology.orchestra.run.smithi171.stdout:LOGO="fedora-logo-icon" 2024-09-15T00:00:24.346 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T00:00:24.346 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2024-09-15T00:00:24.346 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T00:00:24.346 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T00:00:24.346 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T00:00:24.347 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-09-15T00:00:24.378 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2024-09-15T00:00:24.396 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2024-09-15T00:00:24.396 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2024-09-15T00:00:24.454 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2024-09-15T00:00:24.454 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2024-09-15T00:00:24.454 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2024-09-15T00:00:24.454 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2024-09-15T00:00:24.454 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2024-09-15T00:00:24.454 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2024-09-15T00:00:24.454 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T00:00:24.455 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2024-09-15T00:00:24.455 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2024-09-15T00:00:24.455 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T00:00:24.455 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2024-09-15T00:00:24.455 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T00:00:24.455 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T00:00:24.455 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T00:00:24.455 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2024-09-15T00:00:24.477 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T00:00:24.491 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T00:00:24.555 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T00:00:24.555 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-09-15T00:00:24.558 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2024-09-15T00:00:24.575 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T00:00:24.584 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T00:00:24.584 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-09-15T00:00:24.616 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2024-09-15T00:00:24.736 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T00:00:24.839 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T00:00:24.875 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-15T00:00:24.875 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-15T00:00:24.875 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T00:00:24.876 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T00:00:24.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:00:24.894 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-09-15T00:00:24.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:00:24.895 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-09-15T00:00:24.952 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-503.el9.x86_64 2024-09-15T00:00:24.953 INFO:teuthology.task.kernel:Running kernel on smithi184: 5.14.0-503.el9.x86_64 2024-09-15T00:00:24.953 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum install -y kernel 2024-09-15T00:00:24.954 INFO:teuthology.orchestra.run.smithi171.stdout:5.14.0-503.el9.x86_64 2024-09-15T00:00:24.954 INFO:teuthology.task.kernel:Running kernel on smithi171: 5.14.0-503.el9.x86_64 2024-09-15T00:00:24.954 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum install -y kernel 2024-09-15T00:00:28.828 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.2 MB 00:01 2024-09-15T00:00:28.990 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.2 MB 00:02 2024-09-15T00:00:30.512 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-15T00:00:31.000 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-15T00:00:34.821 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-15T00:00:36.371 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 19 kB 00:00 2024-09-15T00:00:37.283 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-15T00:00:37.318 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-15T00:00:39.042 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-15T00:00:39.821 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-15T00:00:42.823 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-15T00:00:44.838 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T00:00:44.838 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T00:00:44.936 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:00:44.951 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:00:44.951 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-09-15T00:00:44.951 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:00:44.951 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:00:44.951 INFO:teuthology.orchestra.run.smithi184.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T00:00:44.951 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-15T00:00:44.951 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T00:00:44.952 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T00:00:44.952 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T00:00:44.952 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:00:44.952 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:00:44.952 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:00:44.952 INFO:teuthology.orchestra.run.smithi184.stdout:Install 4 Packages 2024-09-15T00:00:44.952 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:00:44.953 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 79 M 2024-09-15T00:00:44.953 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 126 M 2024-09-15T00:00:44.953 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:00:45.058 INFO:teuthology.orchestra.run.smithi184.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 269 kB/s | 23 kB 00:00 2024-09-15T00:00:45.277 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-15T00:00:47.317 INFO:teuthology.orchestra.run.smithi171.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T00:00:47.317 INFO:teuthology.orchestra.run.smithi171.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T00:00:47.408 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:00:47.415 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:00:47.415 INFO:teuthology.orchestra.run.smithi171.stdout: Package Architecture Version Repository Size 2024-09-15T00:00:47.415 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:00:47.415 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:00:47.415 INFO:teuthology.orchestra.run.smithi171.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T00:00:47.415 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-15T00:00:47.415 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T00:00:47.415 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T00:00:47.415 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T00:00:47.415 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:00:47.416 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:00:47.416 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:00:47.416 INFO:teuthology.orchestra.run.smithi171.stdout:Install 4 Packages 2024-09-15T00:00:47.416 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:00:47.416 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 79 M 2024-09-15T00:00:47.416 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 126 M 2024-09-15T00:00:47.417 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:00:47.528 INFO:teuthology.orchestra.run.smithi171.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 270 kB/s | 23 kB 00:00 2024-09-15T00:00:48.014 INFO:teuthology.orchestra.run.smithi171.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-15T00:00:48.681 INFO:teuthology.orchestra.run.smithi171.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-15T00:00:48.924 INFO:teuthology.orchestra.run.smithi184.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 4.0 MB/s | 16 MB 00:03 2024-09-15T00:00:49.123 INFO:teuthology.orchestra.run.smithi171.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 21 MB/s | 35 MB 00:01 2024-09-15T00:00:49.123 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:00:49.123 INFO:teuthology.orchestra.run.smithi171.stdout:Total 46 MB/s | 79 MB 00:01 2024-09-15T00:00:49.806 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:00:50.169 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:00:50.169 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:00:51.168 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:00:51.168 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:00:51.316 INFO:teuthology.orchestra.run.smithi184.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 4.6 MB/s | 29 MB 00:06 2024-09-15T00:00:52.424 INFO:teuthology.orchestra.run.smithi184.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 4.6 MB/s | 35 MB 00:07 2024-09-15T00:00:52.425 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:00:52.425 INFO:teuthology.orchestra.run.smithi184.stdout:Total 11 MB/s | 79 MB 00:07 2024-09-15T00:00:53.108 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:00:53.440 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:00:53.440 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:00:53.634 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:00:54.266 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T00:00:54.381 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T00:00:54.405 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:00:54.405 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:00:56.852 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:00:57.188 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T00:00:57.315 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T00:00:57.567 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T00:00:57.660 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T00:01:00.436 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T00:01:00.571 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T00:01:02.178 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T00:01:02.216 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T00:01:05.405 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T00:01:05.447 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T00:01:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T00:01:10.291 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T00:01:45.488 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T00:01:45.527 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T00:01:45.886 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T00:01:45.886 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T00:01:45.887 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T00:01:45.887 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T00:01:46.750 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T00:01:46.750 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:01:46.750 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:01:46.751 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T00:01:46.751 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T00:01:46.751 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T00:01:46.751 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T00:01:46.751 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:01:46.751 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:01:47.066 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T00:01:47.155 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T00:01:47.156 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T00:01:47.156 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T00:01:47.156 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T00:01:47.156 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T00:01:47.157 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-15T00:01:47.157 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi171.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T00:01:47.157 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T00:01:47.216 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T00:01:47.217 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T00:01:47.217 DEBUG:teuthology.orchestra.run.smithi171:> sudo rpm -qi grub2-tools 2024-09-15T00:01:47.251 INFO:teuthology.orchestra.run.smithi171.stdout:Name : grub2-tools 2024-09-15T00:01:47.252 INFO:teuthology.orchestra.run.smithi171.stdout:Epoch : 1 2024-09-15T00:01:47.252 INFO:teuthology.orchestra.run.smithi171.stdout:Version : 2.06 2024-09-15T00:01:47.252 INFO:teuthology.orchestra.run.smithi171.stdout:Release : 82.el9 2024-09-15T00:01:47.252 INFO:teuthology.orchestra.run.smithi171.stdout:Architecture: x86_64 2024-09-15T00:01:47.252 INFO:teuthology.orchestra.run.smithi171.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T00:01:47.252 INFO:teuthology.orchestra.run.smithi171.stdout:Group : Unspecified 2024-09-15T00:01:47.252 INFO:teuthology.orchestra.run.smithi171.stdout:Size : 8274670 2024-09-15T00:01:47.252 INFO:teuthology.orchestra.run.smithi171.stdout:License : GPLv3+ 2024-09-15T00:01:47.252 INFO:teuthology.orchestra.run.smithi171.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T00:01:47.252 INFO:teuthology.orchestra.run.smithi171.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T00:01:47.252 INFO:teuthology.orchestra.run.smithi171.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T00:01:47.252 INFO:teuthology.orchestra.run.smithi171.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T00:01:47.253 INFO:teuthology.orchestra.run.smithi171.stdout:Packager : builder@centos.org 2024-09-15T00:01:47.253 INFO:teuthology.orchestra.run.smithi171.stdout:Vendor : CentOS 2024-09-15T00:01:47.253 INFO:teuthology.orchestra.run.smithi171.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T00:01:47.253 INFO:teuthology.orchestra.run.smithi171.stdout:Summary : Support tools for GRUB. 2024-09-15T00:01:47.253 INFO:teuthology.orchestra.run.smithi171.stdout:Description : 2024-09-15T00:01:47.253 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:01:47.253 INFO:teuthology.orchestra.run.smithi171.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T00:01:47.253 INFO:teuthology.orchestra.run.smithi171.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T00:01:47.253 INFO:teuthology.orchestra.run.smithi171.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T00:01:47.254 INFO:teuthology.orchestra.run.smithi171.stdout:hardware devices. 2024-09-15T00:01:47.254 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:01:47.254 INFO:teuthology.orchestra.run.smithi171.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T00:01:47.255 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T00:01:47.255 INFO:teuthology.task.kernel:Updating grub on smithi171 to boot 5.14.0-505.el9.x86_64 2024-09-15T00:01:47.255 DEBUG:teuthology.orchestra.run.smithi171:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T00:01:47.866 INFO:teuthology.orchestra.run.smithi171.stderr:Generating grub configuration file ... 2024-09-15T00:01:48.337 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T00:01:48.376 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T00:01:48.724 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T00:01:48.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T00:01:48.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T00:01:48.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T00:01:49.190 INFO:teuthology.orchestra.run.smithi171.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T00:01:49.227 INFO:teuthology.orchestra.run.smithi171.stderr:done 2024-09-15T00:01:49.230 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-09-15T00:01:49.245 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.cirV3971gB 2024-09-15T00:01:49.246 DEBUG:teuthology.orchestra.run.smithi171:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.cirV3971gB 2024-09-15T00:01:49.317 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.cirV3971gB 2024-09-15T00:01:49.448 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.cirV3971gB is 6KB 2024-09-15T00:01:49.495 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.cirV3971gB 2024-09-15T00:01:49.511 DEBUG:teuthology.orchestra.run.smithi171:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T00:01:49.573 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T00:01:49.573 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T00:01:49.574 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T00:01:49.574 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T00:01:49.575 DEBUG:teuthology.orchestra.run.smithi171:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T00:01:49.604 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T00:01:49.604 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:01:49.604 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:01:49.604 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T00:01:49.604 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T00:01:49.604 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T00:01:49.604 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T00:01:49.604 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:01:49.604 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:01:49.731 DEBUG:teuthology.orchestra.run.smithi171:> sudo shutdown -r now 2024-09-15T00:01:49.961 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T00:01:50.050 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T00:01:50.050 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T00:01:50.050 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T00:01:50.050 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T00:01:50.050 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T00:01:50.051 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-15T00:01:50.051 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi184.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T00:01:50.051 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T00:01:50.110 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T00:01:50.110 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T00:01:50.110 DEBUG:teuthology.orchestra.run.smithi184:> sudo rpm -qi grub2-tools 2024-09-15T00:01:50.142 INFO:teuthology.orchestra.run.smithi184.stdout:Name : grub2-tools 2024-09-15T00:01:50.142 INFO:teuthology.orchestra.run.smithi184.stdout:Epoch : 1 2024-09-15T00:01:50.142 INFO:teuthology.orchestra.run.smithi184.stdout:Version : 2.06 2024-09-15T00:01:50.142 INFO:teuthology.orchestra.run.smithi184.stdout:Release : 82.el9 2024-09-15T00:01:50.143 INFO:teuthology.orchestra.run.smithi184.stdout:Architecture: x86_64 2024-09-15T00:01:50.143 INFO:teuthology.orchestra.run.smithi184.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T00:01:50.143 INFO:teuthology.orchestra.run.smithi184.stdout:Group : Unspecified 2024-09-15T00:01:50.143 INFO:teuthology.orchestra.run.smithi184.stdout:Size : 8274670 2024-09-15T00:01:50.143 INFO:teuthology.orchestra.run.smithi184.stdout:License : GPLv3+ 2024-09-15T00:01:50.143 INFO:teuthology.orchestra.run.smithi184.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T00:01:50.143 INFO:teuthology.orchestra.run.smithi184.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T00:01:50.143 INFO:teuthology.orchestra.run.smithi184.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T00:01:50.143 INFO:teuthology.orchestra.run.smithi184.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T00:01:50.143 INFO:teuthology.orchestra.run.smithi184.stdout:Packager : builder@centos.org 2024-09-15T00:01:50.143 INFO:teuthology.orchestra.run.smithi184.stdout:Vendor : CentOS 2024-09-15T00:01:50.144 INFO:teuthology.orchestra.run.smithi184.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T00:01:50.144 INFO:teuthology.orchestra.run.smithi184.stdout:Summary : Support tools for GRUB. 2024-09-15T00:01:50.144 INFO:teuthology.orchestra.run.smithi184.stdout:Description : 2024-09-15T00:01:50.144 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:01:50.144 INFO:teuthology.orchestra.run.smithi184.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T00:01:50.144 INFO:teuthology.orchestra.run.smithi184.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T00:01:50.144 INFO:teuthology.orchestra.run.smithi184.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T00:01:50.144 INFO:teuthology.orchestra.run.smithi184.stdout:hardware devices. 2024-09-15T00:01:50.144 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:01:50.144 INFO:teuthology.orchestra.run.smithi184.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T00:01:50.145 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T00:01:50.145 INFO:teuthology.task.kernel:Updating grub on smithi184 to boot 5.14.0-505.el9.x86_64 2024-09-15T00:01:50.145 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T00:01:50.804 INFO:teuthology.orchestra.run.smithi184.stderr:Generating grub configuration file ... 2024-09-15T00:01:52.132 INFO:teuthology.orchestra.run.smithi184.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T00:01:52.161 INFO:teuthology.orchestra.run.smithi184.stderr:done 2024-09-15T00:01:52.164 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-09-15T00:01:52.180 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.xdduapp2l0 2024-09-15T00:01:52.180 DEBUG:teuthology.orchestra.run.smithi184:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xdduapp2l0 2024-09-15T00:01:52.248 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.xdduapp2l0 2024-09-15T00:01:52.405 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.xdduapp2l0 is 6KB 2024-09-15T00:01:52.453 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.xdduapp2l0 2024-09-15T00:01:52.468 DEBUG:teuthology.orchestra.run.smithi184:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T00:01:52.542 INFO:teuthology.orchestra.run.smithi184.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T00:01:52.542 INFO:teuthology.orchestra.run.smithi184.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T00:01:52.542 INFO:teuthology.orchestra.run.smithi184.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T00:01:52.542 INFO:teuthology.orchestra.run.smithi184.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T00:01:52.544 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T00:01:52.733 DEBUG:teuthology.orchestra.run.smithi184:> sudo shutdown -r now 2024-09-15T00:02:19.761 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-15T00:02:19.762 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-15T00:02:19.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:02:22.740 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-15T00:02:22.741 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-15T00:02:22.741 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:02:38.269 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-09-15T00:02:41.085 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2024-09-15T00:02:47.276 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-15T00:02:47.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:02:50.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-15T00:02:50.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:02:50.333 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-09-15T00:03:02.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-15T00:03:02.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:03:50.137 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T00:04:02.149 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-15T00:04:02.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:04:02.349 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T00:04:02.608 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-09-15T00:04:02.977 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-15T00:04:02.977 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-15T00:04:02.977 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-09-15T00:04:02.995 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-505.el9.x86_64 2024-09-15T00:04:02.995 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T00:04:02.995 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T00:04:02.996 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T00:04:03.997 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-15T00:04:03.997 DEBUG:teuthology.orchestra.run.smithi184:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T00:04:04.068 INFO:teuthology.orchestra.run.smithi184.stdout:ttyS1 2024-09-15T00:04:04.095 DEBUG:teuthology.parallel:result is None 2024-09-15T00:04:17.349 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-15T00:04:17.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:04:17.726 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-09-15T00:04:18.090 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-15T00:04:18.090 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-15T00:04:18.090 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-09-15T00:04:18.107 INFO:teuthology.orchestra.run.smithi171.stdout:5.14.0-505.el9.x86_64 2024-09-15T00:04:18.107 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T00:04:18.107 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T00:04:18.107 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T00:04:19.108 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-15T00:04:19.108 DEBUG:teuthology.orchestra.run.smithi171:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T00:04:19.183 INFO:teuthology.orchestra.run.smithi171.stdout:ttyS1 2024-09-15T00:04:19.208 DEBUG:teuthology.parallel:result is None 2024-09-15T00:04:19.209 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T00:04:19.216 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T00:04:19.216 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T00:04:19.220 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T00:04:19.239 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T00:04:19.246 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T00:04:19.252 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T00:04:19.252 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T00:04:19.278 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T00:04:19.328 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T00:04:19.338 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T00:04:19.338 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T00:04:19.341 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T00:04:19.374 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T00:04:19.385 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T00:04:19.396 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T00:04:19.408 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T00:04:19.409 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T00:04:19.439 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T00:04:19.439 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T00:04:19.442 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T00:04:19.479 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T00:04:19.487 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T00:04:19.487 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T00:04:19.513 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T00:04:19.536 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T00:04:19.608 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T00:04:19.660 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T00:04:19.746 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T00:04:19.785 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:04:19.785 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T00:04:19.851 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T00:04:19.894 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T00:04:19.949 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T00:04:20.036 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T00:04:20.077 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:04:20.078 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T00:04:20.144 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-09-15T00:04:20.147 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2024-09-15T00:04:20.192 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T00:04:20.225 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T00:04:20.582 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T00:04:20.589 INFO:teuthology.task.internal:Starting timer... 2024-09-15T00:04:20.589 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T00:04:20.598 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T00:04:20.606 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-15T00:04:20.606 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2024-09-15T00:04:20.656 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2024-09-15T00:04:20.658 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2024-09-15T00:04:20.716 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: 2024-09-15T00:04:20.719 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T00:04:20.719 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2024-09-15T00:04:20.744 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2024-09-15T00:04:20.744 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2024-09-15T00:04:20.765 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2024-09-15T00:04:20.765 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi171.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2024-09-15T00:04:20.765 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T00:04:20.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:04:20.810 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T00:04:20.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:04:20.837 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T00:04:20.837 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2024-09-15T00:04:20.883 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2024-09-15T00:04:20.910 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T00:04:20.932 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T00:04:20.949 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T00:04:20.950 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi171.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T00:08:04.335 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi171.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2024-09-15T00:08:04.337 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-15T00:08:04.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:08:04.419 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-09-15T00:08:04.500 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-15T00:08:04.501 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-15T00:08:04.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:08:04.581 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-09-15T00:08:04.659 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-15T00:08:04.659 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T00:08:04.747 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T00:08:04.747 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T00:08:04.748 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T00:08:04.751 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T00:08:04.751 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T00:08:04.778 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T00:08:04.780 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T00:08:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T00:08:04.794 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T00:08:04.817 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2024-09-15T00:08:04.818 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2024-09-15T00:08:04.828 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2024-09-15T00:08:04.829 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2024-09-15T00:08:04.843 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T00:08:04.844 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T00:08:04.858 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T00:08:04.858 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T00:08:04.913 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-09-15T00:08:04.915 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-09-15T00:08:04.916 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T00:08:04.916 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-09-15T00:08:04.916 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:08:04.917 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:08:04.917 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:08:04.917 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:08:04.919 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T00:08:04.919 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-09-15T00:08:04.919 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:08:04.919 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:08:04.919 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:08:04.919 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:08:04.920 INFO:teuthology.run_tasks:Running task install... 2024-09-15T00:08:04.928 DEBUG:teuthology.task.install:project ceph 2024-09-15T00:08:04.929 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}} 2024-09-15T00:08:04.929 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'} 2024-09-15T00:08:04.929 INFO:teuthology.task.install:Using flavor: default 2024-09-15T00:08:04.935 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T00:08:04.935 INFO:teuthology.task.install:extra packages: [] 2024-09-15T00:08:04.935 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'tag': None, 'wait_for_package': False} 2024-09-15T00:08:04.936 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-15T00:08:04.938 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'tag': None, 'wait_for_package': False} 2024-09-15T00:08:04.938 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-15T00:08:05.086 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/ 2024-09-15T00:08:05.087 INFO:teuthology.task.install.rpm:Package version is 19.1.1-324.gecab7be5 2024-09-15T00:08:05.088 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/ 2024-09-15T00:08:05.088 INFO:teuthology.task.install.rpm:Package version is 19.1.1-324.gecab7be5 2024-09-15T00:08:05.197 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T00:08:05.198 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:08:05.198 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T00:08:05.207 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T00:08:05.208 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:08:05.208 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T00:08:05.223 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-15T00:08:05.223 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ecab7be58963a5bd03910cd73eef66405d7fdd50/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T00:08:05.233 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-15T00:08:05.233 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ecab7be58963a5bd03910cd73eef66405d7fdd50/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T00:08:05.290 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T00:08:05.299 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T00:08:05.364 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T00:08:05.375 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T00:08:05.428 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2024-09-15T00:08:05.430 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2024-09-15T00:08:05.439 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2024-09-15T00:08:05.440 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2024-09-15T00:08:05.666 INFO:teuthology.orchestra.run.smithi184.stdout:45 files removed 2024-09-15T00:08:05.674 INFO:teuthology.orchestra.run.smithi171.stdout:45 files removed 2024-09-15T00:08:05.693 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw 2024-09-15T00:08:05.703 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-radosgw 2024-09-15T00:08:06.253 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 321 kB/s | 84 kB 00:00 2024-09-15T00:08:06.256 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 297 kB/s | 84 kB 00:00 2024-09-15T00:08:06.537 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-15T00:08:06.548 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-15T00:08:06.804 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-15T00:08:06.824 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-15T00:08:07.257 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 21 MB/s | 8.2 MB 00:00 2024-09-15T00:08:08.530 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.2 MB 00:01 2024-09-15T00:08:10.831 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 17 MB/s | 20 MB 00:01 2024-09-15T00:08:11.567 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 20 MB 00:03 2024-09-15T00:08:15.449 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.5 MB 00:00 2024-09-15T00:08:16.992 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-15T00:08:17.093 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-15T00:08:17.863 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-15T00:08:18.526 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 88 kB/s | 19 kB 00:00 2024-09-15T00:08:19.314 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-15T00:08:23.478 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-15T00:08:24.828 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-09-15T00:08:25.879 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:08:25.898 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:08:25.898 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-15T00:08:25.898 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:08:25.898 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:08:25.898 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 11 M 2024-09-15T00:08:25.898 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-09-15T00:08:25.898 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.4 M 2024-09-15T00:08:25.898 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.2 M 2024-09-15T00:08:25.898 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-15T00:08:25.898 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T00:08:25.898 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.4 M 2024-09-15T00:08:25.899 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 21 M 2024-09-15T00:08:25.899 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:19.1.1-324.gecab7be5.el9 ceph 25 k 2024-09-15T00:08:25.899 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T00:08:25.899 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T00:08:25.899 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T00:08:25.899 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 752 k 2024-09-15T00:08:25.899 INFO:teuthology.orchestra.run.smithi171.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T00:08:25.899 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T00:08:25.899 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T00:08:25.899 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 507 k 2024-09-15T00:08:25.899 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T00:08:25.899 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.3 M 2024-09-15T00:08:25.899 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T00:08:25.900 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T00:08:25.900 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T00:08:25.900 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 45 k 2024-09-15T00:08:25.900 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 136 k 2024-09-15T00:08:25.900 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:19.1.1-324.gecab7be5.el9 ceph 162 k 2024-09-15T00:08:25.900 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T00:08:25.900 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:19.1.1-324.gecab7be5.el9 ceph 321 k 2024-09-15T00:08:25.900 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 301 k 2024-09-15T00:08:25.900 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 99 k 2024-09-15T00:08:25.900 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T00:08:25.900 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T00:08:25.900 INFO:teuthology.orchestra.run.smithi171.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T00:08:25.901 INFO:teuthology.orchestra.run.smithi171.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T00:08:25.901 INFO:teuthology.orchestra.run.smithi171.stdout:Installing weak dependencies: 2024-09-15T00:08:25.901 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T00:08:25.901 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:08:25.901 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:08:25.901 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:08:25.901 INFO:teuthology.orchestra.run.smithi171.stdout:Install 30 Packages 2024-09-15T00:08:25.901 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 2 Packages 2024-09-15T00:08:25.901 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:08:25.902 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 60 M 2024-09-15T00:08:25.902 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:08:27.212 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:08:27.224 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:08:27.225 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-15T00:08:27.225 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:08:27.225 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:08:27.225 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 11 M 2024-09-15T00:08:27.225 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2024-09-15T00:08:27.225 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.4 M 2024-09-15T00:08:27.225 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.2 M 2024-09-15T00:08:27.225 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-15T00:08:27.225 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T00:08:27.225 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.4 M 2024-09-15T00:08:27.225 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 21 M 2024-09-15T00:08:27.225 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:19.1.1-324.gecab7be5.el9 ceph 25 k 2024-09-15T00:08:27.226 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T00:08:27.226 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T00:08:27.226 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T00:08:27.226 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 752 k 2024-09-15T00:08:27.226 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T00:08:27.226 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T00:08:27.226 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T00:08:27.226 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 507 k 2024-09-15T00:08:27.226 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T00:08:27.226 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.3 M 2024-09-15T00:08:27.226 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T00:08:27.226 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T00:08:27.226 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T00:08:27.227 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 45 k 2024-09-15T00:08:27.227 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 136 k 2024-09-15T00:08:27.227 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:19.1.1-324.gecab7be5.el9 ceph 162 k 2024-09-15T00:08:27.227 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T00:08:27.227 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:19.1.1-324.gecab7be5.el9 ceph 321 k 2024-09-15T00:08:27.227 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 301 k 2024-09-15T00:08:27.227 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 99 k 2024-09-15T00:08:27.227 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T00:08:27.227 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T00:08:27.227 INFO:teuthology.orchestra.run.smithi184.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T00:08:27.227 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T00:08:27.227 INFO:teuthology.orchestra.run.smithi184.stdout:Installing weak dependencies: 2024-09-15T00:08:27.228 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T00:08:27.228 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:08:27.228 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:08:27.228 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:08:27.228 INFO:teuthology.orchestra.run.smithi184.stdout:Install 30 Packages 2024-09-15T00:08:27.228 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 2 Packages 2024-09-15T00:08:27.228 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:08:27.228 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 60 M 2024-09-15T00:08:27.228 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:08:27.438 INFO:teuthology.orchestra.run.smithi171.stdout:(1/32): ceph-base-19.1.1-324.gecab7be5.el9.x86_ 4.1 MB/s | 5.4 MB 00:01 2024-09-15T00:08:27.480 INFO:teuthology.orchestra.run.smithi171.stdout:(2/32): ceph-selinux-19.1.1-324.gecab7be5.el9.x 606 kB/s | 25 kB 00:00 2024-09-15T00:08:27.705 INFO:teuthology.orchestra.run.smithi171.stdout:(3/32): ceph-radosgw-19.1.1-324.gecab7be5.el9.x 6.6 MB/s | 11 MB 00:01 2024-09-15T00:08:27.780 INFO:teuthology.orchestra.run.smithi171.stdout:(4/32): libcephfs2-19.1.1-324.gecab7be5.el9.x86 2.4 MB/s | 752 kB 00:00 2024-09-15T00:08:27.813 INFO:teuthology.orchestra.run.smithi184.stdout:(1/32): ceph-base-19.1.1-324.gecab7be5.el9.x86_ 14 MB/s | 5.4 MB 00:00 2024-09-15T00:08:27.847 INFO:teuthology.orchestra.run.smithi171.stdout:(5/32): libradosstriper1-19.1.1-324.gecab7be5.e 3.5 MB/s | 507 kB 00:00 2024-09-15T00:08:27.871 INFO:teuthology.orchestra.run.smithi184.stdout:(2/32): ceph-selinux-19.1.1-324.gecab7be5.el9.x 432 kB/s | 25 kB 00:00 2024-09-15T00:08:27.889 INFO:teuthology.orchestra.run.smithi171.stdout:(6/32): python3-ceph-argparse-19.1.1-324.gecab7 1.1 MB/s | 45 kB 00:00 2024-09-15T00:08:27.948 INFO:teuthology.orchestra.run.smithi171.stdout:(7/32): python3-ceph-common-19.1.1-324.gecab7be 2.3 MB/s | 136 kB 00:00 2024-09-15T00:08:27.963 INFO:teuthology.orchestra.run.smithi184.stdout:(3/32): libcephfs2-19.1.1-324.gecab7be5.el9.x86 8.0 MB/s | 752 kB 00:00 2024-09-15T00:08:27.998 INFO:teuthology.orchestra.run.smithi171.stdout:(8/32): python3-cephfs-19.1.1-324.gecab7be5.el9 3.2 MB/s | 162 kB 00:00 2024-09-15T00:08:28.047 INFO:teuthology.orchestra.run.smithi184.stdout:(4/32): libradosstriper1-19.1.1-324.gecab7be5.e 6.0 MB/s | 507 kB 00:00 2024-09-15T00:08:28.082 INFO:teuthology.orchestra.run.smithi171.stdout:(9/32): python3-rados-19.1.1-324.gecab7be5.el9. 3.8 MB/s | 321 kB 00:00 2024-09-15T00:08:28.257 INFO:teuthology.orchestra.run.smithi171.stdout:(10/32): python3-rbd-19.1.1-324.gecab7be5.el9.x 1.7 MB/s | 301 kB 00:00 2024-09-15T00:08:28.308 INFO:teuthology.orchestra.run.smithi171.stdout:(11/32): python3-rgw-19.1.1-324.gecab7be5.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-15T00:08:28.314 INFO:teuthology.orchestra.run.smithi184.stdout:(5/32): librgw2-19.1.1-324.gecab7be5.el9.x86_64 20 MB/s | 5.3 MB 00:00 2024-09-15T00:08:28.372 INFO:teuthology.orchestra.run.smithi184.stdout:(6/32): python3-ceph-argparse-19.1.1-324.gecab7 778 kB/s | 45 kB 00:00 2024-09-15T00:08:28.449 INFO:teuthology.orchestra.run.smithi184.stdout:(7/32): python3-ceph-common-19.1.1-324.gecab7be 1.8 MB/s | 136 kB 00:00 2024-09-15T00:08:28.499 INFO:teuthology.orchestra.run.smithi171.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 558 kB/s | 107 kB 00:00 2024-09-15T00:08:28.583 INFO:teuthology.orchestra.run.smithi171.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-15T00:08:28.634 INFO:teuthology.orchestra.run.smithi171.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-09-15T00:08:28.673 INFO:teuthology.orchestra.run.smithi184.stdout:(8/32): ceph-common-19.1.1-324.gecab7be5.el9.x8 17 MB/s | 21 MB 00:01 2024-09-15T00:08:28.698 INFO:teuthology.orchestra.run.smithi184.stdout:(9/32): python3-cephfs-19.1.1-324.gecab7be5.el9 646 kB/s | 162 kB 00:00 2024-09-15T00:08:28.726 INFO:teuthology.orchestra.run.smithi171.stdout:(15/32): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-09-15T00:08:28.731 INFO:teuthology.orchestra.run.smithi184.stdout:(10/32): python3-rados-19.1.1-324.gecab7be5.el9 5.4 MB/s | 321 kB 00:00 2024-09-15T00:08:28.757 INFO:teuthology.orchestra.run.smithi184.stdout:(11/32): python3-rbd-19.1.1-324.gecab7be5.el9.x 5.0 MB/s | 301 kB 00:00 2024-09-15T00:08:28.799 INFO:teuthology.orchestra.run.smithi184.stdout:(12/32): python3-rgw-19.1.1-324.gecab7be5.el9.x 1.4 MB/s | 99 kB 00:00 2024-09-15T00:08:28.801 INFO:teuthology.orchestra.run.smithi171.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-15T00:08:28.860 INFO:teuthology.orchestra.run.smithi171.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 726 kB/s | 42 kB 00:00 2024-09-15T00:08:28.916 INFO:teuthology.orchestra.run.smithi184.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 673 kB/s | 107 kB 00:00 2024-09-15T00:08:28.918 INFO:teuthology.orchestra.run.smithi171.stdout:(18/32): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-15T00:08:28.949 INFO:teuthology.orchestra.run.smithi184.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-15T00:08:28.969 INFO:teuthology.orchestra.run.smithi171.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 720 kB/s | 36 kB 00:00 2024-09-15T00:08:28.975 INFO:teuthology.orchestra.run.smithi184.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-09-15T00:08:29.019 INFO:teuthology.orchestra.run.smithi171.stdout:(20/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-15T00:08:29.070 INFO:teuthology.orchestra.run.smithi171.stdout:(21/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 6.0 MB/s | 308 kB 00:00 2024-09-15T00:08:29.092 INFO:teuthology.orchestra.run.smithi184.stdout:(16/32): ceph-radosgw-19.1.1-324.gecab7be5.el9. 6.3 MB/s | 11 MB 00:01 2024-09-15T00:08:29.151 INFO:teuthology.orchestra.run.smithi184.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-15T00:08:29.175 INFO:teuthology.orchestra.run.smithi184.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.4 MB/s | 292 kB 00:00 2024-09-15T00:08:29.178 INFO:teuthology.orchestra.run.smithi171.stdout:(22/32): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-15T00:08:29.195 INFO:teuthology.orchestra.run.smithi171.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-15T00:08:29.209 INFO:teuthology.orchestra.run.smithi184.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-15T00:08:29.212 INFO:teuthology.orchestra.run.smithi171.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-15T00:08:29.234 INFO:teuthology.orchestra.run.smithi184.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 612 kB/s | 36 kB 00:00 2024-09-15T00:08:29.238 INFO:teuthology.orchestra.run.smithi171.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-15T00:08:29.260 INFO:teuthology.orchestra.run.smithi184.stdout:(21/32): python3-prettytable-0.7.2-27.el9.noarc 252 kB/s | 42 kB 00:00 2024-09-15T00:08:29.271 INFO:teuthology.orchestra.run.smithi171.stdout:(26/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 25 MB/s | 839 kB 00:00 2024-09-15T00:08:29.285 INFO:teuthology.orchestra.run.smithi184.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-09-15T00:08:29.297 INFO:teuthology.orchestra.run.smithi171.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-15T00:08:29.343 INFO:teuthology.orchestra.run.smithi184.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-15T00:08:29.347 INFO:teuthology.orchestra.run.smithi171.stdout:(28/32): thrift-0.15.0-2.el9.x86_64.rpm 32 MB/s | 1.6 MB 00:00 2024-09-15T00:08:29.369 INFO:teuthology.orchestra.run.smithi184.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 377 kB/s | 26 kB 00:00 2024-09-15T00:08:29.394 INFO:teuthology.orchestra.run.smithi184.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 971 kB/s | 49 kB 00:00 2024-09-15T00:08:29.419 INFO:teuthology.orchestra.run.smithi184.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-15T00:08:29.453 INFO:teuthology.orchestra.run.smithi184.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 5.7 MB/s | 191 kB 00:00 2024-09-15T00:08:29.503 INFO:teuthology.orchestra.run.smithi184.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-15T00:08:29.539 INFO:teuthology.orchestra.run.smithi171.stdout:(29/32): librgw2-19.1.1-324.gecab7be5.el9.x86_6 3.0 MB/s | 5.3 MB 00:01 2024-09-15T00:08:29.570 INFO:teuthology.orchestra.run.smithi184.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-15T00:08:29.629 INFO:teuthology.orchestra.run.smithi184.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-15T00:08:29.746 INFO:teuthology.orchestra.run.smithi184.stdout:(31/32): librados2-19.1.1-324.gecab7be5.el9.x86 14 MB/s | 3.4 MB 00:00 2024-09-15T00:08:29.829 INFO:teuthology.orchestra.run.smithi184.stdout:(32/32): librbd1-19.1.1-324.gecab7be5.el9.x86_6 12 MB/s | 3.2 MB 00:00 2024-09-15T00:08:29.835 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:08:29.835 INFO:teuthology.orchestra.run.smithi184.stdout:Total 23 MB/s | 60 MB 00:02 2024-09-15T00:08:29.940 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:08:29.940 INFO:teuthology.orchestra.run.smithi171.stdout:(30/32): librados2-19.1.1-324.gecab7be5.el9.x86 5.8 MB/s | 3.4 MB 00:00 2024-09-15T00:08:29.975 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:08:29.975 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:08:30.165 INFO:teuthology.orchestra.run.smithi171.stdout:(31/32): librbd1-19.1.1-324.gecab7be5.el9.x86_6 5.1 MB/s | 3.2 MB 00:00 2024-09-15T00:08:30.653 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:08:30.653 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:08:30.673 INFO:teuthology.orchestra.run.smithi171.stdout:(32/32): ceph-common-19.1.1-324.gecab7be5.el9.x 4.6 MB/s | 21 MB 00:04 2024-09-15T00:08:30.678 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:08:30.678 INFO:teuthology.orchestra.run.smithi171.stdout:Total 13 MB/s | 60 MB 00:04 2024-09-15T00:08:30.787 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:08:30.825 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:08:30.825 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:08:31.515 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:08:31.516 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:08:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:08:32.213 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-15T00:08:32.251 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-15T00:08:32.276 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-15T00:08:32.299 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-15T00:08:32.326 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-15T00:08:32.543 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-15T00:08:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-15T00:08:32.919 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-15T00:08:32.985 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:08:33.036 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-15T00:08:33.066 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-15T00:08:33.096 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-15T00:08:33.119 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-15T00:08:33.121 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-15T00:08:33.135 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-15T00:08:33.161 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-15T00:08:33.206 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 8/34 2024-09-15T00:08:33.246 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-15T00:08:33.309 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-15T00:08:33.337 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-15T00:08:33.338 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-15T00:08:33.381 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 11/34 2024-09-15T00:08:33.392 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-15T00:08:33.449 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-15T00:08:33.473 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-15T00:08:33.537 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-15T00:08:33.572 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-15T00:08:33.688 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-15T00:08:33.723 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-15T00:08:33.744 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-15T00:08:33.772 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-15T00:08:33.804 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-15T00:08:33.833 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-15T00:08:33.866 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-15T00:08:33.866 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-15T00:08:33.959 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-15T00:08:34.049 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 8/34 2024-09-15T00:08:34.076 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-15T00:08:34.139 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-15T00:08:34.168 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-15T00:08:34.182 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-15T00:08:34.220 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-15T00:08:34.220 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 11/34 2024-09-15T00:08:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-15T00:08:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-15T00:08:34.315 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-15T00:08:34.342 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 23/34 2024-09-15T00:08:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-15T00:08:34.386 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-15T00:08:34.402 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-15T00:08:34.512 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-15T00:08:34.545 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-15T00:08:34.596 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-15T00:08:34.628 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-15T00:08:34.663 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-15T00:08:34.690 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-15T00:08:34.865 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-15T00:08:34.889 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-15T00:08:34.960 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-15T00:08:35.006 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-15T00:08:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 27/34 2024-09-15T00:08:35.053 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-15T00:08:35.066 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 28/34 2024-09-15T00:08:35.131 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-15T00:08:35.158 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 23/34 2024-09-15T00:08:35.203 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-15T00:08:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-15T00:08:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-15T00:08:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-15T00:08:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 27/34 2024-09-15T00:08:35.867 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 28/34 2024-09-15T00:08:37.335 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-15T00:08:37.380 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-15T00:08:37.950 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-15T00:08:37.992 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-15T00:08:38.083 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-15T00:08:38.084 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T00:08:38.084 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T00:08:38.084 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:08:38.118 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-15T00:08:38.185 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-15T00:08:38.224 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-15T00:08:38.789 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-15T00:08:38.837 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-15T00:08:38.931 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-15T00:08:38.931 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T00:08:38.931 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T00:08:38.931 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:08:38.972 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-15T00:08:48.196 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-15T00:08:48.196 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2024-09-15T00:08:48.196 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2024-09-15T00:08:48.196 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2024-09-15T00:08:48.196 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2024-09-15T00:08:48.196 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2024-09-15T00:08:48.196 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /root 2024-09-15T00:08:48.197 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2024-09-15T00:08:48.197 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:08:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-15T00:08:48.884 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /sys 2024-09-15T00:08:48.884 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /proc 2024-09-15T00:08:48.884 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /mnt 2024-09-15T00:08:48.884 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /var/tmp 2024-09-15T00:08:48.884 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /home 2024-09-15T00:08:48.885 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /root 2024-09-15T00:08:48.885 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /tmp 2024-09-15T00:08:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:08:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-15T00:08:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-15T00:08:49.217 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:08:49.217 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T00:08:49.218 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T00:08:49.218 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T00:08:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:08:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-15T00:08:49.303 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-15T00:08:49.303 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-15T00:08:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-15T00:08:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-15T00:08:49.877 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:08:49.877 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T00:08:49.877 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T00:08:49.878 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T00:08:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:08:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-15T00:08:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-15T00:08:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-15T00:08:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-15T00:08:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 1/34 2024-09-15T00:08:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2/34 2024-09-15T00:08:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 3/34 2024-09-15T00:08:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 4/34 2024-09-15T00:08:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 5/34 2024-09-15T00:08:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 6/34 2024-09-15T00:08:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-15T00:08:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 8/34 2024-09-15T00:08:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 9/34 2024-09-15T00:08:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 10/34 2024-09-15T00:08:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 11/34 2024-09-15T00:08:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-15T00:08:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-15T00:08:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-15T00:08:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-15T00:08:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-15T00:08:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-15T00:08:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-15T00:08:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-15T00:08:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-15T00:08:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-15T00:08:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-15T00:08:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-15T00:08:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-15T00:08:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-15T00:08:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-15T00:08:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-15T00:08:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-15T00:08:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-15T00:08:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-15T00:08:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-15T00:08:50.499 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-15T00:08:50.499 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 33/34 2024-09-15T00:08:51.266 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-15T00:08:51.266 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 1/34 2024-09-15T00:08:51.266 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2/34 2024-09-15T00:08:51.267 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 3/34 2024-09-15T00:08:51.267 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 4/34 2024-09-15T00:08:51.267 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 5/34 2024-09-15T00:08:51.267 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 6/34 2024-09-15T00:08:51.267 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-15T00:08:51.267 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 8/34 2024-09-15T00:08:51.267 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 9/34 2024-09-15T00:08:51.267 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 10/34 2024-09-15T00:08:51.267 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 11/34 2024-09-15T00:08:51.267 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-15T00:08:51.267 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-15T00:08:51.268 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-15T00:08:51.268 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-15T00:08:51.268 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-15T00:08:51.268 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-15T00:08:51.268 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-15T00:08:51.268 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-15T00:08:51.268 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-15T00:08:51.268 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-15T00:08:51.268 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-15T00:08:51.268 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-15T00:08:51.268 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-15T00:08:51.269 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-15T00:08:51.269 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-15T00:08:51.269 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-15T00:08:51.269 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-15T00:08:51.269 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-15T00:08:51.269 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-15T00:08:51.269 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-15T00:08:51.269 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-15T00:08:51.269 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 33/34 2024-09-15T00:08:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-15T00:08:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:08:51.535 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2024-09-15T00:08:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:51.535 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:08:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T00:08:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T00:08:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T00:08:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T00:08:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T00:08:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T00:08:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T00:08:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T00:08:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T00:08:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T00:08:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T00:08:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T00:08:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T00:08:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T00:08:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T00:08:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T00:08:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T00:08:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:08:51.537 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:08:51.899 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-test 2024-09-15T00:08:52.349 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-15T00:08:52.349 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:08:52.349 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-09-15T00:08:52.349 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:52.349 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:52.349 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:08:52.349 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T00:08:52.349 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:52.350 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:52.350 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:52.350 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:52.350 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T00:08:52.350 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T00:08:52.350 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T00:08:52.350 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:52.350 INFO:teuthology.orchestra.run.smithi171.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T00:08:52.350 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T00:08:52.350 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T00:08:52.350 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:52.350 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T00:08:52.350 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:52.350 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T00:08:52.351 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T00:08:52.351 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T00:08:52.351 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:52.351 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:52.351 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:52.351 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T00:08:52.351 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:52.351 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:52.351 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:08:52.351 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T00:08:52.351 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T00:08:52.351 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T00:08:52.351 INFO:teuthology.orchestra.run.smithi171.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T00:08:52.351 INFO:teuthology.orchestra.run.smithi171.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T00:08:52.352 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:08:52.352 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:08:52.471 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:28 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:08:52.612 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:08:52.613 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:08:52.613 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-15T00:08:52.613 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:08:52.613 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:08:52.613 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:19.1.1-324.gecab7be5.el9 ceph 49 M 2024-09-15T00:08:52.613 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-15T00:08:52.613 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:19.1.1-324.gecab7be5.el9 ceph 163 k 2024-09-15T00:08:52.613 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T00:08:52.613 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T00:08:52.614 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:08:52.614 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:08:52.614 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:08:52.614 INFO:teuthology.orchestra.run.smithi184.stdout:Install 4 Packages 2024-09-15T00:08:52.614 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:08:52.614 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 49 M 2024-09-15T00:08:52.615 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 209 M 2024-09-15T00:08:52.615 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:08:52.687 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-test 2024-09-15T00:08:52.992 INFO:teuthology.orchestra.run.smithi184.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-15T00:08:53.025 INFO:teuthology.orchestra.run.smithi184.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2024-09-15T00:08:53.075 INFO:teuthology.orchestra.run.smithi184.stdout:(3/4): libcephsqlite-19.1.1-324.gecab7be5.el9.x 786 kB/s | 163 kB 00:00 2024-09-15T00:08:53.256 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:30 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:08:53.389 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:08:53.390 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:08:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-15T00:08:53.390 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:08:53.390 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:08:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:19.1.1-324.gecab7be5.el9 ceph 49 M 2024-09-15T00:08:53.390 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-15T00:08:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:19.1.1-324.gecab7be5.el9 ceph 163 k 2024-09-15T00:08:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T00:08:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T00:08:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:08:53.391 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:08:53.391 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:08:53.391 INFO:teuthology.orchestra.run.smithi171.stdout:Install 4 Packages 2024-09-15T00:08:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:08:53.391 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 49 M 2024-09-15T00:08:53.391 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 209 M 2024-09-15T00:08:53.391 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:08:54.040 INFO:teuthology.orchestra.run.smithi171.stdout:(1/4): libcephsqlite-19.1.1-324.gecab7be5.el9.x 419 kB/s | 163 kB 00:00 2024-09-15T00:08:54.199 INFO:teuthology.orchestra.run.smithi171.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 554 kB/s | 304 kB 00:00 2024-09-15T00:08:54.392 INFO:teuthology.orchestra.run.smithi184.stdout:(4/4): ceph-test-19.1.1-324.gecab7be5.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-15T00:08:54.394 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:08:54.395 INFO:teuthology.orchestra.run.smithi184.stdout:Total 28 MB/s | 49 MB 00:01 2024-09-15T00:08:54.408 INFO:teuthology.orchestra.run.smithi171.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 173 kB/s | 64 kB 00:00 2024-09-15T00:08:54.422 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:08:54.441 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:08:54.441 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:08:54.832 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:08:54.833 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:08:55.292 INFO:teuthology.orchestra.run.smithi171.stdout:(4/4): ceph-test-19.1.1-324.gecab7be5.el9.x86_6 30 MB/s | 49 MB 00:01 2024-09-15T00:08:55.294 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:08:55.294 INFO:teuthology.orchestra.run.smithi171.stdout:Total 26 MB/s | 49 MB 00:01 2024-09-15T00:08:55.321 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:08:55.342 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:08:55.342 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:08:55.358 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:08:55.444 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-15T00:08:55.484 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-15T00:08:55.511 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-15T00:08:55.742 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:08:55.742 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:08:56.265 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:08:56.368 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-15T00:08:56.408 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-15T00:08:56.449 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-15T00:09:00.117 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-15T00:09:00.192 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-15T00:09:01.102 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-15T00:09:01.194 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-15T00:09:02.800 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-15T00:09:02.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 1/4 2024-09-15T00:09:02.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2/4 2024-09-15T00:09:02.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-15T00:09:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-15T00:09:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:03.136 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:09:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T00:09:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T00:09:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:03.136 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:09:03.404 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph 2024-09-15T00:09:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-15T00:09:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 1/4 2024-09-15T00:09:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2/4 2024-09-15T00:09:03.552 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-15T00:09:03.862 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-15T00:09:03.862 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:03.862 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:09:03.862 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:03.862 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:03.862 INFO:teuthology.orchestra.run.smithi171.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T00:09:03.862 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T00:09:03.862 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:03.862 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:09:03.997 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:39 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:09:04.155 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:09:04.158 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:04.158 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-15T00:09:04.158 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:04.158 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:09:04.158 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:19.1.1-324.gecab7be5.el9 ceph 6.5 k 2024-09-15T00:09:04.158 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-15T00:09:04.158 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:19.1.1-324.gecab7be5.el9 ceph 2.4 M 2024-09-15T00:09:04.158 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:19.1.1-324.gecab7be5.el9 ceph 1.5 M 2024-09-15T00:09:04.159 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 249 k 2024-09-15T00:09:04.159 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:19.1.1-324.gecab7be5.el9 ceph 4.7 M 2024-09-15T00:09:04.159 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 17 M 2024-09-15T00:09:04.159 INFO:teuthology.orchestra.run.smithi184.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T00:09:04.159 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T00:09:04.159 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T00:09:04.159 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T00:09:04.159 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T00:09:04.159 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T00:09:04.159 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T00:09:04.159 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T00:09:04.159 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T00:09:04.159 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T00:09:04.159 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T00:09:04.160 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T00:09:04.160 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T00:09:04.160 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T00:09:04.160 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T00:09:04.160 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T00:09:04.160 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T00:09:04.160 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T00:09:04.160 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T00:09:04.160 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T00:09:04.160 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T00:09:04.160 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T00:09:04.160 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T00:09:04.161 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T00:09:04.161 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T00:09:04.161 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T00:09:04.161 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T00:09:04.161 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T00:09:04.161 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T00:09:04.161 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T00:09:04.161 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T00:09:04.161 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:04.161 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:09:04.161 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:04.161 INFO:teuthology.orchestra.run.smithi184.stdout:Install 37 Packages 2024-09-15T00:09:04.161 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:04.162 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 30 M 2024-09-15T00:09:04.162 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 108 M 2024-09-15T00:09:04.162 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:09:04.216 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph 2024-09-15T00:09:04.808 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:41 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:09:04.864 INFO:teuthology.orchestra.run.smithi184.stdout:(1/37): ceph-19.1.1-324.gecab7be5.el9.x86_64.rp 45 kB/s | 6.5 kB 00:00 2024-09-15T00:09:04.952 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:09:04.955 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:04.955 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-15T00:09:04.955 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:04.955 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:09:04.955 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:19.1.1-324.gecab7be5.el9 ceph 6.5 k 2024-09-15T00:09:04.955 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-15T00:09:04.955 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:19.1.1-324.gecab7be5.el9 ceph 2.4 M 2024-09-15T00:09:04.955 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:19.1.1-324.gecab7be5.el9 ceph 1.5 M 2024-09-15T00:09:04.955 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 249 k 2024-09-15T00:09:04.955 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:19.1.1-324.gecab7be5.el9 ceph 4.7 M 2024-09-15T00:09:04.955 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 17 M 2024-09-15T00:09:04.956 INFO:teuthology.orchestra.run.smithi171.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T00:09:04.956 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T00:09:04.956 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T00:09:04.956 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T00:09:04.956 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T00:09:04.956 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T00:09:04.956 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T00:09:04.956 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T00:09:04.956 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T00:09:04.956 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T00:09:04.956 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T00:09:04.956 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T00:09:04.956 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T00:09:04.957 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T00:09:04.957 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T00:09:04.957 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T00:09:04.957 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T00:09:04.957 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T00:09:04.957 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T00:09:04.957 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T00:09:04.957 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T00:09:04.957 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T00:09:04.957 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T00:09:04.957 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T00:09:04.958 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T00:09:04.958 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T00:09:04.958 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T00:09:04.958 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T00:09:04.958 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T00:09:04.958 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T00:09:04.958 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T00:09:04.958 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:04.958 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:09:04.958 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:04.958 INFO:teuthology.orchestra.run.smithi171.stdout:Install 37 Packages 2024-09-15T00:09:04.958 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:04.959 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 30 M 2024-09-15T00:09:04.959 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 108 M 2024-09-15T00:09:04.959 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:09:05.458 INFO:teuthology.orchestra.run.smithi171.stdout:(1/37): ceph-19.1.1-324.gecab7be5.el9.x86_64.rp 25 kB/s | 6.5 kB 00:00 2024-09-15T00:09:05.714 INFO:teuthology.orchestra.run.smithi184.stdout:(2/37): ceph-mgr-19.1.1-324.gecab7be5.el9.x86_6 1.5 MB/s | 1.5 MB 00:00 2024-09-15T00:09:05.798 INFO:teuthology.orchestra.run.smithi184.stdout:(3/37): ceph-mds-19.1.1-324.gecab7be5.el9.x86_6 2.2 MB/s | 2.4 MB 00:01 2024-09-15T00:09:05.889 INFO:teuthology.orchestra.run.smithi184.stdout:(4/37): ceph-mgr-modules-core-19.1.1-324.gecab7 2.7 MB/s | 249 kB 00:00 2024-09-15T00:09:05.899 INFO:teuthology.orchestra.run.smithi171.stdout:(2/37): ceph-mgr-19.1.1-324.gecab7be5.el9.x86_6 2.2 MB/s | 1.5 MB 00:00 2024-09-15T00:09:05.983 INFO:teuthology.orchestra.run.smithi171.stdout:(3/37): ceph-mds-19.1.1-324.gecab7be5.el9.x86_6 3.0 MB/s | 2.4 MB 00:00 2024-09-15T00:09:06.034 INFO:teuthology.orchestra.run.smithi171.stdout:(4/37): ceph-mgr-modules-core-19.1.1-324.gecab7 4.9 MB/s | 249 kB 00:00 2024-09-15T00:09:06.090 INFO:teuthology.orchestra.run.smithi184.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-15T00:09:06.209 INFO:teuthology.orchestra.run.smithi171.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-15T00:09:06.231 INFO:teuthology.orchestra.run.smithi184.stdout:(6/37): ceph-mon-19.1.1-324.gecab7be5.el9.x86_6 3.4 MB/s | 4.7 MB 00:01 2024-09-15T00:09:06.274 INFO:teuthology.orchestra.run.smithi184.stdout:(7/37): python3-cryptography-36.0.1-4.el9.x86_6 6.7 MB/s | 1.2 MB 00:00 2024-09-15T00:09:06.276 INFO:teuthology.orchestra.run.smithi171.stdout:(6/37): ceph-mon-19.1.1-324.gecab7be5.el9.x86_6 5.8 MB/s | 4.7 MB 00:00 2024-09-15T00:09:06.332 INFO:teuthology.orchestra.run.smithi184.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.3 MB/s | 135 kB 00:00 2024-09-15T00:09:06.343 INFO:teuthology.orchestra.run.smithi171.stdout:(7/37): python3-cryptography-36.0.1-4.el9.x86_6 9.2 MB/s | 1.2 MB 00:00 2024-09-15T00:09:06.382 INFO:teuthology.orchestra.run.smithi184.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-15T00:09:06.393 INFO:teuthology.orchestra.run.smithi171.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 911 kB/s | 106 kB 00:00 2024-09-15T00:09:06.416 INFO:teuthology.orchestra.run.smithi184.stdout:(10/37): python3-ply-3.11-14.el9.noarch.rpm 578 kB/s | 106 kB 00:00 2024-09-15T00:09:06.419 INFO:teuthology.orchestra.run.smithi171.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.8 MB/s | 135 kB 00:00 2024-09-15T00:09:06.441 INFO:teuthology.orchestra.run.smithi184.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-15T00:09:06.477 INFO:teuthology.orchestra.run.smithi171.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 1.5 MB/s | 125 kB 00:00 2024-09-15T00:09:06.503 INFO:teuthology.orchestra.run.smithi171.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-15T00:09:06.553 INFO:teuthology.orchestra.run.smithi171.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-15T00:09:06.566 INFO:teuthology.orchestra.run.smithi184.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-15T00:09:06.578 INFO:teuthology.orchestra.run.smithi171.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-15T00:09:06.592 INFO:teuthology.orchestra.run.smithi184.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-09-15T00:09:06.629 INFO:teuthology.orchestra.run.smithi171.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 461 kB/s | 35 kB 00:00 2024-09-15T00:09:06.634 INFO:teuthology.orchestra.run.smithi184.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 519 kB/s | 35 kB 00:00 2024-09-15T00:09:06.670 INFO:teuthology.orchestra.run.smithi171.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 838 kB/s | 77 kB 00:00 2024-09-15T00:09:06.676 INFO:teuthology.orchestra.run.smithi184.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 924 kB/s | 77 kB 00:00 2024-09-15T00:09:06.696 INFO:teuthology.orchestra.run.smithi171.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 760 kB/s | 51 kB 00:00 2024-09-15T00:09:06.701 INFO:teuthology.orchestra.run.smithi184.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 761 kB/s | 51 kB 00:00 2024-09-15T00:09:06.751 INFO:teuthology.orchestra.run.smithi184.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-09-15T00:09:06.785 INFO:teuthology.orchestra.run.smithi184.stdout:(18/37): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-09-15T00:09:06.827 INFO:teuthology.orchestra.run.smithi184.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-15T00:09:06.852 INFO:teuthology.orchestra.run.smithi184.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 668 kB/s | 43 kB 00:00 2024-09-15T00:09:06.863 INFO:teuthology.orchestra.run.smithi171.stdout:(17/37): ceph-osd-19.1.1-324.gecab7be5.el9.x86_ 17 MB/s | 17 MB 00:00 2024-09-15T00:09:06.886 INFO:teuthology.orchestra.run.smithi184.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.0 MB/s | 172 kB 00:00 2024-09-15T00:09:06.888 INFO:teuthology.orchestra.run.smithi171.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 191 kB/s | 42 kB 00:00 2024-09-15T00:09:06.911 INFO:teuthology.orchestra.run.smithi184.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 422 kB/s | 11 kB 00:00 2024-09-15T00:09:06.930 INFO:teuthology.orchestra.run.smithi171.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 95 kB/s | 22 kB 00:00 2024-09-15T00:09:06.936 INFO:teuthology.orchestra.run.smithi184.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 709 kB/s | 18 kB 00:00 2024-09-15T00:09:06.956 INFO:teuthology.orchestra.run.smithi171.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 648 kB/s | 43 kB 00:00 2024-09-15T00:09:06.962 INFO:teuthology.orchestra.run.smithi184.stdout:(24/37): python3-cherrypy-18.6.1-2.el9.noarch.r 3.2 MB/s | 358 kB 00:00 2024-09-15T00:09:06.978 INFO:teuthology.orchestra.run.smithi184.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 553 kB/s | 23 kB 00:00 2024-09-15T00:09:06.994 INFO:teuthology.orchestra.run.smithi171.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-09-15T00:09:06.996 INFO:teuthology.orchestra.run.smithi184.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 583 kB/s | 19 kB 00:00 2024-09-15T00:09:07.011 INFO:teuthology.orchestra.run.smithi171.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 708 kB/s | 11 kB 00:00 2024-09-15T00:09:07.012 INFO:teuthology.orchestra.run.smithi184.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 586 kB/s | 20 kB 00:00 2024-09-15T00:09:07.028 INFO:teuthology.orchestra.run.smithi171.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 1.0 MB/s | 18 kB 00:00 2024-09-15T00:09:07.029 INFO:teuthology.orchestra.run.smithi184.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 1.3 MB/s | 46 kB 00:00 2024-09-15T00:09:07.053 INFO:teuthology.orchestra.run.smithi171.stdout:(24/37): python3-cherrypy-18.6.1-2.el9.noarch.r 3.9 MB/s | 358 kB 00:00 2024-09-15T00:09:07.055 INFO:teuthology.orchestra.run.smithi184.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-09-15T00:09:07.070 INFO:teuthology.orchestra.run.smithi171.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 549 kB/s | 23 kB 00:00 2024-09-15T00:09:07.072 INFO:teuthology.orchestra.run.smithi184.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 1.0 MB/s | 16 kB 00:00 2024-09-15T00:09:07.096 INFO:teuthology.orchestra.run.smithi171.stdout:(26/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-15T00:09:07.105 INFO:teuthology.orchestra.run.smithi184.stdout:(31/37): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-15T00:09:07.121 INFO:teuthology.orchestra.run.smithi171.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 288 kB/s | 19 kB 00:00 2024-09-15T00:09:07.130 INFO:teuthology.orchestra.run.smithi184.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-15T00:09:07.155 INFO:teuthology.orchestra.run.smithi171.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 234 kB/s | 20 kB 00:00 2024-09-15T00:09:07.156 INFO:teuthology.orchestra.run.smithi184.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 713 kB/s | 36 kB 00:00 2024-09-15T00:09:07.180 INFO:teuthology.orchestra.run.smithi171.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 790 kB/s | 46 kB 00:00 2024-09-15T00:09:07.181 INFO:teuthology.orchestra.run.smithi184.stdout:(34/37): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-15T00:09:07.205 INFO:teuthology.orchestra.run.smithi171.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 939 kB/s | 79 kB 00:00 2024-09-15T00:09:07.231 INFO:teuthology.orchestra.run.smithi171.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 325 kB/s | 16 kB 00:00 2024-09-15T00:09:07.256 INFO:teuthology.orchestra.run.smithi171.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-15T00:09:07.281 INFO:teuthology.orchestra.run.smithi171.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 713 kB/s | 36 kB 00:00 2024-09-15T00:09:07.307 INFO:teuthology.orchestra.run.smithi171.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-15T00:09:07.341 INFO:teuthology.orchestra.run.smithi171.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 591 kB/s | 20 kB 00:00 2024-09-15T00:09:07.357 INFO:teuthology.orchestra.run.smithi184.stdout:(35/37): ceph-osd-19.1.1-324.gecab7be5.el9.x86_ 10 MB/s | 17 MB 00:01 2024-09-15T00:09:07.366 INFO:teuthology.orchestra.run.smithi171.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-15T00:09:07.382 INFO:teuthology.orchestra.run.smithi184.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 100 kB/s | 20 kB 00:00 2024-09-15T00:09:07.399 INFO:teuthology.orchestra.run.smithi171.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.5 MB/s | 427 kB 00:00 2024-09-15T00:09:07.404 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:07.404 INFO:teuthology.orchestra.run.smithi171.stdout:Total 12 MB/s | 30 MB 00:02 2024-09-15T00:09:07.407 INFO:teuthology.orchestra.run.smithi184.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.7 MB/s | 427 kB 00:00 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi184.stdout:Total 9.3 MB/s | 30 MB 00:03 2024-09-15T00:09:07.552 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:09:07.555 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:09:07.592 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:09:07.593 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:09:07.594 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:09:07.594 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:09:08.065 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:09:08.066 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:09:08.140 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:09:08.140 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:09:09.473 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:09:09.536 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-15T00:09:09.538 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:09:09.571 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-15T00:09:09.601 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-15T00:09:09.614 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-15T00:09:09.655 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-15T00:09:09.685 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-15T00:09:09.696 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-15T00:09:09.740 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-15T00:09:09.767 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-15T00:09:09.771 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-15T00:09:09.811 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-15T00:09:09.844 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-15T00:09:09.863 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-15T00:09:09.937 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-15T00:09:09.967 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-15T00:09:10.007 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-15T00:09:10.008 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-15T00:09:10.044 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-15T00:09:10.055 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-15T00:09:10.073 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-15T00:09:10.090 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-15T00:09:10.108 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-15T00:09:10.120 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-15T00:09:10.142 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-15T00:09:10.155 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-15T00:09:10.182 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-15T00:09:10.188 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-15T00:09:10.217 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-15T00:09:10.228 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-15T00:09:10.252 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-15T00:09:10.263 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-15T00:09:10.298 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-15T00:09:10.300 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-15T00:09:10.333 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-15T00:09:10.346 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-15T00:09:10.379 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-15T00:09:10.406 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-15T00:09:10.452 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-15T00:09:10.546 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-15T00:09:10.601 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-15T00:09:10.634 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-15T00:09:10.687 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-15T00:09:10.697 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-15T00:09:10.726 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-15T00:09:10.750 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-15T00:09:10.775 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-15T00:09:10.789 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-15T00:09:10.831 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-15T00:09:10.845 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-15T00:09:10.901 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-15T00:09:11.016 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-15T00:09:11.073 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-15T00:09:11.096 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-15T00:09:11.142 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-15T00:09:11.151 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-15T00:09:11.219 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-15T00:09:11.248 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-15T00:09:11.298 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-15T00:09:11.309 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-15T00:09:11.341 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-15T00:09:11.413 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-15T00:09:11.458 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-15T00:09:11.578 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 32/37 2024-09-15T00:09:11.609 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-15T00:09:11.621 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 32/37 2024-09-15T00:09:11.635 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-15T00:09:11.635 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:09:11.635 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T00:09:11.635 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T00:09:11.636 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T00:09:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:11.652 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-15T00:09:11.678 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-15T00:09:11.678 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:09:11.678 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T00:09:11.678 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T00:09:11.678 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T00:09:11.678 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:13.077 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-15T00:09:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-15T00:09:13.105 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:09:13.105 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T00:09:13.105 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T00:09:13.105 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T00:09:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:13.111 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-15T00:09:13.139 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-15T00:09:13.139 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:09:13.139 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T00:09:13.139 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T00:09:13.139 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T00:09:13.139 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:13.545 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-15T00:09:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-15T00:09:13.569 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:09:13.569 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T00:09:13.569 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T00:09:13.569 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T00:09:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:13.587 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-15T00:09:13.610 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-15T00:09:13.611 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:09:13.611 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T00:09:13.611 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T00:09:13.611 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T00:09:13.611 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:13.795 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-15T00:09:13.819 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-15T00:09:13.819 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:09:13.820 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T00:09:13.820 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T00:09:13.820 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T00:09:13.820 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:13.846 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-15T00:09:13.869 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-15T00:09:13.869 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:09:13.869 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T00:09:13.870 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T00:09:13.870 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T00:09:13.870 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:13.883 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-15T00:09:13.934 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-15T00:09:15.627 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-15T00:09:15.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 1/37 2024-09-15T00:09:15.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2/37 2024-09-15T00:09:15.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 3/37 2024-09-15T00:09:15.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 4/37 2024-09-15T00:09:15.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 5/37 2024-09-15T00:09:15.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 6/37 2024-09-15T00:09:15.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-15T00:09:15.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-15T00:09:15.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-15T00:09:15.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-15T00:09:15.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-15T00:09:15.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-15T00:09:15.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-15T00:09:15.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-15T00:09:15.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-15T00:09:15.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-15T00:09:15.629 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-15T00:09:15.629 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-15T00:09:15.629 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-15T00:09:15.629 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-15T00:09:15.629 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-15T00:09:15.629 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-15T00:09:15.629 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-15T00:09:15.629 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-15T00:09:15.629 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-15T00:09:15.629 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-15T00:09:15.631 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-15T00:09:15.631 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-15T00:09:15.631 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-15T00:09:15.631 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-15T00:09:15.631 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-15T00:09:15.631 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-15T00:09:15.631 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-15T00:09:15.631 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-15T00:09:15.631 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-15T00:09:15.631 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 1/37 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2/37 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 3/37 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 4/37 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 5/37 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 6/37 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-15T00:09:15.825 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-15T00:09:15.825 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-15T00:09:16.652 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-15T00:09:16.652 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:16.652 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:09:16.652 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:16.652 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:16.652 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:16.652 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:16.653 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:16.653 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:16.653 INFO:teuthology.orchestra.run.smithi171.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T00:09:16.653 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T00:09:16.653 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T00:09:16.653 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T00:09:16.653 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T00:09:16.653 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T00:09:16.653 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T00:09:16.653 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T00:09:16.653 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T00:09:16.653 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T00:09:16.654 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T00:09:16.654 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T00:09:16.654 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T00:09:16.654 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T00:09:16.654 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T00:09:16.654 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T00:09:16.654 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T00:09:16.654 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T00:09:16.654 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T00:09:16.654 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T00:09:16.654 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T00:09:16.654 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T00:09:16.654 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T00:09:16.655 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T00:09:16.655 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T00:09:16.655 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T00:09:16.655 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T00:09:16.655 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T00:09:16.655 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T00:09:16.655 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T00:09:16.655 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T00:09:16.655 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:16.655 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:09:16.816 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-15T00:09:16.816 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:16.816 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:09:16.816 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:16.816 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:16.816 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:16.816 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:16.817 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:16.817 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:16.817 INFO:teuthology.orchestra.run.smithi184.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T00:09:16.817 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T00:09:16.817 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T00:09:16.817 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T00:09:16.817 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T00:09:16.817 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T00:09:16.817 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T00:09:16.817 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T00:09:16.817 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T00:09:16.817 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T00:09:16.817 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T00:09:16.817 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T00:09:16.818 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T00:09:16.818 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T00:09:16.818 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T00:09:16.818 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T00:09:16.818 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T00:09:16.818 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T00:09:16.818 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T00:09:16.818 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T00:09:16.818 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T00:09:16.818 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T00:09:16.818 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T00:09:16.818 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T00:09:16.818 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T00:09:16.818 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T00:09:16.819 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T00:09:16.819 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T00:09:16.819 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T00:09:16.819 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T00:09:16.819 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T00:09:16.819 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:16.819 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:09:17.056 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-base 2024-09-15T00:09:17.204 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-base 2024-09-15T00:09:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:54 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:09:17.724 INFO:teuthology.orchestra.run.smithi171.stdout:Package ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:09:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:09:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-15T00:09:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:09:17.769 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:53 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:09:17.841 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install cephadm 2024-09-15T00:09:17.860 INFO:teuthology.orchestra.run.smithi184.stdout:Package ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:09:17.902 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:09:17.903 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-15T00:09:17.903 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:09:17.975 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install cephadm 2024-09-15T00:09:18.405 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:55 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:09:18.538 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:54 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:09:18.539 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:09:18.539 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:18.539 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-15T00:09:18.539 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:18.539 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:09:18.539 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 765 k 2024-09-15T00:09:18.540 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:18.540 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:09:18.540 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:18.540 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-15T00:09:18.540 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:18.540 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 765 k 2024-09-15T00:09:18.540 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 771 k 2024-09-15T00:09:18.540 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:09:18.671 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:09:18.671 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:18.672 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-15T00:09:18.672 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:18.672 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:09:18.672 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 765 k 2024-09-15T00:09:18.672 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:18.672 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:09:18.672 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:18.672 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-09-15T00:09:18.672 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:18.672 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 765 k 2024-09-15T00:09:18.672 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 771 k 2024-09-15T00:09:18.672 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:09:18.808 INFO:teuthology.orchestra.run.smithi171.stdout:cephadm-19.1.1-324.gecab7be5.el9.noarch.rpm 2.8 MB/s | 765 kB 00:00 2024-09-15T00:09:18.808 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:18.808 INFO:teuthology.orchestra.run.smithi171.stdout:Total 2.8 MB/s | 765 kB 00:00 2024-09-15T00:09:18.809 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:09:18.813 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:09:18.814 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:09:18.819 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:09:18.820 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:09:18.894 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:09:18.937 INFO:teuthology.orchestra.run.smithi184.stdout:cephadm-19.1.1-324.gecab7be5.el9.noarch.rpm 2.8 MB/s | 765 kB 00:00 2024-09-15T00:09:18.938 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:18.938 INFO:teuthology.orchestra.run.smithi184.stdout:Total 2.8 MB/s | 765 kB 00:00 2024-09-15T00:09:18.938 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:09:18.943 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:09:18.944 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:09:18.950 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:09:18.950 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:09:19.026 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:09:19.327 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:09:19.413 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:09:19.431 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:09:19.524 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:09:19.767 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:09:19.866 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:09:20.012 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:09:20.012 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:20.012 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:09:20.012 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:20.012 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:20.012 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:09:20.141 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:09:20.141 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:20.141 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:09:20.141 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:20.141 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:20.141 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:09:20.231 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T00:09:20.367 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T00:09:20.811 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:57 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:09:20.933 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:56 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:09:20.946 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:09:20.947 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:20.947 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repo Size 2024-09-15T00:09:20.947 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:20.947 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:09:20.947 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-324.gecab7be5.el9 ceph 145 k 2024-09-15T00:09:20.947 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:20.947 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:09:20.947 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:20.947 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-15T00:09:20.947 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:20.948 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 145 k 2024-09-15T00:09:20.948 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 435 k 2024-09-15T00:09:20.948 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:09:21.067 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:09:21.068 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:21.068 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-09-15T00:09:21.068 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:21.068 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:09:21.068 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-324.gecab7be5.el9 ceph 145 k 2024-09-15T00:09:21.068 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:21.068 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:09:21.068 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:21.068 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-09-15T00:09:21.068 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:21.068 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 145 k 2024-09-15T00:09:21.069 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 435 k 2024-09-15T00:09:21.069 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:09:21.160 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-immutable-object-cache-19.1.1-324.gecab7be 684 kB/s | 145 kB 00:00 2024-09-15T00:09:21.161 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:21.161 INFO:teuthology.orchestra.run.smithi171.stdout:Total 679 kB/s | 145 kB 00:00 2024-09-15T00:09:21.161 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:09:21.169 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:09:21.169 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:09:21.227 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:09:21.227 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:09:21.269 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-immutable-object-cache-19.1.1-324.gecab7be 726 kB/s | 145 kB 00:00 2024-09-15T00:09:21.269 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:21.270 INFO:teuthology.orchestra.run.smithi184.stdout:Total 720 kB/s | 145 kB 00:00 2024-09-15T00:09:21.270 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:09:21.277 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:09:21.277 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:09:21.335 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:09:21.335 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:09:21.389 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:09:21.462 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-15T00:09:21.486 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:21.578 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-15T00:09:21.603 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-15T00:09:21.603 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:09:21.603 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T00:09:21.603 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-15T00:09:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:22.404 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:09:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:22.404 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:09:22.609 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-15T00:09:22.609 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:22.609 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:09:22.609 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:22.609 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:22.609 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:09:22.675 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-mgr 2024-09-15T00:09:22.845 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr 2024-09-15T00:09:23.260 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:00 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:09:23.352 INFO:teuthology.orchestra.run.smithi171.stdout:Package ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:09:23.396 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:09:23.397 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-15T00:09:23.397 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:09:23.418 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:59 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:09:23.469 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T00:09:23.513 INFO:teuthology.orchestra.run.smithi184.stdout:Package ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:09:23.555 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:09:23.570 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-15T00:09:23.570 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:09:23.629 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T00:09:24.030 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:01 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:09:24.163 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:09:24.164 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 3.6 M 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 32 k 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 16 k 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi171.stdout:Install 12 Packages 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 8.7 M 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 104 M 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:09:24.192 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:00 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:09:24.325 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:09:24.326 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:24.326 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-15T00:09:24.326 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:24.326 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:09:24.326 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 3.6 M 2024-09-15T00:09:24.326 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-15T00:09:24.326 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T00:09:24.326 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 32 k 2024-09-15T00:09:24.326 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 16 k 2024-09-15T00:09:24.327 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T00:09:24.327 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T00:09:24.327 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T00:09:24.327 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T00:09:24.327 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T00:09:24.327 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T00:09:24.327 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T00:09:24.327 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T00:09:24.327 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:24.327 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:09:24.327 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:24.327 INFO:teuthology.orchestra.run.smithi184.stdout:Install 12 Packages 2024-09-15T00:09:24.327 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:24.328 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 8.7 M 2024-09-15T00:09:24.328 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 104 M 2024-09-15T00:09:24.328 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:09:24.559 INFO:teuthology.orchestra.run.smithi171.stdout:(1/12): ceph-prometheus-alerts-19.1.1-324.gecab 106 kB/s | 16 kB 00:00 2024-09-15T00:09:24.584 INFO:teuthology.orchestra.run.smithi171.stdout:(2/12): ceph-grafana-dashboards-19.1.1-324.geca 176 kB/s | 32 kB 00:00 2024-09-15T00:09:24.726 INFO:teuthology.orchestra.run.smithi171.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.9 MB/s | 269 kB 00:00 2024-09-15T00:09:24.777 INFO:teuthology.orchestra.run.smithi171.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.6 MB/s | 1.0 MB 00:00 2024-09-15T00:09:24.798 INFO:teuthology.orchestra.run.smithi184.stdout:(1/12): ceph-grafana-dashboards-19.1.1-324.geca 203 kB/s | 32 kB 00:00 2024-09-15T00:09:24.823 INFO:teuthology.orchestra.run.smithi184.stdout:(2/12): ceph-prometheus-alerts-19.1.1-324.gecab 90 kB/s | 16 kB 00:00 2024-09-15T00:09:24.852 INFO:teuthology.orchestra.run.smithi171.stdout:(5/12): ceph-mgr-dashboard-19.1.1-324.gecab7be5 7.9 MB/s | 3.6 MB 00:00 2024-09-15T00:09:24.878 INFO:teuthology.orchestra.run.smithi171.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 5.3 MB/s | 548 kB 00:00 2024-09-15T00:09:24.911 INFO:teuthology.orchestra.run.smithi171.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 329 kB/s | 19 kB 00:00 2024-09-15T00:09:24.956 INFO:teuthology.orchestra.run.smithi184.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.0 MB/s | 269 kB 00:00 2024-09-15T00:09:24.961 INFO:teuthology.orchestra.run.smithi171.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 27 MB/s | 2.0 MB 00:00 2024-09-15T00:09:24.987 INFO:teuthology.orchestra.run.smithi171.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.9 MB/s | 144 kB 00:00 2024-09-15T00:09:25.007 INFO:teuthology.orchestra.run.smithi184.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-09-15T00:09:25.021 INFO:teuthology.orchestra.run.smithi171.stdout:(10/12): protobuf-compiler-3.14.0-13.el9.x86_64 2.9 MB/s | 863 kB 00:00 2024-09-15T00:09:25.046 INFO:teuthology.orchestra.run.smithi171.stdout:(11/12): python3-repoze-lru-0.7-16.el9.noarch.r 366 kB/s | 31 kB 00:00 2024-09-15T00:09:25.057 INFO:teuthology.orchestra.run.smithi184.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-15T00:09:25.071 INFO:teuthology.orchestra.run.smithi171.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.2 MB/s | 188 kB 00:00 2024-09-15T00:09:25.076 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:25.076 INFO:teuthology.orchestra.run.smithi171.stdout:Total 9.5 MB/s | 8.7 MB 00:00 2024-09-15T00:09:25.082 INFO:teuthology.orchestra.run.smithi184.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-15T00:09:25.121 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:09:25.138 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:09:25.139 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:09:25.141 INFO:teuthology.orchestra.run.smithi184.stdout:(7/12): ceph-mgr-dashboard-19.1.1-324.gecab7be5 7.1 MB/s | 3.6 MB 00:00 2024-09-15T00:09:25.183 INFO:teuthology.orchestra.run.smithi184.stdout:(8/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.7 MB/s | 863 kB 00:00 2024-09-15T00:09:25.233 INFO:teuthology.orchestra.run.smithi184.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.5 MB/s | 144 kB 00:00 2024-09-15T00:09:25.258 INFO:teuthology.orchestra.run.smithi184.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 450 kB/s | 31 kB 00:00 2024-09-15T00:09:25.292 INFO:teuthology.orchestra.run.smithi184.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.1 MB/s | 188 kB 00:00 2024-09-15T00:09:25.334 INFO:teuthology.orchestra.run.smithi184.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 8.1 MB/s | 2.0 MB 00:00 2024-09-15T00:09:25.337 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:25.337 INFO:teuthology.orchestra.run.smithi184.stdout:Total 8.6 MB/s | 8.7 MB 00:01 2024-09-15T00:09:25.350 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:09:25.351 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:09:25.385 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:09:25.402 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:09:25.402 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:09:25.617 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:09:25.618 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:09:25.894 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:09:25.980 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-15T00:09:26.024 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-15T00:09:26.070 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-15T00:09:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-15T00:09:26.158 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:09:26.200 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-15T00:09:26.234 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-15T00:09:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-15T00:09:26.325 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-15T00:09:26.329 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-15T00:09:26.363 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-15T00:09:26.430 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-15T00:09:26.464 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-15T00:09:26.474 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-15T00:09:26.504 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-15T00:09:26.535 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 10/12 2024-09-15T00:09:26.588 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-15T00:09:26.675 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-15T00:09:26.734 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-15T00:09:26.763 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-15T00:09:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 10/12 2024-09-15T00:09:27.590 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 11/12 2024-09-15T00:09:27.629 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-15T00:09:27.851 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 11/12 2024-09-15T00:09:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 1/12 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 2/12 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 3/12 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-15T00:09:29.152 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-15T00:09:29.152 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:29.152 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:09:29.152 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T00:09:29.152 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:29.152 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:29.152 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:29.152 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T00:09:29.152 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T00:09:29.152 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T00:09:29.152 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T00:09:29.153 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T00:09:29.153 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T00:09:29.153 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T00:09:29.153 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T00:09:29.153 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:29.153 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:09:29.245 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-15T00:09:29.245 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 1/12 2024-09-15T00:09:29.245 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 2/12 2024-09-15T00:09:29.245 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 3/12 2024-09-15T00:09:29.245 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-15T00:09:29.245 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-15T00:09:29.245 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-15T00:09:29.246 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-15T00:09:29.246 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-15T00:09:29.246 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-15T00:09:29.246 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-15T00:09:29.246 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-15T00:09:29.379 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T00:09:29.566 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-15T00:09:29.566 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:29.566 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:09:29.566 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T00:09:29.566 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:29.566 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:29.567 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:29.567 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T00:09:29.567 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T00:09:29.567 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T00:09:29.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T00:09:29.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T00:09:29.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T00:09:29.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T00:09:29.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T00:09:29.567 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:29.567 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:09:29.836 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T00:09:29.946 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:06 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:09:30.079 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:09:30.080 INFO:teuthology.orchestra.run.smithi171.stdout:====================================================================================== 2024-09-15T00:09:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-15T00:09:30.080 INFO:teuthology.orchestra.run.smithi171.stdout:====================================================================================== 2024-09-15T00:09:30.080 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:09:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 7.4 M 2024-09-15T00:09:30.080 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-15T00:09:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T00:09:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T00:09:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T00:09:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T00:09:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T00:09:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T00:09:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T00:09:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T00:09:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T00:09:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T00:09:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T00:09:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:30.081 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:09:30.081 INFO:teuthology.orchestra.run.smithi171.stdout:====================================================================================== 2024-09-15T00:09:30.082 INFO:teuthology.orchestra.run.smithi171.stdout:Install 12 Packages 2024-09-15T00:09:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:30.082 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 38 M 2024-09-15T00:09:30.082 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 212 M 2024-09-15T00:09:30.082 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:09:30.397 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:06 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:09:30.530 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:09:30.532 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-09-15T00:09:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-15T00:09:30.532 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-09-15T00:09:30.532 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:09:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 7.4 M 2024-09-15T00:09:30.532 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-15T00:09:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T00:09:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T00:09:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T00:09:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T00:09:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T00:09:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T00:09:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T00:09:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T00:09:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T00:09:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T00:09:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T00:09:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:30.533 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:09:30.533 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-09-15T00:09:30.533 INFO:teuthology.orchestra.run.smithi184.stdout:Install 12 Packages 2024-09-15T00:09:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:30.534 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 38 M 2024-09-15T00:09:30.534 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 212 M 2024-09-15T00:09:30.534 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:09:30.626 INFO:teuthology.orchestra.run.smithi171.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 899 kB/s | 191 kB 00:00 2024-09-15T00:09:30.693 INFO:teuthology.orchestra.run.smithi171.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-09-15T00:09:30.718 INFO:teuthology.orchestra.run.smithi171.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.6 MB/s | 801 kB 00:00 2024-09-15T00:09:30.786 INFO:teuthology.orchestra.run.smithi171.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 273 kB/s | 18 kB 00:00 2024-09-15T00:09:30.853 INFO:teuthology.orchestra.run.smithi171.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 19 MB/s | 3.0 MB 00:00 2024-09-15T00:09:30.870 INFO:teuthology.orchestra.run.smithi171.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 459 kB/s | 38 kB 00:00 2024-09-15T00:09:30.978 INFO:teuthology.orchestra.run.smithi171.stdout:(7/12): ceph-mgr-diskprediction-local-19.1.1-32 13 MB/s | 7.4 MB 00:00 2024-09-15T00:09:31.005 INFO:teuthology.orchestra.run.smithi171.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-15T00:09:31.079 INFO:teuthology.orchestra.run.smithi171.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 22 MB/s | 4.9 MB 00:00 2024-09-15T00:09:31.121 INFO:teuthology.orchestra.run.smithi171.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.1 MB/s | 250 kB 00:00 2024-09-15T00:09:31.184 INFO:teuthology.orchestra.run.smithi184.stdout:(1/12): ceph-mgr-diskprediction-local-19.1.1-32 15 MB/s | 7.4 MB 00:00 2024-09-15T00:09:31.259 INFO:teuthology.orchestra.run.smithi184.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 335 kB/s | 191 kB 00:00 2024-09-15T00:09:31.284 INFO:teuthology.orchestra.run.smithi184.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-09-15T00:09:31.328 INFO:teuthology.orchestra.run.smithi184.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 437 kB/s | 18 kB 00:00 2024-09-15T00:09:31.368 INFO:teuthology.orchestra.run.smithi184.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 923 kB/s | 38 kB 00:00 2024-09-15T00:09:31.413 INFO:teuthology.orchestra.run.smithi171.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 49 MB/s | 16 MB 00:00 2024-09-15T00:09:31.435 INFO:teuthology.orchestra.run.smithi184.stdout:(6/12): libgfortran-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 801 kB 00:00 2024-09-15T00:09:31.519 INFO:teuthology.orchestra.run.smithi184.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-09-15T00:09:31.619 INFO:teuthology.orchestra.run.smithi184.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.3 MB/s | 245 kB 00:00 2024-09-15T00:09:31.719 INFO:teuthology.orchestra.run.smithi184.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-09-15T00:09:31.744 INFO:teuthology.orchestra.run.smithi184.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.0 MB/s | 250 kB 00:00 2024-09-15T00:09:31.811 INFO:teuthology.orchestra.run.smithi184.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 11 MB/s | 4.9 MB 00:00 2024-09-15T00:09:31.846 INFO:teuthology.orchestra.run.smithi171.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.9 MB/s | 5.1 MB 00:00 2024-09-15T00:09:31.847 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:31.848 INFO:teuthology.orchestra.run.smithi171.stdout:Total 22 MB/s | 38 MB 00:01 2024-09-15T00:09:32.095 INFO:teuthology.orchestra.run.smithi184.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-09-15T00:09:32.095 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:32.095 INFO:teuthology.orchestra.run.smithi184.stdout:Total 24 MB/s | 38 MB 00:01 2024-09-15T00:09:32.201 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:09:32.228 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:09:32.229 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:09:32.449 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:09:32.475 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:09:32.475 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:09:32.654 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:09:32.654 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:09:32.900 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:09:32.901 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:09:33.420 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:09:33.490 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T00:09:33.538 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T00:09:33.637 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T00:09:33.659 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:09:33.672 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T00:09:33.722 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T00:09:33.759 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T00:09:33.856 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T00:09:33.856 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T00:09:33.887 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T00:09:33.894 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T00:09:33.964 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T00:09:34.079 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T00:09:34.109 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T00:09:34.185 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T00:09:34.624 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T00:09:34.736 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T00:09:34.829 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T00:09:34.954 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T00:09:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T00:09:36.178 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T00:09:37.022 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T00:09:37.043 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-15T00:09:37.211 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T00:09:37.250 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-15T00:09:39.732 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-15T00:09:39.732 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 1/12 2024-09-15T00:09:39.732 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T00:09:39.732 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T00:09:39.732 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T00:09:39.733 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T00:09:39.733 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T00:09:39.733 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T00:09:39.733 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T00:09:39.733 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T00:09:39.733 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T00:09:39.733 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T00:09:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-15T00:09:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 1/12 2024-09-15T00:09:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T00:09:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T00:09:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T00:09:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T00:09:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T00:09:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T00:09:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T00:09:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T00:09:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T00:09:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T00:09:40.143 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T00:09:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:40.144 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:09:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T00:09:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T00:09:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T00:09:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T00:09:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T00:09:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T00:09:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T00:09:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T00:09:40.144 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T00:09:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T00:09:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T00:09:40.145 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:40.145 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:09:40.259 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T00:09:40.259 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:40.259 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:09:40.259 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:40.259 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T00:09:40.259 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T00:09:40.259 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T00:09:40.260 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T00:09:40.260 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T00:09:40.260 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T00:09:40.260 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T00:09:40.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T00:09:40.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T00:09:40.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T00:09:40.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T00:09:40.260 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:40.260 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:09:40.524 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-mgr-rook 2024-09-15T00:09:40.661 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-rook 2024-09-15T00:09:41.101 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:18 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:09:41.236 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:09:41.238 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-15T00:09:41.238 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:41.238 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:09:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 49 k 2024-09-15T00:09:41.238 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-15T00:09:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T00:09:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T00:09:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T00:09:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T00:09:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T00:09:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T00:09:41.238 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T00:09:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T00:09:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T00:09:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T00:09:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T00:09:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T00:09:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:41.239 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:09:41.239 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:41.239 INFO:teuthology.orchestra.run.smithi171.stdout:Install 13 Packages 2024-09-15T00:09:41.239 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:41.240 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 2.2 M 2024-09-15T00:09:41.240 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 26 M 2024-09-15T00:09:41.240 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:09:41.245 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:17 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:09:41.379 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:09:41.380 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:41.380 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-15T00:09:41.380 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:41.380 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:09:41.380 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 49 k 2024-09-15T00:09:41.380 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-15T00:09:41.380 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T00:09:41.380 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T00:09:41.380 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T00:09:41.380 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T00:09:41.380 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T00:09:41.380 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T00:09:41.381 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T00:09:41.381 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T00:09:41.381 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T00:09:41.381 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T00:09:41.381 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T00:09:41.381 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T00:09:41.381 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:41.381 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:09:41.381 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:41.381 INFO:teuthology.orchestra.run.smithi184.stdout:Install 13 Packages 2024-09-15T00:09:41.381 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:41.382 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 2.2 M 2024-09-15T00:09:41.382 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 26 M 2024-09-15T00:09:41.382 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:09:41.516 INFO:teuthology.orchestra.run.smithi171.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 169 kB/s | 26 kB 00:00 2024-09-15T00:09:41.541 INFO:teuthology.orchestra.run.smithi171.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 108 kB/s | 19 kB 00:00 2024-09-15T00:09:41.567 INFO:teuthology.orchestra.run.smithi171.stdout:(3/13): ceph-mgr-rook-19.1.1-324.gecab7be5.el9. 243 kB/s | 49 kB 00:00 2024-09-15T00:09:41.650 INFO:teuthology.orchestra.run.smithi171.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 222 kB 00:00 2024-09-15T00:09:41.676 INFO:teuthology.orchestra.run.smithi171.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-15T00:09:41.701 INFO:teuthology.orchestra.run.smithi171.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-15T00:09:41.726 INFO:teuthology.orchestra.run.smithi171.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 639 kB/s | 32 kB 00:00 2024-09-15T00:09:41.760 INFO:teuthology.orchestra.run.smithi171.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 247 kB/s | 14 kB 00:00 2024-09-15T00:09:41.785 INFO:teuthology.orchestra.run.smithi171.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.2 MB/s | 279 kB 00:00 2024-09-15T00:09:41.819 INFO:teuthology.orchestra.run.smithi171.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.4 MB/s | 220 kB 00:00 2024-09-15T00:09:41.845 INFO:teuthology.orchestra.run.smithi171.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.8 MB/s | 90 kB 00:00 2024-09-15T00:09:41.870 INFO:teuthology.orchestra.run.smithi171.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 706 kB/s | 59 kB 00:00 2024-09-15T00:09:41.903 INFO:teuthology.orchestra.run.smithi171.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.1 MB/s | 1.0 MB 00:00 2024-09-15T00:09:41.906 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:41.906 INFO:teuthology.orchestra.run.smithi171.stdout:Total 3.3 MB/s | 2.2 MB 00:00 2024-09-15T00:09:41.910 INFO:teuthology.orchestra.run.smithi184.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 201 kB/s | 19 kB 00:00 2024-09-15T00:09:41.935 INFO:teuthology.orchestra.run.smithi184.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 213 kB/s | 26 kB 00:00 2024-09-15T00:09:41.971 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:09:41.986 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:09:41.986 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:09:41.993 INFO:teuthology.orchestra.run.smithi184.stdout:(3/13): ceph-mgr-rook-19.1.1-324.gecab7be5.el9. 275 kB/s | 49 kB 00:00 2024-09-15T00:09:42.044 INFO:teuthology.orchestra.run.smithi184.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-15T00:09:42.069 INFO:teuthology.orchestra.run.smithi184.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-09-15T00:09:42.077 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:09:42.078 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:09:42.103 INFO:teuthology.orchestra.run.smithi184.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 912 kB/s | 54 kB 00:00 2024-09-15T00:09:42.136 INFO:teuthology.orchestra.run.smithi184.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 480 kB/s | 32 kB 00:00 2024-09-15T00:09:42.162 INFO:teuthology.orchestra.run.smithi184.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 248 kB/s | 14 kB 00:00 2024-09-15T00:09:42.187 INFO:teuthology.orchestra.run.smithi184.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 4.4 MB/s | 220 kB 00:00 2024-09-15T00:09:42.212 INFO:teuthology.orchestra.run.smithi184.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-15T00:09:42.254 INFO:teuthology.orchestra.run.smithi184.stdout:(11/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.0 MB/s | 279 kB 00:00 2024-09-15T00:09:42.279 INFO:teuthology.orchestra.run.smithi184.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-09-15T00:09:42.313 INFO:teuthology.orchestra.run.smithi184.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-15T00:09:42.315 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:42.315 INFO:teuthology.orchestra.run.smithi184.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-09-15T00:09:42.379 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:09:42.394 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:09:42.394 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:09:42.484 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:09:42.484 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:09:42.593 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:09:42.770 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T00:09:42.840 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T00:09:42.894 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T00:09:42.931 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T00:09:42.973 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T00:09:42.987 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:09:43.084 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T00:09:43.162 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T00:09:43.195 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T00:09:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T00:09:43.260 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T00:09:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T00:09:43.315 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T00:09:43.356 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T00:09:43.467 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T00:09:43.577 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T00:09:43.633 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T00:09:43.989 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T00:09:44.044 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T00:09:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T00:09:44.124 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T00:09:44.153 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-15T00:09:44.360 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T00:09:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T00:09:44.456 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T00:09:44.504 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T00:09:44.534 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-15T00:09:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-15T00:09:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 1/13 2024-09-15T00:09:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T00:09:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T00:09:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T00:09:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T00:09:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T00:09:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T00:09:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T00:09:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T00:09:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T00:09:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T00:09:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T00:09:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T00:09:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:45.308 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:09:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T00:09:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T00:09:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T00:09:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T00:09:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T00:09:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T00:09:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T00:09:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T00:09:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T00:09:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T00:09:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T00:09:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T00:09:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:45.309 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:09:45.384 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-15T00:09:45.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 1/13 2024-09-15T00:09:45.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T00:09:45.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T00:09:45.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T00:09:45.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T00:09:45.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T00:09:45.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T00:09:45.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T00:09:45.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T00:09:45.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T00:09:45.385 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T00:09:45.385 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T00:09:45.575 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T00:09:45.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T00:09:45.721 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:45.721 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:09:45.721 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:45.721 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T00:09:45.721 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T00:09:45.721 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T00:09:45.721 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T00:09:45.722 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T00:09:45.722 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T00:09:45.722 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T00:09:45.722 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T00:09:45.722 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T00:09:45.722 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T00:09:45.722 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T00:09:45.722 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T00:09:45.722 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:45.722 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:09:45.995 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T00:09:46.142 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:23 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:09:46.276 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:09:46.277 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:46.277 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-15T00:09:46.277 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:46.277 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:09:46.277 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 146 k 2024-09-15T00:09:46.277 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-15T00:09:46.277 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T00:09:46.277 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T00:09:46.277 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T00:09:46.277 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T00:09:46.277 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T00:09:46.277 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:46.278 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:09:46.278 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:46.278 INFO:teuthology.orchestra.run.smithi171.stdout:Install 6 Packages 2024-09-15T00:09:46.278 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:46.278 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 7.0 M 2024-09-15T00:09:46.278 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 33 M 2024-09-15T00:09:46.278 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:09:46.558 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:22 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:09:46.691 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:09:46.692 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:46.692 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-15T00:09:46.692 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:46.692 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 146 k 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi184.stdout:Install 6 Packages 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:46.694 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 7.0 M 2024-09-15T00:09:46.694 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 33 M 2024-09-15T00:09:46.694 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:09:46.845 INFO:teuthology.orchestra.run.smithi171.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.9 MB/s | 248 kB 00:00 2024-09-15T00:09:46.895 INFO:teuthology.orchestra.run.smithi171.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-15T00:09:46.920 INFO:teuthology.orchestra.run.smithi171.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.2 MB/s | 58 kB 00:00 2024-09-15T00:09:46.940 INFO:teuthology.orchestra.run.smithi184.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.6 MB/s | 248 kB 00:00 2024-09-15T00:09:46.946 INFO:teuthology.orchestra.run.smithi171.stdout:(4/6): ceph-mgr-cephadm-19.1.1-324.gecab7be5.el 639 kB/s | 146 kB 00:00 2024-09-15T00:09:46.971 INFO:teuthology.orchestra.run.smithi171.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-15T00:09:46.990 INFO:teuthology.orchestra.run.smithi184.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-15T00:09:47.016 INFO:teuthology.orchestra.run.smithi184.stdout:(3/6): ceph-mgr-cephadm-19.1.1-324.gecab7be5.el 653 kB/s | 146 kB 00:00 2024-09-15T00:09:47.041 INFO:teuthology.orchestra.run.smithi184.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-15T00:09:47.063 INFO:teuthology.orchestra.run.smithi171.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-15T00:09:47.065 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:47.065 INFO:teuthology.orchestra.run.smithi171.stdout:Total 8.9 MB/s | 7.0 MB 00:00 2024-09-15T00:09:47.083 INFO:teuthology.orchestra.run.smithi184.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-15T00:09:47.136 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:09:47.148 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:09:47.148 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:09:47.167 INFO:teuthology.orchestra.run.smithi184.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-15T00:09:47.169 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:47.170 INFO:teuthology.orchestra.run.smithi184.stdout:Total 15 MB/s | 7.0 MB 00:00 2024-09-15T00:09:47.241 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:09:47.248 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:09:47.252 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:09:47.252 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:09:47.352 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:09:47.353 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:09:47.555 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:09:47.666 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:09:47.729 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T00:09:47.782 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T00:09:47.825 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T00:09:47.877 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T00:09:48.299 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T00:09:48.394 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T00:09:48.405 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T00:09:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T00:09:48.487 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-15T00:09:48.488 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T00:09:48.551 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T00:09:48.581 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-15T00:09:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-15T00:09:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/6 2024-09-15T00:09:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T00:09:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T00:09:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T00:09:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T00:09:49.386 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-15T00:09:49.386 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/6 2024-09-15T00:09:49.386 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T00:09:49.386 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T00:09:49.386 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T00:09:49.386 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T00:09:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T00:09:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:49.532 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:09:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T00:09:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T00:09:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T00:09:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T00:09:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T00:09:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:49.533 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:09:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T00:09:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:49.662 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:09:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T00:09:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T00:09:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T00:09:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T00:09:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T00:09:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:49.663 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:09:49.791 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-fuse 2024-09-15T00:09:49.923 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-fuse 2024-09-15T00:09:50.361 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:27 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:09:50.489 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:26 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:09:50.496 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:09:50.496 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:50.497 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-15T00:09:50.497 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:50.497 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:09:50.497 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 896 k 2024-09-15T00:09:50.497 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-15T00:09:50.497 INFO:teuthology.orchestra.run.smithi171.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T00:09:50.497 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:50.497 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:09:50.497 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:50.497 INFO:teuthology.orchestra.run.smithi171.stdout:Install 2 Packages 2024-09-15T00:09:50.498 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:50.498 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 976 k 2024-09-15T00:09:50.498 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 2.9 M 2024-09-15T00:09:50.498 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:09:50.624 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:09:50.625 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:50.625 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-15T00:09:50.625 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:50.625 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:09:50.625 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 896 k 2024-09-15T00:09:50.626 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-15T00:09:50.626 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T00:09:50.626 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:50.626 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:09:50.626 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:50.626 INFO:teuthology.orchestra.run.smithi184.stdout:Install 2 Packages 2024-09-15T00:09:50.626 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:50.627 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 976 k 2024-09-15T00:09:50.627 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 2.9 M 2024-09-15T00:09:50.627 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:09:50.793 INFO:teuthology.orchestra.run.smithi184.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 662 kB/s | 80 kB 00:00 2024-09-15T00:09:50.951 INFO:teuthology.orchestra.run.smithi171.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 488 kB/s | 80 kB 00:00 2024-09-15T00:09:50.968 INFO:teuthology.orchestra.run.smithi184.stdout:(2/2): ceph-fuse-19.1.1-324.gecab7be5.el9.x86_6 3.0 MB/s | 896 kB 00:00 2024-09-15T00:09:50.969 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:50.969 INFO:teuthology.orchestra.run.smithi184.stdout:Total 2.8 MB/s | 976 kB 00:00 2024-09-15T00:09:50.981 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:09:51.003 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:09:51.003 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:09:51.069 INFO:teuthology.orchestra.run.smithi171.stdout:(2/2): ceph-fuse-19.1.1-324.gecab7be5.el9.x86_6 3.1 MB/s | 896 kB 00:00 2024-09-15T00:09:51.071 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:51.071 INFO:teuthology.orchestra.run.smithi171.stdout:Total 1.7 MB/s | 976 kB 00:00 2024-09-15T00:09:51.082 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:09:51.085 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:09:51.086 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:09:51.104 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:09:51.104 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:09:51.185 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:09:51.186 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:09:51.267 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:09:51.382 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:09:51.419 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T00:09:51.492 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-15T00:09:51.504 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T00:09:51.568 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-15T00:09:52.226 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-15T00:09:52.227 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/2 2024-09-15T00:09:52.264 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-15T00:09:52.264 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/2 2024-09-15T00:09:52.503 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T00:09:52.503 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:09:52.530 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:09:52.775 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-volume 2024-09-15T00:09:52.789 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-volume 2024-09-15T00:09:53.366 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:29 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:09:53.378 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:30 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:09:53.505 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:09:53.505 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:53.506 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-15T00:09:53.506 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:53.506 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:09:53.506 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 285 k 2024-09-15T00:09:53.506 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:53.506 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:09:53.506 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:53.506 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-09-15T00:09:53.506 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:53.506 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 285 k 2024-09-15T00:09:53.506 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 1.3 M 2024-09-15T00:09:53.507 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:09:53.518 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:09:53.519 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:53.519 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-15T00:09:53.520 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:53.520 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:09:53.520 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 285 k 2024-09-15T00:09:53.520 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:53.520 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:09:53.520 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:53.520 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-15T00:09:53.520 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:53.520 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 285 k 2024-09-15T00:09:53.520 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 1.3 M 2024-09-15T00:09:53.520 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:09:53.714 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-volume-19.1.1-324.gecab7be5.el9.noarch.rpm 1.3 MB/s | 285 kB 00:00 2024-09-15T00:09:53.715 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:53.715 INFO:teuthology.orchestra.run.smithi184.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-15T00:09:53.715 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:09:53.723 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:09:53.723 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:09:53.752 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-volume-19.1.1-324.gecab7be5.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-15T00:09:53.753 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:53.754 INFO:teuthology.orchestra.run.smithi171.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-15T00:09:53.754 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:09:53.757 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:09:53.757 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:09:53.763 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:09:53.763 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:09:53.791 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:09:53.792 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:09:53.984 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:09:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:09:54.050 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:09:54.065 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:09:54.066 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:09:54.066 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T00:09:54.066 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:09:54.113 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:09:54.113 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:09:54.113 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T00:09:54.113 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:55.141 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:09:55.141 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:55.141 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:09:55.141 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:55.141 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:55.141 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:09:55.152 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:09:55.152 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:55.152 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:09:55.152 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:09:55.152 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:55.152 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:09:55.412 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install librados-devel 2024-09-15T00:09:55.435 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install librados-devel 2024-09-15T00:09:55.999 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:32 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:09:56.019 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:32 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:09:56.138 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:09:56.138 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:56.138 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repo Size 2024-09-15T00:09:56.138 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:56.138 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:09:56.138 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 127 k 2024-09-15T00:09:56.138 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:56.138 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:09:56.138 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:56.138 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-15T00:09:56.138 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:56.139 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 127 k 2024-09-15T00:09:56.139 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 456 k 2024-09-15T00:09:56.139 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:09:56.155 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:09:56.155 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:56.155 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-09-15T00:09:56.155 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:56.155 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:09:56.155 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 127 k 2024-09-15T00:09:56.155 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:56.156 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:09:56.156 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:56.156 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-09-15T00:09:56.156 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:56.156 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 127 k 2024-09-15T00:09:56.156 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 456 k 2024-09-15T00:09:56.156 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:09:56.344 INFO:teuthology.orchestra.run.smithi184.stdout:librados-devel-19.1.1-324.gecab7be5.el9.x86_64. 678 kB/s | 127 kB 00:00 2024-09-15T00:09:56.344 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:56.345 INFO:teuthology.orchestra.run.smithi184.stdout:Total 673 kB/s | 127 kB 00:00 2024-09-15T00:09:56.345 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:09:56.346 INFO:teuthology.orchestra.run.smithi171.stdout:librados-devel-19.1.1-324.gecab7be5.el9.x86_64. 606 kB/s | 127 kB 00:00 2024-09-15T00:09:56.346 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:56.347 INFO:teuthology.orchestra.run.smithi171.stdout:Total 602 kB/s | 127 kB 00:00 2024-09-15T00:09:56.347 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:09:56.351 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:09:56.351 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:09:56.352 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:09:56.352 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:09:56.414 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:09:56.414 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:09:56.415 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:09:56.416 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:09:56.561 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:09:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:09:56.668 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:09:56.676 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:09:57.128 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:09:57.386 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:09:57.386 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:57.386 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:09:57.386 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:57.386 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:57.387 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:09:57.398 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:09:57.398 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:57.398 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:09:57.399 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:09:57.399 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:57.399 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:09:57.645 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install libcephfs2 2024-09-15T00:09:57.654 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install libcephfs2 2024-09-15T00:09:58.217 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:34 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:09:58.237 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:35 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:09:58.310 INFO:teuthology.orchestra.run.smithi184.stdout:Package libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:09:58.331 INFO:teuthology.orchestra.run.smithi171.stdout:Package libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:09:58.353 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:09:58.354 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-15T00:09:58.354 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:09:58.377 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:09:58.378 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-15T00:09:58.378 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:09:58.434 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install libcephfs-devel 2024-09-15T00:09:58.447 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install libcephfs-devel 2024-09-15T00:09:58.999 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:34 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:09:59.011 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:36 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:09:59.132 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:09:59.132 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:59.132 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-09-15T00:09:59.132 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:59.133 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:09:59.133 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 32 k 2024-09-15T00:09:59.133 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:59.133 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:09:59.133 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:09:59.133 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-09-15T00:09:59.133 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:09:59.134 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 32 k 2024-09-15T00:09:59.134 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 142 k 2024-09-15T00:09:59.134 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:09:59.145 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:09:59.146 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repo Size 2024-09-15T00:09:59.146 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:59.146 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:09:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 32 k 2024-09-15T00:09:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:59.146 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:09:59.146 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:09:59.147 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-15T00:09:59.147 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:09:59.147 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 32 k 2024-09-15T00:09:59.147 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 142 k 2024-09-15T00:09:59.147 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:09:59.297 INFO:teuthology.orchestra.run.smithi184.stdout:libcephfs-devel-19.1.1-324.gecab7be5.el9.x86_64 194 kB/s | 32 kB 00:00 2024-09-15T00:09:59.298 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:59.298 INFO:teuthology.orchestra.run.smithi184.stdout:Total 192 kB/s | 32 kB 00:00 2024-09-15T00:09:59.298 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:09:59.302 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:09:59.302 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:09:59.312 INFO:teuthology.orchestra.run.smithi171.stdout:libcephfs-devel-19.1.1-324.gecab7be5.el9.x86_64 193 kB/s | 32 kB 00:00 2024-09-15T00:09:59.313 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:09:59.313 INFO:teuthology.orchestra.run.smithi171.stdout:Total 191 kB/s | 32 kB 00:00 2024-09-15T00:09:59.313 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:09:59.316 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:09:59.316 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:09:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:09:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:09:59.334 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:09:59.334 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:09:59.406 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:09:59.427 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:09:59.502 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:09:59.535 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:09:59.875 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:09:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:00.112 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:00.113 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:00.113 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:10:00.113 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:10:00.113 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:00.113 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:10:00.127 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:00.127 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:00.127 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:10:00.127 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:10:00.127 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:00.127 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:10:00.310 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install librados2 2024-09-15T00:10:00.327 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install librados2 2024-09-15T00:10:00.884 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:37 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:10:00.897 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:36 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:10:00.975 INFO:teuthology.orchestra.run.smithi171.stdout:Package librados2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:10:00.988 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:10:01.019 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:10:01.020 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-15T00:10:01.020 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:10:01.032 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:10:01.034 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-15T00:10:01.034 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:10:01.090 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install librbd1 2024-09-15T00:10:01.109 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install librbd1 2024-09-15T00:10:01.653 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:38 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:10:01.677 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:37 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:10:01.744 INFO:teuthology.orchestra.run.smithi171.stdout:Package librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:10:01.768 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:10:01.788 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:10:01.789 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-15T00:10:01.789 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:10:01.811 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:10:01.813 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-15T00:10:01.813 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:10:01.855 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install python3-rados 2024-09-15T00:10:01.891 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-rados 2024-09-15T00:10:02.425 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:39 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:10:02.463 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:38 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:10:02.515 INFO:teuthology.orchestra.run.smithi171.stdout:Package python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:10:02.554 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:10:02.559 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:10:02.560 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-15T00:10:02.560 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:10:02.599 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:10:02.600 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-15T00:10:02.600 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:10:02.636 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install python3-rgw 2024-09-15T00:10:02.673 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-rgw 2024-09-15T00:10:03.207 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:40 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:10:03.242 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:39 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:10:03.298 INFO:teuthology.orchestra.run.smithi171.stdout:Package python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:10:03.335 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:10:03.342 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:10:03.343 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-15T00:10:03.343 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:10:03.378 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:10:03.379 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-15T00:10:03.379 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:10:03.421 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install python3-cephfs 2024-09-15T00:10:03.454 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-cephfs 2024-09-15T00:10:03.985 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:40 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:10:04.023 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:40 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:10:04.075 INFO:teuthology.orchestra.run.smithi171.stdout:Package python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:10:04.116 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:10:04.118 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:10:04.119 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-15T00:10:04.120 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:10:04.159 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:10:04.160 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-15T00:10:04.160 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:10:04.196 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install python3-rbd 2024-09-15T00:10:04.233 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-rbd 2024-09-15T00:10:04.764 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:41 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:10:04.799 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:40 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:10:04.855 INFO:teuthology.orchestra.run.smithi171.stdout:Package python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:10:04.890 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:10:04.898 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:10:04.899 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-15T00:10:04.900 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:10:04.934 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:10:04.935 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-15T00:10:04.935 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:10:04.971 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install rbd-fuse 2024-09-15T00:10:05.004 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install rbd-fuse 2024-09-15T00:10:05.536 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:42 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:10:05.567 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:41 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:10:05.669 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:10:05.669 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:10:05.669 INFO:teuthology.orchestra.run.smithi171.stdout: Package Architecture Version Repository Size 2024-09-15T00:10:05.670 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:10:05.670 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:10:05.670 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 85 k 2024-09-15T00:10:05.670 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:05.670 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:10:05.670 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:10:05.670 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-15T00:10:05.670 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:05.671 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 85 k 2024-09-15T00:10:05.671 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 231 k 2024-09-15T00:10:05.671 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:10:05.701 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:10:05.702 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:10:05.702 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-09-15T00:10:05.702 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:10:05.702 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:10:05.702 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 85 k 2024-09-15T00:10:05.702 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:05.703 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:10:05.703 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:10:05.703 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-09-15T00:10:05.703 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:05.703 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 85 k 2024-09-15T00:10:05.703 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 231 k 2024-09-15T00:10:05.703 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:10:05.874 INFO:teuthology.orchestra.run.smithi171.stdout:rbd-fuse-19.1.1-324.gecab7be5.el9.x86_64.rpm 420 kB/s | 85 kB 00:00 2024-09-15T00:10:05.874 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:10:05.875 INFO:teuthology.orchestra.run.smithi171.stdout:Total 417 kB/s | 85 kB 00:00 2024-09-15T00:10:05.875 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:10:05.881 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:10:05.881 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:10:05.902 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-fuse-19.1.1-324.gecab7be5.el9.x86_64.rpm 429 kB/s | 85 kB 00:00 2024-09-15T00:10:05.902 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:10:05.902 INFO:teuthology.orchestra.run.smithi184.stdout:Total 426 kB/s | 85 kB 00:00 2024-09-15T00:10:05.903 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:10:05.909 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:10:05.909 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:10:05.944 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:10:05.944 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:10:05.972 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:10:05.972 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:10:06.092 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:10:06.125 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:10:06.187 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:06.231 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:06.566 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:06.638 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:10:06.901 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:06.901 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:06.901 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:10:06.901 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:10:06.901 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:06.901 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:10:07.087 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install rbd-mirror 2024-09-15T00:10:07.168 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install rbd-mirror 2024-09-15T00:10:07.678 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:44 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:10:07.758 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:43 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:10:07.822 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:10:07.822 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:10:07.822 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repo Size 2024-09-15T00:10:07.823 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:10:07.823 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:10:07.823 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.1 M 2024-09-15T00:10:07.823 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:07.823 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:10:07.823 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:10:07.823 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-15T00:10:07.823 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:07.823 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 3.1 M 2024-09-15T00:10:07.823 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 13 M 2024-09-15T00:10:07.823 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:10:07.900 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:10:07.900 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:10:07.900 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-09-15T00:10:07.901 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:10:07.901 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:10:07.901 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.1 M 2024-09-15T00:10:07.901 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:07.901 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:10:07.901 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:10:07.902 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-09-15T00:10:07.902 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:07.902 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 3.1 M 2024-09-15T00:10:07.902 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 13 M 2024-09-15T00:10:07.902 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:10:08.166 INFO:teuthology.orchestra.run.smithi171.stdout:rbd-mirror-19.1.1-324.gecab7be5.el9.x86_64.rpm 9.1 MB/s | 3.1 MB 00:00 2024-09-15T00:10:08.166 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:10:08.167 INFO:teuthology.orchestra.run.smithi171.stdout:Total 9.0 MB/s | 3.1 MB 00:00 2024-09-15T00:10:08.167 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:10:08.176 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:10:08.176 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:10:08.230 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-mirror-19.1.1-324.gecab7be5.el9.x86_64.rpm 9.5 MB/s | 3.1 MB 00:00 2024-09-15T00:10:08.230 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:10:08.231 INFO:teuthology.orchestra.run.smithi184.stdout:Total 9.4 MB/s | 3.1 MB 00:00 2024-09-15T00:10:08.231 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:10:08.240 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:10:08.240 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:10:08.257 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:10:08.257 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:10:08.324 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:10:08.324 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:10:08.699 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:10:08.766 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:08.777 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:10:08.792 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:08.792 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:10:08.792 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T00:10:08.792 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T00:10:08.793 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T00:10:08.793 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:08.891 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:10:08.891 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T00:10:08.891 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T00:10:08.891 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T00:10:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:09.920 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:09.920 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:09.920 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:10:09.920 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:10:09.920 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:09.920 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:10:10.067 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:10.067 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:10.067 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:10:10.068 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:10:10.068 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:10.068 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:10:10.178 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install rbd-nbd 2024-09-15T00:10:10.317 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install rbd-nbd 2024-09-15T00:10:10.759 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:47 ago on Sun 15 Sep 2024 12:08:23 AM UTC. 2024-09-15T00:10:10.895 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-15T00:10:10.895 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:10:10.895 INFO:teuthology.orchestra.run.smithi171.stdout: Package Architecture Version Repository Size 2024-09-15T00:10:10.895 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:10:10.895 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-15T00:10:10.895 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 170 k 2024-09-15T00:10:10.895 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:10.896 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-15T00:10:10.896 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-15T00:10:10.896 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-15T00:10:10.896 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:10.896 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 170 k 2024-09-15T00:10:10.896 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 494 k 2024-09-15T00:10:10.896 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-15T00:10:10.903 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:46 ago on Sun 15 Sep 2024 12:08:24 AM UTC. 2024-09-15T00:10:11.038 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-15T00:10:11.038 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:10:11.038 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-09-15T00:10:11.039 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:10:11.039 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-15T00:10:11.039 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 170 k 2024-09-15T00:10:11.039 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:11.039 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-15T00:10:11.039 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-15T00:10:11.039 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-09-15T00:10:11.039 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:11.040 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 170 k 2024-09-15T00:10:11.040 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 494 k 2024-09-15T00:10:11.040 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-15T00:10:11.111 INFO:teuthology.orchestra.run.smithi171.stdout:rbd-nbd-19.1.1-324.gecab7be5.el9.x86_64.rpm 796 kB/s | 170 kB 00:00 2024-09-15T00:10:11.111 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:10:11.111 INFO:teuthology.orchestra.run.smithi171.stdout:Total 791 kB/s | 170 kB 00:00 2024-09-15T00:10:11.112 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-15T00:10:11.119 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-15T00:10:11.119 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-15T00:10:11.183 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-15T00:10:11.183 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-15T00:10:11.261 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-nbd-19.1.1-324.gecab7be5.el9.x86_64.rpm 770 kB/s | 170 kB 00:00 2024-09-15T00:10:11.261 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:10:11.262 INFO:teuthology.orchestra.run.smithi184.stdout:Total 765 kB/s | 170 kB 00:00 2024-09-15T00:10:11.262 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-15T00:10:11.270 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-15T00:10:11.270 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-15T00:10:11.333 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-15T00:10:11.334 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-15T00:10:11.340 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-15T00:10:11.441 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:11.483 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-15T00:10:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:12.121 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-15T00:10:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:10:12.121 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:12.121 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-15T00:10:12.293 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:10:12.293 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:12.293 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-15T00:10:12.293 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:10:12.293 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:12.293 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-15T00:10:12.355 DEBUG:teuthology.parallel:result is None 2024-09-15T00:10:12.530 DEBUG:teuthology.parallel:result is None 2024-09-15T00:10:12.530 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-15T00:10:12.677 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T00:10:12.698 INFO:teuthology.orchestra.run.smithi171.stdout:19.1.1-324.gecab7be5.el9 2024-09-15T00:10:12.699 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-324.gecab7be5.el9 2024-09-15T00:10:12.699 INFO:teuthology.task.install:The correct ceph version 19.1.1-324.gecab7be5 is installed. 2024-09-15T00:10:12.701 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-15T00:10:12.837 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T00:10:12.859 INFO:teuthology.orchestra.run.smithi184.stdout:19.1.1-324.gecab7be5.el9 2024-09-15T00:10:12.860 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-324.gecab7be5.el9 2024-09-15T00:10:12.860 INFO:teuthology.task.install:The correct ceph version 19.1.1-324.gecab7be5 is installed. 2024-09-15T00:10:12.862 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T00:10:12.862 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:12.862 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T00:10:12.889 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:12.889 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T00:10:12.925 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T00:10:12.926 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:12.926 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T00:10:12.955 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T00:10:13.019 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:13.020 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T00:10:13.044 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T00:10:13.110 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T00:10:13.110 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:13.110 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T00:10:13.155 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T00:10:13.221 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:13.221 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T00:10:13.246 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T00:10:13.311 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T00:10:13.311 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:13.311 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T00:10:13.336 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T00:10:13.402 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:13.402 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T00:10:13.426 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T00:10:13.491 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T00:10:13.589 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T00:10:13.589 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /var/log/ceph 2024-09-15T00:10:13.592 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /var/log/ceph 2024-09-15T00:10:13.616 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T00:10:13.616 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T00:10:13.657 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T00:10:13.682 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T00:10:13.682 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T00:10:13.722 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T00:10:13.756 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T00:10:13.756 INFO:tasks.ceph:config {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-15T00:10:13.757 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904819', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7904819', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 4271, 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'sleep_before_teardown': 0, 'subset': '97/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'targets': {'smithi171.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMtslxk6quUC5fU9BkATtHtvsvtS1wu46HhC7ZY9r9OWe1xR2JwhXHSmmiB9W7oT/lnIkMti9pbfICeXcQMUQBI=', 'smithi184.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHyccrCZFFuU+xC5jjUDWpy2Q+vd4VdSvu4eMPM91ukqCHVLMqDzfjZIe71D8rklHFi/SBAmHNvJ/hSBkvv+mHw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-14_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T00:10:13.757 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T00:10:13.788 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T00:10:13.803 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T00:10:13.843 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T00:10:13.868 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:13.869 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T00:10:13.922 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T00:10:13.922 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2024-09-15T00:10:13.978 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T00:10:13.978 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:10:13.978 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-15T00:10:13.978 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:10:13.978 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:10:13.978 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-15 00:10:09.048356688 +0000 2024-09-15T00:10:13.978 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-15 00:07:44.324850132 +0000 2024-09-15T00:10:13.978 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-15 00:07:44.324850132 +0000 2024-09-15T00:10:13.978 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-15 00:07:44.324850132 +0000 2024-09-15T00:10:13.979 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T00:10:14.041 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-15T00:10:14.041 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-15T00:10:14.041 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000209616 s, 2.4 MB/s 2024-09-15T00:10:14.042 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T00:10:14.098 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2024-09-15T00:10:14.154 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T00:10:14.154 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:10:14.154 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-15T00:10:14.154 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:10:14.154 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:10:14.154 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-15 00:10:09.048356688 +0000 2024-09-15T00:10:14.154 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-15 00:07:44.654844160 +0000 2024-09-15T00:10:14.154 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-15 00:07:44.654844160 +0000 2024-09-15T00:10:14.154 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-15 00:07:44.654844160 +0000 2024-09-15T00:10:14.154 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T00:10:14.216 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-15T00:10:14.216 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-15T00:10:14.216 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000163201 s, 3.1 MB/s 2024-09-15T00:10:14.218 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T00:10:14.274 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2024-09-15T00:10:14.328 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T00:10:14.329 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:10:14.329 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-15T00:10:14.329 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:10:14.329 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:10:14.329 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-15 00:10:09.048356688 +0000 2024-09-15T00:10:14.329 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-15 00:07:44.982838224 +0000 2024-09-15T00:10:14.329 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-15 00:07:44.982838224 +0000 2024-09-15T00:10:14.329 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-15 00:07:44.982838224 +0000 2024-09-15T00:10:14.329 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T00:10:14.392 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-15T00:10:14.392 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-15T00:10:14.392 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000230986 s, 2.2 MB/s 2024-09-15T00:10:14.393 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T00:10:14.450 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2024-09-15T00:10:14.506 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T00:10:14.506 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:10:14.506 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-15T00:10:14.506 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:10:14.506 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:10:14.506 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-15 00:10:09.048356688 +0000 2024-09-15T00:10:14.506 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-15 00:07:45.313832234 +0000 2024-09-15T00:10:14.506 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-15 00:07:45.313832234 +0000 2024-09-15T00:10:14.506 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-15 00:07:45.313832234 +0000 2024-09-15T00:10:14.507 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T00:10:14.569 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-15T00:10:14.570 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-15T00:10:14.570 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000187714 s, 2.7 MB/s 2024-09-15T00:10:14.571 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T00:10:14.628 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T00:10:14.628 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:14.628 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T00:10:14.646 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T00:10:14.646 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2024-09-15T00:10:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T00:10:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:10:14.703 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-15T00:10:14.703 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:10:14.703 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:10:14.703 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-15 00:10:09.143634545 +0000 2024-09-15T00:10:14.703 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-15 00:07:44.313931434 +0000 2024-09-15T00:10:14.703 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-15 00:07:44.313931434 +0000 2024-09-15T00:10:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-15 00:07:44.313931434 +0000 2024-09-15T00:10:14.703 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T00:10:14.771 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-15T00:10:14.771 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-15T00:10:14.771 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000250264 s, 2.0 MB/s 2024-09-15T00:10:14.772 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T00:10:14.838 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2024-09-15T00:10:14.896 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T00:10:14.896 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:10:14.896 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-15T00:10:14.896 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:10:14.897 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:10:14.897 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-15 00:10:09.144634525 +0000 2024-09-15T00:10:14.897 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-15 00:07:44.637924841 +0000 2024-09-15T00:10:14.897 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-15 00:07:44.637924841 +0000 2024-09-15T00:10:14.897 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-15 00:07:44.637924841 +0000 2024-09-15T00:10:14.897 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T00:10:14.963 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-15T00:10:14.963 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-15T00:10:14.963 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000221616 s, 2.3 MB/s 2024-09-15T00:10:14.964 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T00:10:15.022 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-15 00:10:09.144634525 +0000 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-15 00:07:44.978917901 +0000 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-15 00:07:44.978917901 +0000 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-15 00:07:44.978917901 +0000 2024-09-15T00:10:15.079 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T00:10:15.144 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-15T00:10:15.144 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-15T00:10:15.145 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000220515 s, 2.3 MB/s 2024-09-15T00:10:15.146 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T00:10:15.203 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2024-09-15T00:10:15.260 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T00:10:15.260 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:10:15.260 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-15T00:10:15.260 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:10:15.260 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:10:15.261 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-15 00:10:09.144634525 +0000 2024-09-15T00:10:15.261 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-15 00:07:45.300911348 +0000 2024-09-15T00:10:15.261 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-15 00:07:45.300911348 +0000 2024-09-15T00:10:15.261 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-15 00:07:45.300911348 +0000 2024-09-15T00:10:15.261 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T00:10:15.325 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-15T00:10:15.325 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-15T00:10:15.325 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000222416 s, 2.3 MB/s 2024-09-15T00:10:15.327 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T00:10:15.383 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T00:10:15.384 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi171.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-15T00:10:15.384 INFO:tasks.ceph:Generating config... 2024-09-15T00:10:15.385 INFO:tasks.ceph:[client] rbd qos iops burst = 100 2024-09-15T00:10:15.385 INFO:tasks.ceph:[client] rbd qos iops limit = 50 2024-09-15T00:10:15.385 INFO:tasks.ceph:[client] rbd qos schedule tick min = 100 2024-09-15T00:10:15.385 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-15T00:10:15.385 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-15T00:10:15.385 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-15T00:10:15.385 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T00:10:15.385 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T00:10:15.386 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T00:10:15.386 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T00:10:15.386 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T00:10:15.386 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-15T00:10:15.386 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2024-09-15T00:10:15.386 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-15T00:10:15.386 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-15T00:10:15.386 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-15T00:10:15.386 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-15T00:10:15.386 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T00:10:15.386 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T00:10:15.386 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-15T00:10:15.386 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-15T00:10:15.386 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-15T00:10:15.386 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-15T00:10:15.387 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-15T00:10:15.387 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-15T00:10:15.387 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-15T00:10:15.387 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-15T00:10:15.387 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-15T00:10:15.387 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T00:10:15.387 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T00:10:15.433 INFO:teuthology.orchestra.run.smithi171.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T00:10:15.436 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T00:10:15.478 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T00:10:15.543 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.171'), ('mon.c', '[v2:172.21.15.171:3301,v1:172.21.15.171:6790]'), ('mon.b', '172.21.15.184')] 2024-09-15T00:10:15.544 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790],172.21.15.184', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-15T00:10:15.544 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:15.544 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T00:10:15.599 DEBUG:teuthology.orchestra.run.smithi171:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.171 --addv c '[v2:172.21.15.171:3301,v1:172.21.15.171:6790]' --add b 172.21.15.184 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:10:15.687 INFO:teuthology.orchestra.run.smithi171.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:10:15.687 INFO:teuthology.orchestra.run.smithi171.stdout:monmaptool: generated fsid bad2533c-9c09-4af4-ab93-147afbacb705 2024-09-15T00:10:15.687 INFO:teuthology.orchestra.run.smithi171.stdout:setting min_mon_release = quincy 2024-09-15T00:10:15.687 INFO:teuthology.orchestra.run.smithi171.stdout:epoch 0 2024-09-15T00:10:15.687 INFO:teuthology.orchestra.run.smithi171.stdout:fsid bad2533c-9c09-4af4-ab93-147afbacb705 2024-09-15T00:10:15.687 INFO:teuthology.orchestra.run.smithi171.stdout:last_changed 2024-09-15T00:10:15.685927+0000 2024-09-15T00:10:15.687 INFO:teuthology.orchestra.run.smithi171.stdout:created 2024-09-15T00:10:15.685927+0000 2024-09-15T00:10:15.687 INFO:teuthology.orchestra.run.smithi171.stdout:min_mon_release 17 (quincy) 2024-09-15T00:10:15.687 INFO:teuthology.orchestra.run.smithi171.stdout:election_strategy: 1 2024-09-15T00:10:15.688 INFO:teuthology.orchestra.run.smithi171.stdout:0: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.a 2024-09-15T00:10:15.688 INFO:teuthology.orchestra.run.smithi171.stdout:1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.b 2024-09-15T00:10:15.688 INFO:teuthology.orchestra.run.smithi171.stdout:2: [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] mon.c 2024-09-15T00:10:15.688 INFO:teuthology.orchestra.run.smithi171.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-15T00:10:15.689 DEBUG:teuthology.orchestra.run.smithi171:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T00:10:15.704 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID bad2533c-9c09-4af4-ab93-147afbacb705... 2024-09-15T00:10:15.705 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T00:10:15.747 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T00:10:15.788 INFO:teuthology.orchestra.run.smithi171.stdout:[global] 2024-09-15T00:10:15.788 INFO:teuthology.orchestra.run.smithi171.stdout: chdir = "" 2024-09-15T00:10:15.788 INFO:teuthology.orchestra.run.smithi171.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T00:10:15.788 INFO:teuthology.orchestra.run.smithi171.stdout: auth supported = cephx 2024-09-15T00:10:15.788 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.788 INFO:teuthology.orchestra.run.smithi171.stdout: filestore xattr use omap = true 2024-09-15T00:10:15.788 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.788 INFO:teuthology.orchestra.run.smithi171.stdout: mon clock drift allowed = 1.000 2024-09-15T00:10:15.788 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.788 INFO:teuthology.orchestra.run.smithi171.stdout: osd crush chooseleaf type = 0 2024-09-15T00:10:15.789 INFO:teuthology.orchestra.run.smithi171.stdout: auth debug = true 2024-09-15T00:10:15.789 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.789 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on old message = true 2024-09-15T00:10:15.789 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on bug = true 2024-09-15T00:10:15.789 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.789 INFO:teuthology.orchestra.run.smithi171.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T00:10:15.789 INFO:teuthology.orchestra.run.smithi171.stdout: mon pg warn max object skew = 0 2024-09-15T00:10:15.789 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.789 INFO:teuthology.orchestra.run.smithi171.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T00:10:15.789 INFO:teuthology.orchestra.run.smithi171.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T00:10:15.789 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.789 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default size = 2 2024-09-15T00:10:15.789 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.789 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow primary affinity = true 2024-09-15T00:10:15.790 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow pg remap = true 2024-09-15T00:10:15.790 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on legacy crush tunables = false 2024-09-15T00:10:15.790 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on crush straw calc version zero = false 2024-09-15T00:10:15.790 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on no sortbitwise = false 2024-09-15T00:10:15.790 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on osd down out interval zero = false 2024-09-15T00:10:15.790 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on too few osds = false 2024-09-15T00:10:15.790 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T00:10:15.790 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T00:10:15.790 INFO:teuthology.orchestra.run.smithi171.stdout: mon_allow_pool_size_one = true 2024-09-15T00:10:15.790 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.790 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T00:10:15.790 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.791 INFO:teuthology.orchestra.run.smithi171.stdout: osd default data pool replay window = 5 2024-09-15T00:10:15.791 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.791 INFO:teuthology.orchestra.run.smithi171.stdout: mon allow pool delete = true 2024-09-15T00:10:15.791 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.791 INFO:teuthology.orchestra.run.smithi171.stdout: mon cluster log file level = debug 2024-09-15T00:10:15.791 INFO:teuthology.orchestra.run.smithi171.stdout: debug asserts on shutdown = true 2024-09-15T00:10:15.791 INFO:teuthology.orchestra.run.smithi171.stdout: mon health detail to clog = false 2024-09-15T00:10:15.791 INFO:teuthology.orchestra.run.smithi171.stdout: mon host = "172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790],172.21.15.184" 2024-09-15T00:10:15.791 INFO:teuthology.orchestra.run.smithi171.stdout: mon client directed command retry = 5 2024-09-15T00:10:15.791 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on pool no app = False 2024-09-15T00:10:15.791 INFO:teuthology.orchestra.run.smithi171.stdout: ms inject socket failures = 5000 2024-09-15T00:10:15.791 INFO:teuthology.orchestra.run.smithi171.stdout: fsid = bad2533c-9c09-4af4-ab93-147afbacb705 2024-09-15T00:10:15.791 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.791 INFO:teuthology.orchestra.run.smithi171.stdout:[osd] 2024-09-15T00:10:15.791 INFO:teuthology.orchestra.run.smithi171.stdout: osd journal size = 100 2024-09-15T00:10:15.792 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.792 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub load threshold = 5.0 2024-09-15T00:10:15.792 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub max interval = 600 2024-09-15T00:10:15.792 INFO:teuthology.orchestra.run.smithi171.stdout: osd mclock profile = high_recovery_ops 2024-09-15T00:10:15.792 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.792 INFO:teuthology.orchestra.run.smithi171.stdout: osd recover clone overlap = true 2024-09-15T00:10:15.792 INFO:teuthology.orchestra.run.smithi171.stdout: osd recovery max chunk = 1048576 2024-09-15T00:10:15.792 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.792 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug shutdown = true 2024-09-15T00:10:15.792 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug op order = true 2024-09-15T00:10:15.792 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug verify stray on activate = true 2024-09-15T00:10:15.792 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug trim objects = true 2024-09-15T00:10:15.792 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.793 INFO:teuthology.orchestra.run.smithi171.stdout: osd open classes on start = true 2024-09-15T00:10:15.793 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug pg log writeout = true 2024-09-15T00:10:15.793 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.793 INFO:teuthology.orchestra.run.smithi171.stdout: osd deep scrub update digest min age = 30 2024-09-15T00:10:15.793 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.793 INFO:teuthology.orchestra.run.smithi171.stdout: osd map max advance = 10 2024-09-15T00:10:15.793 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.793 INFO:teuthology.orchestra.run.smithi171.stdout: journal zero on create = true 2024-09-15T00:10:15.793 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.793 INFO:teuthology.orchestra.run.smithi171.stdout: filestore ondisk finisher threads = 3 2024-09-15T00:10:15.793 INFO:teuthology.orchestra.run.smithi171.stdout: filestore apply finisher threads = 3 2024-09-15T00:10:15.793 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.793 INFO:teuthology.orchestra.run.smithi171.stdout: bdev debug aio = true 2024-09-15T00:10:15.793 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug misdirected ops = true 2024-09-15T00:10:15.793 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore block size = 96636764160 2024-09-15T00:10:15.794 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore compression algorithm = zlib 2024-09-15T00:10:15.794 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore compression mode = aggressive 2024-09-15T00:10:15.794 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore fsck on mount = True 2024-09-15T00:10:15.794 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluefs = 1/20 2024-09-15T00:10:15.794 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluestore = 1/20 2024-09-15T00:10:15.794 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-15T00:10:15.794 INFO:teuthology.orchestra.run.smithi171.stdout: debug osd = 20 2024-09-15T00:10:15.794 INFO:teuthology.orchestra.run.smithi171.stdout: debug rocksdb = 4/10 2024-09-15T00:10:15.794 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T00:10:15.794 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd full ratio = 0.9 2024-09-15T00:10:15.794 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd nearfull ratio = 0.8 2024-09-15T00:10:15.794 INFO:teuthology.orchestra.run.smithi171.stdout: osd blocked scrub grace period = 3600 2024-09-15T00:10:15.794 INFO:teuthology.orchestra.run.smithi171.stdout: osd failsafe full ratio = 0.95 2024-09-15T00:10:15.794 INFO:teuthology.orchestra.run.smithi171.stdout: osd max markdown count = 1000 2024-09-15T00:10:15.794 INFO:teuthology.orchestra.run.smithi171.stdout: osd objectstore = bluestore 2024-09-15T00:10:15.795 INFO:teuthology.orchestra.run.smithi171.stdout: osd shutdown pgref assert = True 2024-09-15T00:10:15.795 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.795 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr] 2024-09-15T00:10:15.795 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-15T00:10:15.795 INFO:teuthology.orchestra.run.smithi171.stdout: debug mgr = 20 2024-09-15T00:10:15.795 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-15T00:10:15.795 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-15T00:10:15.795 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-15T00:10:15.795 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-15T00:10:15.795 INFO:teuthology.orchestra.run.smithi171.stdout: mgr/telemetry/nag = false 2024-09-15T00:10:15.795 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.795 INFO:teuthology.orchestra.run.smithi171.stdout:[mon] 2024-09-15T00:10:15.795 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-15T00:10:15.795 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-15T00:10:15.796 INFO:teuthology.orchestra.run.smithi171.stdout: debug paxos = 20 2024-09-15T00:10:15.796 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-15T00:10:15.796 INFO:teuthology.orchestra.run.smithi171.stdout: mon data avail warn = 5 2024-09-15T00:10:15.796 INFO:teuthology.orchestra.run.smithi171.stdout: mon mgr mkfs grace = 240 2024-09-15T00:10:15.796 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-15T00:10:15.796 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd reporter subtree level = osd 2024-09-15T00:10:15.796 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd prime pg temp = true 2024-09-15T00:10:15.796 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-15T00:10:15.796 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.796 INFO:teuthology.orchestra.run.smithi171.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T00:10:15.796 INFO:teuthology.orchestra.run.smithi171.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T00:10:15.796 INFO:teuthology.orchestra.run.smithi171.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T00:10:15.797 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.797 INFO:teuthology.orchestra.run.smithi171.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T00:10:15.797 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T00:10:15.797 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T00:10:15.797 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.797 INFO:teuthology.orchestra.run.smithi171.stdout: # 1m isn't quite enough 2024-09-15T00:10:15.797 INFO:teuthology.orchestra.run.smithi171.stdout: mon_down_mkfs_grace = 2m 2024-09-15T00:10:15.797 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.797 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_filestore_osds = false 2024-09-15T00:10:15.797 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:15.797 INFO:teuthology.orchestra.run.smithi171.stdout:[client] 2024-09-15T00:10:15.797 INFO:teuthology.orchestra.run.smithi171.stdout: rgw cache enabled = true 2024-09-15T00:10:15.797 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable ops log = true 2024-09-15T00:10:15.797 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable usage log = true 2024-09-15T00:10:15.798 INFO:teuthology.orchestra.run.smithi171.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T00:10:15.798 INFO:teuthology.orchestra.run.smithi171.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T00:10:15.798 INFO:teuthology.orchestra.run.smithi171.stdout: rbd qos iops burst = 100 2024-09-15T00:10:15.798 INFO:teuthology.orchestra.run.smithi171.stdout: rbd qos iops limit = 50 2024-09-15T00:10:15.798 INFO:teuthology.orchestra.run.smithi171.stdout: rbd qos schedule tick min = 100 2024-09-15T00:10:15.798 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.a] 2024-09-15T00:10:15.798 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.c] 2024-09-15T00:10:15.798 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.b] 2024-09-15T00:10:15.798 INFO:teuthology.orchestra.run.smithi184.stdout:[global] 2024-09-15T00:10:15.799 INFO:teuthology.orchestra.run.smithi184.stdout: chdir = "" 2024-09-15T00:10:15.799 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T00:10:15.799 INFO:teuthology.orchestra.run.smithi184.stdout: auth supported = cephx 2024-09-15T00:10:15.799 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.799 INFO:teuthology.orchestra.run.smithi184.stdout: filestore xattr use omap = true 2024-09-15T00:10:15.799 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.799 INFO:teuthology.orchestra.run.smithi184.stdout: mon clock drift allowed = 1.000 2024-09-15T00:10:15.799 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.799 INFO:teuthology.orchestra.run.smithi184.stdout: osd crush chooseleaf type = 0 2024-09-15T00:10:15.799 INFO:teuthology.orchestra.run.smithi184.stdout: auth debug = true 2024-09-15T00:10:15.799 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.799 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on old message = true 2024-09-15T00:10:15.799 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on bug = true 2024-09-15T00:10:15.800 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.800 INFO:teuthology.orchestra.run.smithi184.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T00:10:15.800 INFO:teuthology.orchestra.run.smithi184.stdout: mon pg warn max object skew = 0 2024-09-15T00:10:15.800 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.800 INFO:teuthology.orchestra.run.smithi184.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T00:10:15.800 INFO:teuthology.orchestra.run.smithi184.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T00:10:15.800 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.800 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default size = 2 2024-09-15T00:10:15.800 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.800 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow primary affinity = true 2024-09-15T00:10:15.800 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow pg remap = true 2024-09-15T00:10:15.800 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on legacy crush tunables = false 2024-09-15T00:10:15.800 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on crush straw calc version zero = false 2024-09-15T00:10:15.800 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on no sortbitwise = false 2024-09-15T00:10:15.801 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on osd down out interval zero = false 2024-09-15T00:10:15.801 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on too few osds = false 2024-09-15T00:10:15.801 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T00:10:15.801 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T00:10:15.801 INFO:teuthology.orchestra.run.smithi184.stdout: mon_allow_pool_size_one = true 2024-09-15T00:10:15.801 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.801 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T00:10:15.801 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.801 INFO:teuthology.orchestra.run.smithi184.stdout: osd default data pool replay window = 5 2024-09-15T00:10:15.801 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.801 INFO:teuthology.orchestra.run.smithi184.stdout: mon allow pool delete = true 2024-09-15T00:10:15.801 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.801 INFO:teuthology.orchestra.run.smithi184.stdout: mon cluster log file level = debug 2024-09-15T00:10:15.801 INFO:teuthology.orchestra.run.smithi184.stdout: debug asserts on shutdown = true 2024-09-15T00:10:15.802 INFO:teuthology.orchestra.run.smithi184.stdout: mon health detail to clog = false 2024-09-15T00:10:15.802 INFO:teuthology.orchestra.run.smithi184.stdout: mon host = "172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790],172.21.15.184" 2024-09-15T00:10:15.802 INFO:teuthology.orchestra.run.smithi184.stdout: mon client directed command retry = 5 2024-09-15T00:10:15.802 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on pool no app = False 2024-09-15T00:10:15.802 INFO:teuthology.orchestra.run.smithi184.stdout: ms inject socket failures = 5000 2024-09-15T00:10:15.802 INFO:teuthology.orchestra.run.smithi184.stdout: fsid = bad2533c-9c09-4af4-ab93-147afbacb705 2024-09-15T00:10:15.802 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.802 INFO:teuthology.orchestra.run.smithi184.stdout:[osd] 2024-09-15T00:10:15.802 INFO:teuthology.orchestra.run.smithi184.stdout: osd journal size = 100 2024-09-15T00:10:15.802 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.802 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub load threshold = 5.0 2024-09-15T00:10:15.802 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub max interval = 600 2024-09-15T00:10:15.802 INFO:teuthology.orchestra.run.smithi184.stdout: osd mclock profile = high_recovery_ops 2024-09-15T00:10:15.803 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.803 INFO:teuthology.orchestra.run.smithi184.stdout: osd recover clone overlap = true 2024-09-15T00:10:15.803 INFO:teuthology.orchestra.run.smithi184.stdout: osd recovery max chunk = 1048576 2024-09-15T00:10:15.803 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.803 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug shutdown = true 2024-09-15T00:10:15.803 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug op order = true 2024-09-15T00:10:15.803 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug verify stray on activate = true 2024-09-15T00:10:15.803 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug trim objects = true 2024-09-15T00:10:15.803 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.803 INFO:teuthology.orchestra.run.smithi184.stdout: osd open classes on start = true 2024-09-15T00:10:15.803 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug pg log writeout = true 2024-09-15T00:10:15.803 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.803 INFO:teuthology.orchestra.run.smithi184.stdout: osd deep scrub update digest min age = 30 2024-09-15T00:10:15.803 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.804 INFO:teuthology.orchestra.run.smithi184.stdout: osd map max advance = 10 2024-09-15T00:10:15.804 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.804 INFO:teuthology.orchestra.run.smithi184.stdout: journal zero on create = true 2024-09-15T00:10:15.804 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.804 INFO:teuthology.orchestra.run.smithi184.stdout: filestore ondisk finisher threads = 3 2024-09-15T00:10:15.804 INFO:teuthology.orchestra.run.smithi184.stdout: filestore apply finisher threads = 3 2024-09-15T00:10:15.804 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.804 INFO:teuthology.orchestra.run.smithi184.stdout: bdev debug aio = true 2024-09-15T00:10:15.804 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug misdirected ops = true 2024-09-15T00:10:15.804 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore block size = 96636764160 2024-09-15T00:10:15.804 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore compression algorithm = zlib 2024-09-15T00:10:15.804 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore compression mode = aggressive 2024-09-15T00:10:15.804 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore fsck on mount = True 2024-09-15T00:10:15.804 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluefs = 1/20 2024-09-15T00:10:15.805 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluestore = 1/20 2024-09-15T00:10:15.805 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-09-15T00:10:15.805 INFO:teuthology.orchestra.run.smithi184.stdout: debug osd = 20 2024-09-15T00:10:15.805 INFO:teuthology.orchestra.run.smithi184.stdout: debug rocksdb = 4/10 2024-09-15T00:10:15.805 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T00:10:15.805 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd full ratio = 0.9 2024-09-15T00:10:15.805 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd nearfull ratio = 0.8 2024-09-15T00:10:15.805 INFO:teuthology.orchestra.run.smithi184.stdout: osd blocked scrub grace period = 3600 2024-09-15T00:10:15.805 INFO:teuthology.orchestra.run.smithi184.stdout: osd failsafe full ratio = 0.95 2024-09-15T00:10:15.805 INFO:teuthology.orchestra.run.smithi184.stdout: osd max markdown count = 1000 2024-09-15T00:10:15.805 INFO:teuthology.orchestra.run.smithi184.stdout: osd objectstore = bluestore 2024-09-15T00:10:15.805 INFO:teuthology.orchestra.run.smithi184.stdout: osd shutdown pgref assert = True 2024-09-15T00:10:15.806 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.806 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr] 2024-09-15T00:10:15.806 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-09-15T00:10:15.806 INFO:teuthology.orchestra.run.smithi184.stdout: debug mgr = 20 2024-09-15T00:10:15.806 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2024-09-15T00:10:15.806 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2024-09-15T00:10:15.806 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2024-09-15T00:10:15.806 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2024-09-15T00:10:15.806 INFO:teuthology.orchestra.run.smithi184.stdout: mgr/telemetry/nag = false 2024-09-15T00:10:15.806 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.806 INFO:teuthology.orchestra.run.smithi184.stdout:[mon] 2024-09-15T00:10:15.806 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-09-15T00:10:15.806 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2024-09-15T00:10:15.806 INFO:teuthology.orchestra.run.smithi184.stdout: debug paxos = 20 2024-09-15T00:10:15.807 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2024-09-15T00:10:15.807 INFO:teuthology.orchestra.run.smithi184.stdout: mon data avail warn = 5 2024-09-15T00:10:15.807 INFO:teuthology.orchestra.run.smithi184.stdout: mon mgr mkfs grace = 240 2024-09-15T00:10:15.807 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2024-09-15T00:10:15.807 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd reporter subtree level = osd 2024-09-15T00:10:15.807 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd prime pg temp = true 2024-09-15T00:10:15.807 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2024-09-15T00:10:15.807 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.807 INFO:teuthology.orchestra.run.smithi184.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T00:10:15.807 INFO:teuthology.orchestra.run.smithi184.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T00:10:15.807 INFO:teuthology.orchestra.run.smithi184.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T00:10:15.807 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.807 INFO:teuthology.orchestra.run.smithi184.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T00:10:15.807 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T00:10:15.808 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T00:10:15.808 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.808 INFO:teuthology.orchestra.run.smithi184.stdout: # 1m isn't quite enough 2024-09-15T00:10:15.808 INFO:teuthology.orchestra.run.smithi184.stdout: mon_down_mkfs_grace = 2m 2024-09-15T00:10:15.808 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.808 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_filestore_osds = false 2024-09-15T00:10:15.808 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-15T00:10:15.808 INFO:teuthology.orchestra.run.smithi184.stdout:[client] 2024-09-15T00:10:15.808 INFO:teuthology.orchestra.run.smithi184.stdout: rgw cache enabled = true 2024-09-15T00:10:15.808 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable ops log = true 2024-09-15T00:10:15.808 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable usage log = true 2024-09-15T00:10:15.808 INFO:teuthology.orchestra.run.smithi184.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T00:10:15.808 INFO:teuthology.orchestra.run.smithi184.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T00:10:15.809 INFO:teuthology.orchestra.run.smithi184.stdout: rbd qos iops burst = 100 2024-09-15T00:10:15.809 INFO:teuthology.orchestra.run.smithi184.stdout: rbd qos iops limit = 50 2024-09-15T00:10:15.809 INFO:teuthology.orchestra.run.smithi184.stdout: rbd qos schedule tick min = 100 2024-09-15T00:10:15.809 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.a] 2024-09-15T00:10:15.809 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.c] 2024-09-15T00:10:15.809 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.b] 2024-09-15T00:10:15.810 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T00:10:15.811 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T00:10:15.905 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T00:10:15.905 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:15.905 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T00:10:15.921 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:15.921 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T00:10:15.976 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi171.front.sepia.ceph.com 2024-09-15T00:10:15.977 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:15.977 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T00:10:15.977 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T00:10:16.050 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:16.050 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:10:16.106 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi184.front.sepia.ceph.com 2024-09-15T00:10:16.106 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:16.106 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T00:10:16.106 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T00:10:16.143 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:16.143 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:10:16.199 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T00:10:16.200 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T00:10:16.200 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-15T00:10:16.251 INFO:teuthology.orchestra.run.smithi171.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-15T00:10:16.254 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T00:10:16.312 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T00:10:16.315 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T00:10:16.315 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T00:10:16.316 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T00:10:16.356 INFO:teuthology.orchestra.run.smithi171.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-15T00:10:16.369 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-15T00:10:16.411 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-15T00:10:16.424 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T00:10:16.424 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi171.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-15T00:10:16.424 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T00:10:16.448 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T00:10:16.449 INFO:tasks.ceph:role: osd.0 2024-09-15T00:10:16.449 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-15T00:10:16.449 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T00:10:16.520 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T00:10:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T00:10:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T00:10:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T00:10:16.521 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T00:10:16.521 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-15T00:10:16.521 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T00:10:16.521 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T00:10:16.521 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T00:10:16.521 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T00:10:17.021 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-09-15T00:10:17.023 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-15T00:10:17.023 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T00:10:17.155 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T00:10:17.190 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T00:10:17.256 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T00:10:17.256 INFO:tasks.ceph:role: osd.1 2024-09-15T00:10:17.256 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-15T00:10:17.256 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T00:10:17.329 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T00:10:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T00:10:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T00:10:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T00:10:17.329 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T00:10:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-15T00:10:17.329 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T00:10:17.329 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T00:10:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T00:10:17.329 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T00:10:17.792 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-09-15T00:10:17.794 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-15T00:10:17.794 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-15T00:10:17.888 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T00:10:17.918 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T00:10:17.983 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T00:10:17.984 INFO:tasks.ceph:role: osd.2 2024-09-15T00:10:17.984 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-15T00:10:17.984 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T00:10:18.056 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T00:10:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T00:10:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T00:10:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T00:10:18.057 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T00:10:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-15T00:10:18.057 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T00:10:18.057 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T00:10:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T00:10:18.057 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T00:10:18.568 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-09-15T00:10:18.570 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-15T00:10:18.570 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-15T00:10:18.719 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T00:10:18.749 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-15T00:10:18.815 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T00:10:18.815 INFO:tasks.ceph:role: osd.3 2024-09-15T00:10:18.815 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-15T00:10:18.815 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T00:10:18.888 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T00:10:18.888 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T00:10:18.888 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T00:10:18.888 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T00:10:18.888 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T00:10:18.888 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-15T00:10:18.888 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T00:10:18.888 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T00:10:18.889 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T00:10:18.889 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T00:10:19.392 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-09-15T00:10:19.394 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-15T00:10:19.395 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-15T00:10:19.511 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-15T00:10:19.543 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:10:19.638 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:19.636+0000 7facc8b1b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-15T00:10:19.638 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:19.637+0000 7facc8b1b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-15T00:10:19.639 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:19.637+0000 7facc8b1b640 -1 bdev(0x555cb8263800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-15T00:10:19.639 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:19.637+0000 7facc8b1b640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-15T00:10:22.798 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T00:10:22.828 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:10:22.920 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:22.918+0000 7fc7339cc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-15T00:10:22.920 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:22.918+0000 7fc7339cc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-15T00:10:22.920 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:22.918+0000 7fc7339cc640 -1 bdev(0x56521023d800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-15T00:10:22.920 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:22.918+0000 7fc7339cc640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-15T00:10:25.824 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T00:10:25.852 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:10:25.940 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:25.938+0000 7f01b42e8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-15T00:10:25.940 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:25.939+0000 7f01b42e8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-15T00:10:25.941 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:25.939+0000 7f01b42e8640 -1 bdev(0x56413a7f1800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-15T00:10:25.941 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:25.939+0000 7f01b42e8640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-15T00:10:29.354 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T00:10:29.383 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:10:29.474 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:29.471+0000 7fb81a00d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-15T00:10:29.475 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:29.472+0000 7fb81a00d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-15T00:10:29.475 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:29.472+0000 7fb81a00d640 -1 bdev(0x55669d57b800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-15T00:10:29.475 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:10:29.472+0000 7fb81a00d640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-15T00:10:32.628 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-15T00:10:32.658 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-15T00:10:32.689 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T00:10:32.689 INFO:tasks.ceph:role: osd.4 2024-09-15T00:10:32.689 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-15T00:10:32.689 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T00:10:32.764 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T00:10:32.765 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T00:10:32.765 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T00:10:32.765 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T00:10:32.765 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T00:10:32.765 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-09-15T00:10:32.765 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T00:10:32.765 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T00:10:32.765 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T00:10:32.765 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T00:10:33.245 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-09-15T00:10:33.247 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-09-15T00:10:33.247 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-15T00:10:33.393 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-15T00:10:33.432 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-15T00:10:33.501 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T00:10:33.501 INFO:tasks.ceph:role: osd.5 2024-09-15T00:10:33.501 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-15T00:10:33.501 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T00:10:33.577 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T00:10:33.577 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T00:10:33.577 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T00:10:33.577 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T00:10:33.577 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T00:10:33.577 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-09-15T00:10:33.577 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T00:10:33.577 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T00:10:33.578 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T00:10:33.578 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T00:10:34.036 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-09-15T00:10:34.039 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-09-15T00:10:34.039 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-15T00:10:34.122 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-15T00:10:34.192 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-15T00:10:34.256 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T00:10:34.256 INFO:tasks.ceph:role: osd.6 2024-09-15T00:10:34.256 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-15T00:10:34.257 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T00:10:34.328 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T00:10:34.328 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T00:10:34.329 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T00:10:34.329 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T00:10:34.329 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T00:10:34.329 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-09-15T00:10:34.329 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T00:10:34.329 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T00:10:34.329 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T00:10:34.329 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T00:10:34.822 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-09-15T00:10:34.824 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-09-15T00:10:34.824 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-15T00:10:34.949 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-15T00:10:34.977 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-15T00:10:35.042 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T00:10:35.042 INFO:tasks.ceph:role: osd.7 2024-09-15T00:10:35.042 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-15T00:10:35.042 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T00:10:35.115 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T00:10:35.115 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T00:10:35.115 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T00:10:35.115 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T00:10:35.115 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T00:10:35.115 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-09-15T00:10:35.115 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T00:10:35.115 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T00:10:35.115 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T00:10:35.115 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T00:10:35.652 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-09-15T00:10:35.654 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-09-15T00:10:35.654 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-15T00:10:35.786 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-15T00:10:35.816 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:10:35.915 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:35.913+0000 7fcf94e31640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-15T00:10:35.916 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:35.914+0000 7fcf94e31640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-15T00:10:35.916 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:35.914+0000 7fcf94e31640 -1 bdev(0x56475ff29800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-15T00:10:35.916 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:35.914+0000 7fcf94e31640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-15T00:10:38.826 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-15T00:10:38.855 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:10:38.944 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:38.942+0000 7fc4c0033640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-15T00:10:38.944 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:38.942+0000 7fc4c0033640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-15T00:10:38.944 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:38.942+0000 7fc4c0033640 -1 bdev(0x55fc165f1800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-15T00:10:38.944 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:38.942+0000 7fc4c0033640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-15T00:10:41.852 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-15T00:10:41.880 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:10:41.970 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:41.967+0000 7f46c161b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-15T00:10:41.971 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:41.968+0000 7f46c161b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-15T00:10:41.971 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:41.968+0000 7f46c161b640 -1 bdev(0x55b050bb1800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-15T00:10:41.971 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:41.968+0000 7f46c161b640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-15T00:10:44.886 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-15T00:10:44.910 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:10:44.996 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:44.993+0000 7f7d87026640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-15T00:10:44.996 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:44.993+0000 7f7d87026640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-15T00:10:44.997 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:44.993+0000 7f7d87026640 -1 bdev(0x55c61b21d800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-15T00:10:44.997 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-15T00:10:44.993+0000 7f7d87026640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-15T00:10:47.904 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-15T00:10:47.934 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T00:10:47.934 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:47.935 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-15T00:10:47.965 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:47.965 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T00:10:48.035 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:48.035 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T00:10:48.102 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:48.102 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T00:10:48.169 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:48.169 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-15T00:10:48.236 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:48.236 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T00:10:48.267 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:48.267 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-15T00:10:48.338 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:48.338 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-15T00:10:48.409 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:48.409 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-15T00:10:48.478 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:48.478 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-15T00:10:48.549 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:48.549 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-15T00:10:48.566 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:48.566 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-15T00:10:48.607 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T00:10:48.607 DEBUG:teuthology.orchestra.run.smithi171:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T00:10:48.610 DEBUG:teuthology.orchestra.run.smithi184:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T00:10:48.650 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr.y] 2024-09-15T00:10:48.650 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQDoJeZmtorxDhAAx58D6Z1Fi/K2ndSAWDj56g== 2024-09-15T00:10:48.651 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.0] 2024-09-15T00:10:48.651 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQDrJeZm5hABJhAAZ9gSc7yi7MYDrue1pauLOw== 2024-09-15T00:10:48.651 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.1] 2024-09-15T00:10:48.651 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQDuJeZmBUDJNhAAFy99PPetc24vFFxR/1rTeA== 2024-09-15T00:10:48.651 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.2] 2024-09-15T00:10:48.651 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQDxJeZmky8AOBAAGcCbLI3MRzkbgrvP7DRVcQ== 2024-09-15T00:10:48.651 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.3] 2024-09-15T00:10:48.651 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQD1JeZmAlA7HBAAAXJNQhGPFHTEtX6yFp8AwQ== 2024-09-15T00:10:48.651 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr.x] 2024-09-15T00:10:48.651 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQDoJeZm/guaEhAAIH01GuvhNq6u82CLWJUQFg== 2024-09-15T00:10:48.651 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.4] 2024-09-15T00:10:48.651 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQD7JeZmFbOENhAAM0uWsr0GRRMor93RnXAG8w== 2024-09-15T00:10:48.651 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.5] 2024-09-15T00:10:48.652 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQD+JeZmoZc7OBAAYAckvCrmbajIvugKPr8q6Q== 2024-09-15T00:10:48.652 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.6] 2024-09-15T00:10:48.652 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQABJuZmc4rMORAA0qIGyTQ7eDQ0gEG36bms3g== 2024-09-15T00:10:48.652 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.7] 2024-09-15T00:10:48.652 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAEJuZm1BJWOxAANH3vZ5wUPm/Gxi/BJbpzUg== 2024-09-15T00:10:48.652 INFO:teuthology.orchestra.run.smithi171.stdout:[client.0] 2024-09-15T00:10:48.652 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQDoJeZmmmA6FRAA3mro25pgE92j40DXnYJM6A== 2024-09-15T00:10:48.652 INFO:teuthology.orchestra.run.smithi171.stdout:[client.1] 2024-09-15T00:10:48.652 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQDoJeZmyMd4GBAAZw042J0/zzBExEw0Mtf+YQ== 2024-09-15T00:10:48.675 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.y] 2024-09-15T00:10:48.675 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDoJeZmtorxDhAAx58D6Z1Fi/K2ndSAWDj56g== 2024-09-15T00:10:48.675 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.0] 2024-09-15T00:10:48.675 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDrJeZm5hABJhAAZ9gSc7yi7MYDrue1pauLOw== 2024-09-15T00:10:48.675 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.1] 2024-09-15T00:10:48.675 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDuJeZmBUDJNhAAFy99PPetc24vFFxR/1rTeA== 2024-09-15T00:10:48.675 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.2] 2024-09-15T00:10:48.675 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDxJeZmky8AOBAAGcCbLI3MRzkbgrvP7DRVcQ== 2024-09-15T00:10:48.675 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.3] 2024-09-15T00:10:48.676 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQD1JeZmAlA7HBAAAXJNQhGPFHTEtX6yFp8AwQ== 2024-09-15T00:10:48.676 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.x] 2024-09-15T00:10:48.676 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDoJeZm/guaEhAAIH01GuvhNq6u82CLWJUQFg== 2024-09-15T00:10:48.676 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.4] 2024-09-15T00:10:48.676 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQD7JeZmFbOENhAAM0uWsr0GRRMor93RnXAG8w== 2024-09-15T00:10:48.676 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.5] 2024-09-15T00:10:48.676 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQD+JeZmoZc7OBAAYAckvCrmbajIvugKPr8q6Q== 2024-09-15T00:10:48.676 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.6] 2024-09-15T00:10:48.676 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQABJuZmc4rMORAA0qIGyTQ7eDQ0gEG36bms3g== 2024-09-15T00:10:48.676 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.7] 2024-09-15T00:10:48.676 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAEJuZm1BJWOxAANH3vZ5wUPm/Gxi/BJbpzUg== 2024-09-15T00:10:48.676 INFO:teuthology.orchestra.run.smithi184.stdout:[client.0] 2024-09-15T00:10:48.676 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDoJeZmmmA6FRAA3mro25pgE92j40DXnYJM6A== 2024-09-15T00:10:48.676 INFO:teuthology.orchestra.run.smithi184.stdout:[client.1] 2024-09-15T00:10:48.677 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDoJeZmyMd4GBAAZw042J0/zzBExEw0Mtf+YQ== 2024-09-15T00:10:48.677 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T00:10:48.696 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T00:10:48.776 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:48.778 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:48.832 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:48.835 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:48.888 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:48.891 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:48.941 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:48.945 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:48.996 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T00:10:48.998 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T00:10:49.049 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:49.051 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:49.101 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:49.103 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:49.153 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:49.156 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:49.206 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:49.209 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:10:49.259 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T00:10:49.261 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T00:10:49.315 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T00:10:49.317 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T00:10:49.370 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T00:10:49.370 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T00:10:49.395 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T00:10:49.491 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T00:10:49.515 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-15T00:10:49.580 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T00:10:49.677 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-15T00:10:49.702 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-15T00:10:49.728 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T00:10:49.826 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-15T00:10:49.851 DEBUG:teuthology.orchestra.run.smithi171:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:10:49.853 DEBUG:teuthology.orchestra.run.smithi184:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:10:49.906 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T00:10:49.906 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T00:10:49.907 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T00:10:49.909 INFO:tasks.ceph.mon.a:Started 2024-09-15T00:10:49.909 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T00:10:49.909 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T00:10:49.911 INFO:tasks.ceph.mon.c:Started 2024-09-15T00:10:49.911 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T00:10:49.911 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T00:10:49.949 INFO:tasks.ceph.mon.b:Started 2024-09-15T00:10:49.949 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T00:10:49.949 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-15T00:10:49.949 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-15T00:10:49.952 INFO:tasks.ceph.mgr.y:Started 2024-09-15T00:10:49.952 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T00:10:49.952 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T00:10:49.954 INFO:tasks.ceph.mgr.x:Started 2024-09-15T00:10:49.954 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T00:10:49.954 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T00:10:50.292 INFO:teuthology.orchestra.run.smithi171.stderr:adjusted tunables profile to default 2024-09-15T00:10:50.304 INFO:tasks.ceph:check_enable_crimson: False 2024-09-15T00:10:50.304 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T00:10:50.304 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:50.304 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T00:10:50.309 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:50.307+0000 7fdc5a51f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T00:10:50.329 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:50.329 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T00:10:50.333 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:50.332+0000 7f5842675040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T00:10:50.348 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:50.347+0000 7fdc5a51f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T00:10:50.359 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:50.359 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T00:10:50.378 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:50.376+0000 7f5842675040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T00:10:50.402 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-15T00:10:50.402 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-15T00:10:50.469 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:50.469 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-15T00:10:50.490 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:50.488+0000 7fdc5a51f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T00:10:50.494 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:50.494 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-15T00:10:50.526 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:50.524+0000 7f5842675040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T00:10:50.541 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:50.540+0000 7fdc5a51f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T00:10:50.558 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:50.558 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-15T00:10:50.575 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:50.574+0000 7f5842675040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T00:10:50.627 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-15T00:10:50.627 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-15T00:10:50.696 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 7337111f-0e02-490d-976b-3814a03371f0 0 2024-09-15T00:10:50.741 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:50.739+0000 7f5842675040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T00:10:50.741 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:50.739+0000 7fdc5a51f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T00:10:50.850 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:50.848+0000 7f5842675040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T00:10:50.858 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:50.856+0000 7fdc5a51f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T00:10:50.924 INFO:teuthology.orchestra.run.smithi184.stdout:0 2024-09-15T00:10:50.935 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 64412633-4dd4-4b8d-b3f1-ed3c664ddf48 1 2024-09-15T00:10:51.117 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-15T00:10:51.128 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 0300adee-5ce3-43ce-997b-2c0e970edbc7 2 2024-09-15T00:10:51.168 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:51.166+0000 7f5842675040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T00:10:51.198 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:51.196+0000 7fdc5a51f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T00:10:51.223 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:51.222+0000 7f5842675040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T00:10:51.253 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:51.251+0000 7fdc5a51f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T00:10:51.311 INFO:teuthology.orchestra.run.smithi184.stdout:2 2024-09-15T00:10:51.322 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new ec668801-9b8f-401a-829c-0b1f8d8f08fb 3 2024-09-15T00:10:51.375 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:51.373+0000 7f5842675040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T00:10:51.409 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:51.407+0000 7fdc5a51f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T00:10:51.501 INFO:teuthology.orchestra.run.smithi184.stdout:3 2024-09-15T00:10:51.512 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 9cc48936-5644-44c6-84bb-06510e933e8d 4 2024-09-15T00:10:51.702 INFO:teuthology.orchestra.run.smithi184.stdout:4 2024-09-15T00:10:51.713 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new f6c53297-9b00-476f-9be8-e76ff762b044 5 2024-09-15T00:10:51.889 INFO:teuthology.orchestra.run.smithi184.stdout:5 2024-09-15T00:10:51.901 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 737f08a6-2223-46b3-b9fe-6ae18b067ec3 6 2024-09-15T00:10:52.082 INFO:teuthology.orchestra.run.smithi184.stdout:6 2024-09-15T00:10:52.093 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 5a5fba64-9249-426d-814b-fd251e8aa910 7 2024-09-15T00:10:52.270 INFO:teuthology.orchestra.run.smithi184.stdout:7 2024-09-15T00:10:52.281 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T00:10:52.281 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T00:10:52.284 INFO:tasks.ceph.osd.0:Started 2024-09-15T00:10:52.284 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T00:10:52.284 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T00:10:52.286 INFO:tasks.ceph.osd.1:Started 2024-09-15T00:10:52.286 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T00:10:52.286 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T00:10:52.288 INFO:tasks.ceph.osd.2:Started 2024-09-15T00:10:52.289 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T00:10:52.289 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-15T00:10:52.291 INFO:tasks.ceph.osd.3:Started 2024-09-15T00:10:52.291 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-15T00:10:52.291 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-15T00:10:52.293 INFO:tasks.ceph.osd.4:Started 2024-09-15T00:10:52.293 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-15T00:10:52.294 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-15T00:10:52.296 INFO:tasks.ceph.osd.5:Started 2024-09-15T00:10:52.296 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-15T00:10:52.296 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-15T00:10:52.298 INFO:tasks.ceph.osd.6:Started 2024-09-15T00:10:52.298 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-15T00:10:52.298 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-15T00:10:52.301 INFO:tasks.ceph.osd.7:Started 2024-09-15T00:10:52.301 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T00:10:52.379 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:52.377+0000 7f5842675040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T00:10:52.417 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:52.415+0000 7fdc5a51f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T00:10:52.435 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:52.434+0000 7f5842675040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T00:10:52.470 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:52.468+0000 7fdc5a51f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":10,"fsid":"bad2533c-9c09-4af4-ab93-147afbacb705","created":"2024-09-15T00:10:50.203030+0000","modified":"2024-09-15T00:10:52.267142+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T00:10:52.267142+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7337111f-0e02-490d-976b-3814a03371f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"64412633-4dd4-4b8d-b3f1-ed3c664ddf48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0300adee-5ce3-43ce-997b-2c0e970edbc7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ec668801-9b8f-401a-829c-0b1f8d8f08fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9cc48936-5644-44c6-84bb-06510e933e8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f6c53297-9b00-476f-9be8-e76ff762b044","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"737f08a6-2223-46b3-b9fe-6ae18b067ec3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5a5fba64-9249-426d-814b-fd251e8aa910","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:10:52.512 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T00:10:52.512 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T00:10:52.807 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:52.805+0000 7f5842675040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T00:10:52.831 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:52.829+0000 7fdc5a51f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T00:10:52.890 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:52.888+0000 7f5842675040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T00:10:52.914 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:52.913+0000 7fdc5a51f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T00:10:52.998 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:52.996+0000 7f5842675040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T00:10:53.022 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:53.020+0000 7fdc5a51f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T00:10:53.045 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:53.043+0000 7f5842675040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T00:10:53.067 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:53.066+0000 7fdc5a51f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T00:10:53.156 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:53.154+0000 7f5842675040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T00:10:53.178 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:53.176+0000 7fdc5a51f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T00:10:53.208 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:53.207+0000 7f5842675040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T00:10:53.233 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:53.231+0000 7fdc5a51f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T00:10:53.255 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:53.253+0000 7f5842675040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T00:10:53.278 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:53.276+0000 7fdc5a51f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T00:10:53.319 DEBUG:teuthology.orchestra.run.smithi171:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T00:10:53.370 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:53.368+0000 7f5842675040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T00:10:53.387 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:53.386+0000 7fdc5a51f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T00:10:53.422 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:53.421+0000 7f5842675040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T00:10:53.437 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:53.435+0000 7fdc5a51f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T00:10:53.474 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:53.473+0000 7f5842675040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T00:10:53.475 INFO:teuthology.misc.health.smithi171.stdout: 2024-09-15T00:10:53.475 INFO:teuthology.misc.health.smithi171.stdout:{"epoch":10,"fsid":"bad2533c-9c09-4af4-ab93-147afbacb705","created":"2024-09-15T00:10:50.203030+0000","modified":"2024-09-15T00:10:52.267142+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T00:10:52.267142+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7337111f-0e02-490d-976b-3814a03371f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"64412633-4dd4-4b8d-b3f1-ed3c664ddf48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0300adee-5ce3-43ce-997b-2c0e970edbc7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ec668801-9b8f-401a-829c-0b1f8d8f08fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9cc48936-5644-44c6-84bb-06510e933e8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f6c53297-9b00-476f-9be8-e76ff762b044","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"737f08a6-2223-46b3-b9fe-6ae18b067ec3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5a5fba64-9249-426d-814b-fd251e8aa910","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:10:53.485 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-15T00:10:53.488 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:53.487+0000 7fdc5a51f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T00:10:53.545 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:53.543+0000 7f5842675040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T00:10:53.556 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:53.554+0000 7fdc5a51f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T00:10:53.589 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:53.587+0000 7f5842675040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T00:10:53.600 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:53.598+0000 7fdc5a51f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T00:10:53.633 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:53.631+0000 7f5842675040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T00:10:53.643 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:53.641+0000 7fdc5a51f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T00:10:53.754 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:10:53.752+0000 7f9efc20b640 -1 Falling back to public interface 2024-09-15T00:10:53.766 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:10:53.764+0000 7fc51aa18640 -1 Falling back to public interface 2024-09-15T00:10:53.836 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:10:53.834+0000 7f0f1ebbb640 -1 Falling back to public interface 2024-09-15T00:10:53.992 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:10:53.990+0000 7f801e20b640 -1 Falling back to public interface 2024-09-15T00:10:54.021 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:10:54.019+0000 7f0037919640 -1 Falling back to public interface 2024-09-15T00:10:54.040 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:10:54.038+0000 7fef6ffbb640 -1 Falling back to public interface 2024-09-15T00:10:54.062 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:10:54.060+0000 7fcf12923640 -1 Falling back to public interface 2024-09-15T00:10:54.285 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:54.283+0000 7f5842675040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T00:10:54.287 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:10:54.285+0000 7f23e0210640 -1 Falling back to public interface 2024-09-15T00:10:54.292 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-15T00:10:54.290+0000 7fdc5a51f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T00:10:55.299 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:55.298+0000 7f57e2f38640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T00:10:55.300 INFO:tasks.ceph.mgr.y.smithi171.stderr:2024-09-15T00:10:55.298+0000 7f57e2f38640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T00:10:57.580 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:10:57.578+0000 7fc51aa18640 -1 osd.5 0 log_to_monitors true 2024-09-15T00:10:57.629 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:10:57.627+0000 7f0f1ebbb640 -1 osd.3 0 log_to_monitors true 2024-09-15T00:10:57.743 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:10:57.741+0000 7f801e20b640 -1 osd.6 0 log_to_monitors true 2024-09-15T00:10:57.825 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:10:57.823+0000 7f9efc20b640 -1 osd.4 0 log_to_monitors true 2024-09-15T00:10:57.837 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:10:57.835+0000 7fef6ffbb640 -1 osd.1 0 log_to_monitors true 2024-09-15T00:10:58.034 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:10:58.032+0000 7f0037919640 -1 osd.2 0 log_to_monitors true 2024-09-15T00:10:58.119 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:10:58.117+0000 7fcf12923640 -1 osd.0 0 log_to_monitors true 2024-09-15T00:10:58.572 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:10:58.570+0000 7f23e0210640 -1 osd.7 0 log_to_monitors true 2024-09-15T00:11:00.296 DEBUG:teuthology.orchestra.run.smithi171:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T00:11:00.599 INFO:teuthology.misc.health.smithi171.stdout: 2024-09-15T00:11:00.600 INFO:teuthology.misc.health.smithi171.stdout:{"epoch":13,"fsid":"bad2533c-9c09-4af4-ab93-147afbacb705","created":"2024-09-15T00:10:50.203030+0000","modified":"2024-09-15T00:11:00.306029+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T00:10:52.267142+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7337111f-0e02-490d-976b-3814a03371f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"64412633-4dd4-4b8d-b3f1-ed3c664ddf48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0300adee-5ce3-43ce-997b-2c0e970edbc7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ec668801-9b8f-401a-829c-0b1f8d8f08fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9cc48936-5644-44c6-84bb-06510e933e8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f6c53297-9b00-476f-9be8-e76ff762b044","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"737f08a6-2223-46b3-b9fe-6ae18b067ec3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5a5fba64-9249-426d-814b-fd251e8aa910","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:11:00.614 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-15T00:11:02.437 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:02.435+0000 7f801a45f640 -1 osd.6 0 waiting for initial osdmap 2024-09-15T00:11:02.447 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:02.446+0000 7f8011a6b640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:11:02.461 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:02.459+0000 7f9ef7c31640 -1 osd.4 0 waiting for initial osdmap 2024-09-15T00:11:02.472 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:02.471+0000 7f9eefa51640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:11:02.516 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:02.515+0000 7fc51637a640 -1 osd.5 0 waiting for initial osdmap 2024-09-15T00:11:02.528 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:02.527+0000 7fc50e19a640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:11:02.581 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:02.579+0000 7fcf0e8b2640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T00:11:02.595 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:02.594+0000 7fcf066d2640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:11:02.621 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:02.619+0000 7fef6c75e640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T00:11:02.633 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:02.632+0000 7fef63d6a640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:11:02.642 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:02.640+0000 7f00338a8640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T00:11:02.657 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:02.655+0000 7f002b6c8640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:11:02.709 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:02.707+0000 7f0f1b35e640 -1 osd.3 0 waiting for initial osdmap 2024-09-15T00:11:02.719 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:02.718+0000 7f0f1296a640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:11:03.306 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:03.304+0000 7f23dbbd9640 -1 osd.7 0 waiting for initial osdmap 2024-09-15T00:11:03.317 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:03.316+0000 7f23d39f9640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:11:07.421 DEBUG:teuthology.orchestra.run.smithi171:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T00:11:07.699 INFO:teuthology.misc.health.smithi171.stdout: 2024-09-15T00:11:07.700 INFO:teuthology.misc.health.smithi171.stdout:{"epoch":17,"fsid":"bad2533c-9c09-4af4-ab93-147afbacb705","created":"2024-09-15T00:10:50.203030+0000","modified":"2024-09-15T00:11:06.466415+0000","last_up_change":"2024-09-15T00:11:03.441129+0000","last_in_change":"2024-09-15T00:10:52.267142+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T00:11:04.326356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7337111f-0e02-490d-976b-3814a03371f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6825","nonce":2906515165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6827","nonce":2906515165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6831","nonce":2906515165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6829","nonce":2906515165}]},"public_addr":"172.21.15.171:6825/2906515165","cluster_addr":"172.21.15.171:6827/2906515165","heartbeat_back_addr":"172.21.15.171:6831/2906515165","heartbeat_front_addr":"172.21.15.171:6829/2906515165","state":["exists","up"]},{"osd":1,"uuid":"64412633-4dd4-4b8d-b3f1-ed3c664ddf48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6817","nonce":3039831960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6819","nonce":3039831960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6823","nonce":3039831960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6821","nonce":3039831960}]},"public_addr":"172.21.15.171:6817/3039831960","cluster_addr":"172.21.15.171:6819/3039831960","heartbeat_back_addr":"172.21.15.171:6823/3039831960","heartbeat_front_addr":"172.21.15.171:6821/3039831960","state":["exists","up"]},{"osd":2,"uuid":"0300adee-5ce3-43ce-997b-2c0e970edbc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":3950855930},{"type":"v1","addr":"172.21.15.171:6809","nonce":3950855930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":3950855930},{"type":"v1","addr":"172.21.15.171:6811","nonce":3950855930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":3950855930},{"type":"v1","addr":"172.21.15.171:6815","nonce":3950855930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":3950855930},{"type":"v1","addr":"172.21.15.171:6813","nonce":3950855930}]},"public_addr":"172.21.15.171:6809/3950855930","cluster_addr":"172.21.15.171:6811/3950855930","heartbeat_back_addr":"172.21.15.171:6815/3950855930","heartbeat_front_addr":"172.21.15.171:6813/3950855930","state":["exists","up"]},{"osd":3,"uuid":"ec668801-9b8f-401a-829c-0b1f8d8f08fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6801","nonce":1819098683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6803","nonce":1819098683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6807","nonce":1819098683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6805","nonce":1819098683}]},"public_addr":"172.21.15.171:6801/1819098683","cluster_addr":"172.21.15.171:6803/1819098683","heartbeat_back_addr":"172.21.15.171:6807/1819098683","heartbeat_front_addr":"172.21.15.171:6805/1819098683","state":["exists","up"]},{"osd":4,"uuid":"9cc48936-5644-44c6-84bb-06510e933e8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6801","nonce":2801626173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6803","nonce":2801626173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6807","nonce":2801626173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6805","nonce":2801626173}]},"public_addr":"172.21.15.184:6801/2801626173","cluster_addr":"172.21.15.184:6803/2801626173","heartbeat_back_addr":"172.21.15.184:6807/2801626173","heartbeat_front_addr":"172.21.15.184:6805/2801626173","state":["exists","up"]},{"osd":5,"uuid":"f6c53297-9b00-476f-9be8-e76ff762b044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6809","nonce":3271066302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6811","nonce":3271066302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6815","nonce":3271066302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6813","nonce":3271066302}]},"public_addr":"172.21.15.184:6809/3271066302","cluster_addr":"172.21.15.184:6811/3271066302","heartbeat_back_addr":"172.21.15.184:6815/3271066302","heartbeat_front_addr":"172.21.15.184:6813/3271066302","state":["exists","up"]},{"osd":6,"uuid":"737f08a6-2223-46b3-b9fe-6ae18b067ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6817","nonce":1903994256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6819","nonce":1903994256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6823","nonce":1903994256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6821","nonce":1903994256}]},"public_addr":"172.21.15.184:6817/1903994256","cluster_addr":"172.21.15.184:6819/1903994256","heartbeat_back_addr":"172.21.15.184:6823/1903994256","heartbeat_front_addr":"172.21.15.184:6821/1903994256","state":["exists","up"]},{"osd":7,"uuid":"5a5fba64-9249-426d-814b-fd251e8aa910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6825","nonce":978565004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6827","nonce":978565004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6831","nonce":978565004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6829","nonce":978565004}]},"public_addr":"172.21.15.184:6825/978565004","cluster_addr":"172.21.15.184:6827/978565004","heartbeat_back_addr":"172.21.15.184:6831/978565004","heartbeat_front_addr":"172.21.15.184:6829/978565004","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:59.070337+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.884683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.998882+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.656630+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.802744+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.567865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.739964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:59.565310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:11:07.711 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-15T00:11:07.711 INFO:tasks.ceph:Creating RBD pool 2024-09-15T00:11:07.711 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-15T00:11:08.525 INFO:teuthology.orchestra.run.smithi171.stderr:pool 'rbd' created 2024-09-15T00:11:08.539 DEBUG:teuthology.orchestra.run.smithi171:> rbd --cluster ceph pool init rbd 2024-09-15T00:11:11.552 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T00:11:11.552 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T00:11:11.553 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T00:11:11.553 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T00:11:11.553 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T00:11:11.886 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:11:11.897 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1880765683},{"type":"v1","addr":"172.21.15.171:6833","nonce":1880765683}]},"active_addr":"172.21.15.171:6833/1880765683","active_change":"2024-09-15T00:10:54.285950+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4102,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":3666413485}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":1198395731}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":3225390488}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":1211295947}]}]} 2024-09-15T00:11:11.901 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T00:11:11.901 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T00:11:11.901 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T00:11:12.181 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:11:12.181 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":21,"fsid":"bad2533c-9c09-4af4-ab93-147afbacb705","created":"2024-09-15T00:10:50.203030+0000","modified":"2024-09-15T00:11:11.536030+0000","last_up_change":"2024-09-15T00:11:03.441129+0000","last_in_change":"2024-09-15T00:10:52.267142+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T00:11:04.326356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T00:11:08.005967+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7337111f-0e02-490d-976b-3814a03371f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6825","nonce":2906515165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6827","nonce":2906515165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6831","nonce":2906515165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6829","nonce":2906515165}]},"public_addr":"172.21.15.171:6825/2906515165","cluster_addr":"172.21.15.171:6827/2906515165","heartbeat_back_addr":"172.21.15.171:6831/2906515165","heartbeat_front_addr":"172.21.15.171:6829/2906515165","state":["exists","up"]},{"osd":1,"uuid":"64412633-4dd4-4b8d-b3f1-ed3c664ddf48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6817","nonce":3039831960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6819","nonce":3039831960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6823","nonce":3039831960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6821","nonce":3039831960}]},"public_addr":"172.21.15.171:6817/3039831960","cluster_addr":"172.21.15.171:6819/3039831960","heartbeat_back_addr":"172.21.15.171:6823/3039831960","heartbeat_front_addr":"172.21.15.171:6821/3039831960","state":["exists","up"]},{"osd":2,"uuid":"0300adee-5ce3-43ce-997b-2c0e970edbc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":3950855930},{"type":"v1","addr":"172.21.15.171:6809","nonce":3950855930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":3950855930},{"type":"v1","addr":"172.21.15.171:6811","nonce":3950855930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":3950855930},{"type":"v1","addr":"172.21.15.171:6815","nonce":3950855930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":3950855930},{"type":"v1","addr":"172.21.15.171:6813","nonce":3950855930}]},"public_addr":"172.21.15.171:6809/3950855930","cluster_addr":"172.21.15.171:6811/3950855930","heartbeat_back_addr":"172.21.15.171:6815/3950855930","heartbeat_front_addr":"172.21.15.171:6813/3950855930","state":["exists","up"]},{"osd":3,"uuid":"ec668801-9b8f-401a-829c-0b1f8d8f08fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6801","nonce":1819098683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6803","nonce":1819098683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6807","nonce":1819098683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6805","nonce":1819098683}]},"public_addr":"172.21.15.171:6801/1819098683","cluster_addr":"172.21.15.171:6803/1819098683","heartbeat_back_addr":"172.21.15.171:6807/1819098683","heartbeat_front_addr":"172.21.15.171:6805/1819098683","state":["exists","up"]},{"osd":4,"uuid":"9cc48936-5644-44c6-84bb-06510e933e8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6801","nonce":2801626173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6803","nonce":2801626173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6807","nonce":2801626173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6805","nonce":2801626173}]},"public_addr":"172.21.15.184:6801/2801626173","cluster_addr":"172.21.15.184:6803/2801626173","heartbeat_back_addr":"172.21.15.184:6807/2801626173","heartbeat_front_addr":"172.21.15.184:6805/2801626173","state":["exists","up"]},{"osd":5,"uuid":"f6c53297-9b00-476f-9be8-e76ff762b044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6809","nonce":3271066302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6811","nonce":3271066302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6815","nonce":3271066302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6813","nonce":3271066302}]},"public_addr":"172.21.15.184:6809/3271066302","cluster_addr":"172.21.15.184:6811/3271066302","heartbeat_back_addr":"172.21.15.184:6815/3271066302","heartbeat_front_addr":"172.21.15.184:6813/3271066302","state":["exists","up"]},{"osd":6,"uuid":"737f08a6-2223-46b3-b9fe-6ae18b067ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6817","nonce":1903994256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6819","nonce":1903994256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6823","nonce":1903994256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6821","nonce":1903994256}]},"public_addr":"172.21.15.184:6817/1903994256","cluster_addr":"172.21.15.184:6819/1903994256","heartbeat_back_addr":"172.21.15.184:6823/1903994256","heartbeat_front_addr":"172.21.15.184:6821/1903994256","state":["exists","up"]},{"osd":7,"uuid":"5a5fba64-9249-426d-814b-fd251e8aa910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6825","nonce":978565004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6827","nonce":978565004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6831","nonce":978565004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6829","nonce":978565004}]},"public_addr":"172.21.15.184:6825/978565004","cluster_addr":"172.21.15.184:6827/978565004","heartbeat_back_addr":"172.21.15.184:6831/978565004","heartbeat_front_addr":"172.21.15.184:6829/978565004","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:59.070337+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.884683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.998882+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.656630+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.802744+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.567865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.739964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:59.565310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:11:12.194 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T00:11:12.194 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T00:11:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:11:12.470 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":21,"fsid":"bad2533c-9c09-4af4-ab93-147afbacb705","created":"2024-09-15T00:10:50.203030+0000","modified":"2024-09-15T00:11:11.536030+0000","last_up_change":"2024-09-15T00:11:03.441129+0000","last_in_change":"2024-09-15T00:10:52.267142+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T00:11:04.326356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T00:11:08.005967+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7337111f-0e02-490d-976b-3814a03371f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6825","nonce":2906515165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6827","nonce":2906515165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6831","nonce":2906515165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6829","nonce":2906515165}]},"public_addr":"172.21.15.171:6825/2906515165","cluster_addr":"172.21.15.171:6827/2906515165","heartbeat_back_addr":"172.21.15.171:6831/2906515165","heartbeat_front_addr":"172.21.15.171:6829/2906515165","state":["exists","up"]},{"osd":1,"uuid":"64412633-4dd4-4b8d-b3f1-ed3c664ddf48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6817","nonce":3039831960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6819","nonce":3039831960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6823","nonce":3039831960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6821","nonce":3039831960}]},"public_addr":"172.21.15.171:6817/3039831960","cluster_addr":"172.21.15.171:6819/3039831960","heartbeat_back_addr":"172.21.15.171:6823/3039831960","heartbeat_front_addr":"172.21.15.171:6821/3039831960","state":["exists","up"]},{"osd":2,"uuid":"0300adee-5ce3-43ce-997b-2c0e970edbc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":3950855930},{"type":"v1","addr":"172.21.15.171:6809","nonce":3950855930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":3950855930},{"type":"v1","addr":"172.21.15.171:6811","nonce":3950855930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":3950855930},{"type":"v1","addr":"172.21.15.171:6815","nonce":3950855930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":3950855930},{"type":"v1","addr":"172.21.15.171:6813","nonce":3950855930}]},"public_addr":"172.21.15.171:6809/3950855930","cluster_addr":"172.21.15.171:6811/3950855930","heartbeat_back_addr":"172.21.15.171:6815/3950855930","heartbeat_front_addr":"172.21.15.171:6813/3950855930","state":["exists","up"]},{"osd":3,"uuid":"ec668801-9b8f-401a-829c-0b1f8d8f08fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6801","nonce":1819098683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6803","nonce":1819098683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6807","nonce":1819098683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6805","nonce":1819098683}]},"public_addr":"172.21.15.171:6801/1819098683","cluster_addr":"172.21.15.171:6803/1819098683","heartbeat_back_addr":"172.21.15.171:6807/1819098683","heartbeat_front_addr":"172.21.15.171:6805/1819098683","state":["exists","up"]},{"osd":4,"uuid":"9cc48936-5644-44c6-84bb-06510e933e8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6801","nonce":2801626173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6803","nonce":2801626173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6807","nonce":2801626173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6805","nonce":2801626173}]},"public_addr":"172.21.15.184:6801/2801626173","cluster_addr":"172.21.15.184:6803/2801626173","heartbeat_back_addr":"172.21.15.184:6807/2801626173","heartbeat_front_addr":"172.21.15.184:6805/2801626173","state":["exists","up"]},{"osd":5,"uuid":"f6c53297-9b00-476f-9be8-e76ff762b044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6809","nonce":3271066302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6811","nonce":3271066302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6815","nonce":3271066302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6813","nonce":3271066302}]},"public_addr":"172.21.15.184:6809/3271066302","cluster_addr":"172.21.15.184:6811/3271066302","heartbeat_back_addr":"172.21.15.184:6815/3271066302","heartbeat_front_addr":"172.21.15.184:6813/3271066302","state":["exists","up"]},{"osd":6,"uuid":"737f08a6-2223-46b3-b9fe-6ae18b067ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6817","nonce":1903994256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6819","nonce":1903994256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6823","nonce":1903994256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6821","nonce":1903994256}]},"public_addr":"172.21.15.184:6817/1903994256","cluster_addr":"172.21.15.184:6819/1903994256","heartbeat_back_addr":"172.21.15.184:6823/1903994256","heartbeat_front_addr":"172.21.15.184:6821/1903994256","state":["exists","up"]},{"osd":7,"uuid":"5a5fba64-9249-426d-814b-fd251e8aa910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6825","nonce":978565004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6827","nonce":978565004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6831","nonce":978565004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6829","nonce":978565004}]},"public_addr":"172.21.15.184:6825/978565004","cluster_addr":"172.21.15.184:6827/978565004","heartbeat_back_addr":"172.21.15.184:6831/978565004","heartbeat_front_addr":"172.21.15.184:6829/978565004","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:59.070337+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.884683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.998882+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.656630+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.802744+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.567865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.739964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:59.565310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:11:12.483 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T00:11:12.483 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T00:11:12.484 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T00:11:12.484 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T00:11:12.484 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T00:11:12.485 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T00:11:12.485 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-15T00:11:12.485 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-15T00:11:12.726 INFO:teuthology.orchestra.run.smithi171.stdout:60129542152 2024-09-15T00:11:12.726 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T00:11:12.727 INFO:teuthology.orchestra.run.smithi171.stdout:60129542149 2024-09-15T00:11:12.728 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T00:11:12.731 INFO:teuthology.orchestra.run.smithi171.stdout:60129542150 2024-09-15T00:11:12.731 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T00:11:12.732 INFO:teuthology.orchestra.run.smithi171.stdout:60129542149 2024-09-15T00:11:12.732 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T00:11:12.733 INFO:teuthology.orchestra.run.smithi171.stdout:60129542150 2024-09-15T00:11:12.734 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T00:11:12.735 INFO:teuthology.orchestra.run.smithi171.stdout:60129542147 2024-09-15T00:11:12.735 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T00:11:12.736 INFO:teuthology.orchestra.run.smithi171.stdout:60129542148 2024-09-15T00:11:12.736 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T00:11:12.738 INFO:teuthology.orchestra.run.smithi171.stdout:60129542149 2024-09-15T00:11:12.738 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T00:11:13.305 INFO:teuthology.orchestra.run.smithi171.stdout:60129542148 2024-09-15T00:11:13.315 INFO:teuthology.orchestra.run.smithi171.stdout:60129542149 2024-09-15T00:11:13.321 INFO:teuthology.orchestra.run.smithi171.stdout:60129542149 2024-09-15T00:11:13.324 INFO:teuthology.orchestra.run.smithi171.stdout:60129542151 2024-09-15T00:11:13.326 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.7 2024-09-15T00:11:13.328 INFO:teuthology.orchestra.run.smithi171.stdout:60129542148 2024-09-15T00:11:13.330 INFO:teuthology.orchestra.run.smithi171.stdout:60129542146 2024-09-15T00:11:13.330 INFO:teuthology.orchestra.run.smithi171.stdout:60129542147 2024-09-15T00:11:13.338 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.6 2024-09-15T00:11:13.340 INFO:teuthology.orchestra.run.smithi171.stdout:60129542148 2024-09-15T00:11:13.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.3 2024-09-15T00:11:13.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.4 2024-09-15T00:11:13.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.0 2024-09-15T00:11:13.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.1 2024-09-15T00:11:13.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.5 2024-09-15T00:11:13.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.2 2024-09-15T00:11:14.327 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T00:11:14.338 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T00:11:14.351 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T00:11:14.351 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T00:11:14.355 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T00:11:14.356 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T00:11:14.356 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T00:11:14.359 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T00:11:14.901 INFO:teuthology.orchestra.run.smithi171.stdout:60129542149 2024-09-15T00:11:14.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.7 2024-09-15T00:11:14.921 DEBUG:teuthology.parallel:result is None 2024-09-15T00:11:14.939 INFO:teuthology.orchestra.run.smithi171.stdout:60129542148 2024-09-15T00:11:14.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.1 2024-09-15T00:11:14.953 DEBUG:teuthology.parallel:result is None 2024-09-15T00:11:14.969 INFO:teuthology.orchestra.run.smithi171.stdout:60129542152 2024-09-15T00:11:14.970 INFO:teuthology.orchestra.run.smithi171.stdout:60129542149 2024-09-15T00:11:14.971 INFO:teuthology.orchestra.run.smithi171.stdout:60129542150 2024-09-15T00:11:14.975 INFO:teuthology.orchestra.run.smithi171.stdout:60129542149 2024-09-15T00:11:14.984 INFO:teuthology.orchestra.run.smithi171.stdout:60129542150 2024-09-15T00:11:14.985 INFO:teuthology.orchestra.run.smithi171.stdout:60129542147 2024-09-15T00:11:14.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.4 2024-09-15T00:11:14.988 DEBUG:teuthology.parallel:result is None 2024-09-15T00:11:14.992 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2024-09-15T00:11:14.992 DEBUG:teuthology.parallel:result is None 2024-09-15T00:11:14.993 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.0 2024-09-15T00:11:14.993 DEBUG:teuthology.parallel:result is None 2024-09-15T00:11:14.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.2 2024-09-15T00:11:14.999 DEBUG:teuthology.parallel:result is None 2024-09-15T00:11:15.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.5 2024-09-15T00:11:15.003 DEBUG:teuthology.parallel:result is None 2024-09-15T00:11:15.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2024-09-15T00:11:15.004 DEBUG:teuthology.parallel:result is None 2024-09-15T00:11:15.004 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T00:11:15.004 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:11:15.322 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:11:15.322 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:11:15.334 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-15T00:11:14.999189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218040,"kb_used_data":1784,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756680,"statfs":{"total":773094113280,"available":772870840320,"internally_reserved":0,"allocated":1826816,"data_stored":1758366,"data_compressed":11008,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.472436"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641700+0000","last_change":"2024-09-15T00:11:11.641988+0000","last_active":"2024-09-15T00:11:11.641700+0000","last_peered":"2024-09-15T00:11:11.641700+0000","last_clean":"2024-09-15T00:11:11.641700+0000","last_became_active":"2024-09-15T00:11:09.534794+0000","last_became_peered":"2024-09-15T00:11:09.534794+0000","last_unstale":"2024-09-15T00:11:11.641700+0000","last_undegraded":"2024-09-15T00:11:11.641700+0000","last_fullsized":"2024-09-15T00:11:11.641700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:55:52.135410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.545420+0000","last_change":"2024-09-15T00:11:11.545630+0000","last_active":"2024-09-15T00:11:11.545420+0000","last_peered":"2024-09-15T00:11:11.545420+0000","last_clean":"2024-09-15T00:11:11.545420+0000","last_became_active":"2024-09-15T00:11:09.536615+0000","last_became_peered":"2024-09-15T00:11:09.536615+0000","last_unstale":"2024-09-15T00:11:11.545420+0000","last_undegraded":"2024-09-15T00:11:11.545420+0000","last_fullsized":"2024-09-15T00:11:11.545420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:51:38.340257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035555,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641767+0000","last_change":"2024-09-15T00:11:11.642034+0000","last_active":"2024-09-15T00:11:11.641767+0000","last_peered":"2024-09-15T00:11:11.641767+0000","last_clean":"2024-09-15T00:11:11.641767+0000","last_became_active":"2024-09-15T00:11:09.824182+0000","last_became_peered":"2024-09-15T00:11:09.824182+0000","last_unstale":"2024-09-15T00:11:11.641767+0000","last_undegraded":"2024-09-15T00:11:11.641767+0000","last_fullsized":"2024-09-15T00:11:11.641767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:16.296783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.545471+0000","last_change":"2024-09-15T00:11:11.545675+0000","last_active":"2024-09-15T00:11:11.545471+0000","last_peered":"2024-09-15T00:11:11.545471+0000","last_clean":"2024-09-15T00:11:11.545471+0000","last_became_active":"2024-09-15T00:11:09.823838+0000","last_became_peered":"2024-09-15T00:11:09.823838+0000","last_unstale":"2024-09-15T00:11:11.545471+0000","last_undegraded":"2024-09-15T00:11:11.545471+0000","last_fullsized":"2024-09-15T00:11:11.545471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:24.994222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.543519+0000","last_change":"2024-09-15T00:11:11.543519+0000","last_active":"2024-09-15T00:11:11.543519+0000","last_peered":"2024-09-15T00:11:11.543519+0000","last_clean":"2024-09-15T00:11:11.543519+0000","last_became_active":"2024-09-15T00:11:09.533896+0000","last_became_peered":"2024-09-15T00:11:09.533896+0000","last_unstale":"2024-09-15T00:11:11.543519+0000","last_undegraded":"2024-09-15T00:11:11.543519+0000","last_fullsized":"2024-09-15T00:11:11.543519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:47.613931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.540934+0000","last_change":"2024-09-15T00:11:11.541173+0000","last_active":"2024-09-15T00:11:11.540934+0000","last_peered":"2024-09-15T00:11:11.540934+0000","last_clean":"2024-09-15T00:11:11.540934+0000","last_became_active":"2024-09-15T00:11:09.533270+0000","last_became_peered":"2024-09-15T00:11:09.533270+0000","last_unstale":"2024-09-15T00:11:11.540934+0000","last_undegraded":"2024-09-15T00:11:11.540934+0000","last_fullsized":"2024-09-15T00:11:11.540934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:42.665083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047411000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641732+0000","last_change":"2024-09-15T00:11:11.641995+0000","last_active":"2024-09-15T00:11:11.641732+0000","last_peered":"2024-09-15T00:11:11.641732+0000","last_clean":"2024-09-15T00:11:11.641732+0000","last_became_active":"2024-09-15T00:11:09.533404+0000","last_became_peered":"2024-09-15T00:11:09.533404+0000","last_unstale":"2024-09-15T00:11:11.641732+0000","last_undegraded":"2024-09-15T00:11:11.641732+0000","last_fullsized":"2024-09-15T00:11:11.641732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:20.611551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046024999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.542499+0000","last_change":"2024-09-15T00:11:11.542743+0000","last_active":"2024-09-15T00:11:11.542499+0000","last_peered":"2024-09-15T00:11:11.542499+0000","last_clean":"2024-09-15T00:11:11.542499+0000","last_became_active":"2024-09-15T00:11:09.536357+0000","last_became_peered":"2024-09-15T00:11:09.536357+0000","last_unstale":"2024-09-15T00:11:11.542499+0000","last_undegraded":"2024-09-15T00:11:11.542499+0000","last_fullsized":"2024-09-15T00:11:11.542499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:31:31.968858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641483+0000","last_change":"2024-09-15T00:11:05.468585+0000","last_active":"2024-09-15T00:11:11.641483+0000","last_peered":"2024-09-15T00:11:11.641483+0000","last_clean":"2024-09-15T00:11:11.641483+0000","last_became_active":"2024-09-15T00:11:05.468352+0000","last_became_peered":"2024-09-15T00:11:05.468352+0000","last_unstale":"2024-09-15T00:11:11.641483+0000","last_undegraded":"2024-09-15T00:11:11.641483+0000","last_fullsized":"2024-09-15T00:11:11.641483+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_clean_scrub_stamp":"2024-09-15T00:11:04.453672+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:01:23.254180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11008,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":662567,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":72218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":72218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":662567,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:11:15.335 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:11:15.612 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:11:15.612 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:11:15.624 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-15T00:11:14.999189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218040,"kb_used_data":1784,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756680,"statfs":{"total":773094113280,"available":772870840320,"internally_reserved":0,"allocated":1826816,"data_stored":1758366,"data_compressed":11008,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.472436"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641700+0000","last_change":"2024-09-15T00:11:11.641988+0000","last_active":"2024-09-15T00:11:11.641700+0000","last_peered":"2024-09-15T00:11:11.641700+0000","last_clean":"2024-09-15T00:11:11.641700+0000","last_became_active":"2024-09-15T00:11:09.534794+0000","last_became_peered":"2024-09-15T00:11:09.534794+0000","last_unstale":"2024-09-15T00:11:11.641700+0000","last_undegraded":"2024-09-15T00:11:11.641700+0000","last_fullsized":"2024-09-15T00:11:11.641700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:55:52.135410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.545420+0000","last_change":"2024-09-15T00:11:11.545630+0000","last_active":"2024-09-15T00:11:11.545420+0000","last_peered":"2024-09-15T00:11:11.545420+0000","last_clean":"2024-09-15T00:11:11.545420+0000","last_became_active":"2024-09-15T00:11:09.536615+0000","last_became_peered":"2024-09-15T00:11:09.536615+0000","last_unstale":"2024-09-15T00:11:11.545420+0000","last_undegraded":"2024-09-15T00:11:11.545420+0000","last_fullsized":"2024-09-15T00:11:11.545420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:51:38.340257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035555,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641767+0000","last_change":"2024-09-15T00:11:11.642034+0000","last_active":"2024-09-15T00:11:11.641767+0000","last_peered":"2024-09-15T00:11:11.641767+0000","last_clean":"2024-09-15T00:11:11.641767+0000","last_became_active":"2024-09-15T00:11:09.824182+0000","last_became_peered":"2024-09-15T00:11:09.824182+0000","last_unstale":"2024-09-15T00:11:11.641767+0000","last_undegraded":"2024-09-15T00:11:11.641767+0000","last_fullsized":"2024-09-15T00:11:11.641767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:16.296783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.545471+0000","last_change":"2024-09-15T00:11:11.545675+0000","last_active":"2024-09-15T00:11:11.545471+0000","last_peered":"2024-09-15T00:11:11.545471+0000","last_clean":"2024-09-15T00:11:11.545471+0000","last_became_active":"2024-09-15T00:11:09.823838+0000","last_became_peered":"2024-09-15T00:11:09.823838+0000","last_unstale":"2024-09-15T00:11:11.545471+0000","last_undegraded":"2024-09-15T00:11:11.545471+0000","last_fullsized":"2024-09-15T00:11:11.545471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:24.994222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.543519+0000","last_change":"2024-09-15T00:11:11.543519+0000","last_active":"2024-09-15T00:11:11.543519+0000","last_peered":"2024-09-15T00:11:11.543519+0000","last_clean":"2024-09-15T00:11:11.543519+0000","last_became_active":"2024-09-15T00:11:09.533896+0000","last_became_peered":"2024-09-15T00:11:09.533896+0000","last_unstale":"2024-09-15T00:11:11.543519+0000","last_undegraded":"2024-09-15T00:11:11.543519+0000","last_fullsized":"2024-09-15T00:11:11.543519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:47.613931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.540934+0000","last_change":"2024-09-15T00:11:11.541173+0000","last_active":"2024-09-15T00:11:11.540934+0000","last_peered":"2024-09-15T00:11:11.540934+0000","last_clean":"2024-09-15T00:11:11.540934+0000","last_became_active":"2024-09-15T00:11:09.533270+0000","last_became_peered":"2024-09-15T00:11:09.533270+0000","last_unstale":"2024-09-15T00:11:11.540934+0000","last_undegraded":"2024-09-15T00:11:11.540934+0000","last_fullsized":"2024-09-15T00:11:11.540934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:42.665083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047411000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641732+0000","last_change":"2024-09-15T00:11:11.641995+0000","last_active":"2024-09-15T00:11:11.641732+0000","last_peered":"2024-09-15T00:11:11.641732+0000","last_clean":"2024-09-15T00:11:11.641732+0000","last_became_active":"2024-09-15T00:11:09.533404+0000","last_became_peered":"2024-09-15T00:11:09.533404+0000","last_unstale":"2024-09-15T00:11:11.641732+0000","last_undegraded":"2024-09-15T00:11:11.641732+0000","last_fullsized":"2024-09-15T00:11:11.641732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:20.611551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046024999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.542499+0000","last_change":"2024-09-15T00:11:11.542743+0000","last_active":"2024-09-15T00:11:11.542499+0000","last_peered":"2024-09-15T00:11:11.542499+0000","last_clean":"2024-09-15T00:11:11.542499+0000","last_became_active":"2024-09-15T00:11:09.536357+0000","last_became_peered":"2024-09-15T00:11:09.536357+0000","last_unstale":"2024-09-15T00:11:11.542499+0000","last_undegraded":"2024-09-15T00:11:11.542499+0000","last_fullsized":"2024-09-15T00:11:11.542499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:31:31.968858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641483+0000","last_change":"2024-09-15T00:11:05.468585+0000","last_active":"2024-09-15T00:11:11.641483+0000","last_peered":"2024-09-15T00:11:11.641483+0000","last_clean":"2024-09-15T00:11:11.641483+0000","last_became_active":"2024-09-15T00:11:05.468352+0000","last_became_peered":"2024-09-15T00:11:05.468352+0000","last_unstale":"2024-09-15T00:11:11.641483+0000","last_undegraded":"2024-09-15T00:11:11.641483+0000","last_fullsized":"2024-09-15T00:11:11.641483+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_clean_scrub_stamp":"2024-09-15T00:11:04.453672+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:01:23.254180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11008,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":662567,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":72218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":72218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":662567,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:11:15.625 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T00:11:15.625 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T00:11:15.625 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T00:11:15.626 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T00:11:15.935 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:11:15.936 INFO:teuthology.orchestra.run.smithi171.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T00:11:15.948 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T00:11:15.949 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-15T00:11:15.960 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-15T00:11:15.960 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}} 2024-09-15T00:11:15.960 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-15T00:11:15.960 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-15T00:11:16.257 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-15T00:11:16.257 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T00:11:16.257 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:11:16.531 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:11:16.531 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:11:16.543 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-15T00:11:16.294281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218040,"kb_used_data":1784,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756680,"statfs":{"total":773094113280,"available":772870840320,"internally_reserved":0,"allocated":1826816,"data_stored":1758366,"data_compressed":11008,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001091"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641700+0000","last_change":"2024-09-15T00:11:11.641988+0000","last_active":"2024-09-15T00:11:11.641700+0000","last_peered":"2024-09-15T00:11:11.641700+0000","last_clean":"2024-09-15T00:11:11.641700+0000","last_became_active":"2024-09-15T00:11:09.534794+0000","last_became_peered":"2024-09-15T00:11:09.534794+0000","last_unstale":"2024-09-15T00:11:11.641700+0000","last_undegraded":"2024-09-15T00:11:11.641700+0000","last_fullsized":"2024-09-15T00:11:11.641700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:55:52.135410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.545420+0000","last_change":"2024-09-15T00:11:11.545630+0000","last_active":"2024-09-15T00:11:11.545420+0000","last_peered":"2024-09-15T00:11:11.545420+0000","last_clean":"2024-09-15T00:11:11.545420+0000","last_became_active":"2024-09-15T00:11:09.536615+0000","last_became_peered":"2024-09-15T00:11:09.536615+0000","last_unstale":"2024-09-15T00:11:11.545420+0000","last_undegraded":"2024-09-15T00:11:11.545420+0000","last_fullsized":"2024-09-15T00:11:11.545420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:51:38.340257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035555,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641767+0000","last_change":"2024-09-15T00:11:11.642034+0000","last_active":"2024-09-15T00:11:11.641767+0000","last_peered":"2024-09-15T00:11:11.641767+0000","last_clean":"2024-09-15T00:11:11.641767+0000","last_became_active":"2024-09-15T00:11:09.824182+0000","last_became_peered":"2024-09-15T00:11:09.824182+0000","last_unstale":"2024-09-15T00:11:11.641767+0000","last_undegraded":"2024-09-15T00:11:11.641767+0000","last_fullsized":"2024-09-15T00:11:11.641767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:16.296783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.545471+0000","last_change":"2024-09-15T00:11:11.545675+0000","last_active":"2024-09-15T00:11:11.545471+0000","last_peered":"2024-09-15T00:11:11.545471+0000","last_clean":"2024-09-15T00:11:11.545471+0000","last_became_active":"2024-09-15T00:11:09.823838+0000","last_became_peered":"2024-09-15T00:11:09.823838+0000","last_unstale":"2024-09-15T00:11:11.545471+0000","last_undegraded":"2024-09-15T00:11:11.545471+0000","last_fullsized":"2024-09-15T00:11:11.545471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:24.994222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.543519+0000","last_change":"2024-09-15T00:11:11.543519+0000","last_active":"2024-09-15T00:11:11.543519+0000","last_peered":"2024-09-15T00:11:11.543519+0000","last_clean":"2024-09-15T00:11:11.543519+0000","last_became_active":"2024-09-15T00:11:09.533896+0000","last_became_peered":"2024-09-15T00:11:09.533896+0000","last_unstale":"2024-09-15T00:11:11.543519+0000","last_undegraded":"2024-09-15T00:11:11.543519+0000","last_fullsized":"2024-09-15T00:11:11.543519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:47.613931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.540934+0000","last_change":"2024-09-15T00:11:11.541173+0000","last_active":"2024-09-15T00:11:11.540934+0000","last_peered":"2024-09-15T00:11:11.540934+0000","last_clean":"2024-09-15T00:11:11.540934+0000","last_became_active":"2024-09-15T00:11:09.533270+0000","last_became_peered":"2024-09-15T00:11:09.533270+0000","last_unstale":"2024-09-15T00:11:11.540934+0000","last_undegraded":"2024-09-15T00:11:11.540934+0000","last_fullsized":"2024-09-15T00:11:11.540934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:42.665083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047411000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641732+0000","last_change":"2024-09-15T00:11:11.641995+0000","last_active":"2024-09-15T00:11:11.641732+0000","last_peered":"2024-09-15T00:11:11.641732+0000","last_clean":"2024-09-15T00:11:11.641732+0000","last_became_active":"2024-09-15T00:11:09.533404+0000","last_became_peered":"2024-09-15T00:11:09.533404+0000","last_unstale":"2024-09-15T00:11:11.641732+0000","last_undegraded":"2024-09-15T00:11:11.641732+0000","last_fullsized":"2024-09-15T00:11:11.641732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:20.611551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046024999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.542499+0000","last_change":"2024-09-15T00:11:11.542743+0000","last_active":"2024-09-15T00:11:11.542499+0000","last_peered":"2024-09-15T00:11:11.542499+0000","last_clean":"2024-09-15T00:11:11.542499+0000","last_became_active":"2024-09-15T00:11:09.536357+0000","last_became_peered":"2024-09-15T00:11:09.536357+0000","last_unstale":"2024-09-15T00:11:11.542499+0000","last_undegraded":"2024-09-15T00:11:11.542499+0000","last_fullsized":"2024-09-15T00:11:11.542499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:31:31.968858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641483+0000","last_change":"2024-09-15T00:11:05.468585+0000","last_active":"2024-09-15T00:11:11.641483+0000","last_peered":"2024-09-15T00:11:11.641483+0000","last_clean":"2024-09-15T00:11:11.641483+0000","last_became_active":"2024-09-15T00:11:05.468352+0000","last_became_peered":"2024-09-15T00:11:05.468352+0000","last_unstale":"2024-09-15T00:11:11.641483+0000","last_undegraded":"2024-09-15T00:11:11.641483+0000","last_fullsized":"2024-09-15T00:11:11.641483+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_clean_scrub_stamp":"2024-09-15T00:11:04.453672+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:01:23.254180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11008,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":662567,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":72218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":72218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":662567,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:11:16.544 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:11:16.820 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:11:16.820 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:11:16.832 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-15T00:11:16.294281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218040,"kb_used_data":1784,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756680,"statfs":{"total":773094113280,"available":772870840320,"internally_reserved":0,"allocated":1826816,"data_stored":1758366,"data_compressed":11008,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001091"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641700+0000","last_change":"2024-09-15T00:11:11.641988+0000","last_active":"2024-09-15T00:11:11.641700+0000","last_peered":"2024-09-15T00:11:11.641700+0000","last_clean":"2024-09-15T00:11:11.641700+0000","last_became_active":"2024-09-15T00:11:09.534794+0000","last_became_peered":"2024-09-15T00:11:09.534794+0000","last_unstale":"2024-09-15T00:11:11.641700+0000","last_undegraded":"2024-09-15T00:11:11.641700+0000","last_fullsized":"2024-09-15T00:11:11.641700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:55:52.135410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.545420+0000","last_change":"2024-09-15T00:11:11.545630+0000","last_active":"2024-09-15T00:11:11.545420+0000","last_peered":"2024-09-15T00:11:11.545420+0000","last_clean":"2024-09-15T00:11:11.545420+0000","last_became_active":"2024-09-15T00:11:09.536615+0000","last_became_peered":"2024-09-15T00:11:09.536615+0000","last_unstale":"2024-09-15T00:11:11.545420+0000","last_undegraded":"2024-09-15T00:11:11.545420+0000","last_fullsized":"2024-09-15T00:11:11.545420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:51:38.340257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035555,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641767+0000","last_change":"2024-09-15T00:11:11.642034+0000","last_active":"2024-09-15T00:11:11.641767+0000","last_peered":"2024-09-15T00:11:11.641767+0000","last_clean":"2024-09-15T00:11:11.641767+0000","last_became_active":"2024-09-15T00:11:09.824182+0000","last_became_peered":"2024-09-15T00:11:09.824182+0000","last_unstale":"2024-09-15T00:11:11.641767+0000","last_undegraded":"2024-09-15T00:11:11.641767+0000","last_fullsized":"2024-09-15T00:11:11.641767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:16.296783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.545471+0000","last_change":"2024-09-15T00:11:11.545675+0000","last_active":"2024-09-15T00:11:11.545471+0000","last_peered":"2024-09-15T00:11:11.545471+0000","last_clean":"2024-09-15T00:11:11.545471+0000","last_became_active":"2024-09-15T00:11:09.823838+0000","last_became_peered":"2024-09-15T00:11:09.823838+0000","last_unstale":"2024-09-15T00:11:11.545471+0000","last_undegraded":"2024-09-15T00:11:11.545471+0000","last_fullsized":"2024-09-15T00:11:11.545471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:24.994222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.543519+0000","last_change":"2024-09-15T00:11:11.543519+0000","last_active":"2024-09-15T00:11:11.543519+0000","last_peered":"2024-09-15T00:11:11.543519+0000","last_clean":"2024-09-15T00:11:11.543519+0000","last_became_active":"2024-09-15T00:11:09.533896+0000","last_became_peered":"2024-09-15T00:11:09.533896+0000","last_unstale":"2024-09-15T00:11:11.543519+0000","last_undegraded":"2024-09-15T00:11:11.543519+0000","last_fullsized":"2024-09-15T00:11:11.543519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:47.613931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.540934+0000","last_change":"2024-09-15T00:11:11.541173+0000","last_active":"2024-09-15T00:11:11.540934+0000","last_peered":"2024-09-15T00:11:11.540934+0000","last_clean":"2024-09-15T00:11:11.540934+0000","last_became_active":"2024-09-15T00:11:09.533270+0000","last_became_peered":"2024-09-15T00:11:09.533270+0000","last_unstale":"2024-09-15T00:11:11.540934+0000","last_undegraded":"2024-09-15T00:11:11.540934+0000","last_fullsized":"2024-09-15T00:11:11.540934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:42.665083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047411000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641732+0000","last_change":"2024-09-15T00:11:11.641995+0000","last_active":"2024-09-15T00:11:11.641732+0000","last_peered":"2024-09-15T00:11:11.641732+0000","last_clean":"2024-09-15T00:11:11.641732+0000","last_became_active":"2024-09-15T00:11:09.533404+0000","last_became_peered":"2024-09-15T00:11:09.533404+0000","last_unstale":"2024-09-15T00:11:11.641732+0000","last_undegraded":"2024-09-15T00:11:11.641732+0000","last_fullsized":"2024-09-15T00:11:11.641732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:20.611551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046024999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.542499+0000","last_change":"2024-09-15T00:11:11.542743+0000","last_active":"2024-09-15T00:11:11.542499+0000","last_peered":"2024-09-15T00:11:11.542499+0000","last_clean":"2024-09-15T00:11:11.542499+0000","last_became_active":"2024-09-15T00:11:09.536357+0000","last_became_peered":"2024-09-15T00:11:09.536357+0000","last_unstale":"2024-09-15T00:11:11.542499+0000","last_undegraded":"2024-09-15T00:11:11.542499+0000","last_fullsized":"2024-09-15T00:11:11.542499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:31:31.968858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T00:11:11.641483+0000","last_change":"2024-09-15T00:11:05.468585+0000","last_active":"2024-09-15T00:11:11.641483+0000","last_peered":"2024-09-15T00:11:11.641483+0000","last_clean":"2024-09-15T00:11:11.641483+0000","last_became_active":"2024-09-15T00:11:05.468352+0000","last_became_peered":"2024-09-15T00:11:05.468352+0000","last_unstale":"2024-09-15T00:11:11.641483+0000","last_undegraded":"2024-09-15T00:11:11.641483+0000","last_fullsized":"2024-09-15T00:11:11.641483+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_clean_scrub_stamp":"2024-09-15T00:11:04.453672+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:01:23.254180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11008,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":662567,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":72218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":72218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":662567,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:11:16.833 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T00:11:16.834 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-15T00:11:17.112 INFO:teuthology.orchestra.run.smithi171.stdout:epoch 22 2024-09-15T00:11:17.112 INFO:teuthology.orchestra.run.smithi171.stdout:fsid bad2533c-9c09-4af4-ab93-147afbacb705 2024-09-15T00:11:17.112 INFO:teuthology.orchestra.run.smithi171.stdout:created 2024-09-15T00:10:50.203030+0000 2024-09-15T00:11:17.112 INFO:teuthology.orchestra.run.smithi171.stdout:modified 2024-09-15T00:11:14.992005+0000 2024-09-15T00:11:17.112 INFO:teuthology.orchestra.run.smithi171.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-15T00:11:17.112 INFO:teuthology.orchestra.run.smithi171.stdout:crush_version 5 2024-09-15T00:11:17.113 INFO:teuthology.orchestra.run.smithi171.stdout:full_ratio 0.95 2024-09-15T00:11:17.113 INFO:teuthology.orchestra.run.smithi171.stdout:backfillfull_ratio 0.9 2024-09-15T00:11:17.113 INFO:teuthology.orchestra.run.smithi171.stdout:nearfull_ratio 0.85 2024-09-15T00:11:17.113 INFO:teuthology.orchestra.run.smithi171.stdout:require_min_compat_client luminous 2024-09-15T00:11:17.113 INFO:teuthology.orchestra.run.smithi171.stdout:min_compat_client jewel 2024-09-15T00:11:17.113 INFO:teuthology.orchestra.run.smithi171.stdout:require_osd_release squid 2024-09-15T00:11:17.113 INFO:teuthology.orchestra.run.smithi171.stdout:stretch_mode_enabled false 2024-09-15T00:11:17.113 INFO:teuthology.orchestra.run.smithi171.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-15T00:11:17.113 INFO:teuthology.orchestra.run.smithi171.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-15T00:11:17.113 INFO:teuthology.orchestra.run.smithi171.stdout:max_osd 8 2024-09-15T00:11:17.113 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6824/2906515165,v1:172.21.15.171:6825/2906515165] [v2:172.21.15.171:6826/2906515165,v1:172.21.15.171:6827/2906515165] exists,up 7337111f-0e02-490d-976b-3814a03371f0 2024-09-15T00:11:17.113 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6816/3039831960,v1:172.21.15.171:6817/3039831960] [v2:172.21.15.171:6818/3039831960,v1:172.21.15.171:6819/3039831960] exists,up 64412633-4dd4-4b8d-b3f1-ed3c664ddf48 2024-09-15T00:11:17.113 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6808/3950855930,v1:172.21.15.171:6809/3950855930] [v2:172.21.15.171:6810/3950855930,v1:172.21.15.171:6811/3950855930] exists,up 0300adee-5ce3-43ce-997b-2c0e970edbc7 2024-09-15T00:11:17.113 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/1819098683,v1:172.21.15.171:6801/1819098683] [v2:172.21.15.171:6802/1819098683,v1:172.21.15.171:6803/1819098683] exists,up ec668801-9b8f-401a-829c-0b1f8d8f08fb 2024-09-15T00:11:17.114 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6800/2801626173,v1:172.21.15.184:6801/2801626173] [v2:172.21.15.184:6802/2801626173,v1:172.21.15.184:6803/2801626173] exists,up 9cc48936-5644-44c6-84bb-06510e933e8d 2024-09-15T00:11:17.114 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6808/3271066302,v1:172.21.15.184:6809/3271066302] [v2:172.21.15.184:6810/3271066302,v1:172.21.15.184:6811/3271066302] exists,up f6c53297-9b00-476f-9be8-e76ff762b044 2024-09-15T00:11:17.114 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6816/1903994256,v1:172.21.15.184:6817/1903994256] [v2:172.21.15.184:6818/1903994256,v1:172.21.15.184:6819/1903994256] exists,up 737f08a6-2223-46b3-b9fe-6ae18b067ec3 2024-09-15T00:11:17.114 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6824/978565004,v1:172.21.15.184:6825/978565004] [v2:172.21.15.184:6826/978565004,v1:172.21.15.184:6827/978565004] exists,up 5a5fba64-9249-426d-814b-fd251e8aa910 2024-09-15T00:11:17.125 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6824/2906515165,v1:172.21.15.171:6825/2906515165] [v2:172.21.15.171:6826/2906515165,v1:172.21.15.171:6827/2906515165] exists,up 7337111f-0e02-490d-976b-3814a03371f0', 'osd.1 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6816/3039831960,v1:172.21.15.171:6817/3039831960] [v2:172.21.15.171:6818/3039831960,v1:172.21.15.171:6819/3039831960] exists,up 64412633-4dd4-4b8d-b3f1-ed3c664ddf48', 'osd.2 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6808/3950855930,v1:172.21.15.171:6809/3950855930] [v2:172.21.15.171:6810/3950855930,v1:172.21.15.171:6811/3950855930] exists,up 0300adee-5ce3-43ce-997b-2c0e970edbc7', 'osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/1819098683,v1:172.21.15.171:6801/1819098683] [v2:172.21.15.171:6802/1819098683,v1:172.21.15.171:6803/1819098683] exists,up ec668801-9b8f-401a-829c-0b1f8d8f08fb', 'osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6800/2801626173,v1:172.21.15.184:6801/2801626173] [v2:172.21.15.184:6802/2801626173,v1:172.21.15.184:6803/2801626173] exists,up 9cc48936-5644-44c6-84bb-06510e933e8d', 'osd.5 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6808/3271066302,v1:172.21.15.184:6809/3271066302] [v2:172.21.15.184:6810/3271066302,v1:172.21.15.184:6811/3271066302] exists,up f6c53297-9b00-476f-9be8-e76ff762b044', 'osd.6 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6816/1903994256,v1:172.21.15.184:6817/1903994256] [v2:172.21.15.184:6818/1903994256,v1:172.21.15.184:6819/1903994256] exists,up 737f08a6-2223-46b3-b9fe-6ae18b067ec3', 'osd.7 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6824/978565004,v1:172.21.15.184:6825/978565004] [v2:172.21.15.184:6826/978565004,v1:172.21.15.184:6827/978565004] exists,up 5a5fba64-9249-426d-814b-fd251e8aa910'] 2024-09-15T00:11:17.125 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-15T00:11:17.224 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: "name": "mon.a", 2024-09-15T00:11:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster": "ceph", 2024-09-15T00:11:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-15T00:11:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: "admin_socket_mode": "", 2024-09-15T00:11:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-15T00:11:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: "auth_client_required": "cephx, none", 2024-09-15T00:11:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: "auth_cluster_required": "cephx", 2024-09-15T00:11:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: "auth_debug": "true", 2024-09-15T00:11:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-15T00:11:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-15T00:11:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: "auth_service_required": "cephx", 2024-09-15T00:11:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-15T00:11:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: "auth_supported": "cephx", 2024-09-15T00:11:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_aio": "true", 2024-09-15T00:11:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-15T00:11:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_aio_poll_ms": "250", 2024-09-15T00:11:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_aio_reap_max": "16", 2024-09-15T00:11:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_async_discard": "false", 2024-09-15T00:11:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_block_size": "4096", 2024-09-15T00:11:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_debug_aio": "false", 2024-09-15T00:11:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-15T00:11:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-15T00:11:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_debug_inflight_ios": "false", 2024-09-15T00:11:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_enable_discard": "false", 2024-09-15T00:11:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_flock_retry": "3", 2024-09-15T00:11:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-15T00:11:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_inject_crash": "0", 2024-09-15T00:11:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-15T00:11:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_ioring": "false", 2024-09-15T00:11:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_ioring_hipri": "false", 2024-09-15T00:11:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-15T00:11:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-15T00:11:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-15T00:11:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-15T00:11:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-15T00:11:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-15T00:11:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: "bdev_type": "", 2024-09-15T00:11:17.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_alloc_size": "1048576", 2024-09-15T00:11:17.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_allocator": "hybrid", 2024-09-15T00:11:17.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_buffered_io": "true", 2024-09-15T00:11:17.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_check_for_zeros": "false", 2024-09-15T00:11:17.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-15T00:11:17.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-15T00:11:17.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_compact_log_sync": "false", 2024-09-15T00:11:17.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-15T00:11:17.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-15T00:11:17.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-15T00:11:17.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-15T00:11:17.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_max_log_runway": "4194304", 2024-09-15T00:11:17.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_max_prefetch": "1048576", 2024-09-15T00:11:17.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_min_flush_size": "524288", 2024-09-15T00:11:17.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_min_log_runway": "1048576", 2024-09-15T00:11:17.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_replay_recovery": "false", 2024-09-15T00:11:17.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-15T00:11:17.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-15T00:11:17.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_sync_write": "false", 2024-09-15T00:11:17.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-15T00:11:17.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-15T00:11:17.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-15T00:11:17.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_allocation_from_file": "true", 2024-09-15T00:11:17.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_allocator": "hybrid", 2024-09-15T00:11:17.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-15T00:11:17.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-15T00:11:17.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-15T00:11:17.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-15T00:11:17.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_bdev_label_multi": "true", 2024-09-15T00:11:17.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-15T00:11:17.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-15T00:11:17.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-15T00:11:17.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-15T00:11:17.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-15T00:11:17.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_block_create": "true", 2024-09-15T00:11:17.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_block_db_create": "false", 2024-09-15T00:11:17.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_block_db_path": "", 2024-09-15T00:11:17.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_block_db_size": "0", 2024-09-15T00:11:17.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_block_path": "", 2024-09-15T00:11:17.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_block_preallocate_file": "false", 2024-09-15T00:11:17.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_block_size": "107374182400", 2024-09-15T00:11:17.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_block_wal_create": "false", 2024-09-15T00:11:17.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_block_wal_path": "", 2024-09-15T00:11:17.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_block_wal_size": "100663296", 2024-09-15T00:11:17.230 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_bluefs": "true", 2024-09-15T00:11:17.230 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-15T00:11:17.230 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-15T00:11:17.230 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-15T00:11:17.230 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-15T00:11:17.230 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T00:11:17.230 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T00:11:17.230 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-15T00:11:17.230 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T00:11:17.230 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_autotune": "true", 2024-09-15T00:11:17.230 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-15T00:11:17.230 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-15T00:11:17.230 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-15T00:11:17.230 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-15T00:11:17.230 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_size": "0", 2024-09-15T00:11:17.231 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-15T00:11:17.231 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-15T00:11:17.231 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-15T00:11:17.231 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-15T00:11:17.231 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_type": "2q", 2024-09-15T00:11:17.231 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-15T00:11:17.231 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_clone_cow": "true", 2024-09-15T00:11:17.231 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-15T00:11:17.231 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-15T00:11:17.231 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-15T00:11:17.231 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-15T00:11:17.231 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-15T00:11:17.231 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-15T00:11:17.231 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-15T00:11:17.232 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_compression_mode": "none", 2024-09-15T00:11:17.232 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-15T00:11:17.232 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_csum_type": "crc32c", 2024-09-15T00:11:17.232 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-15T00:11:17.232 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_freelist": "false", 2024-09-15T00:11:17.232 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-15T00:11:17.232 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-15T00:11:17.232 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-15T00:11:17.232 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-15T00:11:17.232 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-15T00:11:17.232 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-15T00:11:17.232 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-15T00:11:17.232 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-15T00:11:17.232 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-15T00:11:17.232 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-15T00:11:17.233 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-15T00:11:17.233 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-15T00:11:17.233 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_small_allocations": "0", 2024-09-15T00:11:17.233 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-15T00:11:17.233 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_default_buffered_read": "true", 2024-09-15T00:11:17.233 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_default_buffered_write": "false", 2024-09-15T00:11:17.233 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-15T00:11:17.233 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-15T00:11:17.233 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-15T00:11:17.233 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-15T00:11:17.233 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-15T00:11:17.233 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-15T00:11:17.233 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-15T00:11:17.233 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-15T00:11:17.234 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-15T00:11:17.234 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fail_eio": "false", 2024-09-15T00:11:17.234 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-15T00:11:17.234 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-15T00:11:17.234 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-15T00:11:17.234 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-15T00:11:17.234 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-15T00:11:17.234 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-15T00:11:17.234 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_on_mount": "false", 2024-09-15T00:11:17.234 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-15T00:11:17.234 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_on_umount": "false", 2024-09-15T00:11:17.234 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-15T00:11:17.234 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-15T00:11:17.235 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-15T00:11:17.235 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-15T00:11:17.235 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-15T00:11:17.235 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-15T00:11:17.235 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-15T00:11:17.235 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-15T00:11:17.235 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_ignore_data_csum": "false", 2024-09-15T00:11:17.235 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-15T00:11:17.235 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-15T00:11:17.235 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-15T00:11:17.235 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-15T00:11:17.235 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_log_op_age": "5.000000", 2024-09-15T00:11:17.235 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_max_alloc_size": "0", 2024-09-15T00:11:17.235 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_max_blob_size": "0", 2024-09-15T00:11:17.236 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-15T00:11:17.236 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-15T00:11:17.236 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-15T00:11:17.236 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_max_deferred_txc": "32", 2024-09-15T00:11:17.236 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_min_alloc_size": "0", 2024-09-15T00:11:17.236 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-15T00:11:17.236 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-15T00:11:17.236 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_nid_prealloc": "1024", 2024-09-15T00:11:17.236 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-15T00:11:17.236 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-15T00:11:17.236 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-15T00:11:17.236 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-15T00:11:17.236 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_retry_disk_reads": "3", 2024-09-15T00:11:17.236 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_rocksdb_cf": "true", 2024-09-15T00:11:17.237 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-15T00:11:17.237 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-15T00:11:17.237 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-15T00:11:17.237 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-15T00:11:17.237 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-15T00:11:17.237 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-15T00:11:17.237 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-15T00:11:17.237 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-15T00:11:17.237 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_spdk_mem": "512", 2024-09-15T00:11:17.237 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-15T00:11:17.237 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-15T00:11:17.237 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-15T00:11:17.238 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-15T00:11:17.238 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-15T00:11:17.238 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-15T00:11:17.238 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-15T00:11:17.238 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_tracing": "false", 2024-09-15T00:11:17.238 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-15T00:11:17.238 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-15T00:11:17.238 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-15T00:11:17.238 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-15T00:11:17.238 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-15T00:11:17.238 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-15T00:11:17.238 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-15T00:11:17.238 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-15T00:11:17.238 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-15T00:11:17.239 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_zero_block_detection": "false", 2024-09-15T00:11:17.239 INFO:teuthology.orchestra.run.smithi171.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-15T00:11:17.239 INFO:teuthology.orchestra.run.smithi171.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-15T00:11:17.239 INFO:teuthology.orchestra.run.smithi171.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-15T00:11:17.239 INFO:teuthology.orchestra.run.smithi171.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-15T00:11:17.239 INFO:teuthology.orchestra.run.smithi171.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-15T00:11:17.239 INFO:teuthology.orchestra.run.smithi171.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-15T00:11:17.239 INFO:teuthology.orchestra.run.smithi171.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-15T00:11:17.239 INFO:teuthology.orchestra.run.smithi171.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-15T00:11:17.239 INFO:teuthology.orchestra.run.smithi171.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-15T00:11:17.239 INFO:teuthology.orchestra.run.smithi171.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-15T00:11:17.239 INFO:teuthology.orchestra.run.smithi171.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-15T00:11:17.239 INFO:teuthology.orchestra.run.smithi171.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-15T00:11:17.240 INFO:teuthology.orchestra.run.smithi171.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-15T00:11:17.240 INFO:teuthology.orchestra.run.smithi171.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-15T00:11:17.240 INFO:teuthology.orchestra.run.smithi171.stdout: "cephx_cluster_require_signatures": "false", 2024-09-15T00:11:17.240 INFO:teuthology.orchestra.run.smithi171.stdout: "cephx_cluster_require_version": "2", 2024-09-15T00:11:17.240 INFO:teuthology.orchestra.run.smithi171.stdout: "cephx_require_signatures": "false", 2024-09-15T00:11:17.240 INFO:teuthology.orchestra.run.smithi171.stdout: "cephx_require_version": "2", 2024-09-15T00:11:17.240 INFO:teuthology.orchestra.run.smithi171.stdout: "cephx_service_require_signatures": "false", 2024-09-15T00:11:17.240 INFO:teuthology.orchestra.run.smithi171.stdout: "cephx_service_require_version": "2", 2024-09-15T00:11:17.240 INFO:teuthology.orchestra.run.smithi171.stdout: "cephx_sign_messages": "true", 2024-09-15T00:11:17.240 INFO:teuthology.orchestra.run.smithi171.stdout: "chdir": "", 2024-09-15T00:11:17.240 INFO:teuthology.orchestra.run.smithi171.stdout: "client_acl_type": "", 2024-09-15T00:11:17.240 INFO:teuthology.orchestra.run.smithi171.stdout: "client_asio_thread_count": "2", 2024-09-15T00:11:17.240 INFO:teuthology.orchestra.run.smithi171.stdout: "client_cache_mid": "0.750000", 2024-09-15T00:11:17.240 INFO:teuthology.orchestra.run.smithi171.stdout: "client_cache_size": "16384", 2024-09-15T00:11:17.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_caps_release_delay": "5", 2024-09-15T00:11:17.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_check_pool_perm": "true", 2024-09-15T00:11:17.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-15T00:11:17.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_debug_force_sync_read": "false", 2024-09-15T00:11:17.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_debug_getattr_caps": "false", 2024-09-15T00:11:17.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_debug_inject_features": "", 2024-09-15T00:11:17.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_debug_inject_tick_delay": "0", 2024-09-15T00:11:17.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-15T00:11:17.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_die_on_failed_remount": "false", 2024-09-15T00:11:17.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_dirsize_rbytes": "true", 2024-09-15T00:11:17.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_force_lazyio": "false", 2024-09-15T00:11:17.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_fs": "", 2024-09-15T00:11:17.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-15T00:11:17.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_inject_release_failure": "false", 2024-09-15T00:11:17.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_max_inline_size": "4096", 2024-09-15T00:11:17.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-15T00:11:17.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_mds_namespace": "", 2024-09-15T00:11:17.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_metadata": "", 2024-09-15T00:11:17.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_mount_gid": "-1", 2024-09-15T00:11:17.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_mount_timeout": "300", 2024-09-15T00:11:17.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_mount_uid": "-1", 2024-09-15T00:11:17.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_mountpoint": "/", 2024-09-15T00:11:17.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_notify_timeout": "10", 2024-09-15T00:11:17.243 INFO:teuthology.orchestra.run.smithi171.stdout: "client_oc": "true", 2024-09-15T00:11:17.243 INFO:teuthology.orchestra.run.smithi171.stdout: "client_oc_max_dirty": "104857600", 2024-09-15T00:11:17.243 INFO:teuthology.orchestra.run.smithi171.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-15T00:11:17.243 INFO:teuthology.orchestra.run.smithi171.stdout: "client_oc_max_objects": "1000", 2024-09-15T00:11:17.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client_oc_size": "209715200", 2024-09-15T00:11:17.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client_oc_target_dirty": "8388608", 2024-09-15T00:11:17.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client_permissions": "true", 2024-09-15T00:11:17.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client_quota": "true", 2024-09-15T00:11:17.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client_quota_df": "true", 2024-09-15T00:11:17.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client_readahead_max_bytes": "0", 2024-09-15T00:11:17.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client_readahead_max_periods": "4", 2024-09-15T00:11:17.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client_readahead_min": "131072", 2024-09-15T00:11:17.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client_reconnect_stale": "false", 2024-09-15T00:11:17.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client_shutdown_timeout": "30", 2024-09-15T00:11:17.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client_snapdir": ".snap", 2024-09-15T00:11:17.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client_tick_interval": "1", 2024-09-15T00:11:17.244 INFO:teuthology.orchestra.run.smithi171.stdout: "client_trace": "", 2024-09-15T00:11:17.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client_try_dentry_invalidate": "false", 2024-09-15T00:11:17.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client_use_faked_inos": "false", 2024-09-15T00:11:17.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client_use_random_mds": "false", 2024-09-15T00:11:17.245 INFO:teuthology.orchestra.run.smithi171.stdout: "clog_to_graylog": "false", 2024-09-15T00:11:17.245 INFO:teuthology.orchestra.run.smithi171.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-15T00:11:17.245 INFO:teuthology.orchestra.run.smithi171.stdout: "clog_to_graylog_port": "12201", 2024-09-15T00:11:17.245 INFO:teuthology.orchestra.run.smithi171.stdout: "clog_to_monitors": "default=true", 2024-09-15T00:11:17.245 INFO:teuthology.orchestra.run.smithi171.stdout: "clog_to_syslog": "false", 2024-09-15T00:11:17.245 INFO:teuthology.orchestra.run.smithi171.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-15T00:11:17.245 INFO:teuthology.orchestra.run.smithi171.stdout: "clog_to_syslog_level": "info", 2024-09-15T00:11:17.245 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "-", 2024-09-15T00:11:17.245 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_network": "", 2024-09-15T00:11:17.245 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_network_interface": "", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "compressor_zlib_isal": "false", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "compressor_zlib_level": "5", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "compressor_zlib_winsize": "-15", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "compressor_zstd_level": "1", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "crimson_alien_op_num_threads": "6", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "crimson_osd_stat_interval": "0", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "crimson_seastar_cpu_cores": "", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "crimson_seastar_num_threads": "0", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_location": "", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_location_hook": "", 2024-09-15T00:11:17.246 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_location_hook_timeout": "10", 2024-09-15T00:11:17.247 INFO:teuthology.orchestra.run.smithi171.stdout: "daemonize": "false", 2024-09-15T00:11:17.247 INFO:teuthology.orchestra.run.smithi171.stdout: "daos_pool": "tank", 2024-09-15T00:11:17.247 INFO:teuthology.orchestra.run.smithi171.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-15T00:11:17.247 INFO:teuthology.orchestra.run.smithi171.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-15T00:11:17.247 INFO:teuthology.orchestra.run.smithi171.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-15T00:11:17.247 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_alienstore": "0/5", 2024-09-15T00:11:17.247 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_allow_any_pool_priority": "false", 2024-09-15T00:11:17.247 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_asok": "1/5", 2024-09-15T00:11:17.247 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_asok_assert_abort": "false", 2024-09-15T00:11:17.247 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_asserts_on_shutdown": "true", 2024-09-15T00:11:17.247 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_auth": "20/20", 2024-09-15T00:11:17.247 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_bdev": "1/3", 2024-09-15T00:11:17.247 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_bluefs": "1/5", 2024-09-15T00:11:17.247 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_bluestore": "1/5", 2024-09-15T00:11:17.247 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_buffer": "0/1", 2024-09-15T00:11:17.248 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_ceph_exporter": "1/5", 2024-09-15T00:11:17.248 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_cephfs_mirror": "0/5", 2024-09-15T00:11:17.248 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_cephsqlite": "0/5", 2024-09-15T00:11:17.248 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_client": "0/5", 2024-09-15T00:11:17.248 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_compressor": "1/5", 2024-09-15T00:11:17.248 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_context": "0/1", 2024-09-15T00:11:17.248 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_crimson_interrupt": "0/5", 2024-09-15T00:11:17.248 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_crush": "1/1", 2024-09-15T00:11:17.248 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_crypto": "1/5", 2024-09-15T00:11:17.248 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_cyanstore": "0/5", 2024-09-15T00:11:17.248 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_deliberately_leak_memory": "false", 2024-09-15T00:11:17.248 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_disable_randomized_ping": "false", 2024-09-15T00:11:17.248 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_dpdk": "1/5", 2024-09-15T00:11:17.249 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_eventtrace": "1/5", 2024-09-15T00:11:17.249 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_filer": "0/1", 2024-09-15T00:11:17.249 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_filestore": "1/3", 2024-09-15T00:11:17.249 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_finisher": "1/1", 2024-09-15T00:11:17.249 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_fuse": "1/5", 2024-09-15T00:11:17.249 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_heartbeat_testing_span": "0", 2024-09-15T00:11:17.249 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_heartbeatmap": "1/5", 2024-09-15T00:11:17.249 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-15T00:11:17.249 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_javaclient": "1/5", 2024-09-15T00:11:17.249 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_journal": "1/3", 2024-09-15T00:11:17.249 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_journaler": "0/5", 2024-09-15T00:11:17.249 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_kstore": "1/5", 2024-09-15T00:11:17.249 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_lockdep": "0/1", 2024-09-15T00:11:17.249 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_mclock": "1/5", 2024-09-15T00:11:17.249 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_mds": "1/5", 2024-09-15T00:11:17.250 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_mds_balancer": "1/5", 2024-09-15T00:11:17.250 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_mds_locker": "1/5", 2024-09-15T00:11:17.250 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_mds_log": "1/5", 2024-09-15T00:11:17.250 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_mds_log_expire": "1/5", 2024-09-15T00:11:17.250 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_mds_migrator": "1/5", 2024-09-15T00:11:17.250 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_mds_quiesce": "3/5", 2024-09-15T00:11:17.250 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_memstore": "1/5", 2024-09-15T00:11:17.250 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_mgr": "2/5", 2024-09-15T00:11:17.250 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_mgrc": "1/5", 2024-09-15T00:11:17.250 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_mon": "20/20", 2024-09-15T00:11:17.250 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_monc": "0/10", 2024-09-15T00:11:17.250 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_ms": "1/1", 2024-09-15T00:11:17.250 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_none": "0/5", 2024-09-15T00:11:17.250 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_objclass": "0/5", 2024-09-15T00:11:17.250 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_objectcacher": "0/5", 2024-09-15T00:11:17.251 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_objecter": "0/1", 2024-09-15T00:11:17.251 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_optracker": "0/5", 2024-09-15T00:11:17.251 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_osd": "1/5", 2024-09-15T00:11:17.251 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_paxos": "20/20", 2024-09-15T00:11:17.251 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_perfcounter": "1/5", 2024-09-15T00:11:17.251 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_prioritycache": "1/5", 2024-09-15T00:11:17.251 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_rados": "0/5", 2024-09-15T00:11:17.251 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_rbd": "0/5", 2024-09-15T00:11:17.251 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_rbd_mirror": "0/5", 2024-09-15T00:11:17.251 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_rbd_pwl": "0/5", 2024-09-15T00:11:17.251 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_rbd_replay": "0/5", 2024-09-15T00:11:17.251 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_refs": "0/0", 2024-09-15T00:11:17.251 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_reserver": "1/1", 2024-09-15T00:11:17.251 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_rgw": "1/5", 2024-09-15T00:11:17.251 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_rgw_access": "1/5", 2024-09-15T00:11:17.252 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_rgw_datacache": "1/5", 2024-09-15T00:11:17.252 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_rgw_dbstore": "1/5", 2024-09-15T00:11:17.252 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_rgw_flight": "1/5", 2024-09-15T00:11:17.252 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-15T00:11:17.252 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_rgw_sync": "1/5", 2024-09-15T00:11:17.252 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_rocksdb": "4/5", 2024-09-15T00:11:17.252 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_seastore": "0/5", 2024-09-15T00:11:17.252 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_seastore_backref": "0/5", 2024-09-15T00:11:17.252 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_seastore_cache": "0/5", 2024-09-15T00:11:17.252 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_seastore_cleaner": "0/5", 2024-09-15T00:11:17.252 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_seastore_device": "0/5", 2024-09-15T00:11:17.252 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_seastore_epm": "0/5", 2024-09-15T00:11:17.252 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-15T00:11:17.253 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_seastore_journal": "0/5", 2024-09-15T00:11:17.253 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_seastore_lba": "0/5", 2024-09-15T00:11:17.253 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_seastore_odata": "0/5", 2024-09-15T00:11:17.253 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_seastore_omap": "0/5", 2024-09-15T00:11:17.253 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_seastore_onode": "0/5", 2024-09-15T00:11:17.253 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_seastore_t": "0/5", 2024-09-15T00:11:17.253 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_seastore_tm": "0/5", 2024-09-15T00:11:17.253 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_striper": "0/1", 2024-09-15T00:11:17.253 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_test": "0/5", 2024-09-15T00:11:17.253 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_throttle": "1/1", 2024-09-15T00:11:17.253 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_timer": "0/1", 2024-09-15T00:11:17.253 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_tp": "0/5", 2024-09-15T00:11:17.253 INFO:teuthology.orchestra.run.smithi171.stdout: "debug_trace": "1/5", 2024-09-15T00:11:17.253 INFO:teuthology.orchestra.run.smithi171.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-15T00:11:17.254 INFO:teuthology.orchestra.run.smithi171.stdout: "device_failure_prediction_mode": "none", 2024-09-15T00:11:17.254 INFO:teuthology.orchestra.run.smithi171.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-15T00:11:17.254 INFO:teuthology.orchestra.run.smithi171.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-15T00:11:17.254 INFO:teuthology.orchestra.run.smithi171.stdout: "err_to_graylog": "false", 2024-09-15T00:11:17.254 INFO:teuthology.orchestra.run.smithi171.stdout: "err_to_journald": "false", 2024-09-15T00:11:17.254 INFO:teuthology.orchestra.run.smithi171.stdout: "err_to_stderr": "true", 2024-09-15T00:11:17.254 INFO:teuthology.orchestra.run.smithi171.stdout: "err_to_syslog": "false", 2024-09-15T00:11:17.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event_tracing": "false", 2024-09-15T00:11:17.254 INFO:teuthology.orchestra.run.smithi171.stdout: "exporter_addr": "0.0.0.0", 2024-09-15T00:11:17.254 INFO:teuthology.orchestra.run.smithi171.stdout: "exporter_http_port": "9926", 2024-09-15T00:11:17.255 INFO:teuthology.orchestra.run.smithi171.stdout: "exporter_prio_limit": "5", 2024-09-15T00:11:17.255 INFO:teuthology.orchestra.run.smithi171.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-15T00:11:17.255 INFO:teuthology.orchestra.run.smithi171.stdout: "exporter_sort_metrics": "true", 2024-09-15T00:11:17.255 INFO:teuthology.orchestra.run.smithi171.stdout: "exporter_stats_period": "5", 2024-09-15T00:11:17.255 INFO:teuthology.orchestra.run.smithi171.stdout: "fake_statfs_for_testing": "0", 2024-09-15T00:11:17.255 INFO:teuthology.orchestra.run.smithi171.stdout: "fatal_signal_handlers": "true", 2024-09-15T00:11:17.255 INFO:teuthology.orchestra.run.smithi171.stdout: "filer_max_purge_ops": "10", 2024-09-15T00:11:17.255 INFO:teuthology.orchestra.run.smithi171.stdout: "filer_max_truncate_ops": "128", 2024-09-15T00:11:17.255 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_apply_finisher_threads": "1", 2024-09-15T00:11:17.255 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_blackhole": "false", 2024-09-15T00:11:17.255 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_btrfs_clone_range": "true", 2024-09-15T00:11:17.256 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_btrfs_snap": "true", 2024-09-15T00:11:17.256 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_caller_concurrency": "10", 2024-09-15T00:11:17.256 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_collect_device_partition_information": "true", 2024-09-15T00:11:17.256 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_commit_timeout": "600.000000", 2024-09-15T00:11:17.256 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_debug_inject_read_err": "false", 2024-09-15T00:11:17.256 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_debug_omap_check": "false", 2024-09-15T00:11:17.256 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-15T00:11:17.256 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_debug_verify_split": "false", 2024-09-15T00:11:17.256 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_dump_file": "", 2024-09-15T00:11:17.256 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-15T00:11:17.256 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-15T00:11:17.257 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_fadvise": "true", 2024-09-15T00:11:17.257 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_fail_eio": "true", 2024-09-15T00:11:17.257 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_fd_cache_shards": "16", 2024-09-15T00:11:17.257 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_fd_cache_size": "128", 2024-09-15T00:11:17.257 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_fiemap": "false", 2024-09-15T00:11:17.257 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_fiemap_threshold": "4096", 2024-09-15T00:11:17.257 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-15T00:11:17.257 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-15T00:11:17.257 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_inject_stall": "0", 2024-09-15T00:11:17.257 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_journal_parallel": "false", 2024-09-15T00:11:17.257 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_journal_trailing": "false", 2024-09-15T00:11:17.257 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_journal_writeahead": "false", 2024-09-15T00:11:17.257 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_kill_at": "0", 2024-09-15T00:11:17.258 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-15T00:11:17.258 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-15T00:11:17.258 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-15T00:11:17.258 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-15T00:11:17.258 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-15T00:11:17.258 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_max_inline_xattrs": "0", 2024-09-15T00:11:17.258 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-15T00:11:17.258 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-15T00:11:17.258 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-15T00:11:17.258 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-15T00:11:17.258 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_max_xattr_value_size": "0", 2024-09-15T00:11:17.258 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-15T00:11:17.258 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-15T00:11:17.259 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-15T00:11:17.259 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_merge_threshold": "-10", 2024-09-15T00:11:17.259 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-15T00:11:17.259 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_odsync_write": "false", 2024-09-15T00:11:17.259 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_omap_backend": "rocksdb", 2024-09-15T00:11:17.259 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_omap_backend_path": "", 2024-09-15T00:11:17.259 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-15T00:11:17.259 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-15T00:11:17.259 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-15T00:11:17.259 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_op_thread_timeout": "60", 2024-09-15T00:11:17.259 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_op_threads": "2", 2024-09-15T00:11:17.259 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_punch_hole": "false", 2024-09-15T00:11:17.259 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-15T00:11:17.259 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-15T00:11:17.260 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-15T00:11:17.260 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-15T00:11:17.260 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-15T00:11:17.260 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-15T00:11:17.260 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-15T00:11:17.260 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-15T00:11:17.260 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-15T00:11:17.260 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_queue_max_ops": "50", 2024-09-15T00:11:17.260 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-15T00:11:17.260 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_seek_data_hole": "false", 2024-09-15T00:11:17.260 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_sloppy_crc": "false", 2024-09-15T00:11:17.260 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-15T00:11:17.260 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_splice": "false", 2024-09-15T00:11:17.261 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_split_multiple": "2", 2024-09-15T00:11:17.261 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_split_rand_factor": "20", 2024-09-15T00:11:17.261 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_update_to": "1000", 2024-09-15T00:11:17.261 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-15T00:11:17.261 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-15T00:11:17.261 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-15T00:11:17.261 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-15T00:11:17.261 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-15T00:11:17.261 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-15T00:11:17.261 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_wbthrottle_enable": "true", 2024-09-15T00:11:17.261 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-15T00:11:17.261 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-15T00:11:17.261 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-15T00:11:17.262 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-15T00:11:17.262 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-15T00:11:17.262 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-15T00:11:17.262 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_xfs_extsize": "false", 2024-09-15T00:11:17.262 INFO:teuthology.orchestra.run.smithi171.stdout: "filestore_zfs_snap": "false", 2024-09-15T00:11:17.262 INFO:teuthology.orchestra.run.smithi171.stdout: "fio_dir": "/tmp/fio", 2024-09-15T00:11:17.262 INFO:teuthology.orchestra.run.smithi171.stdout: "fsid": "bad2533c-9c09-4af4-ab93-147afbacb705", 2024-09-15T00:11:17.262 INFO:teuthology.orchestra.run.smithi171.stdout: "fuse_allow_other": "true", 2024-09-15T00:11:17.262 INFO:teuthology.orchestra.run.smithi171.stdout: "fuse_atomic_o_trunc": "true", 2024-09-15T00:11:17.262 INFO:teuthology.orchestra.run.smithi171.stdout: "fuse_big_writes": "true", 2024-09-15T00:11:17.262 INFO:teuthology.orchestra.run.smithi171.stdout: "fuse_debug": "false", 2024-09-15T00:11:17.262 INFO:teuthology.orchestra.run.smithi171.stdout: "fuse_default_permissions": "false", 2024-09-15T00:11:17.262 INFO:teuthology.orchestra.run.smithi171.stdout: "fuse_disable_pagecache": "false", 2024-09-15T00:11:17.262 INFO:teuthology.orchestra.run.smithi171.stdout: "fuse_max_write": "0", 2024-09-15T00:11:17.263 INFO:teuthology.orchestra.run.smithi171.stdout: "fuse_multithreaded": "true", 2024-09-15T00:11:17.263 INFO:teuthology.orchestra.run.smithi171.stdout: "fuse_require_active_mds": "true", 2024-09-15T00:11:17.263 INFO:teuthology.orchestra.run.smithi171.stdout: "fuse_set_user_groups": "true", 2024-09-15T00:11:17.263 INFO:teuthology.orchestra.run.smithi171.stdout: "fuse_splice_move": "true", 2024-09-15T00:11:17.263 INFO:teuthology.orchestra.run.smithi171.stdout: "fuse_splice_read": "true", 2024-09-15T00:11:17.263 INFO:teuthology.orchestra.run.smithi171.stdout: "fuse_splice_write": "true", 2024-09-15T00:11:17.263 INFO:teuthology.orchestra.run.smithi171.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-15T00:11:17.263 INFO:teuthology.orchestra.run.smithi171.stdout: "fuse_use_invalidate_cb": "true", 2024-09-15T00:11:17.263 INFO:teuthology.orchestra.run.smithi171.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-15T00:11:17.263 INFO:teuthology.orchestra.run.smithi171.stdout: "gss_target_name": "ceph", 2024-09-15T00:11:17.263 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_file": "", 2024-09-15T00:11:17.263 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_inject_failure": "0", 2024-09-15T00:11:17.263 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_interval": "5", 2024-09-15T00:11:17.263 INFO:teuthology.orchestra.run.smithi171.stdout: "host": "smithi171", 2024-09-15T00:11:17.264 INFO:teuthology.orchestra.run.smithi171.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-15T00:11:17.264 INFO:teuthology.orchestra.run.smithi171.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-15T00:11:17.264 INFO:teuthology.orchestra.run.smithi171.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-15T00:11:17.264 INFO:teuthology.orchestra.run.smithi171.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-15T00:11:17.264 INFO:teuthology.orchestra.run.smithi171.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-15T00:11:17.264 INFO:teuthology.orchestra.run.smithi171.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-15T00:11:17.264 INFO:teuthology.orchestra.run.smithi171.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-15T00:11:17.264 INFO:teuthology.orchestra.run.smithi171.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-15T00:11:17.264 INFO:teuthology.orchestra.run.smithi171.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-15T00:11:17.264 INFO:teuthology.orchestra.run.smithi171.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-15T00:11:17.264 INFO:teuthology.orchestra.run.smithi171.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-15T00:11:17.264 INFO:teuthology.orchestra.run.smithi171.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-15T00:11:17.265 INFO:teuthology.orchestra.run.smithi171.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-15T00:11:17.265 INFO:teuthology.orchestra.run.smithi171.stdout: "inject_early_sigterm": "false", 2024-09-15T00:11:17.265 INFO:teuthology.orchestra.run.smithi171.stdout: "jaeger_agent_port": "6799", 2024-09-15T00:11:17.265 INFO:teuthology.orchestra.run.smithi171.stdout: "jaeger_tracing_enable": "false", 2024-09-15T00:11:17.265 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_aio": "true", 2024-09-15T00:11:17.265 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_align_min_size": "65536", 2024-09-15T00:11:17.265 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_block_align": "true", 2024-09-15T00:11:17.265 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_block_size": "4096", 2024-09-15T00:11:17.265 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_dio": "true", 2024-09-15T00:11:17.265 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_discard": "false", 2024-09-15T00:11:17.265 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_force_aio": "false", 2024-09-15T00:11:17.265 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_ignore_corruption": "false", 2024-09-15T00:11:17.266 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_max_write_bytes": "10485760", 2024-09-15T00:11:17.266 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_max_write_entries": "100", 2024-09-15T00:11:17.266 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_replay_from": "0", 2024-09-15T00:11:17.266 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-15T00:11:17.266 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-15T00:11:17.266 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-15T00:11:17.266 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-15T00:11:17.266 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_write_header_frequency": "0", 2024-09-15T00:11:17.266 INFO:teuthology.orchestra.run.smithi171.stdout: "journal_zero_on_create": "false", 2024-09-15T00:11:17.266 INFO:teuthology.orchestra.run.smithi171.stdout: "journaler_prefetch_periods": "10", 2024-09-15T00:11:17.266 INFO:teuthology.orchestra.run.smithi171.stdout: "journaler_prezero_periods": "5", 2024-09-15T00:11:17.266 INFO:teuthology.orchestra.run.smithi171.stdout: "journaler_write_head_interval": "15", 2024-09-15T00:11:17.266 INFO:teuthology.orchestra.run.smithi171.stdout: "key": "", 2024-09-15T00:11:17.267 INFO:teuthology.orchestra.run.smithi171.stdout: "keyfile": "", 2024-09-15T00:11:17.267 INFO:teuthology.orchestra.run.smithi171.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-15T00:11:17.267 INFO:teuthology.orchestra.run.smithi171.stdout: "kstore_backend": "rocksdb", 2024-09-15T00:11:17.267 INFO:teuthology.orchestra.run.smithi171.stdout: "kstore_default_stripe_size": "65536", 2024-09-15T00:11:17.267 INFO:teuthology.orchestra.run.smithi171.stdout: "kstore_fsck_on_mount": "false", 2024-09-15T00:11:17.267 INFO:teuthology.orchestra.run.smithi171.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-15T00:11:17.267 INFO:teuthology.orchestra.run.smithi171.stdout: "kstore_max_bytes": "67108864", 2024-09-15T00:11:17.267 INFO:teuthology.orchestra.run.smithi171.stdout: "kstore_max_ops": "512", 2024-09-15T00:11:17.267 INFO:teuthology.orchestra.run.smithi171.stdout: "kstore_nid_prealloc": "1024", 2024-09-15T00:11:17.267 INFO:teuthology.orchestra.run.smithi171.stdout: "kstore_onode_map_size": "1024", 2024-09-15T00:11:17.267 INFO:teuthology.orchestra.run.smithi171.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-15T00:11:17.267 INFO:teuthology.orchestra.run.smithi171.stdout: "kstore_sync_submit_transaction": "false", 2024-09-15T00:11:17.267 INFO:teuthology.orchestra.run.smithi171.stdout: "kstore_sync_transaction": "false", 2024-09-15T00:11:17.268 INFO:teuthology.orchestra.run.smithi171.stdout: "librados_thread_count": "2", 2024-09-15T00:11:17.268 INFO:teuthology.orchestra.run.smithi171.stdout: "lockdep": "false", 2024-09-15T00:11:17.268 INFO:teuthology.orchestra.run.smithi171.stdout: "lockdep_force_backtrace": "false", 2024-09-15T00:11:17.268 INFO:teuthology.orchestra.run.smithi171.stdout: "log_coarse_timestamps": "true", 2024-09-15T00:11:17.268 INFO:teuthology.orchestra.run.smithi171.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-15T00:11:17.268 INFO:teuthology.orchestra.run.smithi171.stdout: "log_flush_on_exit": "false", 2024-09-15T00:11:17.268 INFO:teuthology.orchestra.run.smithi171.stdout: "log_graylog_host": "127.0.0.1", 2024-09-15T00:11:17.268 INFO:teuthology.orchestra.run.smithi171.stdout: "log_graylog_port": "12201", 2024-09-15T00:11:17.268 INFO:teuthology.orchestra.run.smithi171.stdout: "log_max_new": "1000", 2024-09-15T00:11:17.268 INFO:teuthology.orchestra.run.smithi171.stdout: "log_max_recent": "10000", 2024-09-15T00:11:17.268 INFO:teuthology.orchestra.run.smithi171.stdout: "log_stderr_prefix": "", 2024-09-15T00:11:17.269 INFO:teuthology.orchestra.run.smithi171.stdout: "log_stop_at_utilization": "0.970000", 2024-09-15T00:11:17.269 INFO:teuthology.orchestra.run.smithi171.stdout: "log_to_file": "true", 2024-09-15T00:11:17.269 INFO:teuthology.orchestra.run.smithi171.stdout: "log_to_graylog": "false", 2024-09-15T00:11:17.269 INFO:teuthology.orchestra.run.smithi171.stdout: "log_to_journald": "false", 2024-09-15T00:11:17.269 INFO:teuthology.orchestra.run.smithi171.stdout: "log_to_stderr": "false", 2024-09-15T00:11:17.269 INFO:teuthology.orchestra.run.smithi171.stdout: "log_to_syslog": "false", 2024-09-15T00:11:17.269 INFO:teuthology.orchestra.run.smithi171.stdout: "max_rotating_auth_attempts": "10", 2024-09-15T00:11:17.269 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-15T00:11:17.269 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_action_on_write_error": "1", 2024-09-15T00:11:17.269 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_alternate_name_max": "8192", 2024-09-15T00:11:17.269 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_asio_thread_count": "2", 2024-09-15T00:11:17.269 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_export_pin": "true", 2024-09-15T00:11:17.269 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_fragment_dirs": "true", 2024-09-15T00:11:17.269 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-15T00:11:17.270 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_fragment_interval": "5", 2024-09-15T00:11:17.270 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-15T00:11:17.270 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-15T00:11:17.270 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_interval": "10", 2024-09-15T00:11:17.270 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_max": "-1", 2024-09-15T00:11:17.270 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_max_until": "-1", 2024-09-15T00:11:17.270 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_merge_size": "50", 2024-09-15T00:11:17.270 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_midchunk": "0.300000", 2024-09-15T00:11:17.270 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-15T00:11:17.270 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_min_start": "0.200000", 2024-09-15T00:11:17.270 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_minchunk": "0.001000", 2024-09-15T00:11:17.270 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_mode": "0", 2024-09-15T00:11:17.270 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_need_max": "1.200000", 2024-09-15T00:11:17.270 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_need_min": "0.800000", 2024-09-15T00:11:17.271 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_overload_epochs": "2", 2024-09-15T00:11:17.271 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-15T00:11:17.271 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-15T00:11:17.271 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_split_bits": "3", 2024-09-15T00:11:17.271 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-15T00:11:17.271 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_split_size": "10000", 2024-09-15T00:11:17.271 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-15T00:11:17.271 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_target_decay": "10.000000", 2024-09-15T00:11:17.271 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-15T00:11:17.271 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_beacon_grace": "15.000000", 2024-09-15T00:11:17.271 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_beacon_interval": "4.000000", 2024-09-15T00:11:17.271 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-15T00:11:17.271 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-15T00:11:17.271 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache_mid": "0.700000", 2024-09-15T00:11:17.272 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-15T00:11:17.272 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache_quiesce_delay": "0", 2024-09-15T00:11:17.272 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-15T00:11:17.272 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-15T00:11:17.272 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-15T00:11:17.272 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache_release_free_interval": "10", 2024-09-15T00:11:17.272 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache_reservation": "0.050000", 2024-09-15T00:11:17.272 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-15T00:11:17.272 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache_trim_interval": "1", 2024-09-15T00:11:17.272 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache_trim_threshold": "262144", 2024-09-15T00:11:17.272 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-15T00:11:17.272 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-15T00:11:17.273 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-15T00:11:17.273 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_client_prealloc_inos": "1000", 2024-09-15T00:11:17.273 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-15T00:11:17.273 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_connect_bootstrapping": "false", 2024-09-15T00:11:17.273 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_damage_table_max_entries": "10000", 2024-09-15T00:11:17.273 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-15T00:11:17.273 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_debug_auth_pins": "false", 2024-09-15T00:11:17.273 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_debug_frag": "false", 2024-09-15T00:11:17.273 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_debug_scatterstat": "false", 2024-09-15T00:11:17.273 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_debug_subtrees": "false", 2024-09-15T00:11:17.273 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_decay_halflife": "5.000000", 2024-09-15T00:11:17.273 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_default_dir_hash": "2", 2024-09-15T00:11:17.273 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_defer_session_stale": "true", 2024-09-15T00:11:17.273 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_deny_all_reconnect": "false", 2024-09-15T00:11:17.274 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_dir_keys_per_op": "16384", 2024-09-15T00:11:17.274 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_dir_max_commit_size": "10", 2024-09-15T00:11:17.274 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_dir_max_entries": "0", 2024-09-15T00:11:17.274 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_dir_prefetch": "true", 2024-09-15T00:11:17.274 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-15T00:11:17.274 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-15T00:11:17.274 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_dump_cache_on_map": "false", 2024-09-15T00:11:17.274 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-15T00:11:17.274 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-15T00:11:17.274 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_early_reply": "true", 2024-09-15T00:11:17.274 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_enable_op_tracker": "true", 2024-09-15T00:11:17.274 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_enforce_unique_name": "true", 2024-09-15T00:11:17.275 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-15T00:11:17.275 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-15T00:11:17.275 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_export_ephemeral_random": "true", 2024-09-15T00:11:17.275 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-15T00:11:17.275 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-15T00:11:17.275 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-15T00:11:17.275 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-15T00:11:17.275 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-15T00:11:17.275 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-15T00:11:17.275 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-15T00:11:17.275 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-15T00:11:17.275 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_health_summarize_threshold": "10", 2024-09-15T00:11:17.275 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-15T00:11:17.275 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-15T00:11:17.275 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_inject_health_dummy": "false", 2024-09-15T00:11:17.276 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-15T00:11:17.276 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_inject_migrator_session_race": "false", 2024-09-15T00:11:17.276 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-15T00:11:17.276 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-15T00:11:17.276 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-15T00:11:17.276 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_join_fs": "", 2024-09-15T00:11:17.276 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_journal_format": "1", 2024-09-15T00:11:17.276 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-15T00:11:17.276 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_kill_create_at": "0", 2024-09-15T00:11:17.276 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_kill_export_at": "0", 2024-09-15T00:11:17.276 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_kill_import_at": "0", 2024-09-15T00:11:17.276 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_kill_journal_at": "0", 2024-09-15T00:11:17.277 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_kill_journal_expire_at": "0", 2024-09-15T00:11:17.277 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_kill_journal_replay_at": "0", 2024-09-15T00:11:17.277 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_kill_link_at": "0", 2024-09-15T00:11:17.277 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_kill_mdstable_at": "0", 2024-09-15T00:11:17.277 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_kill_openc_at": "0", 2024-09-15T00:11:17.277 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_kill_rename_at": "0", 2024-09-15T00:11:17.277 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_kill_shutdown_at": "0", 2024-09-15T00:11:17.277 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log_event_large_threshold": "524288", 2024-09-15T00:11:17.277 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log_events_per_segment": "1024", 2024-09-15T00:11:17.277 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-15T00:11:17.277 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log_max_events": "-1", 2024-09-15T00:11:17.277 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log_max_segments": "128", 2024-09-15T00:11:17.277 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log_pause": "false", 2024-09-15T00:11:17.277 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log_segment_size": "0", 2024-09-15T00:11:17.278 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-15T00:11:17.278 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-15T00:11:17.278 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-15T00:11:17.278 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log_trim_threshold": "128", 2024-09-15T00:11:17.278 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-15T00:11:17.278 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log_warn_factor": "2.000000", 2024-09-15T00:11:17.278 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_max_caps_per_client": "1048576", 2024-09-15T00:11:17.278 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_max_completed_flushes": "100000", 2024-09-15T00:11:17.278 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_max_completed_requests": "100000", 2024-09-15T00:11:17.278 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_max_export_size": "20971520", 2024-09-15T00:11:17.278 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_max_file_recover": "32", 2024-09-15T00:11:17.278 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_max_purge_files": "64", 2024-09-15T00:11:17.278 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_max_purge_ops": "8192", 2024-09-15T00:11:17.278 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-15T00:11:17.279 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-15T00:11:17.279 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_max_snaps_per_dir": "100", 2024-09-15T00:11:17.279 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_metrics_update_interval": "2", 2024-09-15T00:11:17.279 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_min_caps_per_client": "100", 2024-09-15T00:11:17.279 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_min_caps_working_set": "10000", 2024-09-15T00:11:17.279 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-15T00:11:17.279 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_numa_node": "-1", 2024-09-15T00:11:17.279 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-15T00:11:17.279 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_op_complaint_time": "30.000000", 2024-09-15T00:11:17.279 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_op_history_duration": "600", 2024-09-15T00:11:17.279 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_op_history_size": "20", 2024-09-15T00:11:17.279 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_op_history_slow_op_size": "20", 2024-09-15T00:11:17.279 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-15T00:11:17.279 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_op_log_threshold": "5", 2024-09-15T00:11:17.280 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_ping_grace": "15", 2024-09-15T00:11:17.280 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_ping_interval": "5", 2024-09-15T00:11:17.280 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-15T00:11:17.280 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-15T00:11:17.280 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_recall_max_caps": "30000", 2024-09-15T00:11:17.280 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-15T00:11:17.280 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-15T00:11:17.280 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-15T00:11:17.280 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_recall_warning_threshold": "262144", 2024-09-15T00:11:17.280 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-15T00:11:17.280 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_replay_interval": "1.000000", 2024-09-15T00:11:17.280 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-15T00:11:17.280 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-15T00:11:17.280 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_root_ino_gid": "0", 2024-09-15T00:11:17.281 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_root_ino_uid": "0", 2024-09-15T00:11:17.281 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-15T00:11:17.281 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-15T00:11:17.281 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-15T00:11:17.281 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-15T00:11:17.281 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-15T00:11:17.281 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-15T00:11:17.281 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-15T00:11:17.281 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-15T00:11:17.281 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-15T00:11:17.281 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-15T00:11:17.281 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_shutdown_check": "0", 2024-09-15T00:11:17.281 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_skip_ino": "0", 2024-09-15T00:11:17.282 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-15T00:11:17.282 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_snap_max_uid": "4294967294", 2024-09-15T00:11:17.282 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_snap_min_uid": "0", 2024-09-15T00:11:17.282 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_snap_rstat": "false", 2024-09-15T00:11:17.282 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_standby_replay_damaged": "false", 2024-09-15T00:11:17.282 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_symlink_recovery": "true", 2024-09-15T00:11:17.282 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-15T00:11:17.282 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_thrash_exports": "0", 2024-09-15T00:11:17.282 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_thrash_fragments": "0", 2024-09-15T00:11:17.282 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_tick_interval": "5.000000", 2024-09-15T00:11:17.282 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_valgrind_exit": "false", 2024-09-15T00:11:17.282 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_verify_backtrace": "1", 2024-09-15T00:11:17.282 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_verify_scatter": "false", 2024-09-15T00:11:17.283 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-15T00:11:17.283 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_wipe_sessions": "false", 2024-09-15T00:11:17.283 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool_debug": "false", 2024-09-15T00:11:17.283 INFO:teuthology.orchestra.run.smithi171.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-15T00:11:17.283 INFO:teuthology.orchestra.run.smithi171.stdout: "memstore_device_bytes": "1073741824", 2024-09-15T00:11:17.283 INFO:teuthology.orchestra.run.smithi171.stdout: "memstore_page_set": "false", 2024-09-15T00:11:17.283 INFO:teuthology.orchestra.run.smithi171.stdout: "memstore_page_size": "65536", 2024-09-15T00:11:17.283 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_client_bytes": "134217728", 2024-09-15T00:11:17.283 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_client_messages": "512", 2024-09-15T00:11:17.283 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-15T00:11:17.283 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-15T00:11:17.283 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-15T00:11:17.283 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-15T00:11:17.284 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_disabled_modules": "", 2024-09-15T00:11:17.284 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_enable_op_tracker": "true", 2024-09-15T00:11:17.284 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-15T00:11:17.284 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_max_pg_num_change": "128", 2024-09-15T00:11:17.284 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_mds_bytes": "134217728", 2024-09-15T00:11:17.284 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_mds_messages": "128", 2024-09-15T00:11:17.284 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-15T00:11:17.284 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_mon_bytes": "134217728", 2024-09-15T00:11:17.284 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_mon_messages": "128", 2024-09-15T00:11:17.284 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-15T00:11:17.284 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-15T00:11:17.284 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_op_history_duration": "600", 2024-09-15T00:11:17.284 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_op_history_size": "20", 2024-09-15T00:11:17.284 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-15T00:11:17.285 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-15T00:11:17.285 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_op_log_threshold": "5", 2024-09-15T00:11:17.285 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_osd_bytes": "536870912", 2024-09-15T00:11:17.285 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_osd_messages": "8192", 2024-09-15T00:11:17.285 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_pool": "true", 2024-09-15T00:11:17.285 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-15T00:11:17.285 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_standby_modules": "true", 2024-09-15T00:11:17.285 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_stats_period": "5", 2024-09-15T00:11:17.285 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_stats_threshold": "5", 2024-09-15T00:11:17.285 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_tick_period": "2", 2024-09-15T00:11:17.285 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-15T00:11:17.285 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-15T00:11:17.285 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_allow_pool_delete": "true", 2024-09-15T00:11:17.285 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_allow_pool_size_one": "true", 2024-09-15T00:11:17.286 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_auth_validate_all_caps": "true", 2024-09-15T00:11:17.286 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-15T00:11:17.286 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-15T00:11:17.286 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_client_bytes": "104857600", 2024-09-15T00:11:17.286 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_client_directed_command_retry": "5", 2024-09-15T00:11:17.286 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-15T00:11:17.286 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-15T00:11:17.286 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-15T00:11:17.286 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-15T00:11:17.286 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_client_hunt_parallel": "3", 2024-09-15T00:11:17.286 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_client_log_interval": "1.000000", 2024-09-15T00:11:17.286 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-15T00:11:17.286 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_client_ping_interval": "10.000000", 2024-09-15T00:11:17.286 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-15T00:11:17.287 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_client_target_rank": "-1", 2024-09-15T00:11:17.287 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-15T00:11:17.287 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-15T00:11:17.287 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-15T00:11:17.287 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_cluster_log_file_level": "debug", 2024-09-15T00:11:17.287 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_cluster_log_to_file": "true", 2024-09-15T00:11:17.287 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-15T00:11:17.287 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-15T00:11:17.287 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-15T00:11:17.287 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_cluster_log_to_journald": "false", 2024-09-15T00:11:17.287 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-15T00:11:17.287 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-15T00:11:17.287 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-15T00:11:17.288 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-15T00:11:17.288 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_compact_on_bootstrap": "false", 2024-09-15T00:11:17.288 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_compact_on_start": "false", 2024-09-15T00:11:17.288 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_compact_on_trim": "true", 2024-09-15T00:11:17.288 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-15T00:11:17.288 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-15T00:11:17.288 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-15T00:11:17.288 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_cpu_threads": "4", 2024-09-15T00:11:17.288 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_crush_min_required_version": "hammer", 2024-09-15T00:11:17.288 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_daemon_bytes": "419430400", 2024-09-15T00:11:17.288 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-15T00:11:17.288 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_data_avail_crit": "5", 2024-09-15T00:11:17.288 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_data_avail_warn": "5", 2024-09-15T00:11:17.289 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_data_size_warn": "16106127360", 2024-09-15T00:11:17.289 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-15T00:11:17.289 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-15T00:11:17.289 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_debug_dump_json": "false", 2024-09-15T00:11:17.289 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-15T00:11:17.289 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_debug_dump_transactions": "false", 2024-09-15T00:11:17.289 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_debug_extra_checks": "false", 2024-09-15T00:11:17.289 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-15T00:11:17.289 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-15T00:11:17.289 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_debug_no_require_reef": "false", 2024-09-15T00:11:17.289 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_debug_no_require_squid": "false", 2024-09-15T00:11:17.289 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-15T00:11:17.289 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-15T00:11:17.289 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-15T00:11:17.290 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_down_mkfs_grace": "120", 2024-09-15T00:11:17.290 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_down_uptime_grace": "60", 2024-09-15T00:11:17.290 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_election_default_strategy": "1", 2024-09-15T00:11:17.290 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_election_timeout": "5.000000", 2024-09-15T00:11:17.290 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-15T00:11:17.290 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_elector_ping_divisor": "2", 2024-09-15T00:11:17.290 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-15T00:11:17.290 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_enable_op_tracker": "true", 2024-09-15T00:11:17.290 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_fake_pool_delete": "false", 2024-09-15T00:11:17.290 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_force_quorum_join": "false", 2024-09-15T00:11:17.290 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-15T00:11:17.290 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_globalid_prealloc": "10000", 2024-09-15T00:11:17.290 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_health_detail_to_clog": "false", 2024-09-15T00:11:17.291 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_health_log_update_period": "5", 2024-09-15T00:11:17.291 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_health_max_detail": "50", 2024-09-15T00:11:17.291 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_health_to_clog": "true", 2024-09-15T00:11:17.291 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_health_to_clog_interval": "600", 2024-09-15T00:11:17.291 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-15T00:11:17.291 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_host": "172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790],172.21.15.184", 2024-09-15T00:11:17.291 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_host_override": "", 2024-09-15T00:11:17.291 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_initial_members": "", 2024-09-15T00:11:17.291 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-15T00:11:17.291 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-15T00:11:17.291 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-15T00:11:17.291 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-15T00:11:17.291 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-15T00:11:17.291 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_lease": "5.000000", 2024-09-15T00:11:17.292 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-15T00:11:17.292 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-15T00:11:17.292 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_log_full_interval": "50", 2024-09-15T00:11:17.292 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_log_max": "10000", 2024-09-15T00:11:17.292 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_log_max_summary": "50", 2024-09-15T00:11:17.292 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-15T00:11:17.292 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_max_log_epochs": "500", 2024-09-15T00:11:17.292 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-15T00:11:17.292 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-15T00:11:17.292 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_max_osd": "10000", 2024-09-15T00:11:17.292 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_max_pg_per_osd": "10000", 2024-09-15T00:11:17.292 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_max_pool_pg_num": "65536", 2024-09-15T00:11:17.292 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-15T00:11:17.293 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-15T00:11:17.293 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_mds_force_trim_to": "0", 2024-09-15T00:11:17.293 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_mds_skip_sanity": "false", 2024-09-15T00:11:17.293 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_memory_autotune": "true", 2024-09-15T00:11:17.293 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_memory_target": "2147483648", 2024-09-15T00:11:17.293 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_mgr_beacon_grace": "30", 2024-09-15T00:11:17.293 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-15T00:11:17.293 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_mgr_digest_period": "5", 2024-09-15T00:11:17.293 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_mgr_inactive_grace": "60", 2024-09-15T00:11:17.293 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-15T00:11:17.293 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-15T00:11:17.293 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_min_osdmap_epochs": "500", 2024-09-15T00:11:17.293 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_op_complaint_time": "30", 2024-09-15T00:11:17.294 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_op_history_duration": "600", 2024-09-15T00:11:17.294 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_op_history_size": "20", 2024-09-15T00:11:17.294 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_op_history_slow_op_size": "20", 2024-09-15T00:11:17.294 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-15T00:11:17.294 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_op_log_threshold": "5", 2024-09-15T00:11:17.294 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-15T00:11:17.294 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-15T00:11:17.294 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-15T00:11:17.294 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_auto_mark_in": "false", 2024-09-15T00:11:17.294 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-15T00:11:17.294 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-15T00:11:17.294 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-15T00:11:17.294 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_cache_size": "500", 2024-09-15T00:11:17.294 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-15T00:11:17.295 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-15T00:11:17.295 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-15T00:11:17.295 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_down_out_interval": "600", 2024-09-15T00:11:17.295 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-15T00:11:17.295 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-15T00:11:17.295 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_force_trim_to": "0", 2024-09-15T00:11:17.295 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-15T00:11:17.295 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-15T00:11:17.295 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-15T00:11:17.295 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-15T00:11:17.295 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-15T00:11:17.295 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-15T00:11:17.295 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-15T00:11:17.296 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-15T00:11:17.296 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_min_down_reporters": "2", 2024-09-15T00:11:17.296 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-15T00:11:17.296 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-15T00:11:17.296 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-15T00:11:17.296 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-15T00:11:17.296 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-15T00:11:17.296 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-15T00:11:17.296 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_report_timeout": "900", 2024-09-15T00:11:17.296 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-15T00:11:17.296 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-15T00:11:17.296 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-15T00:11:17.296 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-15T00:11:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-15T00:11:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-15T00:11:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-15T00:11:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-15T00:11:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-15T00:11:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_pg_stuck_threshold": "60", 2024-09-15T00:11:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-15T00:11:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-15T00:11:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-15T00:11:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-15T00:11:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-15T00:11:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-15T00:11:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_probe_timeout": "2.000000", 2024-09-15T00:11:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_reweight_max_change": "0.050000", 2024-09-15T00:11:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_reweight_max_osds": "4", 2024-09-15T00:11:17.298 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-15T00:11:17.298 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-15T00:11:17.298 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-15T00:11:17.298 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-15T00:11:17.298 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-15T00:11:17.298 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_scrub_interval": "86400", 2024-09-15T00:11:17.298 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_scrub_max_keys": "100", 2024-09-15T00:11:17.298 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_scrub_timeout": "300", 2024-09-15T00:11:17.298 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_session_timeout": "300", 2024-09-15T00:11:17.298 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_smart_report_timeout": "5", 2024-09-15T00:11:17.298 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_stat_smooth_intervals": "6", 2024-09-15T00:11:17.298 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-15T00:11:17.298 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_stretch_pool_min_size": "2", 2024-09-15T00:11:17.299 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_stretch_pool_size": "4", 2024-09-15T00:11:17.299 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-15T00:11:17.299 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-15T00:11:17.299 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_sync_debug": "false", 2024-09-15T00:11:17.299 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-15T00:11:17.299 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-15T00:11:17.299 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_sync_provider_kill_at": "0", 2024-09-15T00:11:17.299 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_sync_requester_kill_at": "0", 2024-09-15T00:11:17.299 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_sync_timeout": "60.000000", 2024-09-15T00:11:17.299 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_target_pg_per_osd": "100", 2024-09-15T00:11:17.299 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_tick_interval": "5", 2024-09-15T00:11:17.299 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_timecheck_interval": "300.000000", 2024-09-15T00:11:17.299 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-15T00:11:17.300 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_use_min_delay_socket": "false", 2024-09-15T00:11:17.300 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_older_version_delay": "604800", 2024-09-15T00:11:17.300 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-15T00:11:17.300 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-15T00:11:17.300 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-15T00:11:17.300 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-15T00:11:17.300 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-15T00:11:17.300 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-15T00:11:17.300 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-15T00:11:17.300 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_misplaced": "false", 2024-09-15T00:11:17.300 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-15T00:11:17.300 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_older_version": "true", 2024-09-15T00:11:17.300 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-15T00:11:17.300 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-15T00:11:17.301 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-15T00:11:17.301 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-15T00:11:17.301 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-15T00:11:17.301 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-15T00:11:17.301 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-15T00:11:17.301 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-15T00:11:17.301 INFO:teuthology.orchestra.run.smithi171.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-15T00:11:17.301 INFO:teuthology.orchestra.run.smithi171.stdout: "monmap": "", 2024-09-15T00:11:17.301 INFO:teuthology.orchestra.run.smithi171.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-15T00:11:17.301 INFO:teuthology.orchestra.run.smithi171.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-15T00:11:17.301 INFO:teuthology.orchestra.run.smithi171.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-15T00:11:17.301 INFO:teuthology.orchestra.run.smithi171.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-15T00:11:17.301 INFO:teuthology.orchestra.run.smithi171.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-15T00:11:17.301 INFO:teuthology.orchestra.run.smithi171.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-15T00:11:17.301 INFO:teuthology.orchestra.run.smithi171.stdout: "motr_tracing_enabled": "false", 2024-09-15T00:11:17.302 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_op_threads": "3", 2024-09-15T00:11:17.302 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-15T00:11:17.302 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_cm": "false", 2024-09-15T00:11:17.302 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_device_name": "", 2024-09-15T00:11:17.302 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_dscp": "96", 2024-09-15T00:11:17.302 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-15T00:11:17.302 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-15T00:11:17.302 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_local_gid": "", 2024-09-15T00:11:17.302 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-15T00:11:17.302 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_port_num": "1", 2024-09-15T00:11:17.302 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-15T00:11:17.302 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-15T00:11:17.302 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-15T00:11:17.303 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-15T00:11:17.303 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_sl": "3", 2024-09-15T00:11:17.303 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_support_srq": "true", 2024-09-15T00:11:17.303 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_rdma_type": "ib", 2024-09-15T00:11:17.303 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_async_reap_threshold": "5", 2024-09-15T00:11:17.303 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_bind_before_connect": "false", 2024-09-15T00:11:17.303 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_bind_ipv4": "true", 2024-09-15T00:11:17.303 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_bind_ipv6": "false", 2024-09-15T00:11:17.303 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_bind_msgr1": "true", 2024-09-15T00:11:17.303 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_bind_msgr2": "true", 2024-09-15T00:11:17.303 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_bind_port_max": "7568", 2024-09-15T00:11:17.303 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_bind_port_min": "6800", 2024-09-15T00:11:17.303 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-15T00:11:17.303 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_bind_retry_count": "3", 2024-09-15T00:11:17.304 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_bind_retry_delay": "5", 2024-09-15T00:11:17.304 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_blackhole_client": "false", 2024-09-15T00:11:17.304 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_blackhole_mds": "false", 2024-09-15T00:11:17.304 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_blackhole_mgr": "false", 2024-09-15T00:11:17.304 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_blackhole_mon": "false", 2024-09-15T00:11:17.304 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_blackhole_osd": "false", 2024-09-15T00:11:17.304 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_client_mode": "crc secure", 2024-09-15T00:11:17.304 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_cluster_mode": "crc secure", 2024-09-15T00:11:17.304 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_cluster_type": "", 2024-09-15T00:11:17.304 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_compress_secure": "false", 2024-09-15T00:11:17.304 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_connection_idle_timeout": "900", 2024-09-15T00:11:17.304 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_connection_ready_timeout": "10", 2024-09-15T00:11:17.304 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_crc_data": "true", 2024-09-15T00:11:17.304 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_crc_header": "true", 2024-09-15T00:11:17.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_die_on_bad_msg": "false", 2024-09-15T00:11:17.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_die_on_bug": "true", 2024-09-15T00:11:17.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_die_on_old_message": "true", 2024-09-15T00:11:17.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_die_on_skipped_message": "false", 2024-09-15T00:11:17.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-15T00:11:17.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-15T00:11:17.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dpdk_coremask": "0xF", 2024-09-15T00:11:17.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-15T00:11:17.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-15T00:11:17.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dpdk_enable_tso": "true", 2024-09-15T00:11:17.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-15T00:11:17.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-15T00:11:17.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dpdk_hugepages": "", 2024-09-15T00:11:17.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-15T00:11:17.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-15T00:11:17.306 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dpdk_lro": "true", 2024-09-15T00:11:17.306 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dpdk_memory_channel": "4", 2024-09-15T00:11:17.306 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-15T00:11:17.306 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dpdk_pmd": "", 2024-09-15T00:11:17.306 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dpdk_port_id": "0", 2024-09-15T00:11:17.306 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-15T00:11:17.306 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-15T00:11:17.306 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_dump_on_send": "false", 2024-09-15T00:11:17.306 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_initial_backoff": "0.200000", 2024-09-15T00:11:17.306 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_inject_delay_max": "1.000000", 2024-09-15T00:11:17.306 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-15T00:11:17.306 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_inject_delay_type": "", 2024-09-15T00:11:17.306 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-15T00:11:17.307 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_inject_network_congestion": "0", 2024-09-15T00:11:17.307 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_inject_socket_failures": "5000", 2024-09-15T00:11:17.307 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_learn_addr_from_peer": "true", 2024-09-15T00:11:17.307 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_max_accept_failures": "4", 2024-09-15T00:11:17.307 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_max_backoff": "15.000000", 2024-09-15T00:11:17.307 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_mon_client_mode": "secure crc", 2024-09-15T00:11:17.307 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-15T00:11:17.307 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_mon_service_mode": "secure crc", 2024-09-15T00:11:17.307 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_osd_compress_min_size": "1024", 2024-09-15T00:11:17.307 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_osd_compress_mode": "none", 2024-09-15T00:11:17.307 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-15T00:11:17.307 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-15T00:11:17.307 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_pq_min_cost": "65536", 2024-09-15T00:11:17.308 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_public_type": "", 2024-09-15T00:11:17.308 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_service_mode": "crc secure", 2024-09-15T00:11:17.308 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_tcp_listen_backlog": "512", 2024-09-15T00:11:17.308 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_tcp_nodelay": "true", 2024-09-15T00:11:17.308 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-15T00:11:17.308 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_tcp_rcvbuf": "0", 2024-09-15T00:11:17.308 INFO:teuthology.orchestra.run.smithi171.stdout: "ms_type": "async+posix", 2024-09-15T00:11:17.308 INFO:teuthology.orchestra.run.smithi171.stdout: "no_config_file": "false", 2024-09-15T00:11:17.308 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter_completion_locks_per_session": "32", 2024-09-15T00:11:17.308 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-15T00:11:17.308 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-15T00:11:17.308 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter_inflight_ops": "1024", 2024-09-15T00:11:17.308 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-15T00:11:17.308 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-15T00:11:17.308 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter_tick_interval": "5.000000", 2024-09-15T00:11:17.309 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter_timeout": "10.000000", 2024-09-15T00:11:17.309 INFO:teuthology.orchestra.run.smithi171.stdout: "objectstore_blackhole": "false", 2024-09-15T00:11:17.309 INFO:teuthology.orchestra.run.smithi171.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-15T00:11:17.309 INFO:teuthology.orchestra.run.smithi171.stdout: "openssl_engine_opts": "", 2024-09-15T00:11:17.309 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_agent_delay_time": "5.000000", 2024-09-15T00:11:17.309 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_agent_hist_halflife": "1000", 2024-09-15T00:11:17.309 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_agent_max_low_ops": "2", 2024-09-15T00:11:17.309 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_agent_max_ops": "4", 2024-09-15T00:11:17.309 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-15T00:11:17.309 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-15T00:11:17.309 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_agent_slop": "0.020000", 2024-09-15T00:11:17.309 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-15T00:11:17.309 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-15T00:11:17.309 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_asio_thread_count": "2", 2024-09-15T00:11:17.310 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_async_recovery_min_cost": "100", 2024-09-15T00:11:17.310 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-15T00:11:17.310 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-15T00:11:17.310 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_backfill_scan_max": "512", 2024-09-15T00:11:17.310 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_backfill_scan_min": "64", 2024-09-15T00:11:17.310 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_backoff_on_degraded": "false", 2024-09-15T00:11:17.310 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_backoff_on_peering": "false", 2024-09-15T00:11:17.310 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_backoff_on_unfound": "true", 2024-09-15T00:11:17.310 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_beacon_report_interval": "300", 2024-09-15T00:11:17.310 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bench_duration": "30", 2024-09-15T00:11:17.310 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-15T00:11:17.310 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bench_max_block_size": "67108864", 2024-09-15T00:11:17.310 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-15T00:11:17.310 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_blkin_trace_all": "false", 2024-09-15T00:11:17.311 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-15T00:11:17.311 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-15T00:11:17.311 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-15T00:11:17.311 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-15T00:11:17.311 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_check_for_log_corruption": "false", 2024-09-15T00:11:17.311 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-15T00:11:17.311 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T00:11:17.311 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-15T00:11:17.311 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T00:11:17.311 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_class_update_on_start": "true", 2024-09-15T00:11:17.311 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_client_message_cap": "256", 2024-09-15T00:11:17.311 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_client_message_size_cap": "524288000", 2024-09-15T00:11:17.311 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_client_op_priority": "63", 2024-09-15T00:11:17.311 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_client_watch_timeout": "30", 2024-09-15T00:11:17.312 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_command_max_records": "256", 2024-09-15T00:11:17.312 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_compact_on_start": "false", 2024-09-15T00:11:17.312 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-15T00:11:17.312 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-15T00:11:17.312 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-15T00:11:17.312 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_crush_update_on_start": "true", 2024-09-15T00:11:17.312 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_crush_update_weight_set": "true", 2024-09-15T00:11:17.312 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-15T00:11:17.312 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-15T00:11:17.312 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-15T00:11:17.312 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-15T00:11:17.312 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-15T00:11:17.312 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_feed_pullee": "-1", 2024-09-15T00:11:17.312 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-15T00:11:17.313 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-15T00:11:17.313 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-15T00:11:17.313 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_misdirected_ops": "false", 2024-09-15T00:11:17.313 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_no_acting_change": "false", 2024-09-15T00:11:17.313 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_no_purge_strays": "false", 2024-09-15T00:11:17.313 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_op_order": "false", 2024-09-15T00:11:17.313 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-15T00:11:17.313 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-15T00:11:17.313 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-15T00:11:17.313 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-15T00:11:17.313 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_shutdown": "false", 2024-09-15T00:11:17.313 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-15T00:11:17.313 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-15T00:11:17.314 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_trim_objects": "false", 2024-09-15T00:11:17.314 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-15T00:11:17.314 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-15T00:11:17.314 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_verify_snaps": "false", 2024-09-15T00:11:17.314 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-15T00:11:17.314 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-15T00:11:17.314 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_deep_scrub_keys": "1024", 2024-09-15T00:11:17.314 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-15T00:11:17.314 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-15T00:11:17.314 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-15T00:11:17.314 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_deep_scrub_stride": "524288", 2024-09-15T00:11:17.314 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-15T00:11:17.314 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-15T00:11:17.314 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_default_notify_timeout": "30", 2024-09-15T00:11:17.314 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_delete_sleep": "0.000000", 2024-09-15T00:11:17.315 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-15T00:11:17.315 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-15T00:11:17.315 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-15T00:11:17.315 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_discard_disconnected_ops": "true", 2024-09-15T00:11:17.315 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_enable_op_tracker": "true", 2024-09-15T00:11:17.315 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-15T00:11:17.315 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-15T00:11:17.315 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-15T00:11:17.315 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-15T00:11:17.315 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_fast_info": "true", 2024-09-15T00:11:17.315 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_fast_shutdown": "true", 2024-09-15T00:11:17.315 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-15T00:11:17.315 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-15T00:11:17.315 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-15T00:11:17.316 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-15T00:11:17.316 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-15T00:11:17.316 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_function_tracing": "false", 2024-09-15T00:11:17.316 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_heartbeat_grace": "20", 2024-09-15T00:11:17.316 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_heartbeat_interval": "6", 2024-09-15T00:11:17.316 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-15T00:11:17.316 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_heartbeat_min_peers": "10", 2024-09-15T00:11:17.316 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_heartbeat_min_size": "2000", 2024-09-15T00:11:17.316 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_heartbeat_stale": "600", 2024-09-15T00:11:17.316 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-15T00:11:17.316 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_hit_set_max_size": "100000", 2024-09-15T00:11:17.316 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_hit_set_min_size": "1000", 2024-09-15T00:11:17.316 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-15T00:11:17.316 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-15T00:11:17.316 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-15T00:11:17.317 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-15T00:11:17.317 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-15T00:11:17.317 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-15T00:11:17.317 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_journal_size": "5120", 2024-09-15T00:11:17.317 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_kill_backfill_at": "0", 2024-09-15T00:11:17.317 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-15T00:11:17.317 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_map_cache_size": "50", 2024-09-15T00:11:17.317 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_map_dedup": "true", 2024-09-15T00:11:17.317 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_map_message_max": "40", 2024-09-15T00:11:17.317 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-15T00:11:17.317 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_map_share_max_epochs": "40", 2024-09-15T00:11:17.317 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_attr_name_len": "100", 2024-09-15T00:11:17.317 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_attr_size": "0", 2024-09-15T00:11:17.317 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_backfills": "1", 2024-09-15T00:11:17.318 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_markdown_count": "5", 2024-09-15T00:11:17.318 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_markdown_period": "600", 2024-09-15T00:11:17.318 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_object_name_len": "2048", 2024-09-15T00:11:17.318 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_object_namespace_len": "256", 2024-09-15T00:11:17.318 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_object_size": "134217728", 2024-09-15T00:11:17.318 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-15T00:11:17.318 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-15T00:11:17.318 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_pg_blocked_by": "16", 2024-09-15T00:11:17.318 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_pg_log_entries": "10000", 2024-09-15T00:11:17.318 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-15T00:11:17.318 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_pgls": "1024", 2024-09-15T00:11:17.318 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_push_cost": "8388608", 2024-09-15T00:11:17.318 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_push_objects": "10", 2024-09-15T00:11:17.318 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_scrubs": "3", 2024-09-15T00:11:17.318 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-15T00:11:17.319 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_trimming_pgs": "2", 2024-09-15T00:11:17.319 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_write_op_reply_len": "64", 2024-09-15T00:11:17.319 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_max_write_size": "90", 2024-09-15T00:11:17.319 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-15T00:11:17.319 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-15T00:11:17.319 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-15T00:11:17.319 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-15T00:11:17.319 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-15T00:11:17.319 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-15T00:11:17.319 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-15T00:11:17.319 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-15T00:11:17.319 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_profile": "balanced", 2024-09-15T00:11:17.319 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-15T00:11:17.319 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-15T00:11:17.319 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-15T00:11:17.320 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-15T00:11:17.320 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-15T00:11:17.320 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-15T00:11:17.320 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-15T00:11:17.320 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-15T00:11:17.320 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-15T00:11:17.320 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-15T00:11:17.320 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-15T00:11:17.320 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_memory_base": "805306368", 2024-09-15T00:11:17.320 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_memory_cache_min": "134217728", 2024-09-15T00:11:17.320 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-15T00:11:17.320 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-15T00:11:17.320 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_memory_target": "4294967296", 2024-09-15T00:11:17.321 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_memory_target_autotune": "false", 2024-09-15T00:11:17.321 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-15T00:11:17.321 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_min_pg_log_entries": "250", 2024-09-15T00:11:17.321 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_min_recovery_priority": "0", 2024-09-15T00:11:17.321 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-15T00:11:17.321 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-15T00:11:17.321 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mon_report_interval": "5", 2024-09-15T00:11:17.321 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-15T00:11:17.321 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-15T00:11:17.321 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_num_cache_shards": "32", 2024-09-15T00:11:17.321 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_num_op_tracker_shard": "32", 2024-09-15T00:11:17.321 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_numa_auto_affinity": "true", 2024-09-15T00:11:17.321 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_numa_node": "-1", 2024-09-15T00:11:17.321 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_numa_prefer_iface": "true", 2024-09-15T00:11:17.322 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-15T00:11:17.322 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_objecter_finishers": "1", 2024-09-15T00:11:17.322 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_objectstore": "bluestore", 2024-09-15T00:11:17.322 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_objectstore_fuse": "false", 2024-09-15T00:11:17.322 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_objectstore_tracing": "false", 2024-09-15T00:11:17.322 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_complaint_time": "30.000000", 2024-09-15T00:11:17.322 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_history_duration": "600", 2024-09-15T00:11:17.322 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_history_size": "20", 2024-09-15T00:11:17.322 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_history_slow_op_size": "20", 2024-09-15T00:11:17.322 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-15T00:11:17.322 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_log_threshold": "5", 2024-09-15T00:11:17.322 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_num_shards": "0", 2024-09-15T00:11:17.322 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_num_shards_hdd": "5", 2024-09-15T00:11:17.322 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_num_shards_ssd": "8", 2024-09-15T00:11:17.322 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-15T00:11:17.323 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-15T00:11:17.323 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-15T00:11:17.323 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-15T00:11:17.323 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_pq_min_cost": "65536", 2024-09-15T00:11:17.323 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-15T00:11:17.323 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_queue_cut_off": "high", 2024-09-15T00:11:17.323 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-15T00:11:17.323 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_op_thread_timeout": "15", 2024-09-15T00:11:17.323 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_open_classes_on_start": "true", 2024-09-15T00:11:17.323 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_os_flags": "0", 2024-09-15T00:11:17.323 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_peering_op_priority": "255", 2024-09-15T00:11:17.323 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pg_delete_cost": "1048576", 2024-09-15T00:11:17.323 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pg_delete_priority": "5", 2024-09-15T00:11:17.323 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-15T00:11:17.323 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-15T00:11:17.324 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-15T00:11:17.324 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pg_log_trim_max": "10000", 2024-09-15T00:11:17.324 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pg_log_trim_min": "100", 2024-09-15T00:11:17.324 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-15T00:11:17.324 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-15T00:11:17.324 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-15T00:11:17.324 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-15T00:11:17.324 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-15T00:11:17.324 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-15T00:11:17.324 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-15T00:11:17.324 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-15T00:11:17.324 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-15T00:11:17.324 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-15T00:11:17.324 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_crimson": "false", 2024-09-15T00:11:17.325 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-15T00:11:17.325 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-15T00:11:17.325 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-15T00:11:17.325 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-15T00:11:17.325 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-15T00:11:17.325 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-15T00:11:17.325 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-15T00:11:17.325 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-15T00:11:17.325 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_flags": "0", 2024-09-15T00:11:17.325 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-15T00:11:17.325 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_min_size": "0", 2024-09-15T00:11:17.325 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-15T00:11:17.325 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_pg_num": "32", 2024-09-15T00:11:17.325 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_pgp_num": "0", 2024-09-15T00:11:17.325 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_read_ratio": "70", 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_size": "2", 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_default_type": "replicated", 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_push_per_object_cost": "1000", 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recover_clone_overlap": "true", 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_cost": "20971520", 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_max_active": "0", 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-15T00:11:17.327 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-15T00:11:17.327 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-15T00:11:17.327 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_max_single_start": "1", 2024-09-15T00:11:17.327 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_op_priority": "3", 2024-09-15T00:11:17.327 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-15T00:11:17.327 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_priority": "5", 2024-09-15T00:11:17.327 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-15T00:11:17.327 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_sleep": "0.000000", 2024-09-15T00:11:17.327 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-15T00:11:17.327 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-15T00:11:17.327 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-15T00:11:17.327 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_repair_during_recovery": "false", 2024-09-15T00:11:17.327 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_requested_scrub_priority": "120", 2024-09-15T00:11:17.327 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-15T00:11:17.327 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-15T00:11:17.328 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_auto_repair": "false", 2024-09-15T00:11:17.328 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-15T00:11:17.328 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-15T00:11:17.328 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_begin_hour": "0", 2024-09-15T00:11:17.328 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_begin_week_day": "0", 2024-09-15T00:11:17.328 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_chunk_max": "25", 2024-09-15T00:11:17.328 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_chunk_min": "5", 2024-09-15T00:11:17.328 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_cost": "52428800", 2024-09-15T00:11:17.328 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-15T00:11:17.328 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_during_recovery": "false", 2024-09-15T00:11:17.328 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_end_hour": "0", 2024-09-15T00:11:17.328 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_end_week_day": "0", 2024-09-15T00:11:17.328 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_event_cost": "4096", 2024-09-15T00:11:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-15T00:11:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-15T00:11:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_invalid_stats": "true", 2024-09-15T00:11:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-15T00:11:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-15T00:11:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_max_preemptions": "5", 2024-09-15T00:11:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-15T00:11:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_priority": "5", 2024-09-15T00:11:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_scrub_sleep": "0.000000", 2024-09-15T00:11:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-15T00:11:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-15T00:11:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-15T00:11:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_skip_data_digest": "false", 2024-09-15T00:11:17.329 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_smart_report_timeout": "5", 2024-09-15T00:11:17.330 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_snap_trim_cost": "1048576", 2024-09-15T00:11:17.330 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_snap_trim_priority": "5", 2024-09-15T00:11:17.330 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-15T00:11:17.330 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-15T00:11:17.330 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-15T00:11:17.330 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-15T00:11:17.330 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-15T00:11:17.330 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-15T00:11:17.330 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-15T00:11:17.330 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_target_transaction_size": "30", 2024-09-15T00:11:17.330 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-15T00:11:17.330 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-15T00:11:17.330 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-15T00:11:17.330 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-15T00:11:17.331 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-15T00:11:17.331 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-15T00:11:17.331 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-15T00:11:17.331 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-15T00:11:17.331 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-15T00:11:17.331 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-15T00:11:17.331 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_tracing": "false", 2024-09-15T00:11:17.331 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_use_stale_snap": "false", 2024-09-15T00:11:17.331 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-15T00:11:17.331 INFO:teuthology.orchestra.run.smithi171.stdout: "osdc_blkin_trace_all": "false", 2024-09-15T00:11:17.331 INFO:teuthology.orchestra.run.smithi171.stdout: "paxos_kill_at": "0", 2024-09-15T00:11:17.331 INFO:teuthology.orchestra.run.smithi171.stdout: "paxos_max_join_drift": "10", 2024-09-15T00:11:17.331 INFO:teuthology.orchestra.run.smithi171.stdout: "paxos_min": "500", 2024-09-15T00:11:17.331 INFO:teuthology.orchestra.run.smithi171.stdout: "paxos_min_wait": "0.050000", 2024-09-15T00:11:17.332 INFO:teuthology.orchestra.run.smithi171.stdout: "paxos_propose_interval": "1.000000", 2024-09-15T00:11:17.332 INFO:teuthology.orchestra.run.smithi171.stdout: "paxos_service_trim_max": "500", 2024-09-15T00:11:17.332 INFO:teuthology.orchestra.run.smithi171.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-15T00:11:17.332 INFO:teuthology.orchestra.run.smithi171.stdout: "paxos_service_trim_min": "250", 2024-09-15T00:11:17.332 INFO:teuthology.orchestra.run.smithi171.stdout: "paxos_stash_full_interval": "25", 2024-09-15T00:11:17.332 INFO:teuthology.orchestra.run.smithi171.stdout: "paxos_trim_max": "500", 2024-09-15T00:11:17.332 INFO:teuthology.orchestra.run.smithi171.stdout: "paxos_trim_min": "250", 2024-09-15T00:11:17.332 INFO:teuthology.orchestra.run.smithi171.stdout: "perf": "true", 2024-09-15T00:11:17.332 INFO:teuthology.orchestra.run.smithi171.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-15T00:11:17.332 INFO:teuthology.orchestra.run.smithi171.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-15T00:11:17.332 INFO:teuthology.orchestra.run.smithi171.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-15T00:11:17.332 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "-", 2024-09-15T00:11:17.332 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrv": "", 2024-09-15T00:11:17.332 INFO:teuthology.orchestra.run.smithi171.stdout: "public_bind_addr": "-", 2024-09-15T00:11:17.333 INFO:teuthology.orchestra.run.smithi171.stdout: "public_network": "", 2024-09-15T00:11:17.333 INFO:teuthology.orchestra.run.smithi171.stdout: "public_network_interface": "", 2024-09-15T00:11:17.333 INFO:teuthology.orchestra.run.smithi171.stdout: "qat_compressor_busy_polling": "false", 2024-09-15T00:11:17.333 INFO:teuthology.orchestra.run.smithi171.stdout: "qat_compressor_enabled": "false", 2024-09-15T00:11:17.333 INFO:teuthology.orchestra.run.smithi171.stdout: "qat_compressor_session_max_number": "256", 2024-09-15T00:11:17.333 INFO:teuthology.orchestra.run.smithi171.stdout: "rados_mon_op_timeout": "0", 2024-09-15T00:11:17.333 INFO:teuthology.orchestra.run.smithi171.stdout: "rados_osd_op_timeout": "0", 2024-09-15T00:11:17.333 INFO:teuthology.orchestra.run.smithi171.stdout: "rados_tracing": "false", 2024-09-15T00:11:17.333 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_atime_update_interval": "60", 2024-09-15T00:11:17.333 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-15T00:11:17.333 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_balance_parent_reads": "false", 2024-09-15T00:11:17.333 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_balance_snap_reads": "false", 2024-09-15T00:11:17.333 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_blkin_trace_all": "false", 2024-09-15T00:11:17.333 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-15T00:11:17.334 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-15T00:11:17.334 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_cache": "true", 2024-09-15T00:11:17.334 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-15T00:11:17.334 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-15T00:11:17.334 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-15T00:11:17.334 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-15T00:11:17.334 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_cache_policy": "writearound", 2024-09-15T00:11:17.334 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_cache_size": "33554432", 2024-09-15T00:11:17.334 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-15T00:11:17.334 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-15T00:11:17.334 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_clone_copy_on_read": "false", 2024-09-15T00:11:17.334 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_compression_hint": "none", 2024-09-15T00:11:17.334 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_concurrent_management_ops": "10", 2024-09-15T00:11:17.335 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-15T00:11:17.335 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_default_clone_format": "auto", 2024-09-15T00:11:17.335 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_default_data_pool": "", 2024-09-15T00:11:17.335 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_default_features": "61", 2024-09-15T00:11:17.335 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_default_format": "2", 2024-09-15T00:11:17.335 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_default_map_options": "", 2024-09-15T00:11:17.335 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_default_order": "22", 2024-09-15T00:11:17.335 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_default_pool": "rbd", 2024-09-15T00:11:17.335 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-15T00:11:17.335 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_default_stripe_count": "0", 2024-09-15T00:11:17.335 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_default_stripe_unit": "0", 2024-09-15T00:11:17.335 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-15T00:11:17.335 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-15T00:11:17.335 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-15T00:11:17.335 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_enable_alloc_hint": "true", 2024-09-15T00:11:17.336 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-15T00:11:17.336 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_io_scheduler": "simple", 2024-09-15T00:11:17.336 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-15T00:11:17.336 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-15T00:11:17.336 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-15T00:11:17.336 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-15T00:11:17.336 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-15T00:11:17.336 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-15T00:11:17.336 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-15T00:11:17.336 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-15T00:11:17.336 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-15T00:11:17.336 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_journal_order": "24", 2024-09-15T00:11:17.337 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_journal_pool": "", 2024-09-15T00:11:17.337 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_journal_splay_width": "4", 2024-09-15T00:11:17.337 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_localize_parent_reads": "false", 2024-09-15T00:11:17.337 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_localize_snap_reads": "false", 2024-09-15T00:11:17.337 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-15T00:11:17.337 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-15T00:11:17.337 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-15T00:11:17.337 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-15T00:11:17.337 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-15T00:11:17.337 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-15T00:11:17.337 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-15T00:11:17.337 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-15T00:11:17.337 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-15T00:11:17.338 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-15T00:11:17.338 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-15T00:11:17.338 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-15T00:11:17.338 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-15T00:11:17.338 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-15T00:11:17.338 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-15T00:11:17.338 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-15T00:11:17.338 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-15T00:11:17.338 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-15T00:11:17.338 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-15T00:11:17.338 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-15T00:11:17.338 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-15T00:11:17.339 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-15T00:11:17.339 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-15T00:11:17.339 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-15T00:11:17.339 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-15T00:11:17.339 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-15T00:11:17.339 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-15T00:11:17.339 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-15T00:11:17.339 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-15T00:11:17.339 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-15T00:11:17.339 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-15T00:11:17.339 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_mtime_update_interval": "60", 2024-09-15T00:11:17.339 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_non_blocking_aio": "true", 2024-09-15T00:11:17.339 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_op_thread_timeout": "60", 2024-09-15T00:11:17.339 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_op_threads": "1", 2024-09-15T00:11:17.339 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_parent_cache_enabled": "false", 2024-09-15T00:11:17.340 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-15T00:11:17.340 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-15T00:11:17.340 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-15T00:11:17.340 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_plugins": "", 2024-09-15T00:11:17.340 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_bps_burst": "0", 2024-09-15T00:11:17.340 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-15T00:11:17.340 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_bps_limit": "0", 2024-09-15T00:11:17.340 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_exclude_ops": "0", 2024-09-15T00:11:17.340 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_iops_burst": "0", 2024-09-15T00:11:17.340 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-15T00:11:17.340 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_iops_limit": "0", 2024-09-15T00:11:17.340 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-15T00:11:17.340 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-15T00:11:17.341 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-15T00:11:17.341 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-15T00:11:17.341 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-15T00:11:17.341 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-15T00:11:17.341 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-15T00:11:17.341 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-15T00:11:17.341 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-15T00:11:17.341 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-15T00:11:17.341 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-15T00:11:17.341 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-15T00:11:17.341 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-15T00:11:17.341 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-15T00:11:17.341 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_read_from_replica_policy": "default", 2024-09-15T00:11:17.341 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-15T00:11:17.342 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-15T00:11:17.342 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-15T00:11:17.342 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-15T00:11:17.342 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_skip_partial_discard": "true", 2024-09-15T00:11:17.342 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-15T00:11:17.342 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_tracing": "false", 2024-09-15T00:11:17.342 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_validate_names": "true", 2024-09-15T00:11:17.342 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd_validate_pool": "true", 2024-09-15T00:11:17.342 INFO:teuthology.orchestra.run.smithi171.stdout: "restapi_base_url": "", 2024-09-15T00:11:17.342 INFO:teuthology.orchestra.run.smithi171.stdout: "restapi_log_level": "", 2024-09-15T00:11:17.342 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-15T00:11:17.343 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-15T00:11:17.343 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_acl_grants_max_num": "100", 2024-09-15T00:11:17.343 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_admin_entry": "admin", 2024-09-15T00:11:17.343 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-15T00:11:17.343 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_backend_store": "rados", 2024-09-15T00:11:17.343 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_barbican_url": "", 2024-09-15T00:11:17.343 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_beast_enable_async": "true", 2024-09-15T00:11:17.343 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_bucket_counters_cache": "false", 2024-09-15T00:11:17.343 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-15T00:11:17.343 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-15T00:11:17.343 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-15T00:11:17.343 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-15T00:11:17.343 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-15T00:11:17.343 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-15T00:11:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-15T00:11:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-15T00:11:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_cache_enabled": "true", 2024-09-15T00:11:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_cache_expiry_interval": "900", 2024-09-15T00:11:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_cache_lru_size": "25000", 2024-09-15T00:11:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_config_store": "rados", 2024-09-15T00:11:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_content_length_compat": "false", 2024-09-15T00:11:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_copy_obj_progress": "true", 2024-09-15T00:11:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-15T00:11:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_cors_rules_max_num": "100", 2024-09-15T00:11:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_cross_domain_policy": "", 2024-09-15T00:11:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-15T00:11:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_kmip_addr": "", 2024-09-15T00:11:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-15T00:11:17.345 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-15T00:11:17.345 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-15T00:11:17.345 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-15T00:11:17.345 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_kmip_password": "", 2024-09-15T00:11:17.345 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-15T00:11:17.345 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_kmip_username": "", 2024-09-15T00:11:17.345 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_require_ssl": "true", 2024-09-15T00:11:17.345 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-15T00:11:17.345 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-15T00:11:17.345 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-15T00:11:17.345 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-15T00:11:17.345 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-15T00:11:17.345 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-15T00:11:17.346 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-15T00:11:17.346 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-15T00:11:17.346 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-15T00:11:17.346 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-15T00:11:17.346 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-15T00:11:17.346 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-15T00:11:17.346 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-15T00:11:17.346 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-15T00:11:17.346 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-15T00:11:17.346 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_vault_addr": "", 2024-09-15T00:11:17.346 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_vault_auth": "token", 2024-09-15T00:11:17.346 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_vault_namespace": "", 2024-09-15T00:11:17.346 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_vault_prefix": "", 2024-09-15T00:11:17.346 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-15T00:11:17.347 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-15T00:11:17.347 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-15T00:11:17.347 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-15T00:11:17.347 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_vault_token_file": "", 2024-09-15T00:11:17.347 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-15T00:11:17.347 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_curl_buffersize": "524288", 2024-09-15T00:11:17.347 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-15T00:11:17.347 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_curl_low_speed_time": "300", 2024-09-15T00:11:17.347 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-15T00:11:17.347 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-15T00:11:17.347 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-15T00:11:17.347 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-15T00:11:17.347 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-15T00:11:17.347 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-15T00:11:17.348 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-15T00:11:17.348 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-15T00:11:17.348 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-15T00:11:17.348 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-15T00:11:17.348 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-15T00:11:17.348 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_d4n_port": "6379", 2024-09-15T00:11:17.348 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-15T00:11:17.348 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_data_log_changes_size": "1000", 2024-09-15T00:11:17.348 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_data_log_num_shards": "128", 2024-09-15T00:11:17.348 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-15T00:11:17.348 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_data_log_window": "30", 2024-09-15T00:11:17.348 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-15T00:11:17.348 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-15T00:11:17.348 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-15T00:11:17.349 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-15T00:11:17.349 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-15T00:11:17.349 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-15T00:11:17.349 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-15T00:11:17.349 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-15T00:11:17.349 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-15T00:11:17.349 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-15T00:11:17.349 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-15T00:11:17.349 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-15T00:11:17.349 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-15T00:11:17.349 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-15T00:11:17.349 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-15T00:11:17.349 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-15T00:11:17.349 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-15T00:11:17.350 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-15T00:11:17.350 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-15T00:11:17.350 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-15T00:11:17.350 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-15T00:11:17.350 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-15T00:11:17.350 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-15T00:11:17.350 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-15T00:11:17.350 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-15T00:11:17.350 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_dns_name": "", 2024-09-15T00:11:17.350 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_dns_s3website_name": "", 2024-09-15T00:11:17.350 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_dynamic_resharding": "true", 2024-09-15T00:11:17.350 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-15T00:11:17.350 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_enable_gc_threads": "true", 2024-09-15T00:11:17.350 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_enable_lc_threads": "true", 2024-09-15T00:11:17.350 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_enable_ops_log": "false", 2024-09-15T00:11:17.351 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_enable_quota_threads": "true", 2024-09-15T00:11:17.351 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_enable_static_website": "false", 2024-09-15T00:11:17.351 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_enable_usage_log": "false", 2024-09-15T00:11:17.351 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_enforce_swift_acls": "true", 2024-09-15T00:11:17.351 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_exit_timeout_secs": "120", 2024-09-15T00:11:17.351 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_expose_bucket": "false", 2024-09-15T00:11:17.351 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_extended_http_attrs": "", 2024-09-15T00:11:17.351 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_filter": "none", 2024-09-15T00:11:17.351 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-15T00:11:17.351 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_frontends": "beast port=7480", 2024-09-15T00:11:17.351 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-15T00:11:17.351 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_gc_max_deferred": "50", 2024-09-15T00:11:17.351 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-15T00:11:17.351 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_gc_max_objs": "32", 2024-09-15T00:11:17.352 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-15T00:11:17.352 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-15T00:11:17.352 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-15T00:11:17.352 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-15T00:11:17.352 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_gc_processor_period": "3600", 2024-09-15T00:11:17.352 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-15T00:11:17.352 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-15T00:11:17.352 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-15T00:11:17.352 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-15T00:11:17.352 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_init_timeout": "300", 2024-09-15T00:11:17.352 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-15T00:11:17.352 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-15T00:11:17.352 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_kafka_connection_idle": "300", 2024-09-15T00:11:17.352 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-15T00:11:17.352 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-15T00:11:17.353 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-15T00:11:17.353 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-15T00:11:17.353 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-15T00:11:17.353 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_admin_domain": "", 2024-09-15T00:11:17.353 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_admin_password": "", 2024-09-15T00:11:17.353 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_admin_password_path": "", 2024-09-15T00:11:17.353 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_admin_project": "", 2024-09-15T00:11:17.353 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_admin_tenant": "", 2024-09-15T00:11:17.353 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_admin_token": "", 2024-09-15T00:11:17.353 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_admin_token_path": "", 2024-09-15T00:11:17.353 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_admin_user": "", 2024-09-15T00:11:17.354 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_api_version": "2", 2024-09-15T00:11:17.354 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_barbican_domain": "", 2024-09-15T00:11:17.354 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_barbican_password": "", 2024-09-15T00:11:17.354 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_barbican_project": "", 2024-09-15T00:11:17.354 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-15T00:11:17.354 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_barbican_user": "", 2024-09-15T00:11:17.354 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-15T00:11:17.354 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-15T00:11:17.354 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-15T00:11:17.354 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-15T00:11:17.354 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-15T00:11:17.354 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_url": "", 2024-09-15T00:11:17.354 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-15T00:11:17.354 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_lc_debug_interval": "-1", 2024-09-15T00:11:17.355 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_lc_lock_max_time": "90", 2024-09-15T00:11:17.355 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_lc_max_objs": "32", 2024-09-15T00:11:17.355 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_lc_max_rules": "1000", 2024-09-15T00:11:17.355 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_lc_max_worker": "3", 2024-09-15T00:11:17.355 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-15T00:11:17.355 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_lc_thread_delay": "0", 2024-09-15T00:11:17.355 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-15T00:11:17.355 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_ldap_dnattr": "uid", 2024-09-15T00:11:17.355 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-15T00:11:17.355 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_ldap_searchfilter": "", 2024-09-15T00:11:17.355 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-15T00:11:17.355 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_ldap_uri": "", 2024-09-15T00:11:17.355 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-15T00:11:17.355 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-15T00:11:17.356 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-15T00:11:17.356 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_log_http_headers": "", 2024-09-15T00:11:17.356 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-15T00:11:17.356 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-15T00:11:17.356 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_log_object_name_utc": "false", 2024-09-15T00:11:17.356 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-15T00:11:17.356 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-15T00:11:17.356 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_max_attr_name_len": "0", 2024-09-15T00:11:17.356 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_max_attr_size": "0", 2024-09-15T00:11:17.356 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-15T00:11:17.356 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_max_chunk_size": "4194304", 2024-09-15T00:11:17.357 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-15T00:11:17.357 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-15T00:11:17.357 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-15T00:11:17.357 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_max_listing_results": "1000", 2024-09-15T00:11:17.357 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_max_notify_retries": "10", 2024-09-15T00:11:17.357 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-15T00:11:17.357 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_max_put_param_size": "1048576", 2024-09-15T00:11:17.357 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_max_put_size": "5368709120", 2024-09-15T00:11:17.357 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_max_slo_entries": "1000", 2024-09-15T00:11:17.357 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_md_log_max_shards": "64", 2024-09-15T00:11:17.357 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-15T00:11:17.357 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-15T00:11:17.357 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-15T00:11:17.357 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-15T00:11:17.357 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_mp_lock_max_time": "600", 2024-09-15T00:11:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-15T00:11:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-15T00:11:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-15T00:11:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-15T00:11:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-15T00:11:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-15T00:11:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-15T00:11:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-15T00:11:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_nfs_max_gc": "300", 2024-09-15T00:11:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-15T00:11:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-15T00:11:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-15T00:11:17.358 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-15T00:11:17.359 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-15T00:11:17.359 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-15T00:11:17.359 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-15T00:11:17.359 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_num_async_rados_threads": "32", 2024-09-15T00:11:17.359 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_num_control_oids": "8", 2024-09-15T00:11:17.359 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_numa_node": "-1", 2024-09-15T00:11:17.359 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-15T00:11:17.359 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-15T00:11:17.359 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_objexp_chunk_size": "100", 2024-09-15T00:11:17.359 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_objexp_gc_interval": "600", 2024-09-15T00:11:17.359 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-15T00:11:17.359 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-15T00:11:17.359 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-15T00:11:17.359 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_op_thread_timeout": "600", 2024-09-15T00:11:17.359 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_op_tracing": "false", 2024-09-15T00:11:17.360 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_opa_token": "", 2024-09-15T00:11:17.360 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_opa_url": "", 2024-09-15T00:11:17.360 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_opa_verify_ssl": "true", 2024-09-15T00:11:17.360 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-15T00:11:17.360 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-15T00:11:17.360 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_ops_log_rados": "false", 2024-09-15T00:11:17.360 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_ops_log_socket_path": "", 2024-09-15T00:11:17.360 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-15T00:11:17.360 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-15T00:11:17.360 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-15T00:11:17.360 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_period_push_interval": "2.000000", 2024-09-15T00:11:17.360 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-15T00:11:17.360 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-15T00:11:17.361 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-15T00:11:17.361 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-15T00:11:17.361 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_posix_cache_lanes": "3", 2024-09-15T00:11:17.361 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-15T00:11:17.361 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-15T00:11:17.361 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_posix_cache_partitions": "3", 2024-09-15T00:11:17.361 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-15T00:11:17.361 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_print_continue": "true", 2024-09-15T00:11:17.361 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-15T00:11:17.361 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-15T00:11:17.361 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-15T00:11:17.361 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-15T00:11:17.361 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-15T00:11:17.361 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_rados_tracing": "false", 2024-09-15T00:11:17.362 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_realm": "", 2024-09-15T00:11:17.362 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_realm_id": "", 2024-09-15T00:11:17.362 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-15T00:11:17.362 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_region": "", 2024-09-15T00:11:17.362 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-15T00:11:17.362 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-15T00:11:17.362 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-15T00:11:17.362 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-15T00:11:17.362 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_request_uri": "", 2024-09-15T00:11:17.362 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_reshard_batch_size": "64", 2024-09-15T00:11:17.362 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-15T00:11:17.362 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_reshard_max_aio": "128", 2024-09-15T00:11:17.362 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_reshard_num_logs": "16", 2024-09-15T00:11:17.362 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_reshard_thread_interval": "600", 2024-09-15T00:11:17.363 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_resolve_cname": "false", 2024-09-15T00:11:17.363 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-15T00:11:17.363 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_run_sync_thread": "true", 2024-09-15T00:11:17.363 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-15T00:11:17.363 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-15T00:11:17.363 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-15T00:11:17.363 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-15T00:11:17.363 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-15T00:11:17.363 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-15T00:11:17.363 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-15T00:11:17.363 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-15T00:11:17.363 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-15T00:11:17.363 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_scheduler_type": "throttler", 2024-09-15T00:11:17.364 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_script_uri": "", 2024-09-15T00:11:17.364 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_service_provider_name": "", 2024-09-15T00:11:17.364 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-15T00:11:17.364 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sts_client_id": "", 2024-09-15T00:11:17.364 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sts_client_secret": "", 2024-09-15T00:11:17.364 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sts_entry": "sts", 2024-09-15T00:11:17.364 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sts_key": "sts", 2024-09-15T00:11:17.364 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-15T00:11:17.364 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sts_min_session_duration": "900", 2024-09-15T00:11:17.364 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sts_token_introspection_url": "", 2024-09-15T00:11:17.364 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_swift_account_in_url": "false", 2024-09-15T00:11:17.364 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_swift_auth_entry": "auth", 2024-09-15T00:11:17.364 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_swift_auth_url": "", 2024-09-15T00:11:17.364 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_swift_custom_header": "", 2024-09-15T00:11:17.365 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-15T00:11:17.365 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_swift_need_stats": "true", 2024-09-15T00:11:17.365 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_swift_tenant_name": "", 2024-09-15T00:11:17.365 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_swift_token_expiration": "86400", 2024-09-15T00:11:17.365 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_swift_url": "", 2024-09-15T00:11:17.365 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_swift_url_prefix": "swift", 2024-09-15T00:11:17.365 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-15T00:11:17.365 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-15T00:11:17.365 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-15T00:11:17.365 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sync_lease_period": "120", 2024-09-15T00:11:17.365 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-15T00:11:17.365 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-15T00:11:17.365 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-15T00:11:17.365 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-15T00:11:17.366 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-15T00:11:17.366 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-15T00:11:17.366 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-15T00:11:17.366 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-15T00:11:17.366 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-15T00:11:17.366 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_thread_pool_size": "512", 2024-09-15T00:11:17.366 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-15T00:11:17.366 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-15T00:11:17.366 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-15T00:11:17.366 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-15T00:11:17.366 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_torrent_comment": "", 2024-09-15T00:11:17.366 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_torrent_createby": "", 2024-09-15T00:11:17.366 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_torrent_encoding": "", 2024-09-15T00:11:17.366 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_torrent_flag": "false", 2024-09-15T00:11:17.367 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-15T00:11:17.367 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_torrent_origin": "", 2024-09-15T00:11:17.367 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-15T00:11:17.367 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_torrent_tracker": "", 2024-09-15T00:11:17.367 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_trust_forwarded_https": "false", 2024-09-15T00:11:17.367 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-15T00:11:17.367 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-15T00:11:17.367 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_usage_max_shards": "32", 2024-09-15T00:11:17.367 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_usage_max_user_shards": "1", 2024-09-15T00:11:17.367 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_use_opa_authz": "false", 2024-09-15T00:11:17.367 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_user_counters_cache": "false", 2024-09-15T00:11:17.367 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-15T00:11:17.367 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-15T00:11:17.367 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-15T00:11:17.368 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_user_max_buckets": "1000", 2024-09-15T00:11:17.368 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_user_policies_max_num": "100", 2024-09-15T00:11:17.368 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-15T00:11:17.368 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-15T00:11:17.368 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-15T00:11:17.368 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-15T00:11:17.368 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_user_unique_email": "true", 2024-09-15T00:11:17.368 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_verify_ssl": "true", 2024-09-15T00:11:17.368 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-15T00:11:17.368 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_zone": "", 2024-09-15T00:11:17.368 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_zone_id": "", 2024-09-15T00:11:17.368 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-15T00:11:17.368 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_zonegroup": "", 2024-09-15T00:11:17.368 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_zonegroup_id": "", 2024-09-15T00:11:17.369 INFO:teuthology.orchestra.run.smithi171.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-15T00:11:17.369 INFO:teuthology.orchestra.run.smithi171.stdout: "rgwlc_auto_session_clear": "true", 2024-09-15T00:11:17.369 INFO:teuthology.orchestra.run.smithi171.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-15T00:11:17.369 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_block_size": "4096", 2024-09-15T00:11:17.369 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-15T00:11:17.369 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-15T00:11:17.369 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-15T00:11:17.369 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-15T00:11:17.369 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-15T00:11:17.369 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_cache_size": "536870912", 2024-09-15T00:11:17.369 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-15T00:11:17.369 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-15T00:11:17.369 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-15T00:11:17.370 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-15T00:11:17.370 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-15T00:11:17.370 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-15T00:11:17.370 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-15T00:11:17.370 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-15T00:11:17.370 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_index_type": "binary_search", 2024-09-15T00:11:17.370 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-15T00:11:17.370 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-15T00:11:17.370 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_partition_filters": "false", 2024-09-15T00:11:17.370 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_perf": "false", 2024-09-15T00:11:17.370 INFO:teuthology.orchestra.run.smithi171.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-15T00:11:17.370 INFO:teuthology.orchestra.run.smithi171.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-15T00:11:17.370 INFO:teuthology.orchestra.run.smithi171.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-15T00:11:17.370 INFO:teuthology.orchestra.run.smithi171.stdout: "run_dir": "/var/run/ceph", 2024-09-15T00:11:17.371 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_block_create": "true", 2024-09-15T00:11:17.371 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_cache_lru_size": "67108864", 2024-09-15T00:11:17.371 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-15T00:11:17.371 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-15T00:11:17.371 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_default_max_object_size": "16777216", 2024-09-15T00:11:17.371 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-15T00:11:17.371 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_device_size": "53687091200", 2024-09-15T00:11:17.371 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-15T00:11:17.371 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_full_integrity_check": "false", 2024-09-15T00:11:17.371 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_journal_batch_capacity": "16", 2024-09-15T00:11:17.371 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-15T00:11:17.371 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-15T00:11:17.371 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-15T00:11:17.371 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_main_device_type": "SSD", 2024-09-15T00:11:17.371 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-15T00:11:17.372 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-15T00:11:17.372 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-15T00:11:17.372 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-15T00:11:17.372 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-15T00:11:17.372 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-15T00:11:17.372 INFO:teuthology.orchestra.run.smithi171.stdout: "seastore_segment_size": "67108864", 2024-09-15T00:11:17.372 INFO:teuthology.orchestra.run.smithi171.stdout: "set_keepcaps": "false", 2024-09-15T00:11:17.372 INFO:teuthology.orchestra.run.smithi171.stdout: "setgroup": "", 2024-09-15T00:11:17.372 INFO:teuthology.orchestra.run.smithi171.stdout: "setuser": "", 2024-09-15T00:11:17.372 INFO:teuthology.orchestra.run.smithi171.stdout: "setuser_match_path": "", 2024-09-15T00:11:17.372 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-15T00:11:17.372 INFO:teuthology.orchestra.run.smithi171.stdout: "thp": "false", 2024-09-15T00:11:17.373 INFO:teuthology.orchestra.run.smithi171.stdout: "threadpool_default_timeout": "60", 2024-09-15T00:11:17.373 INFO:teuthology.orchestra.run.smithi171.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-15T00:11:17.373 INFO:teuthology.orchestra.run.smithi171.stdout: "throttler_perf_counter": "true" 2024-09-15T00:11:17.373 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:17.375 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-15T00:11:17.487 INFO:teuthology.orchestra.run.smithi171.stdout:mon.a: {} 2024-09-15T00:11:17.487 INFO:teuthology.orchestra.run.smithi171.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T00:11:17.497 INFO:teuthology.orchestra.run.smithi171.stdout:mon.b: {} 2024-09-15T00:11:17.497 INFO:teuthology.orchestra.run.smithi171.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T00:11:17.505 INFO:teuthology.orchestra.run.smithi171.stdout:mon.c: {} 2024-09-15T00:11:17.505 INFO:teuthology.orchestra.run.smithi171.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T00:11:17.516 DEBUG:teuthology.orchestra.run.smithi184:> type ceph-objectstore-tool 2024-09-15T00:11:17.534 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T00:11:17.535 DEBUG:teuthology.orchestra.run.smithi171:> type ceph-objectstore-tool 2024-09-15T00:11:17.549 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T00:11:17.549 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-15T00:11:17.560 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-15T00:11:17.560 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-15T00:11:17.560 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T00:11:17.560 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:11:17.560 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-15T00:11:17.561 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.000000 2024-09-15T00:11:17.561 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0 2024-09-15T00:11:17.561 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-15T00:11:17.561 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-15T00:11:17.562 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:11:17.562 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-15T00:11:17.562 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:17.562 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-15T00:11:17.562 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:11:17.563 DEBUG:teuthology.orchestra.run.smithi171:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-15T00:11:17.592 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:17.589+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:mapped writes DISABLED 2024-09-15T00:11:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:mapped reads DISABLED 2024-09-15T00:11:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:Seed set to 7632 2024-09-15T00:11:17.672 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:17.671+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:17.717 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:17.717 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:17.717 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:17.717 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:17.729 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:11:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:17.749 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:17.768 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:17.769 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:17.767+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:17.787 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:17.803 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:17.804 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:17.821 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:17.821 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:17.839 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:17.839 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:17.856 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:17.856 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:17.857 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:17.857 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:17.864 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:17.863+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:17.866 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:11:17.873 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:17.873 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:17.965 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:17.963+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:17.976 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:17.977 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:17.985 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:18.065 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:18.063+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:18.089 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:18.089 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:18.098 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:18.166 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:18.164+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:18.202 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:18.202 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:18.211 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:18.268 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:18.266+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:18.315 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:18.315 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:18.324 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:18.408 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:18.407+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:18.468 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:18.469 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:18.469 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:18.467+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:18.477 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:18.570 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:18.569+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:18.583 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is set 2024-09-15T00:11:18.588 INFO:teuthology.orchestra.run.smithi171.stderr:set osd.0 primary-affinity to 0 (802) 2024-09-15T00:11:18.628 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:18.629 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:18.638 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:18.679 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:18.677+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:18.772 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:18.771+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:18.781 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:18.781 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:18.790 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:18.874 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:18.873+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:18.934 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:18.934 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:18.942 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:18.974 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:18.972+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:19.075 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:19.074+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:19.088 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:19.088 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:19.096 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:19.176 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:19.174+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:19.200 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:19.200 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:19.209 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:19.276 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:19.275+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:19.323 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:19.324 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:19.332 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:19.377 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:19.375+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:19.436 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:19.436 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:19.445 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:19.478 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:19.477+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:19.579 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:19.577+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:19.590 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:19.591 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:19.606 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:19.680 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:19.678+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:19.753 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:19.753 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:19.763 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:19.807 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:19.805+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:19.868 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:19.868 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:19.877 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:19.881 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:19.879+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:19.886 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:11:19.982 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:19.980+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:20.009 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:20.009 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:20.024 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:20.024 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:20.025 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:20.026 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:20.033 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:11:20.041 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:20.041 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:20.057 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:20.058 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:20.073 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:20.073 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:20.089 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:20.090 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:20.107 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:20.107 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:20.117 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:20.115+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:20.123 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:20.123 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:20.179 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:20.180 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:20.180 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:20.180 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:20.183 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:20.181+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:20.189 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:11:20.284 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:20.282+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:20.334 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:20.334 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:20.335 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:20.335 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:20.335 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:20.343 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:11:20.385 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:20.383+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:20.486 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:20.485+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:20.489 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:20.489 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:20.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:20.489 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [] 2024-09-15T00:11:20.489 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:20.498 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:11:20.587 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:20.586+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:20.604 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:11:20.605 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:20.605 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:20.605 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:20.606 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:20.614 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:11:20.687 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:20.686+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:20.718 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:20.718 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:20.718 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:20.718 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:20.718 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:20.729 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:11:20.788 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:20.787+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:20.835 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:20.835 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:20.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:20.835 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:20.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.835 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.835 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:05.534145+0000", 2024-09-15T00:11:20.835 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.297285587999999, 2024-09-15T00:11:20.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0080234130000000001, 2024-09-15T00:11:20.836 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.836 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.836 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.836 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.836 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.836 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.836 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 70 2024-09-15T00:11:20.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.836 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.534145+0000", 2024-09-15T00:11:20.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.534145+0000", 2024-09-15T00:11:20.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.534146+0000", 2024-09-15T00:11:20.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7e-07 2024-09-15T00:11:20.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.534148+0000", 2024-09-15T00:11:20.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2879999999999999e-06 2024-09-15T00:11:20.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.534151+0000", 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6669999999999998e-06 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.534155+0000", 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1329999999999999e-06 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.535886+0000", 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001731607 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.535936+0000", 2024-09-15T00:11:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9395999999999999e-05 2024-09-15T00:11:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.536023+0000", 2024-09-15T00:11:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7540000000000006e-05 2024-09-15T00:11:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.537864+0000", 2024-09-15T00:11:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018404140000000001 2024-09-15T00:11:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.542072+0000", 2024-09-15T00:11:20.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042080069999999997 2024-09-15T00:11:20.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.542124+0000", 2024-09-15T00:11:20.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2244000000000003e-05 2024-09-15T00:11:20.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.542169+0000", 2024-09-15T00:11:20.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4746999999999997e-05 2024-09-15T00:11:20.840 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.841 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.841 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.841 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.841 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:05.695268+0000", 2024-09-15T00:11:20.841 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.136163099999999, 2024-09-15T00:11:20.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077182240000000001, 2024-09-15T00:11:20.841 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.841 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.841 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.841 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.841 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.841 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.842 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 107 2024-09-15T00:11:20.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.842 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695268+0000", 2024-09-15T00:11:20.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695268+0000", 2024-09-15T00:11:20.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695268+0000", 2024-09-15T00:11:20.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:11:20.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695270+0000", 2024-09-15T00:11:20.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.232e-06 2024-09-15T00:11:20.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695274+0000", 2024-09-15T00:11:20.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7440000000000001e-06 2024-09-15T00:11:20.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695277+0000", 2024-09-15T00:11:20.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6840000000000001e-06 2024-09-15T00:11:20.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.697215+0000", 2024-09-15T00:11:20.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001937956 2024-09-15T00:11:20.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.697301+0000", 2024-09-15T00:11:20.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6334000000000004e-05 2024-09-15T00:11:20.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.697437+0000", 2024-09-15T00:11:20.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013589799999999999 2024-09-15T00:11:20.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.699438+0000", 2024-09-15T00:11:20.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002000862 2024-09-15T00:11:20.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.702934+0000", 2024-09-15T00:11:20.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034960799999999999 2024-09-15T00:11:20.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.702963+0000", 2024-09-15T00:11:20.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9561999999999999e-05 2024-09-15T00:11:20.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.702986+0000", 2024-09-15T00:11:20.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2532e-05 2024-09-15T00:11:20.848 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.848 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.848 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.848 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.848 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:05.695621+0000", 2024-09-15T00:11:20.848 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.135810207, 2024-09-15T00:11:20.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077216819999999997, 2024-09-15T00:11:20.848 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.848 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.848 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.849 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.849 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.849 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.849 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 111 2024-09-15T00:11:20.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.849 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695621+0000", 2024-09-15T00:11:20.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695621+0000", 2024-09-15T00:11:20.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695621+0000", 2024-09-15T00:11:20.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:11:20.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695625+0000", 2024-09-15T00:11:20.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0260000000000001e-06 2024-09-15T00:11:20.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695632+0000", 2024-09-15T00:11:20.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5899999999999996e-06 2024-09-15T00:11:20.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695635+0000", 2024-09-15T00:11:20.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1580000000000001e-06 2024-09-15T00:11:20.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.699850+0000", 2024-09-15T00:11:20.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042153620000000003 2024-09-15T00:11:20.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.699927+0000", 2024-09-15T00:11:20.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6730999999999994e-05 2024-09-15T00:11:20.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.700013+0000", 2024-09-15T00:11:20.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6267000000000006e-05 2024-09-15T00:11:20.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703167+0000", 2024-09-15T00:11:20.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031542879999999999 2024-09-15T00:11:20.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703298+0000", 2024-09-15T00:11:20.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013085000000000001 2024-09-15T00:11:20.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703324+0000", 2024-09-15T00:11:20.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5346000000000001e-05 2024-09-15T00:11:20.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703342+0000", 2024-09-15T00:11:20.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8658999999999999e-05 2024-09-15T00:11:20.854 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.854 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.854 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.855 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.855 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:05.695639+0000", 2024-09-15T00:11:20.855 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.135791305, 2024-09-15T00:11:20.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007779317, 2024-09-15T00:11:20.855 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.855 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.855 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.855 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.855 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.855 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.855 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 112 2024-09-15T00:11:20.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.855 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695639+0000", 2024-09-15T00:11:20.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695639+0000", 2024-09-15T00:11:20.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695640+0000", 2024-09-15T00:11:20.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6199999999999999e-07 2024-09-15T00:11:20.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695685+0000", 2024-09-15T00:11:20.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5549e-05 2024-09-15T00:11:20.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695693+0000", 2024-09-15T00:11:20.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6009999999999997e-06 2024-09-15T00:11:20.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695698+0000", 2024-09-15T00:11:20.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:11:20.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.700336+0000", 2024-09-15T00:11:20.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0046379059999999998 2024-09-15T00:11:20.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.700383+0000", 2024-09-15T00:11:20.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6585000000000002e-05 2024-09-15T00:11:20.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.700456+0000", 2024-09-15T00:11:20.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3473000000000006e-05 2024-09-15T00:11:20.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703180+0000", 2024-09-15T00:11:20.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0027236119999999998 2024-09-15T00:11:20.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703373+0000", 2024-09-15T00:11:20.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019360300000000001 2024-09-15T00:11:20.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703399+0000", 2024-09-15T00:11:20.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5772999999999999e-05 2024-09-15T00:11:20.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703419+0000", 2024-09-15T00:11:20.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9483999999999998e-05 2024-09-15T00:11:20.861 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.861 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.861 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.861 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.861 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:05.695708+0000", 2024-09-15T00:11:20.861 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.135722564, 2024-09-15T00:11:20.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077838869999999998, 2024-09-15T00:11:20.862 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.862 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.862 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.862 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 113 2024-09-15T00:11:20.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.862 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695708+0000", 2024-09-15T00:11:20.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695708+0000", 2024-09-15T00:11:20.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695709+0000", 2024-09-15T00:11:20.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:11:20.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695711+0000", 2024-09-15T00:11:20.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8109999999999999e-06 2024-09-15T00:11:20.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695717+0000", 2024-09-15T00:11:20.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5300000000000004e-06 2024-09-15T00:11:20.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695720+0000", 2024-09-15T00:11:20.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8940000000000002e-06 2024-09-15T00:11:20.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.700937+0000", 2024-09-15T00:11:20.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052176870000000004 2024-09-15T00:11:20.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.700984+0000", 2024-09-15T00:11:20.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6105e-05 2024-09-15T00:11:20.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.701057+0000", 2024-09-15T00:11:20.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3903e-05 2024-09-15T00:11:20.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703185+0000", 2024-09-15T00:11:20.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002127192 2024-09-15T00:11:20.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703448+0000", 2024-09-15T00:11:20.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00026328099999999998 2024-09-15T00:11:20.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703472+0000", 2024-09-15T00:11:20.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.442e-05 2024-09-15T00:11:20.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703492+0000", 2024-09-15T00:11:20.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9653e-05 2024-09-15T00:11:20.867 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.867 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.867 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.868 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.868 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:05.695725+0000", 2024-09-15T00:11:20.868 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.135705679000001, 2024-09-15T00:11:20.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0078451630000000005, 2024-09-15T00:11:20.868 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.868 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.868 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.868 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.868 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.868 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.868 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 114 2024-09-15T00:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695725+0000", 2024-09-15T00:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695725+0000", 2024-09-15T00:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695725+0000", 2024-09-15T00:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3999999999999998e-07 2024-09-15T00:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695728+0000", 2024-09-15T00:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2699999999999999e-06 2024-09-15T00:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695731+0000", 2024-09-15T00:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6040000000000001e-06 2024-09-15T00:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695734+0000", 2024-09-15T00:11:20.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7410000000000001e-06 2024-09-15T00:11:20.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.701277+0000", 2024-09-15T00:11:20.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055427819999999996 2024-09-15T00:11:20.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.701324+0000", 2024-09-15T00:11:20.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7074999999999999e-05 2024-09-15T00:11:20.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.701399+0000", 2024-09-15T00:11:20.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5562000000000006e-05 2024-09-15T00:11:20.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703191+0000", 2024-09-15T00:11:20.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0017916620000000001 2024-09-15T00:11:20.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703527+0000", 2024-09-15T00:11:20.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00033627400000000001 2024-09-15T00:11:20.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703553+0000", 2024-09-15T00:11:20.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5780000000000001e-05 2024-09-15T00:11:20.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703570+0000", 2024-09-15T00:11:20.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7173e-05 2024-09-15T00:11:20.874 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.874 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.874 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.875 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.875 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:05.695773+0000", 2024-09-15T00:11:20.875 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.135657747, 2024-09-15T00:11:20.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007874107, 2024-09-15T00:11:20.875 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.875 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.875 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.875 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.875 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.875 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.875 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 115 2024-09-15T00:11:20.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.876 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695773+0000", 2024-09-15T00:11:20.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695773+0000", 2024-09-15T00:11:20.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695773+0000", 2024-09-15T00:11:20.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9700000000000003e-07 2024-09-15T00:11:20.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695808+0000", 2024-09-15T00:11:20.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4552999999999999e-05 2024-09-15T00:11:20.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695814+0000", 2024-09-15T00:11:20.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3400000000000003e-06 2024-09-15T00:11:20.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695819+0000", 2024-09-15T00:11:20.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:11:20.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.701610+0000", 2024-09-15T00:11:20.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057910360000000003 2024-09-15T00:11:20.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.701658+0000", 2024-09-15T00:11:20.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7361000000000003e-05 2024-09-15T00:11:20.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.701729+0000", 2024-09-15T00:11:20.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1435999999999998e-05 2024-09-15T00:11:20.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703196+0000", 2024-09-15T00:11:20.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0014666 2024-09-15T00:11:20.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703601+0000", 2024-09-15T00:11:20.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00040563399999999999 2024-09-15T00:11:20.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703627+0000", 2024-09-15T00:11:20.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5321999999999999e-05 2024-09-15T00:11:20.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703647+0000", 2024-09-15T00:11:20.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0389000000000001e-05 2024-09-15T00:11:20.881 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.881 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.882 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.882 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.882 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:05.695836+0000", 2024-09-15T00:11:20.882 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.135595149, 2024-09-15T00:11:20.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007886278, 2024-09-15T00:11:20.882 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.882 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.882 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.883 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 116 2024-09-15T00:11:20.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.883 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695836+0000", 2024-09-15T00:11:20.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695836+0000", 2024-09-15T00:11:20.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695836+0000", 2024-09-15T00:11:20.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4299999999999998e-07 2024-09-15T00:11:20.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695839+0000", 2024-09-15T00:11:20.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6189999999999998e-06 2024-09-15T00:11:20.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695843+0000", 2024-09-15T00:11:20.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1849999999999997e-06 2024-09-15T00:11:20.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695846+0000", 2024-09-15T00:11:20.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.875e-06 2024-09-15T00:11:20.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.702200+0000", 2024-09-15T00:11:20.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0063547250000000003 2024-09-15T00:11:20.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.702245+0000", 2024-09-15T00:11:20.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4912e-05 2024-09-15T00:11:20.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.702345+0000", 2024-09-15T00:11:20.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9701e-05 2024-09-15T00:11:20.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703500+0000", 2024-09-15T00:11:20.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0011548820000000001 2024-09-15T00:11:20.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703677+0000", 2024-09-15T00:11:20.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000176842 2024-09-15T00:11:20.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703702+0000", 2024-09-15T00:11:20.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5434000000000001e-05 2024-09-15T00:11:20.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703722+0000", 2024-09-15T00:11:20.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9660000000000002e-05 2024-09-15T00:11:20.889 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.889 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.889 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.889 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.889 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:05.695850+0000", 2024-09-15T00:11:20.889 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.135580702, 2024-09-15T00:11:20.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0079411719999999998, 2024-09-15T00:11:20.889 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.889 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.889 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.890 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 117 2024-09-15T00:11:20.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.890 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695850+0000", 2024-09-15T00:11:20.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695850+0000", 2024-09-15T00:11:20.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695850+0000", 2024-09-15T00:11:20.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.41e-07 2024-09-15T00:11:20.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695853+0000", 2024-09-15T00:11:20.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.999e-06 2024-09-15T00:11:20.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695857+0000", 2024-09-15T00:11:20.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.087e-06 2024-09-15T00:11:20.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.695861+0000", 2024-09-15T00:11:20.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2940000000000001e-06 2024-09-15T00:11:20.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.702589+0000", 2024-09-15T00:11:20.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0067282590000000003 2024-09-15T00:11:20.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.702639+0000", 2024-09-15T00:11:20.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0373e-05 2024-09-15T00:11:20.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.702720+0000", 2024-09-15T00:11:20.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0876000000000003e-05 2024-09-15T00:11:20.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703744+0000", 2024-09-15T00:11:20.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0010239470000000001 2024-09-15T00:11:20.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703750+0000", 2024-09-15T00:11:20.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9070000000000004e-06 2024-09-15T00:11:20.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703774+0000", 2024-09-15T00:11:20.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4107000000000001e-05 2024-09-15T00:11:20.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.703791+0000", 2024-09-15T00:11:20.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7082e-05 2024-09-15T00:11:20.895 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.896 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.896 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.896 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.896 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056375+0000", 2024-09-15T00:11:20.896 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.775056119, 2024-09-15T00:11:20.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0090326620000000003, 2024-09-15T00:11:20.896 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.896 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.896 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.897 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.897 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.897 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.897 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 136 2024-09-15T00:11:20.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.897 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056375+0000", 2024-09-15T00:11:20.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056375+0000", 2024-09-15T00:11:20.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056375+0000", 2024-09-15T00:11:20.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9900000000000001e-07 2024-09-15T00:11:20.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056379+0000", 2024-09-15T00:11:20.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7759999999999999e-06 2024-09-15T00:11:20.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056390+0000", 2024-09-15T00:11:20.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0720000000000001e-05 2024-09-15T00:11:20.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056395+0000", 2024-09-15T00:11:20.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.558e-06 2024-09-15T00:11:20.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.059634+0000", 2024-09-15T00:11:20.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032390890000000001 2024-09-15T00:11:20.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.059682+0000", 2024-09-15T00:11:20.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7868000000000003e-05 2024-09-15T00:11:20.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.059761+0000", 2024-09-15T00:11:20.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8863999999999996e-05 2024-09-15T00:11:20.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061758+0000", 2024-09-15T00:11:20.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00199731 2024-09-15T00:11:20.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065345+0000", 2024-09-15T00:11:20.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035862989999999998 2024-09-15T00:11:20.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065383+0000", 2024-09-15T00:11:20.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8408999999999998e-05 2024-09-15T00:11:20.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065407+0000", 2024-09-15T00:11:20.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.427e-05 2024-09-15T00:11:20.903 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.904 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.904 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.904 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.904 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056408+0000", 2024-09-15T00:11:20.904 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.775022652000001, 2024-09-15T00:11:20.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0090848679999999994, 2024-09-15T00:11:20.904 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.904 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.904 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.905 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.905 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.905 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.905 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 137 2024-09-15T00:11:20.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.905 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056408+0000", 2024-09-15T00:11:20.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056408+0000", 2024-09-15T00:11:20.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056409+0000", 2024-09-15T00:11:20.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0100000000000005e-07 2024-09-15T00:11:20.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056478+0000", 2024-09-15T00:11:20.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9580999999999997e-05 2024-09-15T00:11:20.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056485+0000", 2024-09-15T00:11:20.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3459999999999996e-06 2024-09-15T00:11:20.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056491+0000", 2024-09-15T00:11:20.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.417e-06 2024-09-15T00:11:20.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.060060+0000", 2024-09-15T00:11:20.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003569511 2024-09-15T00:11:20.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.060108+0000", 2024-09-15T00:11:20.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7788999999999999e-05 2024-09-15T00:11:20.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.060180+0000", 2024-09-15T00:11:20.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2248000000000004e-05 2024-09-15T00:11:20.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065304+0000", 2024-09-15T00:11:20.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051239550000000003 2024-09-15T00:11:20.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065443+0000", 2024-09-15T00:11:20.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000138632 2024-09-15T00:11:20.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065472+0000", 2024-09-15T00:11:20.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8459999999999999e-05 2024-09-15T00:11:20.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065493+0000", 2024-09-15T00:11:20.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1427999999999999e-05 2024-09-15T00:11:20.911 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.911 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.911 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.911 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.911 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056499+0000", 2024-09-15T00:11:20.911 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.774931904000001, 2024-09-15T00:11:20.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0090724829999999992, 2024-09-15T00:11:20.911 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.911 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.911 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.912 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.912 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.912 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.912 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 138 2024-09-15T00:11:20.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.912 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056499+0000", 2024-09-15T00:11:20.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056499+0000", 2024-09-15T00:11:20.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056499+0000", 2024-09-15T00:11:20.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:11:20.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056502+0000", 2024-09-15T00:11:20.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4739999999999999e-06 2024-09-15T00:11:20.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056506+0000", 2024-09-15T00:11:20.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2470000000000002e-06 2024-09-15T00:11:20.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056509+0000", 2024-09-15T00:11:20.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8729999999999999e-06 2024-09-15T00:11:20.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061051+0000", 2024-09-15T00:11:20.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045423410000000001 2024-09-15T00:11:20.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061105+0000", 2024-09-15T00:11:20.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4101000000000003e-05 2024-09-15T00:11:20.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061178+0000", 2024-09-15T00:11:20.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3104999999999999e-05 2024-09-15T00:11:20.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065310+0000", 2024-09-15T00:11:20.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041314990000000003 2024-09-15T00:11:20.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065525+0000", 2024-09-15T00:11:20.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000214947 2024-09-15T00:11:20.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065551+0000", 2024-09-15T00:11:20.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5955000000000001e-05 2024-09-15T00:11:20.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065571+0000", 2024-09-15T00:11:20.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0594e-05 2024-09-15T00:11:20.918 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.918 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.918 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.918 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.918 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056512+0000", 2024-09-15T00:11:20.918 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.774918273000001, 2024-09-15T00:11:20.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.009307727, 2024-09-15T00:11:20.918 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.918 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.919 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.919 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.919 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.919 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.919 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 139 2024-09-15T00:11:20.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.919 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056512+0000", 2024-09-15T00:11:20.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056512+0000", 2024-09-15T00:11:20.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056513+0000", 2024-09-15T00:11:20.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5100000000000001e-07 2024-09-15T00:11:20.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056515+0000", 2024-09-15T00:11:20.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.26e-06 2024-09-15T00:11:20.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056519+0000", 2024-09-15T00:11:20.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0300000000000004e-06 2024-09-15T00:11:20.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056523+0000", 2024-09-15T00:11:20.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2379999999999999e-06 2024-09-15T00:11:20.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061418+0000", 2024-09-15T00:11:20.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048945630000000002 2024-09-15T00:11:20.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061474+0000", 2024-09-15T00:11:20.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6688999999999999e-05 2024-09-15T00:11:20.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061554+0000", 2024-09-15T00:11:20.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9810999999999999e-05 2024-09-15T00:11:20.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065502+0000", 2024-09-15T00:11:20.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039479720000000001 2024-09-15T00:11:20.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065776+0000", 2024-09-15T00:11:20.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00027390599999999998 2024-09-15T00:11:20.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065801+0000", 2024-09-15T00:11:20.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5219999999999999e-05 2024-09-15T00:11:20.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065820+0000", 2024-09-15T00:11:20.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8787000000000002e-05 2024-09-15T00:11:20.925 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.925 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.925 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.925 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.925 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:20.926 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.774903068, 2024-09-15T00:11:20.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0096064029999999995, 2024-09-15T00:11:20.926 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.926 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.926 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.926 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.926 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.926 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.926 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 140 2024-09-15T00:11:20.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.926 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:20.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:20.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:20.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4499999999999998e-07 2024-09-15T00:11:20.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056583+0000", 2024-09-15T00:11:20.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5377000000000002e-05 2024-09-15T00:11:20.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056588+0000", 2024-09-15T00:11:20.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0209999999999999e-06 2024-09-15T00:11:20.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056591+0000", 2024-09-15T00:11:20.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1089999999999998e-06 2024-09-15T00:11:20.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061790+0000", 2024-09-15T00:11:20.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051988759999999998 2024-09-15T00:11:20.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061837+0000", 2024-09-15T00:11:20.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6233000000000003e-05 2024-09-15T00:11:20.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061927+0000", 2024-09-15T00:11:20.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0525999999999994e-05 2024-09-15T00:11:20.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065584+0000", 2024-09-15T00:11:20.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036566630000000001 2024-09-15T00:11:20.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066084+0000", 2024-09-15T00:11:20.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000500557 2024-09-15T00:11:20.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066116+0000", 2024-09-15T00:11:20.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1272000000000001e-05 2024-09-15T00:11:20.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066134+0000", 2024-09-15T00:11:20.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8524000000000001e-05 2024-09-15T00:11:20.932 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.932 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.932 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.932 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.932 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:20.933 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.774833686000001, 2024-09-15T00:11:20.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0096621280000000007, 2024-09-15T00:11:20.933 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.933 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.933 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.933 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.933 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.934 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.934 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 141 2024-09-15T00:11:20.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.934 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:20.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:20.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:20.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9299999999999999e-07 2024-09-15T00:11:20.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056600+0000", 2024-09-15T00:11:20.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5330000000000001e-06 2024-09-15T00:11:20.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056606+0000", 2024-09-15T00:11:20.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5030000000000002e-06 2024-09-15T00:11:20.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056610+0000", 2024-09-15T00:11:20.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2279999999999998e-06 2024-09-15T00:11:20.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.064980+0000", 2024-09-15T00:11:20.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008370097 2024-09-15T00:11:20.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065030+0000", 2024-09-15T00:11:20.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0268000000000001e-05 2024-09-15T00:11:20.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065102+0000", 2024-09-15T00:11:20.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2120999999999996e-05 2024-09-15T00:11:20.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066159+0000", 2024-09-15T00:11:20.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001056881 2024-09-15T00:11:20.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066217+0000", 2024-09-15T00:11:20.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8344999999999998e-05 2024-09-15T00:11:20.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066241+0000", 2024-09-15T00:11:20.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3773000000000001e-05 2024-09-15T00:11:20.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066259+0000", 2024-09-15T00:11:20.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8085999999999999e-05 2024-09-15T00:11:20.939 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.939 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.939 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.940 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:155 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.940 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523404+0000", 2024-09-15T00:11:20.940 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.308026505999999, 2024-09-15T00:11:20.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092202580000000003, 2024-09-15T00:11:20.940 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.940 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.940 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.940 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.941 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 155 2024-09-15T00:11:20.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.941 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523404+0000", 2024-09-15T00:11:20.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523404+0000", 2024-09-15T00:11:20.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523405+0000", 2024-09-15T00:11:20.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:11:20.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523523+0000", 2024-09-15T00:11:20.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00011842500000000001 2024-09-15T00:11:20.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523531+0000", 2024-09-15T00:11:20.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7649999999999993e-06 2024-09-15T00:11:20.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523544+0000", 2024-09-15T00:11:20.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3243e-05 2024-09-15T00:11:20.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.524482+0000", 2024-09-15T00:11:20.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00093750599999999995 2024-09-15T00:11:20.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.524527+0000", 2024-09-15T00:11:20.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5296000000000001e-05 2024-09-15T00:11:20.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.524600+0000", 2024-09-15T00:11:20.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3271999999999999e-05 2024-09-15T00:11:20.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527438+0000", 2024-09-15T00:11:20.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002837783 2024-09-15T00:11:20.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532545+0000", 2024-09-15T00:11:20.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051066690000000003 2024-09-15T00:11:20.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532590+0000", 2024-09-15T00:11:20.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4929999999999998e-05 2024-09-15T00:11:20.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532624+0000", 2024-09-15T00:11:20.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4838000000000001e-05 2024-09-15T00:11:20.946 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.946 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.946 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.946 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.947 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523563+0000", 2024-09-15T00:11:20.947 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.307867787999999, 2024-09-15T00:11:20.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0091540000000000007, 2024-09-15T00:11:20.947 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.947 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.947 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.947 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.947 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.947 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.947 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 156 2024-09-15T00:11:20.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.948 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523563+0000", 2024-09-15T00:11:20.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523563+0000", 2024-09-15T00:11:20.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523564+0000", 2024-09-15T00:11:20.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9800000000000003e-07 2024-09-15T00:11:20.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523572+0000", 2024-09-15T00:11:20.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7180000000000002e-06 2024-09-15T00:11:20.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523580+0000", 2024-09-15T00:11:20.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9799999999999998e-06 2024-09-15T00:11:20.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523586+0000", 2024-09-15T00:11:20.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.251e-06 2024-09-15T00:11:20.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525453+0000", 2024-09-15T00:11:20.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001866604 2024-09-15T00:11:20.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525496+0000", 2024-09-15T00:11:20.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3222000000000001e-05 2024-09-15T00:11:20.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525569+0000", 2024-09-15T00:11:20.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2408000000000002e-05 2024-09-15T00:11:20.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527444+0000", 2024-09-15T00:11:20.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001875294 2024-09-15T00:11:20.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532664+0000", 2024-09-15T00:11:20.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052197759999999998 2024-09-15T00:11:20.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532694+0000", 2024-09-15T00:11:20.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0020000000000001e-05 2024-09-15T00:11:20.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532717+0000", 2024-09-15T00:11:20.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3129000000000002e-05 2024-09-15T00:11:20.953 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.953 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.953 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.954 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.954 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:20.954 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.307832828, 2024-09-15T00:11:20.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092491780000000003, 2024-09-15T00:11:20.954 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.954 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.954 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.954 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.954 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.954 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.954 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 157 2024-09-15T00:11:20.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.954 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:20.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:20.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:20.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0500000000000004e-07 2024-09-15T00:11:20.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523602+0000", 2024-09-15T00:11:20.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5930000000000002e-06 2024-09-15T00:11:20.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523609+0000", 2024-09-15T00:11:20.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9970000000000001e-06 2024-09-15T00:11:20.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523614+0000", 2024-09-15T00:11:20.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8640000000000004e-06 2024-09-15T00:11:20.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525797+0000", 2024-09-15T00:11:20.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021830500000000002 2024-09-15T00:11:20.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525880+0000", 2024-09-15T00:11:20.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3394999999999997e-05 2024-09-15T00:11:20.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525997+0000", 2024-09-15T00:11:20.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000116971 2024-09-15T00:11:20.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527448+0000", 2024-09-15T00:11:20.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0014509690000000001 2024-09-15T00:11:20.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532758+0000", 2024-09-15T00:11:20.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053093319999999999 2024-09-15T00:11:20.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532804+0000", 2024-09-15T00:11:20.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5995000000000003e-05 2024-09-15T00:11:20.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532847+0000", 2024-09-15T00:11:20.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3507000000000002e-05 2024-09-15T00:11:20.959 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.959 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.959 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.960 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.960 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523622+0000", 2024-09-15T00:11:20.960 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.307808353, 2024-09-15T00:11:20.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0093097089999999993, 2024-09-15T00:11:20.960 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.960 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.960 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.960 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.960 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.960 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.960 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 158 2024-09-15T00:11:20.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.961 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523622+0000", 2024-09-15T00:11:20.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523622+0000", 2024-09-15T00:11:20.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523623+0000", 2024-09-15T00:11:20.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5400000000000002e-07 2024-09-15T00:11:20.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523689+0000", 2024-09-15T00:11:20.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6434999999999997e-05 2024-09-15T00:11:20.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523698+0000", 2024-09-15T00:11:20.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2229999999999996e-06 2024-09-15T00:11:20.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523705+0000", 2024-09-15T00:11:20.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6969999999999996e-06 2024-09-15T00:11:20.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526370+0000", 2024-09-15T00:11:20.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026647989999999998 2024-09-15T00:11:20.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526415+0000", 2024-09-15T00:11:20.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5278999999999998e-05 2024-09-15T00:11:20.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526484+0000", 2024-09-15T00:11:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8487999999999999e-05 2024-09-15T00:11:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.528098+0000", 2024-09-15T00:11:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0016147480000000001 2024-09-15T00:11:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532887+0000", 2024-09-15T00:11:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047889860000000003 2024-09-15T00:11:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532912+0000", 2024-09-15T00:11:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4694000000000001e-05 2024-09-15T00:11:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532932+0000", 2024-09-15T00:11:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9905999999999999e-05 2024-09-15T00:11:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.30770004, 2024-09-15T00:11:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0093142290000000003, 2024-09-15T00:11:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 159 2024-09-15T00:11:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6899999999999997e-07 2024-09-15T00:11:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523735+0000", 2024-09-15T00:11:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7359999999999999e-06 2024-09-15T00:11:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523745+0000", 2024-09-15T00:11:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0149000000000001e-05 2024-09-15T00:11:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523752+0000", 2024-09-15T00:11:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7220000000000002e-06 2024-09-15T00:11:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526967+0000", 2024-09-15T00:11:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032149280000000001 2024-09-15T00:11:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527039+0000", 2024-09-15T00:11:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2261000000000005e-05 2024-09-15T00:11:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527148+0000", 2024-09-15T00:11:20.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000108743 2024-09-15T00:11:20.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:20.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.528988+0000", 2024-09-15T00:11:20.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001840008 2024-09-15T00:11:20.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:20.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532965+0000", 2024-09-15T00:11:20.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039768980000000004 2024-09-15T00:11:20.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:20.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.533008+0000", 2024-09-15T00:11:20.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3673e-05 2024-09-15T00:11:20.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:20.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:20.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:20.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.533045+0000", 2024-09-15T00:11:20.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6541999999999997e-05 2024-09-15T00:11:20.971 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.971 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.971 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.971 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:20.971 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:20.971 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:20.973 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:20.974 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:20.971+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:21.074 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:21.072+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:21.077 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:21.077 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:21.078 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:21.078 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:21.086 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:11:21.175 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:21.174+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:21.189 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:21.189 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:21.189 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:21.190 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:21.190 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:21.198 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:11:21.276 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:21.275+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:21.303 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-15T00:11:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:11:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.765403574, 2024-09-15T00:11:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0035574980000000001, 2024-09-15T00:11:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:11:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:11:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535370+0000", 2024-09-15T00:11:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-15T00:11:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535438+0000", 2024-09-15T00:11:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199000000000002e-05 2024-09-15T00:11:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535446+0000", 2024-09-15T00:11:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1640000000000001e-06 2024-09-15T00:11:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535450+0000", 2024-09-15T00:11:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1520000000000002e-06 2024-09-15T00:11:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535491+0000", 2024-09-15T00:11:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0969e-05 2024-09-15T00:11:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535510+0000", 2024-09-15T00:11:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9031000000000001e-05 2024-09-15T00:11:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.538893+0000", 2024-09-15T00:11:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033823870000000002 2024-09-15T00:11:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.538927+0000", 2024-09-15T00:11:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4085999999999999e-05 2024-09-15T00:11:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:70 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-09-15T00:11:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:11:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.764663354, 2024-09-15T00:11:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057871490000000001, 2024-09-15T00:11:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:11:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:11:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536110+0000", 2024-09-15T00:11:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-15T00:11:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536115+0000", 2024-09-15T00:11:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8960000000000002e-06 2024-09-15T00:11:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536123+0000", 2024-09-15T00:11:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3979999999999999e-06 2024-09-15T00:11:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536128+0000", 2024-09-15T00:11:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5569999999999997e-06 2024-09-15T00:11:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537011+0000", 2024-09-15T00:11:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088276299999999995 2024-09-15T00:11:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537034+0000", 2024-09-15T00:11:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3448999999999999e-05 2024-09-15T00:11:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541867+0000", 2024-09-15T00:11:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048332949999999996 2024-09-15T00:11:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541896+0000", 2024-09-15T00:11:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9179999999999998e-05 2024-09-15T00:11:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-15T00:11:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.737044901000001, 2024-09-15T00:11:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027567730000000001, 2024-09-15T00:11:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5599999999999995e-07 2024-09-15T00:11:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563787+0000", 2024-09-15T00:11:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9092000000000003e-05 2024-09-15T00:11:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563796+0000", 2024-09-15T00:11:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0059999999999998e-06 2024-09-15T00:11:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563801+0000", 2024-09-15T00:11:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3290000000000004e-06 2024-09-15T00:11:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563830+0000", 2024-09-15T00:11:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9369999999999998e-05 2024-09-15T00:11:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563842+0000", 2024-09-15T00:11:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1646e-05 2024-09-15T00:11:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.566459+0000", 2024-09-15T00:11:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026174549999999999 2024-09-15T00:11:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.566485+0000", 2024-09-15T00:11:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5318999999999999e-05 2024-09-15T00:11:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:106 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-09-15T00:11:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:11:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.604142438, 2024-09-15T00:11:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029384250000000002, 2024-09-15T00:11:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:11:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:11:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696631+0000", 2024-09-15T00:11:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4299999999999998e-07 2024-09-15T00:11:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696692+0000", 2024-09-15T00:11:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1229999999999995e-05 2024-09-15T00:11:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696699+0000", 2024-09-15T00:11:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5399999999999998e-06 2024-09-15T00:11:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696705+0000", 2024-09-15T00:11:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5770000000000001e-06 2024-09-15T00:11:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696732+0000", 2024-09-15T00:11:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7022e-05 2024-09-15T00:11:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696743+0000", 2024-09-15T00:11:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1426999999999999e-05 2024-09-15T00:11:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699542+0000", 2024-09-15T00:11:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002798521 2024-09-15T00:11:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699569+0000", 2024-09-15T00:11:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6665e-05 2024-09-15T00:11:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:107 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-09-15T00:11:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.603247008, 2024-09-15T00:11:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048949229999999998, 2024-09-15T00:11:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5799999999999999e-07 2024-09-15T00:11:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697531+0000", 2024-09-15T00:11:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3100000000000002e-06 2024-09-15T00:11:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697539+0000", 2024-09-15T00:11:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6410000000000008e-06 2024-09-15T00:11:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697543+0000", 2024-09-15T00:11:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1060000000000002e-06 2024-09-15T00:11:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697573+0000", 2024-09-15T00:11:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9978e-05 2024-09-15T00:11:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697585+0000", 2024-09-15T00:11:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1279e-05 2024-09-15T00:11:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702398+0000", 2024-09-15T00:11:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004812981 2024-09-15T00:11:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702421+0000", 2024-09-15T00:11:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3070000000000001e-05 2024-09-15T00:11:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:108 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'66)", 2024-09-15T00:11:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:11:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.602353517999999, 2024-09-15T00:11:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040179439999999999, 2024-09-15T00:11:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:11:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:11:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698420+0000", 2024-09-15T00:11:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0900000000000002e-07 2024-09-15T00:11:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698423+0000", 2024-09-15T00:11:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8319999999999999e-06 2024-09-15T00:11:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698431+0000", 2024-09-15T00:11:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8949999999999998e-06 2024-09-15T00:11:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698435+0000", 2024-09-15T00:11:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5939999999999998e-06 2024-09-15T00:11:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698469+0000", 2024-09-15T00:11:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4258000000000003e-05 2024-09-15T00:11:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698481+0000", 2024-09-15T00:11:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1763000000000001e-05 2024-09-15T00:11:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702422+0000", 2024-09-15T00:11:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003940603 2024-09-15T00:11:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702437+0000", 2024-09-15T00:11:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5489999999999999e-05 2024-09-15T00:11:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:109 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'66)", 2024-09-15T00:11:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.601823081999999, 2024-09-15T00:11:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038555479999999999, 2024-09-15T00:11:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.13e-07 2024-09-15T00:11:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699018+0000", 2024-09-15T00:11:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7540999999999996e-05 2024-09-15T00:11:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699024+0000", 2024-09-15T00:11:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4869999999999999e-06 2024-09-15T00:11:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699044+0000", 2024-09-15T00:11:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9400999999999999e-05 2024-09-15T00:11:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699104+0000", 2024-09-15T00:11:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0331999999999998e-05 2024-09-15T00:11:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699117+0000", 2024-09-15T00:11:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2789000000000001e-05 2024-09-15T00:11:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702777+0000", 2024-09-15T00:11:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0036604900000000002 2024-09-15T00:11:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702805+0000", 2024-09-15T00:11:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7994999999999999e-05 2024-09-15T00:11:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:110 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'72, mlcod=16'66)", 2024-09-15T00:11:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:11:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.601063548999999, 2024-09-15T00:11:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031108189999999999, 2024-09-15T00:11:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:11:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:11:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699710+0000", 2024-09-15T00:11:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:11:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699714+0000", 2024-09-15T00:11:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0010000000000003e-06 2024-09-15T00:11:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699722+0000", 2024-09-15T00:11:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4840000000000004e-06 2024-09-15T00:11:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699726+0000", 2024-09-15T00:11:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:11:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699755+0000", 2024-09-15T00:11:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8433999999999999e-05 2024-09-15T00:11:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699775+0000", 2024-09-15T00:11:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0017000000000002e-05 2024-09-15T00:11:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702807+0000", 2024-09-15T00:11:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030320239999999999 2024-09-15T00:11:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702820+0000", 2024-09-15T00:11:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3169999999999999e-05 2024-09-15T00:11:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:111 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'73, mlcod=16'66)", 2024-09-15T00:11:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.700088+0000", 2024-09-15T00:11:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.600684668, 2024-09-15T00:11:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027485500000000002, 2024-09-15T00:11:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.700088+0000", 2024-09-15T00:11:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.700088+0000", 2024-09-15T00:11:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.700088+0000", 2024-09-15T00:11:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4999999999999998e-07 2024-09-15T00:11:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.700092+0000", 2024-09-15T00:11:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.095e-06 2024-09-15T00:11:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.700098+0000", 2024-09-15T00:11:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0349999999999998e-06 2024-09-15T00:11:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.700101+0000", 2024-09-15T00:11:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.061e-06 2024-09-15T00:11:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.700125+0000", 2024-09-15T00:11:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4814e-05 2024-09-15T00:11:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.700136+0000", 2024-09-15T00:11:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0533999999999999e-05 2024-09-15T00:11:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702821+0000", 2024-09-15T00:11:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026845269999999999 2024-09-15T00:11:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702837+0000", 2024-09-15T00:11:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6033999999999999e-05 2024-09-15T00:11:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:120 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'82, mlcod=16'78)", 2024-09-15T00:11:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:11:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.595160203000001, 2024-09-15T00:11:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028965829999999999, 2024-09-15T00:11:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:11:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:11:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705613+0000", 2024-09-15T00:11:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1800000000000001e-07 2024-09-15T00:11:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705684+0000", 2024-09-15T00:11:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0903999999999997e-05 2024-09-15T00:11:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705691+0000", 2024-09-15T00:11:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4290000000000004e-06 2024-09-15T00:11:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705695+0000", 2024-09-15T00:11:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4790000000000001e-06 2024-09-15T00:11:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705729+0000", 2024-09-15T00:11:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4415999999999997e-05 2024-09-15T00:11:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705741+0000", 2024-09-15T00:11:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1443999999999999e-05 2024-09-15T00:11:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.708476+0000", 2024-09-15T00:11:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027356630000000002 2024-09-15T00:11:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.708509+0000", 2024-09-15T00:11:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2830000000000002e-05 2024-09-15T00:11:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-15T00:11:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.594373738, 2024-09-15T00:11:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047824809999999999, 2024-09-15T00:11:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:11:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706471+0000", 2024-09-15T00:11:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1496999999999997e-05 2024-09-15T00:11:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706478+0000", 2024-09-15T00:11:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7460000000000003e-06 2024-09-15T00:11:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706481+0000", 2024-09-15T00:11:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.258e-06 2024-09-15T00:11:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706513+0000", 2024-09-15T00:11:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2567000000000002e-05 2024-09-15T00:11:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706524+0000", 2024-09-15T00:11:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1028e-05 2024-09-15T00:11:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.711150+0000", 2024-09-15T00:11:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046259450000000002 2024-09-15T00:11:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.711181+0000", 2024-09-15T00:11:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1041000000000003e-05 2024-09-15T00:11:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:136 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-15T00:11:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.240936804, 2024-09-15T00:11:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031270970000000001, 2024-09-15T00:11:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3000000000000002e-07 2024-09-15T00:11:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059839+0000", 2024-09-15T00:11:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3050000000000001e-06 2024-09-15T00:11:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059847+0000", 2024-09-15T00:11:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.622e-06 2024-09-15T00:11:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059851+0000", 2024-09-15T00:11:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8120000000000001e-06 2024-09-15T00:11:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059884+0000", 2024-09-15T00:11:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2620999999999998e-05 2024-09-15T00:11:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059894+0000", 2024-09-15T00:11:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0924000000000001e-05 2024-09-15T00:11:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062935+0000", 2024-09-15T00:11:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030406830000000002 2024-09-15T00:11:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062963+0000", 2024-09-15T00:11:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7800000000000001e-05 2024-09-15T00:11:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:139 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'92)", 2024-09-15T00:11:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.239149581, 2024-09-15T00:11:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040690240000000001, 2024-09-15T00:11:21.372 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.372 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.372 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.372 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:21.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:21.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:21.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:11:21.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061631+0000", 2024-09-15T00:11:21.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.17e-06 2024-09-15T00:11:21.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061636+0000", 2024-09-15T00:11:21.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7679999999999997e-06 2024-09-15T00:11:21.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061639+0000", 2024-09-15T00:11:21.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9069999999999999e-06 2024-09-15T00:11:21.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062733+0000", 2024-09-15T00:11:21.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010937119999999999 2024-09-15T00:11:21.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062744+0000", 2024-09-15T00:11:21.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1439e-05 2024-09-15T00:11:21.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065658+0000", 2024-09-15T00:11:21.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029134970000000001 2024-09-15T00:11:21.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065692+0000", 2024-09-15T00:11:21.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4147999999999999e-05 2024-09-15T00:11:21.375 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.375 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.375 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.376 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'92)", 2024-09-15T00:11:21.376 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:21.376 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.238760519, 2024-09-15T00:11:21.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003986608, 2024-09-15T00:11:21.376 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.376 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.376 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.376 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:21.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:21.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:21.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7000000000000001e-07 2024-09-15T00:11:21.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062075+0000", 2024-09-15T00:11:21.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2638000000000006e-05 2024-09-15T00:11:21.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062083+0000", 2024-09-15T00:11:21.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4330000000000004e-06 2024-09-15T00:11:21.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062087+0000", 2024-09-15T00:11:21.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3799999999999998e-06 2024-09-15T00:11:21.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.063001+0000", 2024-09-15T00:11:21.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091419500000000002 2024-09-15T00:11:21.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.063012+0000", 2024-09-15T00:11:21.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1117999999999999e-05 2024-09-15T00:11:21.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065977+0000", 2024-09-15T00:11:21.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029650840000000002 2024-09-15T00:11:21.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065999+0000", 2024-09-15T00:11:21.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1489999999999999e-05 2024-09-15T00:11:21.380 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.380 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.380 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.380 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:154 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'108, mlcod=16'105)", 2024-09-15T00:11:21.380 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:21.380 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.776512894, 2024-09-15T00:11:21.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053187770000000002, 2024-09-15T00:11:21.380 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.380 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.380 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.380 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:21.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:21.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:21.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:11:21.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524264+0000", 2024-09-15T00:11:21.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.738e-06 2024-09-15T00:11:21.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524270+0000", 2024-09-15T00:11:21.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9379999999999997e-06 2024-09-15T00:11:21.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524273+0000", 2024-09-15T00:11:21.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6370000000000001e-06 2024-09-15T00:11:21.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524310+0000", 2024-09-15T00:11:21.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6266999999999997e-05 2024-09-15T00:11:21.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524322+0000", 2024-09-15T00:11:21.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2741999999999999e-05 2024-09-15T00:11:21.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.529547+0000", 2024-09-15T00:11:21.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005225001 2024-09-15T00:11:21.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.529579+0000", 2024-09-15T00:11:21.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1065999999999997e-05 2024-09-15T00:11:21.384 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.384 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.384 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.384 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-15T00:11:21.384 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:11:21.384 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.776075256, 2024-09-15T00:11:21.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077021529999999998, 2024-09-15T00:11:21.384 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.385 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.385 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.385 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:11:21.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:11:21.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524698+0000", 2024-09-15T00:11:21.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6300000000000001e-07 2024-09-15T00:11:21.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524778+0000", 2024-09-15T00:11:21.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0598000000000003e-05 2024-09-15T00:11:21.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524785+0000", 2024-09-15T00:11:21.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6359999999999997e-06 2024-09-15T00:11:21.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524788+0000", 2024-09-15T00:11:21.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2540000000000001e-06 2024-09-15T00:11:21.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524817+0000", 2024-09-15T00:11:21.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8784000000000001e-05 2024-09-15T00:11:21.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524828+0000", 2024-09-15T00:11:21.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-15T00:11:21.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532360+0000", 2024-09-15T00:11:21.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007531757 2024-09-15T00:11:21.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532400+0000", 2024-09-15T00:11:21.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9835000000000001e-05 2024-09-15T00:11:21.388 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.388 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.388 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.388 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=16'106)", 2024-09-15T00:11:21.389 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:21.389 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.775133103, 2024-09-15T00:11:21.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067923050000000002, 2024-09-15T00:11:21.389 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.389 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.389 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.389 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:21.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:21.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:21.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-15T00:11:21.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525644+0000", 2024-09-15T00:11:21.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.709e-06 2024-09-15T00:11:21.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525651+0000", 2024-09-15T00:11:21.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7970000000000003e-06 2024-09-15T00:11:21.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525654+0000", 2024-09-15T00:11:21.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.433e-06 2024-09-15T00:11:21.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525685+0000", 2024-09-15T00:11:21.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0970000000000003e-05 2024-09-15T00:11:21.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525697+0000", 2024-09-15T00:11:21.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1759e-05 2024-09-15T00:11:21.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532401+0000", 2024-09-15T00:11:21.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006704425 2024-09-15T00:11:21.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532432+0000", 2024-09-15T00:11:21.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0709e-05 2024-09-15T00:11:21.392 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.392 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.393 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.393 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'106)", 2024-09-15T00:11:21.393 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:21.393 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.774670849, 2024-09-15T00:11:21.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063539190000000004, 2024-09-15T00:11:21.393 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.393 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.393 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.393 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:21.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:21.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:21.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:11:21.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526107+0000", 2024-09-15T00:11:21.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:11:21.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526117+0000", 2024-09-15T00:11:21.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7750000000000004e-06 2024-09-15T00:11:21.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526121+0000", 2024-09-15T00:11:21.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:11:21.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526163+0000", 2024-09-15T00:11:21.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1360999999999999e-05 2024-09-15T00:11:21.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526175+0000", 2024-09-15T00:11:21.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2729e-05 2024-09-15T00:11:21.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532433+0000", 2024-09-15T00:11:21.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062576999999999997 2024-09-15T00:11:21.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532456+0000", 2024-09-15T00:11:21.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2773e-05 2024-09-15T00:11:21.397 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.397 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.397 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.397 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'106)", 2024-09-15T00:11:21.397 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:11:21.397 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.774201446999999, 2024-09-15T00:11:21.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059082689999999998, 2024-09-15T00:11:21.397 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.397 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.397 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.397 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:11:21.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:11:21.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526572+0000", 2024-09-15T00:11:21.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9500000000000003e-07 2024-09-15T00:11:21.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526641+0000", 2024-09-15T00:11:21.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8858999999999999e-05 2024-09-15T00:11:21.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526648+0000", 2024-09-15T00:11:21.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:11:21.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526654+0000", 2024-09-15T00:11:21.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7509999999999997e-06 2024-09-15T00:11:21.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526689+0000", 2024-09-15T00:11:21.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5800000000000003e-05 2024-09-15T00:11:21.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526700+0000", 2024-09-15T00:11:21.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0467e-05 2024-09-15T00:11:21.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532457+0000", 2024-09-15T00:11:21.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057568180000000004 2024-09-15T00:11:21.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532479+0000", 2024-09-15T00:11:21.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2739e-05 2024-09-15T00:11:21.401 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.401 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.401 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.401 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'106)", 2024-09-15T00:11:21.401 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:11:21.401 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.773545181999999, 2024-09-15T00:11:21.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005274801, 2024-09-15T00:11:21.401 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.401 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.402 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.402 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:11:21.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:11:21.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527228+0000", 2024-09-15T00:11:21.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1600000000000001e-07 2024-09-15T00:11:21.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527232+0000", 2024-09-15T00:11:21.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8920000000000002e-06 2024-09-15T00:11:21.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527239+0000", 2024-09-15T00:11:21.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2880000000000004e-06 2024-09-15T00:11:21.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527243+0000", 2024-09-15T00:11:21.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.143e-06 2024-09-15T00:11:21.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527275+0000", 2024-09-15T00:11:21.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1866000000000003e-05 2024-09-15T00:11:21.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527288+0000", 2024-09-15T00:11:21.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469999999999999e-05 2024-09-15T00:11:21.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532480+0000", 2024-09-15T00:11:21.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051927129999999998 2024-09-15T00:11:21.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532502+0000", 2024-09-15T00:11:21.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1912999999999998e-05 2024-09-15T00:11:21.405 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.405 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.405 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.405 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.405 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.405 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:21.407 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:21.407 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:21.405+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:21.507 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:21.505+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:21.511 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:21.511 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:21.511 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:21.511 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:21.520 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:21.593 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is set 2024-09-15T00:11:21.609 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:21.607+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:21.674 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:21.674 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:21.674 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:21.683 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:21.726 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:21.724+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:21.790 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:21.790 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:21.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:21.790 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:21.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.791 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4287.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-15T00:11:21.791 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:10.541930+0000", 2024-09-15T00:11:21.791 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 11.247179198, 2024-09-15T00:11:21.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00078948500000000003, 2024-09-15T00:11:21.791 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.791 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.791 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.791 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.541930+0000", 2024-09-15T00:11:21.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.541930+0000", 2024-09-15T00:11:21.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.541930+0000", 2024-09-15T00:11:21.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.85e-07 2024-09-15T00:11:21.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.541935+0000", 2024-09-15T00:11:21.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7149999999999997e-06 2024-09-15T00:11:21.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.541946+0000", 2024-09-15T00:11:21.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1328999999999999e-05 2024-09-15T00:11:21.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.541956+0000", 2024-09-15T00:11:21.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7860000000000008e-06 2024-09-15T00:11:21.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.542005+0000", 2024-09-15T00:11:21.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8795000000000003e-05 2024-09-15T00:11:21.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.542030+0000", 2024-09-15T00:11:21.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4703e-05 2024-09-15T00:11:21.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.542676+0000", 2024-09-15T00:11:21.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0006466 2024-09-15T00:11:21.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.542720+0000", 2024-09-15T00:11:21.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3371999999999997e-05 2024-09-15T00:11:21.795 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.796 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.796 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4287.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-15T00:11:21.796 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:11.542948+0000", 2024-09-15T00:11:21.796 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.24616151, 2024-09-15T00:11:21.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016400029999999999, 2024-09-15T00:11:21.796 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.796 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.797 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.797 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.542948+0000", 2024-09-15T00:11:21.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.542948+0000", 2024-09-15T00:11:21.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.542948+0000", 2024-09-15T00:11:21.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0100000000000005e-07 2024-09-15T00:11:21.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.542957+0000", 2024-09-15T00:11:21.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6430000000000001e-06 2024-09-15T00:11:21.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.542975+0000", 2024-09-15T00:11:21.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7812000000000002e-05 2024-09-15T00:11:21.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.542989+0000", 2024-09-15T00:11:21.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3898e-05 2024-09-15T00:11:21.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.543067+0000", 2024-09-15T00:11:21.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8046000000000005e-05 2024-09-15T00:11:21.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.543093+0000", 2024-09-15T00:11:21.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6503e-05 2024-09-15T00:11:21.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.544526+0000", 2024-09-15T00:11:21.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014328089999999999 2024-09-15T00:11:21.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.544588+0000", 2024-09-15T00:11:21.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1791000000000005e-05 2024-09-15T00:11:21.801 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.801 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.801 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.801 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:7 3.1a e25/23 3:5cea7035:::rbd_info:head v 25'1, mlcod=0'0)", 2024-09-15T00:11:21.801 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:20.594306+0000", 2024-09-15T00:11:21.801 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.1948035589999999, 2024-09-15T00:11:21.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00066814700000000001, 2024-09-15T00:11:21.801 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:21.801 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:21.802 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:21.802 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:21.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:21.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594306+0000", 2024-09-15T00:11:21.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:21.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594306+0000", 2024-09-15T00:11:21.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:21.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:21.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594306+0000", 2024-09-15T00:11:21.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.34e-07 2024-09-15T00:11:21.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:21.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594311+0000", 2024-09-15T00:11:21.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1939999999999997e-06 2024-09-15T00:11:21.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:21.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594321+0000", 2024-09-15T00:11:21.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8030000000000008e-06 2024-09-15T00:11:21.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:21.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594330+0000", 2024-09-15T00:11:21.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8319999999999995e-06 2024-09-15T00:11:21.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:21.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594371+0000", 2024-09-15T00:11:21.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1470000000000001e-05 2024-09-15T00:11:21.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:21.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594389+0000", 2024-09-15T00:11:21.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7450000000000001e-05 2024-09-15T00:11:21.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:21.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594942+0000", 2024-09-15T00:11:21.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00055328199999999999 2024-09-15T00:11:21.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:21.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:21.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:21.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594974+0000", 2024-09-15T00:11:21.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1881999999999997e-05 2024-09-15T00:11:21.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.805 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.806 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:21.806 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:21.806 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:21.807 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:21.849 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:21.846+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:21.912 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:21.909+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:21.953 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:21.953 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:21.965 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:22.049 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:22.046+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:22.109 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:22.110 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:22.110 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:22.110 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:22.110 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:22.113 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:22.112+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:22.118 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:22.135 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:11:22.214 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:22.213+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:22.269 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4287.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-15T00:11:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:09.537222+0000", 2024-09-15T00:11:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.730861581999999, 2024-09-15T00:11:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00057522300000000003, 2024-09-15T00:11:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:09.537222+0000", 2024-09-15T00:11:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:09.537222+0000", 2024-09-15T00:11:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:09.537222+0000", 2024-09-15T00:11:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.65e-07 2024-09-15T00:11:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:09.537231+0000", 2024-09-15T00:11:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8859999999999993e-06 2024-09-15T00:11:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:09.537242+0000", 2024-09-15T00:11:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1124000000000001e-05 2024-09-15T00:11:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:09.537253+0000", 2024-09-15T00:11:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1219e-05 2024-09-15T00:11:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:09.537292+0000", 2024-09-15T00:11:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8869999999999999e-05 2024-09-15T00:11:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:09.537312+0000", 2024-09-15T00:11:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0475e-05 2024-09-15T00:11:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:09.537768+0000", 2024-09-15T00:11:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045569099999999998 2024-09-15T00:11:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:09.537797+0000", 2024-09-15T00:11:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8693000000000001e-05 2024-09-15T00:11:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:22.274 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:22.274 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:22.281 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:22.290 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:22.291 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:22.311 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:22.311 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:22.314 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:22.312+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:22.327 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:22.327 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:22.344 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:22.344 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:22.361 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:22.361 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:22.378 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:22.378 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:22.388 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:22.388 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:22.395 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:22.395 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:22.397 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:22.412 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:22.412 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:22.415 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:22.412+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:22.516 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:22.513+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:22.517 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:22.517 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:22.526 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:22.616 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:22.614+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:22.675 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:22.676 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:22.685 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:22.729 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:22.727+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:22.805 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:22.805 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:22.815 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:22.818 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:22.816+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:22.918 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:22.916+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:22.960 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:22.960 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:22.969 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:23.019 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:23.017+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:23.071 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:23.072 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:23.080 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:23.124 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:23.122+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:23.183 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:23.183 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:23.192 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:23.222 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:23.219+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:23.309 INFO:teuthology.orchestra.run.smithi171.stdout:1 write 0x3cbeeae thru 0x3cceb01 (0xfc54 bytes) 2024-09-15T00:11:23.313 INFO:teuthology.orchestra.run.smithi171.stdout:3 clone 1 order 24 su 262144 sc 12 2024-09-15T00:11:23.322 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:23.319+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:23.339 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:23.339 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:23.348 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:23.423 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:23.421+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:23.450 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:23.451 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:23.459 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:11:23.524 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:23.523+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:23.562 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:23.562 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:23.570 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:11:23.608 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T00:11:23.608 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:11:23.608 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T00:11:23.609 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-15T00:11:23.615 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:11:23.653 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:23.650+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:23.681 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:23.681 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:23.690 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:11:23.716 INFO:teuthology.orchestra.run.smithi171.stdout:4 read 0x15a73a5 thru 0x15b4f53 (0xdbaf bytes) 2024-09-15T00:11:23.716 INFO:teuthology.orchestra.run.smithi171.stdout:6 punch from 0x14ffbb1 to 0x1502cc5, (0x3114 bytes) 2024-09-15T00:11:23.719 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:23.720 INFO:teuthology.orchestra.run.smithi171.stdout:truncating to largest ever: 0xb48474a 2024-09-15T00:11:23.729 INFO:teuthology.orchestra.run.smithi171.stdout:8 trunc from 0x3cceb02 to 0xb48474a 2024-09-15T00:11:23.733 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:23.732+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:23.734 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:23.743 INFO:teuthology.orchestra.run.smithi171.stdout:11 writesame 0xe455c37 thru 0xe4615ff data_size 0x1(0xb9c9 bytes) 2024-09-15T00:11:23.744 INFO:teuthology.orchestra.run.smithi171.stdout:12 write 0x5fff8c5 thru 0x60045b3 (0x4cef bytes) 2024-09-15T00:11:23.748 INFO:teuthology.orchestra.run.smithi171.stdout:truncating to largest ever: 0xb9896df 2024-09-15T00:11:23.751 INFO:teuthology.orchestra.run.smithi171.stdout:13 trunc from 0xe461600 to 0xb9896df 2024-09-15T00:11:23.771 INFO:teuthology.orchestra.run.smithi171.stdout:14 writesame 0x53079e thru 0x53d421 data_size 0x6(0xcc84 bytes) 2024-09-15T00:11:23.802 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:23.802 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:23.802 INFO:teuthology.orchestra.run.smithi171.stdout:15 write 0xea8d787 thru 0xea9a15d (0xc9d7 bytes) 2024-09-15T00:11:23.805 INFO:teuthology.orchestra.run.smithi171.stdout:16 read 0x165074b thru 0x165a86a (0xa120 bytes) 2024-09-15T00:11:23.805 INFO:teuthology.orchestra.run.smithi171.stdout:17 writesame 0x3458630 thru 0x34680b0 data_size 0x1(0xfa81 bytes) 2024-09-15T00:11:23.811 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:11:23.853 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:23.851+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:23.928 INFO:tasks.ceph.mon.a.smithi171.stderr:2024-09-15T00:11:23.926+0000 7f0425d35640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-15T00:11:23.929 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:23.927+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:23.931 INFO:teuthology.orchestra.run.smithi171.stdout:18 punch from 0xd3c7d57 to 0xd3d6016, (0xe2bf bytes) 2024-09-15T00:11:23.932 INFO:teuthology.orchestra.run.smithi171.stdout:20 write 0x56261cd thru 0x563280a (0xc63e bytes) 2024-09-15T00:11:23.936 INFO:teuthology.orchestra.run.smithi171.stdout:21 read 0x565dc2d thru 0x5666db7 (0x918b bytes) 2024-09-15T00:11:23.936 INFO:teuthology.orchestra.run.smithi171.stdout:22 write 0x6dfafcd thru 0x6e03d35 (0x8d69 bytes) 2024-09-15T00:11:23.946 INFO:teuthology.orchestra.run.smithi171.stdout:24 trunc from 0xea9a15e to 0x85be149 2024-09-15T00:11:23.963 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:23.963 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:23.963 INFO:teuthology.orchestra.run.smithi171.stdout:26 punch from 0x7a321fc to 0x7a40c85, (0xea89 bytes) 2024-09-15T00:11:23.963 INFO:teuthology.orchestra.run.smithi171.stdout:truncating to largest ever: 0xcf055f0 2024-09-15T00:11:23.968 INFO:teuthology.orchestra.run.smithi171.stdout:27 trunc from 0x85be149 to 0xcf055f0 2024-09-15T00:11:23.972 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:11:23.974 INFO:teuthology.orchestra.run.smithi171.stdout:28 write 0x88c5226 thru 0x88d328f (0xe06a bytes) 2024-09-15T00:11:23.979 INFO:teuthology.orchestra.run.smithi171.stdout:29 punch from 0x4662dba to 0x4666396, (0x35dc bytes) 2024-09-15T00:11:23.979 INFO:teuthology.orchestra.run.smithi171.stdout:31 write 0xab6a34f thru 0xab7494d (0xa5ff bytes) 2024-09-15T00:11:23.982 INFO:teuthology.orchestra.run.smithi171.stdout:32 read 0x6715e0a thru 0x6721849 (0xba40 bytes) 2024-09-15T00:11:23.983 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:23.983 INFO:teuthology.orchestra.run.smithi171.stdout:34 read 0x17b78b5 thru 0x17bda16 (0x6162 bytes) 2024-09-15T00:11:23.993 INFO:teuthology.orchestra.run.smithi171.stdout:35 trunc from 0xcf055f0 to 0x4a2bbeb 2024-09-15T00:11:24.004 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:24.004 INFO:teuthology.orchestra.run.smithi171.stdout:37 read 0x46bed00 thru 0x46cd665 (0xe966 bytes) 2024-09-15T00:11:24.005 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:24.005 INFO:teuthology.orchestra.run.smithi171.stdout:39 writesame 0x443259d thru 0x44356ff data_size 0x1(0x3163 bytes) 2024-09-15T00:11:24.025 INFO:teuthology.orchestra.run.smithi171.stdout:40 write 0xa5522ef thru 0xa5590df (0x6df1 bytes) 2024-09-15T00:11:24.027 INFO:teuthology.orchestra.run.smithi171.stdout:41 punch from 0x8c3a114 to 0x8c3c0e4, (0x1fd0 bytes) 2024-09-15T00:11:24.027 INFO:teuthology.orchestra.run.smithi171.stdout:42 punch from 0x8550989 to 0x855e756, (0xddcd bytes) 2024-09-15T00:11:24.027 INFO:teuthology.orchestra.run.smithi171.stdout:43 read 0x9a37e0f thru 0x9a3f2f8 (0x74ea bytes) 2024-09-15T00:11:24.028 INFO:teuthology.orchestra.run.smithi171.stdout:44 write 0x57af7c8 thru 0x57b6a0a (0x7243 bytes) 2024-09-15T00:11:24.030 INFO:teuthology.orchestra.run.smithi171.stdout:45 read 0x28262f6 thru 0x282cc9d (0x69a8 bytes) 2024-09-15T00:11:24.040 INFO:teuthology.orchestra.run.smithi171.stdout:46 writesame 0xd48f6a8 thru 0xd498f5e data_size 0x7(0x98b7 bytes) 2024-09-15T00:11:24.040 INFO:teuthology.orchestra.run.smithi171.stdout:48 writesame 0x275963b thru 0x275ef5a data_size 0x5c(0x5920 bytes) 2024-09-15T00:11:24.042 INFO:teuthology.orchestra.run.smithi171.stdout:49 punch from 0xff86ea to 0xffcb51, (0x4467 bytes) 2024-09-15T00:11:24.042 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:24.042 INFO:teuthology.orchestra.run.smithi171.stdout:51 read 0x77fce84 thru 0x7808c70 (0xbded bytes) 2024-09-15T00:11:24.042 INFO:teuthology.orchestra.run.smithi171.stdout:52 read 0x84d332e thru 0x84dd7fb (0xa4ce bytes) 2024-09-15T00:11:24.042 INFO:teuthology.orchestra.run.smithi171.stdout:53 writesame 0x773df5 thru 0x77995e data_size 0x1(0x5b6a bytes) 2024-09-15T00:11:24.057 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:24.056+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:24.059 INFO:teuthology.orchestra.run.smithi171.stdout:54 read 0x48c9ba0 thru 0x48ca3e1 (0x842 bytes) 2024-09-15T00:11:24.059 INFO:teuthology.orchestra.run.smithi171.stdout:56 read 0x5a3dd21 thru 0x5a45b2e (0x7e0e bytes) 2024-09-15T00:11:24.060 INFO:teuthology.orchestra.run.smithi171.stdout:57 write 0x478d636 thru 0x478e085 (0xa50 bytes) 2024-09-15T00:11:24.062 INFO:teuthology.orchestra.run.smithi171.stdout:58 punch from 0x9b5db6c to 0x9b6ccfb, (0xf18f bytes) 2024-09-15T00:11:24.062 INFO:teuthology.orchestra.run.smithi171.stdout:59 punch from 0xe44ad4 to 0xe4639d, (0x18c9 bytes) 2024-09-15T00:11:24.062 INFO:teuthology.orchestra.run.smithi171.stdout:61 punch from 0x15557c8 to 0x15650c7, (0xf8ff bytes) 2024-09-15T00:11:24.062 INFO:teuthology.orchestra.run.smithi171.stdout:62 punch from 0x7c8e11e to 0x7c957bb, (0x769d bytes) 2024-09-15T00:11:24.062 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:24.062 INFO:teuthology.orchestra.run.smithi171.stdout:64 writesame 0xc622890 thru 0xc625ec8 data_size 0x7(0x3639 bytes) 2024-09-15T00:11:24.066 INFO:teuthology.orchestra.run.smithi171.stdout:66 write 0x2c20f41 thru 0x2c298db (0x899b bytes) 2024-09-15T00:11:24.070 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:24.070 INFO:teuthology.orchestra.run.smithi171.stdout:68 write 0x946d03a thru 0x94736f4 (0x66bb bytes) 2024-09-15T00:11:24.072 INFO:teuthology.orchestra.run.smithi171.stdout:71 writesame 0x2ad2fa0 thru 0x2ae0194 data_size 0x1(0xd1f5 bytes) 2024-09-15T00:11:24.118 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:24.119 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:24.127 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:11:24.130 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:24.128+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:24.177 INFO:teuthology.orchestra.run.smithi171.stdout:73 writesame 0x305f6e1 thru 0x30671df data_size 0x17(0x7aff bytes) 2024-09-15T00:11:24.181 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:24.181 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:24.181 INFO:teuthology.orchestra.run.smithi171.stdout:77 punch from 0x1bd1cf8 to 0x1bd2e26, (0x112e bytes) 2024-09-15T00:11:24.181 INFO:teuthology.orchestra.run.smithi171.stdout:78 punch from 0x8c6ab47 to 0x8c73a89, (0x8f42 bytes) 2024-09-15T00:11:24.181 INFO:teuthology.orchestra.run.smithi171.stdout:79 punch from 0xb2f6374 to 0xb2fdc40, (0x78cc bytes) 2024-09-15T00:11:24.181 INFO:teuthology.orchestra.run.smithi171.stdout:80 read 0x6d331ef thru 0x6d3bf69 (0x8d7b bytes) 2024-09-15T00:11:24.181 INFO:teuthology.orchestra.run.smithi171.stdout:81 read 0x2171f25 thru 0x217c736 (0xa812 bytes) 2024-09-15T00:11:24.181 INFO:teuthology.orchestra.run.smithi171.stdout:83 punch from 0xc4c630c to 0xc4c95e6, (0x32da bytes) 2024-09-15T00:11:24.184 INFO:teuthology.orchestra.run.smithi171.stdout:84 trunc from 0xd498f5f to 0xaf40abc 2024-09-15T00:11:24.204 INFO:teuthology.orchestra.run.smithi171.stdout:85 write 0xd6250f7 thru 0xd625962 (0x86c bytes) 2024-09-15T00:11:24.224 INFO:teuthology.orchestra.run.smithi171.stdout:86 trunc from 0xd625963 to 0x7f2720 2024-09-15T00:11:24.230 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:24.229+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:24.235 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:24.235 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:24.235 INFO:teuthology.orchestra.run.smithi171.stdout:93 flatten 2024-09-15T00:11:24.240 INFO:teuthology.orchestra.run.smithi171.stdout:94 read 0xaffe3 thru 0xbf9a4 (0xf9c2 bytes) 2024-09-15T00:11:24.240 INFO:teuthology.orchestra.run.smithi171.stdout:95 clone 2 order 22 su 65536 sc 2 2024-09-15T00:11:24.274 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:24.274 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:24.282 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:24.331 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:24.329+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:24.388 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:24.388 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:24.396 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:11:24.424 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:11:24.432 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:24.431+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:24.534 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:24.532+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:24.546 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:24.546 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:24.554 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:11:24.561 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:24.561 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:24.578 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:24.578 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:24.594 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:24.595 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:24.611 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:24.611 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:24.619 INFO:teuthology.orchestra.run.smithi171.stderr:marked out osd.0. 2024-09-15T00:11:24.621 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is unset 2024-09-15T00:11:24.634 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:24.632+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:24.636 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:24.636 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:24.654 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:24.654 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:24.671 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:24.671 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:24.672 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:24.672 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:24.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:24.672 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:24.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.673 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-15T00:11:24.673 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:11:24.673 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.134628403000001, 2024-09-15T00:11:24.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0035574980000000001, 2024-09-15T00:11:24.673 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.673 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.673 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.673 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:11:24.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:11:24.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535370+0000", 2024-09-15T00:11:24.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-15T00:11:24.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535438+0000", 2024-09-15T00:11:24.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199000000000002e-05 2024-09-15T00:11:24.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535446+0000", 2024-09-15T00:11:24.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1640000000000001e-06 2024-09-15T00:11:24.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535450+0000", 2024-09-15T00:11:24.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1520000000000002e-06 2024-09-15T00:11:24.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535491+0000", 2024-09-15T00:11:24.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0969e-05 2024-09-15T00:11:24.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535510+0000", 2024-09-15T00:11:24.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9031000000000001e-05 2024-09-15T00:11:24.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.538893+0000", 2024-09-15T00:11:24.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033823870000000002 2024-09-15T00:11:24.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.538927+0000", 2024-09-15T00:11:24.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4085999999999999e-05 2024-09-15T00:11:24.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.677 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.677 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:70 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-09-15T00:11:24.677 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:11:24.677 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.133888183, 2024-09-15T00:11:24.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057871490000000001, 2024-09-15T00:11:24.677 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.677 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.677 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.677 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:11:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:11:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536110+0000", 2024-09-15T00:11:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-15T00:11:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536115+0000", 2024-09-15T00:11:24.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8960000000000002e-06 2024-09-15T00:11:24.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536123+0000", 2024-09-15T00:11:24.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3979999999999999e-06 2024-09-15T00:11:24.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536128+0000", 2024-09-15T00:11:24.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5569999999999997e-06 2024-09-15T00:11:24.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537011+0000", 2024-09-15T00:11:24.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088276299999999995 2024-09-15T00:11:24.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537034+0000", 2024-09-15T00:11:24.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3448999999999999e-05 2024-09-15T00:11:24.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541867+0000", 2024-09-15T00:11:24.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048332949999999996 2024-09-15T00:11:24.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541896+0000", 2024-09-15T00:11:24.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9179999999999998e-05 2024-09-15T00:11:24.681 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.681 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.681 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.681 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-15T00:11:24.681 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:24.681 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.106269730000001, 2024-09-15T00:11:24.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027567730000000001, 2024-09-15T00:11:24.682 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.682 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.682 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.682 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:24.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:24.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:24.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5599999999999995e-07 2024-09-15T00:11:24.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563787+0000", 2024-09-15T00:11:24.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9092000000000003e-05 2024-09-15T00:11:24.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563796+0000", 2024-09-15T00:11:24.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0059999999999998e-06 2024-09-15T00:11:24.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563801+0000", 2024-09-15T00:11:24.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3290000000000004e-06 2024-09-15T00:11:24.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563830+0000", 2024-09-15T00:11:24.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9369999999999998e-05 2024-09-15T00:11:24.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563842+0000", 2024-09-15T00:11:24.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1646e-05 2024-09-15T00:11:24.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.566459+0000", 2024-09-15T00:11:24.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026174549999999999 2024-09-15T00:11:24.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.566485+0000", 2024-09-15T00:11:24.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5318999999999999e-05 2024-09-15T00:11:24.685 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.685 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.685 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.686 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:106 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-09-15T00:11:24.686 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:11:24.686 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.973367267, 2024-09-15T00:11:24.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029384250000000002, 2024-09-15T00:11:24.686 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.686 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.686 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.686 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:11:24.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:11:24.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696631+0000", 2024-09-15T00:11:24.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4299999999999998e-07 2024-09-15T00:11:24.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696692+0000", 2024-09-15T00:11:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1229999999999995e-05 2024-09-15T00:11:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696699+0000", 2024-09-15T00:11:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5399999999999998e-06 2024-09-15T00:11:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696705+0000", 2024-09-15T00:11:24.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5770000000000001e-06 2024-09-15T00:11:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696732+0000", 2024-09-15T00:11:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7022e-05 2024-09-15T00:11:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696743+0000", 2024-09-15T00:11:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1426999999999999e-05 2024-09-15T00:11:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699542+0000", 2024-09-15T00:11:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002798521 2024-09-15T00:11:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699569+0000", 2024-09-15T00:11:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6665e-05 2024-09-15T00:11:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:107 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-09-15T00:11:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:24.690 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.972471837000001, 2024-09-15T00:11:24.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048949229999999998, 2024-09-15T00:11:24.691 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.691 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.691 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.691 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:24.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:24.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:24.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5799999999999999e-07 2024-09-15T00:11:24.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697531+0000", 2024-09-15T00:11:24.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3100000000000002e-06 2024-09-15T00:11:24.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697539+0000", 2024-09-15T00:11:24.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6410000000000008e-06 2024-09-15T00:11:24.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697543+0000", 2024-09-15T00:11:24.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1060000000000002e-06 2024-09-15T00:11:24.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697573+0000", 2024-09-15T00:11:24.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9978e-05 2024-09-15T00:11:24.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697585+0000", 2024-09-15T00:11:24.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1279e-05 2024-09-15T00:11:24.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702398+0000", 2024-09-15T00:11:24.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004812981 2024-09-15T00:11:24.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702421+0000", 2024-09-15T00:11:24.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3070000000000001e-05 2024-09-15T00:11:24.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.695 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.695 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:108 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'66)", 2024-09-15T00:11:24.695 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:11:24.695 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.971578347000001, 2024-09-15T00:11:24.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040179439999999999, 2024-09-15T00:11:24.696 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.696 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.696 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.696 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:11:24.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:11:24.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698420+0000", 2024-09-15T00:11:24.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0900000000000002e-07 2024-09-15T00:11:24.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698423+0000", 2024-09-15T00:11:24.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8319999999999999e-06 2024-09-15T00:11:24.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698431+0000", 2024-09-15T00:11:24.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8949999999999998e-06 2024-09-15T00:11:24.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698435+0000", 2024-09-15T00:11:24.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5939999999999998e-06 2024-09-15T00:11:24.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698469+0000", 2024-09-15T00:11:24.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4258000000000003e-05 2024-09-15T00:11:24.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698481+0000", 2024-09-15T00:11:24.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1763000000000001e-05 2024-09-15T00:11:24.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702422+0000", 2024-09-15T00:11:24.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003940603 2024-09-15T00:11:24.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702437+0000", 2024-09-15T00:11:24.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5489999999999999e-05 2024-09-15T00:11:24.699 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.699 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.699 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.700 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:109 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'66)", 2024-09-15T00:11:24.700 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:24.700 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.971047910999999, 2024-09-15T00:11:24.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038555479999999999, 2024-09-15T00:11:24.700 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.700 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.700 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.700 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:24.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:24.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:24.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.13e-07 2024-09-15T00:11:24.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699018+0000", 2024-09-15T00:11:24.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7540999999999996e-05 2024-09-15T00:11:24.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699024+0000", 2024-09-15T00:11:24.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4869999999999999e-06 2024-09-15T00:11:24.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699044+0000", 2024-09-15T00:11:24.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9400999999999999e-05 2024-09-15T00:11:24.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699104+0000", 2024-09-15T00:11:24.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0331999999999998e-05 2024-09-15T00:11:24.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699117+0000", 2024-09-15T00:11:24.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2789000000000001e-05 2024-09-15T00:11:24.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702777+0000", 2024-09-15T00:11:24.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0036604900000000002 2024-09-15T00:11:24.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702805+0000", 2024-09-15T00:11:24.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7994999999999999e-05 2024-09-15T00:11:24.704 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.704 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.704 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.704 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:110 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'72, mlcod=16'66)", 2024-09-15T00:11:24.704 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:11:24.705 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.970288377999999, 2024-09-15T00:11:24.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031108189999999999, 2024-09-15T00:11:24.705 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.705 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.705 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.705 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:11:24.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:11:24.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699710+0000", 2024-09-15T00:11:24.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:11:24.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699714+0000", 2024-09-15T00:11:24.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0010000000000003e-06 2024-09-15T00:11:24.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699722+0000", 2024-09-15T00:11:24.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4840000000000004e-06 2024-09-15T00:11:24.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699726+0000", 2024-09-15T00:11:24.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:11:24.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699755+0000", 2024-09-15T00:11:24.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8433999999999999e-05 2024-09-15T00:11:24.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699775+0000", 2024-09-15T00:11:24.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0017000000000002e-05 2024-09-15T00:11:24.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702807+0000", 2024-09-15T00:11:24.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030320239999999999 2024-09-15T00:11:24.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702820+0000", 2024-09-15T00:11:24.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3169999999999999e-05 2024-09-15T00:11:24.709 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.709 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.709 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.709 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:120 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'82, mlcod=16'78)", 2024-09-15T00:11:24.709 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:11:24.709 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.964385031999999, 2024-09-15T00:11:24.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028965829999999999, 2024-09-15T00:11:24.709 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.709 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.709 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.709 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:11:24.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:11:24.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705613+0000", 2024-09-15T00:11:24.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1800000000000001e-07 2024-09-15T00:11:24.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705684+0000", 2024-09-15T00:11:24.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0903999999999997e-05 2024-09-15T00:11:24.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705691+0000", 2024-09-15T00:11:24.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4290000000000004e-06 2024-09-15T00:11:24.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705695+0000", 2024-09-15T00:11:24.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4790000000000001e-06 2024-09-15T00:11:24.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705729+0000", 2024-09-15T00:11:24.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4415999999999997e-05 2024-09-15T00:11:24.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705741+0000", 2024-09-15T00:11:24.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1443999999999999e-05 2024-09-15T00:11:24.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.708476+0000", 2024-09-15T00:11:24.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027356630000000002 2024-09-15T00:11:24.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.708509+0000", 2024-09-15T00:11:24.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2830000000000002e-05 2024-09-15T00:11:24.713 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.713 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.713 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.713 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-15T00:11:24.713 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:24.713 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.963598566999998, 2024-09-15T00:11:24.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047824809999999999, 2024-09-15T00:11:24.713 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.713 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.714 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.714 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:24.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:24.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:24.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:11:24.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706471+0000", 2024-09-15T00:11:24.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1496999999999997e-05 2024-09-15T00:11:24.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706478+0000", 2024-09-15T00:11:24.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7460000000000003e-06 2024-09-15T00:11:24.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706481+0000", 2024-09-15T00:11:24.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.258e-06 2024-09-15T00:11:24.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706513+0000", 2024-09-15T00:11:24.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2567000000000002e-05 2024-09-15T00:11:24.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706524+0000", 2024-09-15T00:11:24.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1028e-05 2024-09-15T00:11:24.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.711150+0000", 2024-09-15T00:11:24.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046259450000000002 2024-09-15T00:11:24.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.711181+0000", 2024-09-15T00:11:24.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1041000000000003e-05 2024-09-15T00:11:24.717 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.717 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.717 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.718 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:136 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-15T00:11:24.718 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:24.718 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.610161633000001, 2024-09-15T00:11:24.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031270970000000001, 2024-09-15T00:11:24.718 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.718 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.718 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.718 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:24.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:24.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:24.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3000000000000002e-07 2024-09-15T00:11:24.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059839+0000", 2024-09-15T00:11:24.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3050000000000001e-06 2024-09-15T00:11:24.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059847+0000", 2024-09-15T00:11:24.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.622e-06 2024-09-15T00:11:24.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059851+0000", 2024-09-15T00:11:24.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8120000000000001e-06 2024-09-15T00:11:24.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059884+0000", 2024-09-15T00:11:24.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2620999999999998e-05 2024-09-15T00:11:24.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059894+0000", 2024-09-15T00:11:24.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0924000000000001e-05 2024-09-15T00:11:24.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062935+0000", 2024-09-15T00:11:24.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030406830000000002 2024-09-15T00:11:24.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062963+0000", 2024-09-15T00:11:24.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7800000000000001e-05 2024-09-15T00:11:24.722 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.723 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.723 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.723 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:139 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'92)", 2024-09-15T00:11:24.724 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:24.724 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.60837441, 2024-09-15T00:11:24.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040690240000000001, 2024-09-15T00:11:24.724 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.724 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.724 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.724 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:24.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:24.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:24.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:11:24.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061631+0000", 2024-09-15T00:11:24.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.17e-06 2024-09-15T00:11:24.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061636+0000", 2024-09-15T00:11:24.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7679999999999997e-06 2024-09-15T00:11:24.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061639+0000", 2024-09-15T00:11:24.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9069999999999999e-06 2024-09-15T00:11:24.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062733+0000", 2024-09-15T00:11:24.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010937119999999999 2024-09-15T00:11:24.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062744+0000", 2024-09-15T00:11:24.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1439e-05 2024-09-15T00:11:24.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065658+0000", 2024-09-15T00:11:24.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029134970000000001 2024-09-15T00:11:24.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065692+0000", 2024-09-15T00:11:24.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4147999999999999e-05 2024-09-15T00:11:24.728 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.728 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.729 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.729 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'92)", 2024-09-15T00:11:24.729 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:24.729 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.607985348, 2024-09-15T00:11:24.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003986608, 2024-09-15T00:11:24.729 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.729 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.729 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.730 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:24.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:24.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:24.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7000000000000001e-07 2024-09-15T00:11:24.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062075+0000", 2024-09-15T00:11:24.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2638000000000006e-05 2024-09-15T00:11:24.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062083+0000", 2024-09-15T00:11:24.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4330000000000004e-06 2024-09-15T00:11:24.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062087+0000", 2024-09-15T00:11:24.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3799999999999998e-06 2024-09-15T00:11:24.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.063001+0000", 2024-09-15T00:11:24.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091419500000000002 2024-09-15T00:11:24.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.063012+0000", 2024-09-15T00:11:24.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1117999999999999e-05 2024-09-15T00:11:24.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065977+0000", 2024-09-15T00:11:24.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029650840000000002 2024-09-15T00:11:24.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065999+0000", 2024-09-15T00:11:24.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1489999999999999e-05 2024-09-15T00:11:24.733 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.733 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.733 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.733 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:154 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'108, mlcod=16'105)", 2024-09-15T00:11:24.733 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:24.733 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.145737723, 2024-09-15T00:11:24.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053187770000000002, 2024-09-15T00:11:24.734 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.734 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.734 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.734 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:24.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:24.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:24.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:11:24.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524264+0000", 2024-09-15T00:11:24.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.738e-06 2024-09-15T00:11:24.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524270+0000", 2024-09-15T00:11:24.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9379999999999997e-06 2024-09-15T00:11:24.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524273+0000", 2024-09-15T00:11:24.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6370000000000001e-06 2024-09-15T00:11:24.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524310+0000", 2024-09-15T00:11:24.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6266999999999997e-05 2024-09-15T00:11:24.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524322+0000", 2024-09-15T00:11:24.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2741999999999999e-05 2024-09-15T00:11:24.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.529547+0000", 2024-09-15T00:11:24.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005225001 2024-09-15T00:11:24.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.529579+0000", 2024-09-15T00:11:24.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1065999999999997e-05 2024-09-15T00:11:24.737 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.738 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.738 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.738 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-15T00:11:24.738 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:11:24.738 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.145300084999999, 2024-09-15T00:11:24.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077021529999999998, 2024-09-15T00:11:24.738 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.738 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.738 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.738 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:11:24.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:11:24.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524698+0000", 2024-09-15T00:11:24.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6300000000000001e-07 2024-09-15T00:11:24.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524778+0000", 2024-09-15T00:11:24.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0598000000000003e-05 2024-09-15T00:11:24.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524785+0000", 2024-09-15T00:11:24.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6359999999999997e-06 2024-09-15T00:11:24.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524788+0000", 2024-09-15T00:11:24.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2540000000000001e-06 2024-09-15T00:11:24.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524817+0000", 2024-09-15T00:11:24.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8784000000000001e-05 2024-09-15T00:11:24.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524828+0000", 2024-09-15T00:11:24.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-15T00:11:24.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532360+0000", 2024-09-15T00:11:24.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007531757 2024-09-15T00:11:24.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532400+0000", 2024-09-15T00:11:24.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9835000000000001e-05 2024-09-15T00:11:24.742 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.742 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.742 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.742 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=16'106)", 2024-09-15T00:11:24.742 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:24.742 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.144357931999998, 2024-09-15T00:11:24.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067923050000000002, 2024-09-15T00:11:24.743 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.743 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.743 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.743 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:24.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:24.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:24.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-15T00:11:24.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525644+0000", 2024-09-15T00:11:24.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.709e-06 2024-09-15T00:11:24.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525651+0000", 2024-09-15T00:11:24.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7970000000000003e-06 2024-09-15T00:11:24.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525654+0000", 2024-09-15T00:11:24.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.433e-06 2024-09-15T00:11:24.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525685+0000", 2024-09-15T00:11:24.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0970000000000003e-05 2024-09-15T00:11:24.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525697+0000", 2024-09-15T00:11:24.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1759e-05 2024-09-15T00:11:24.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532401+0000", 2024-09-15T00:11:24.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006704425 2024-09-15T00:11:24.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532432+0000", 2024-09-15T00:11:24.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0709e-05 2024-09-15T00:11:24.746 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.747 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.747 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.747 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'106)", 2024-09-15T00:11:24.747 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:24.747 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.143895678, 2024-09-15T00:11:24.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063539190000000004, 2024-09-15T00:11:24.747 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.747 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.748 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.748 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:24.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:24.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:24.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:11:24.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526107+0000", 2024-09-15T00:11:24.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:11:24.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526117+0000", 2024-09-15T00:11:24.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7750000000000004e-06 2024-09-15T00:11:24.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526121+0000", 2024-09-15T00:11:24.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:11:24.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526163+0000", 2024-09-15T00:11:24.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1360999999999999e-05 2024-09-15T00:11:24.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526175+0000", 2024-09-15T00:11:24.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2729e-05 2024-09-15T00:11:24.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532433+0000", 2024-09-15T00:11:24.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062576999999999997 2024-09-15T00:11:24.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532456+0000", 2024-09-15T00:11:24.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2773e-05 2024-09-15T00:11:24.754 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.754 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.754 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.754 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'106)", 2024-09-15T00:11:24.754 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:11:24.754 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.143426276, 2024-09-15T00:11:24.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059082689999999998, 2024-09-15T00:11:24.755 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.755 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.755 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.755 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:11:24.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:11:24.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526572+0000", 2024-09-15T00:11:24.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9500000000000003e-07 2024-09-15T00:11:24.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526641+0000", 2024-09-15T00:11:24.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8858999999999999e-05 2024-09-15T00:11:24.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526648+0000", 2024-09-15T00:11:24.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:11:24.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526654+0000", 2024-09-15T00:11:24.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7509999999999997e-06 2024-09-15T00:11:24.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526689+0000", 2024-09-15T00:11:24.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5800000000000003e-05 2024-09-15T00:11:24.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526700+0000", 2024-09-15T00:11:24.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0467e-05 2024-09-15T00:11:24.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532457+0000", 2024-09-15T00:11:24.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057568180000000004 2024-09-15T00:11:24.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532479+0000", 2024-09-15T00:11:24.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2739e-05 2024-09-15T00:11:24.761 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.761 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.761 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.761 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'106)", 2024-09-15T00:11:24.761 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:11:24.761 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.142770011, 2024-09-15T00:11:24.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005274801, 2024-09-15T00:11:24.762 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.762 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.762 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.762 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:11:24.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:11:24.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527228+0000", 2024-09-15T00:11:24.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1600000000000001e-07 2024-09-15T00:11:24.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527232+0000", 2024-09-15T00:11:24.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8920000000000002e-06 2024-09-15T00:11:24.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527239+0000", 2024-09-15T00:11:24.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2880000000000004e-06 2024-09-15T00:11:24.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527243+0000", 2024-09-15T00:11:24.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.143e-06 2024-09-15T00:11:24.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527275+0000", 2024-09-15T00:11:24.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1866000000000003e-05 2024-09-15T00:11:24.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527288+0000", 2024-09-15T00:11:24.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469999999999999e-05 2024-09-15T00:11:24.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532480+0000", 2024-09-15T00:11:24.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051927129999999998 2024-09-15T00:11:24.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532502+0000", 2024-09-15T00:11:24.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1912999999999998e-05 2024-09-15T00:11:24.765 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.765 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.765 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.766 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:25 3.10 e26/23 3:0d9e4e9e:::rbd_header.11395e6383ec:head v 26'2, mlcod=0'0)", 2024-09-15T00:11:24.766 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:21.619783+0000", 2024-09-15T00:11:24.766 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.0502143450000001, 2024-09-15T00:11:24.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031992600000000002, 2024-09-15T00:11:24.766 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:24.766 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:24.766 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:24.766 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:24.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:24.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619783+0000", 2024-09-15T00:11:24.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:24.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619783+0000", 2024-09-15T00:11:24.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:24.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:24.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619784+0000", 2024-09-15T00:11:24.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9900000000000009e-07 2024-09-15T00:11:24.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:24.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619795+0000", 2024-09-15T00:11:24.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0689e-05 2024-09-15T00:11:24.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:24.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619809+0000", 2024-09-15T00:11:24.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3757000000000001e-05 2024-09-15T00:11:24.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:24.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619818+0000", 2024-09-15T00:11:24.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4909999999999994e-06 2024-09-15T00:11:24.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:24.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619868+0000", 2024-09-15T00:11:24.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0046999999999997e-05 2024-09-15T00:11:24.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:24.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619887+0000", 2024-09-15T00:11:24.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780999999999999e-05 2024-09-15T00:11:24.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:24.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.622947+0000", 2024-09-15T00:11:24.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030601069999999998 2024-09-15T00:11:24.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:24.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:24.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:24.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.622982+0000", 2024-09-15T00:11:24.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5389000000000003e-05 2024-09-15T00:11:24.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.770 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:24.770 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:24.770 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:24.772 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:24.772 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:24.773 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:24.773 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:24.769+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:24.872 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:24.870+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:24.920 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:24.920 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:24.920 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:24.920 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:24.929 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:24.973 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:24.972+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:25.073 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:25.071+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:25.084 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:25.084 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:25.084 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:25.085 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-15T00:11:25.096 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:25.156 INFO:teuthology.orchestra.run.smithi171.stdout:96 read 0x34d48c thru 0x34e210 (0xd85 bytes) 2024-09-15T00:11:25.156 INFO:teuthology.orchestra.run.smithi171.stdout:97 read 0x5e3ad2 thru 0x5e5cf5 (0x2224 bytes) 2024-09-15T00:11:25.157 INFO:teuthology.orchestra.run.smithi171.stdout:98 punch from 0x7526a8 to 0x75b727, (0x907f bytes) 2024-09-15T00:11:25.174 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:25.172+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:25.180 INFO:teuthology.orchestra.run.smithi171.stdout:99 write 0xba97ddd thru 0xbaa5072 (0xd296 bytes) 2024-09-15T00:11:25.194 INFO:teuthology.orchestra.run.smithi171.stdout:100 write 0xd7c7d66 thru 0xd7ca91d (0x2bb8 bytes) 2024-09-15T00:11:25.198 INFO:teuthology.orchestra.run.smithi171.stdout:101 writesame 0x625295e thru 0x625b35a data_size 0x4b(0x89fd bytes) 2024-09-15T00:11:25.209 INFO:teuthology.orchestra.run.smithi171.stdout:102 write 0xe6c9c52 thru 0xe6d6077 (0xc426 bytes) 2024-09-15T00:11:25.216 INFO:teuthology.orchestra.run.smithi171.stdout:103 punch from 0xfa24fc to 0xfabd85, (0x9889 bytes) 2024-09-15T00:11:25.226 INFO:teuthology.orchestra.run.smithi171.stdout:104 trunc from 0xe6d6078 to 0x639c95a 2024-09-15T00:11:25.240 INFO:teuthology.orchestra.run.smithi171.stdout:106 writesame 0x4245653 thru 0x4252188 data_size 0x26(0xcb36 bytes) 2024-09-15T00:11:25.247 INFO:teuthology.orchestra.run.smithi171.stdout:107 trunc from 0x639c95a to 0x3491352 2024-09-15T00:11:25.257 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:25.258 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:25.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:25.258 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:25.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.258 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4287.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-15T00:11:25.258 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:10.541930+0000", 2024-09-15T00:11:25.258 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.713220872000001, 2024-09-15T00:11:25.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00078948500000000003, 2024-09-15T00:11:25.258 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.258 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.258 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.258 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.541930+0000", 2024-09-15T00:11:25.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.541930+0000", 2024-09-15T00:11:25.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.541930+0000", 2024-09-15T00:11:25.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.85e-07 2024-09-15T00:11:25.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.541935+0000", 2024-09-15T00:11:25.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7149999999999997e-06 2024-09-15T00:11:25.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.541946+0000", 2024-09-15T00:11:25.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1328999999999999e-05 2024-09-15T00:11:25.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.541956+0000", 2024-09-15T00:11:25.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7860000000000008e-06 2024-09-15T00:11:25.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.542005+0000", 2024-09-15T00:11:25.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8795000000000003e-05 2024-09-15T00:11:25.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.542030+0000", 2024-09-15T00:11:25.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4703e-05 2024-09-15T00:11:25.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.542676+0000", 2024-09-15T00:11:25.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0006466 2024-09-15T00:11:25.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:10.542720+0000", 2024-09-15T00:11:25.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3371999999999997e-05 2024-09-15T00:11:25.262 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.262 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.262 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.262 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4287.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-15T00:11:25.262 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:11.542948+0000", 2024-09-15T00:11:25.262 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.712203184, 2024-09-15T00:11:25.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016400029999999999, 2024-09-15T00:11:25.262 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.542948+0000", 2024-09-15T00:11:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.542948+0000", 2024-09-15T00:11:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.542948+0000", 2024-09-15T00:11:25.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0100000000000005e-07 2024-09-15T00:11:25.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.542957+0000", 2024-09-15T00:11:25.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6430000000000001e-06 2024-09-15T00:11:25.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.542975+0000", 2024-09-15T00:11:25.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7812000000000002e-05 2024-09-15T00:11:25.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.542989+0000", 2024-09-15T00:11:25.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3898e-05 2024-09-15T00:11:25.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.543067+0000", 2024-09-15T00:11:25.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8046000000000005e-05 2024-09-15T00:11:25.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.543093+0000", 2024-09-15T00:11:25.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6503e-05 2024-09-15T00:11:25.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.544526+0000", 2024-09-15T00:11:25.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014328089999999999 2024-09-15T00:11:25.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:11.544588+0000", 2024-09-15T00:11:25.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1791000000000005e-05 2024-09-15T00:11:25.266 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.266 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.266 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.267 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:7 3.1a e25/23 3:5cea7035:::rbd_info:head v 25'1, mlcod=0'0)", 2024-09-15T00:11:25.267 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:20.594306+0000", 2024-09-15T00:11:25.267 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6608452329999999, 2024-09-15T00:11:25.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00066814700000000001, 2024-09-15T00:11:25.267 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.267 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.267 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.267 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594306+0000", 2024-09-15T00:11:25.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594306+0000", 2024-09-15T00:11:25.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594306+0000", 2024-09-15T00:11:25.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.34e-07 2024-09-15T00:11:25.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594311+0000", 2024-09-15T00:11:25.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1939999999999997e-06 2024-09-15T00:11:25.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594321+0000", 2024-09-15T00:11:25.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8030000000000008e-06 2024-09-15T00:11:25.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594330+0000", 2024-09-15T00:11:25.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8319999999999995e-06 2024-09-15T00:11:25.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594371+0000", 2024-09-15T00:11:25.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1470000000000001e-05 2024-09-15T00:11:25.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594389+0000", 2024-09-15T00:11:25.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7450000000000001e-05 2024-09-15T00:11:25.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594942+0000", 2024-09-15T00:11:25.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00055328199999999999 2024-09-15T00:11:25.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:20.594974+0000", 2024-09-15T00:11:25.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1881999999999997e-05 2024-09-15T00:11:25.271 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.271 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.271 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.271 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:9 3.1a e26/23 3:5cea7035:::rbd_info:head v 26'2, mlcod=0'0)", 2024-09-15T00:11:25.271 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:21.596795+0000", 2024-09-15T00:11:25.271 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.6583556590000001, 2024-09-15T00:11:25.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000871363, 2024-09-15T00:11:25.271 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.271 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.271 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.271 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.596795+0000", 2024-09-15T00:11:25.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.596795+0000", 2024-09-15T00:11:25.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.596796+0000", 2024-09-15T00:11:25.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3799999999999998e-07 2024-09-15T00:11:25.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.596801+0000", 2024-09-15T00:11:25.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3639999999999998e-06 2024-09-15T00:11:25.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.596812+0000", 2024-09-15T00:11:25.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0423e-05 2024-09-15T00:11:25.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.596822+0000", 2024-09-15T00:11:25.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0766e-05 2024-09-15T00:11:25.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.596867+0000", 2024-09-15T00:11:25.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4437999999999997e-05 2024-09-15T00:11:25.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.596882+0000", 2024-09-15T00:11:25.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5318000000000001e-05 2024-09-15T00:11:25.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.597623+0000", 2024-09-15T00:11:25.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074066699999999997 2024-09-15T00:11:25.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.597667+0000", 2024-09-15T00:11:25.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4048999999999998e-05 2024-09-15T00:11:25.275 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.275 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.275 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.275 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:426 3.1a e28/23 3:5a1c0c11:::rbd_header.1139feddb981:head v 28'23, mlcod=27'21)", 2024-09-15T00:11:25.275 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:24.621290+0000", 2024-09-15T00:11:25.275 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.63386102700000002, 2024-09-15T00:11:25.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00260129, 2024-09-15T00:11:25.275 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.276 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.276 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.276 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.621290+0000", 2024-09-15T00:11:25.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.621290+0000", 2024-09-15T00:11:25.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.621290+0000", 2024-09-15T00:11:25.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1800000000000002e-07 2024-09-15T00:11:25.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.621297+0000", 2024-09-15T00:11:25.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2160000000000003e-06 2024-09-15T00:11:25.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.621309+0000", 2024-09-15T00:11:25.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.225e-05 2024-09-15T00:11:25.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.621319+0000", 2024-09-15T00:11:25.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0386e-05 2024-09-15T00:11:25.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.621389+0000", 2024-09-15T00:11:25.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.004e-05 2024-09-15T00:11:25.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.621414+0000", 2024-09-15T00:11:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4326999999999999e-05 2024-09-15T00:11:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.623843+0000", 2024-09-15T00:11:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002429755 2024-09-15T00:11:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.623891+0000", 2024-09-15T00:11:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7997999999999997e-05 2024-09-15T00:11:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.280 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:440 3.1a e28/23 3:5a1c0c11:::rbd_header.1139feddb981:head v 28'24, mlcod=27'22)", 2024-09-15T00:11:25.280 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:24.636745+0000", 2024-09-15T00:11:25.280 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.61840588699999999, 2024-09-15T00:11:25.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000788264, 2024-09-15T00:11:25.280 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.280 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.280 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.280 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.636745+0000", 2024-09-15T00:11:25.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.636745+0000", 2024-09-15T00:11:25.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.636747+0000", 2024-09-15T00:11:25.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.624e-06 2024-09-15T00:11:25.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.636765+0000", 2024-09-15T00:11:25.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8389999999999998e-05 2024-09-15T00:11:25.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.636784+0000", 2024-09-15T00:11:25.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069e-05 2024-09-15T00:11:25.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.636794+0000", 2024-09-15T00:11:25.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2969999999999992e-06 2024-09-15T00:11:25.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.636872+0000", 2024-09-15T00:11:25.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8858999999999998e-05 2024-09-15T00:11:25.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.636898+0000", 2024-09-15T00:11:25.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5282999999999998e-05 2024-09-15T00:11:25.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.637500+0000", 2024-09-15T00:11:25.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00060229800000000003 2024-09-15T00:11:25.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.637533+0000", 2024-09-15T00:11:25.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3444e-05 2024-09-15T00:11:25.283 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.284 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.284 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.284 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.18 29/28 [PushOp(3:1a690a65:::rbd_data.1139feddb981.0000000000000008:head, version: 27'10, data_included: [460702~52356], data_size: 52356, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:1a690a65:::rbd_data.1139feddb981.0000000000000008:head@27'10, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:11:25.284 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.007964+0000", 2024-09-15T00:11:25.284 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.24718743700000001, 2024-09-15T00:11:25.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00085299400000000004, 2024-09-15T00:11:25.284 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.284 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.284 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:25.284 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.007964+0000", 2024-09-15T00:11:25.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.007964+0000", 2024-09-15T00:11:25.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.007964+0000", 2024-09-15T00:11:25.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-15T00:11:25.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.008024+0000", 2024-09-15T00:11:25.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0257000000000003e-05 2024-09-15T00:11:25.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.008046+0000", 2024-09-15T00:11:25.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1378000000000001e-05 2024-09-15T00:11:25.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.008068+0000", 2024-09-15T00:11:25.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2368999999999999e-05 2024-09-15T00:11:25.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.008168+0000", 2024-09-15T00:11:25.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000100098 2024-09-15T00:11:25.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.008183+0000", 2024-09-15T00:11:25.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5085e-05 2024-09-15T00:11:25.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.008817+0000", 2024-09-15T00:11:25.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00063319600000000004 2024-09-15T00:11:25.287 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.287 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.288 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.288 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:462 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:25.288 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.015785+0000", 2024-09-15T00:11:25.288 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.239365727, 2024-09-15T00:11:25.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042325740000000002, 2024-09-15T00:11:25.288 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.288 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.288 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:25.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:25.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:25.288 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 462 2024-09-15T00:11:25.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.289 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.015785+0000", 2024-09-15T00:11:25.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.015785+0000", 2024-09-15T00:11:25.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.015787+0000", 2024-09-15T00:11:25.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3209999999999999e-06 2024-09-15T00:11:25.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.015793+0000", 2024-09-15T00:11:25.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4309999999999999e-06 2024-09-15T00:11:25.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.015805+0000", 2024-09-15T00:11:25.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1671000000000001e-05 2024-09-15T00:11:25.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.015810+0000", 2024-09-15T00:11:25.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5210000000000002e-06 2024-09-15T00:11:25.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.015859+0000", 2024-09-15T00:11:25.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8498999999999998e-05 2024-09-15T00:11:25.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.015962+0000", 2024-09-15T00:11:25.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010357300000000001 2024-09-15T00:11:25.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:25.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016092+0000", 2024-09-15T00:11:25.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012920999999999999 2024-09-15T00:11:25.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:25.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019435+0000", 2024-09-15T00:11:25.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003343356 2024-09-15T00:11:25.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:25.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019954+0000", 2024-09-15T00:11:25.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000519025 2024-09-15T00:11:25.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019987+0000", 2024-09-15T00:11:25.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2969999999999998e-05 2024-09-15T00:11:25.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020018+0000", 2024-09-15T00:11:25.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0997000000000002e-05 2024-09-15T00:11:25.293 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.294 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.294 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.294 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:466 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:25.294 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.035821+0000", 2024-09-15T00:11:25.294 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.21933022199999999, 2024-09-15T00:11:25.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032551500000000002, 2024-09-15T00:11:25.294 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.294 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.294 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:25.294 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:25.294 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:25.295 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:25.295 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 466 2024-09-15T00:11:25.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.295 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.035821+0000", 2024-09-15T00:11:25.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.035821+0000", 2024-09-15T00:11:25.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.035823+0000", 2024-09-15T00:11:25.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6690000000000002e-06 2024-09-15T00:11:25.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.035832+0000", 2024-09-15T00:11:25.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7639999999999994e-06 2024-09-15T00:11:25.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.035855+0000", 2024-09-15T00:11:25.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2708999999999999e-05 2024-09-15T00:11:25.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.035865+0000", 2024-09-15T00:11:25.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5559999999999996e-06 2024-09-15T00:11:25.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.035943+0000", 2024-09-15T00:11:25.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8588e-05 2024-09-15T00:11:25.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.036043+0000", 2024-09-15T00:11:25.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9451999999999996e-05 2024-09-15T00:11:25.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.036146+0000", 2024-09-15T00:11:25.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010377700000000001 2024-09-15T00:11:25.298 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.298 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.298 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.298 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:467 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:25.298 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.036238+0000", 2024-09-15T00:11:25.298 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.21891271900000001, 2024-09-15T00:11:25.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00037985699999999999, 2024-09-15T00:11:25.299 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.299 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.299 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:25.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:25.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:25.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:25.299 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 467 2024-09-15T00:11:25.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.299 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.036238+0000", 2024-09-15T00:11:25.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.036238+0000", 2024-09-15T00:11:25.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.036239+0000", 2024-09-15T00:11:25.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999995e-07 2024-09-15T00:11:25.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.036245+0000", 2024-09-15T00:11:25.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0379999999999996e-06 2024-09-15T00:11:25.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.036255+0000", 2024-09-15T00:11:25.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.662e-06 2024-09-15T00:11:25.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.036260+0000", 2024-09-15T00:11:25.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:11:25.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.036303+0000", 2024-09-15T00:11:25.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3300000000000002e-05 2024-09-15T00:11:25.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.036368+0000", 2024-09-15T00:11:25.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4923000000000002e-05 2024-09-15T00:11:25.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.036618+0000", 2024-09-15T00:11:25.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025033599999999998 2024-09-15T00:11:25.303 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.303 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.303 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.303 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:496 3.1a e29/23 3:5a1c0c11:::rbd_header.1139feddb981:head v 29'25, mlcod=28'23)", 2024-09-15T00:11:25.303 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.060365+0000", 2024-09-15T00:11:25.303 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.19478578399999999, 2024-09-15T00:11:25.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00057899099999999995, 2024-09-15T00:11:25.303 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.303 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.303 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.303 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.060365+0000", 2024-09-15T00:11:25.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.060365+0000", 2024-09-15T00:11:25.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.060366+0000", 2024-09-15T00:11:25.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.001e-06 2024-09-15T00:11:25.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.060374+0000", 2024-09-15T00:11:25.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0199999999999994e-06 2024-09-15T00:11:25.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.060388+0000", 2024-09-15T00:11:25.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.414e-05 2024-09-15T00:11:25.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.060395+0000", 2024-09-15T00:11:25.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9750000000000001e-06 2024-09-15T00:11:25.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.060450+0000", 2024-09-15T00:11:25.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4938000000000002e-05 2024-09-15T00:11:25.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.060465+0000", 2024-09-15T00:11:25.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.506e-05 2024-09-15T00:11:25.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.060915+0000", 2024-09-15T00:11:25.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000449278 2024-09-15T00:11:25.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.060944+0000", 2024-09-15T00:11:25.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9578999999999999e-05 2024-09-15T00:11:25.307 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.307 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.307 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.307 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:498 3.1a e29/23 3:5a1c0c11:::rbd_header.1139feddb981:head v 29'26, mlcod=28'24)", 2024-09-15T00:11:25.307 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.061567+0000", 2024-09-15T00:11:25.308 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.19358376499999999, 2024-09-15T00:11:25.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00043012200000000001, 2024-09-15T00:11:25.308 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.308 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.308 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.308 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.061567+0000", 2024-09-15T00:11:25.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.061567+0000", 2024-09-15T00:11:25.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.061568+0000", 2024-09-15T00:11:25.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:11:25.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.061576+0000", 2024-09-15T00:11:25.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.473e-06 2024-09-15T00:11:25.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.061584+0000", 2024-09-15T00:11:25.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1980000000000001e-06 2024-09-15T00:11:25.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.061589+0000", 2024-09-15T00:11:25.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.313e-06 2024-09-15T00:11:25.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.061624+0000", 2024-09-15T00:11:25.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5539e-05 2024-09-15T00:11:25.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.061637+0000", 2024-09-15T00:11:25.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2199e-05 2024-09-15T00:11:25.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.061974+0000", 2024-09-15T00:11:25.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033715600000000002 2024-09-15T00:11:25.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.061997+0000", 2024-09-15T00:11:25.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3705000000000001e-05 2024-09-15T00:11:25.311 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.311 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.311 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.312 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:552 3.6 3:678920ff:::rbd_data.11395e6383ec.000000000000000d:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:25.312 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.099995+0000", 2024-09-15T00:11:25.312 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.155155763, 2024-09-15T00:11:25.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000218235, 2024-09-15T00:11:25.312 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.312 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.312 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:11:25.312 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:25.312 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:25.312 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:25.312 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 552 2024-09-15T00:11:25.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.313 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.099995+0000", 2024-09-15T00:11:25.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.099995+0000", 2024-09-15T00:11:25.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.099997+0000", 2024-09-15T00:11:25.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0779999999999998e-06 2024-09-15T00:11:25.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.100010+0000", 2024-09-15T00:11:25.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2499e-05 2024-09-15T00:11:25.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.100032+0000", 2024-09-15T00:11:25.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1772000000000001e-05 2024-09-15T00:11:25.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.100041+0000", 2024-09-15T00:11:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3959999999999995e-06 2024-09-15T00:11:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.100112+0000", 2024-09-15T00:11:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1320000000000002e-05 2024-09-15T00:11:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.100214+0000", 2024-09-15T00:11:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010117 2024-09-15T00:11:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.316 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.316 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.316 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:589 3.8 e29/23 3:17096be3:::rbd_data.11397bff333d.0000000000000034:head v 29'6, mlcod=27'4)", 2024-09-15T00:11:25.316 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.196298+0000", 2024-09-15T00:11:25.316 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.058852873, 2024-09-15T00:11:25.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00095785499999999997, 2024-09-15T00:11:25.316 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.316 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.316 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.316 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.196298+0000", 2024-09-15T00:11:25.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.196298+0000", 2024-09-15T00:11:25.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.196299+0000", 2024-09-15T00:11:25.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.122e-06 2024-09-15T00:11:25.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.196322+0000", 2024-09-15T00:11:25.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3098000000000001e-05 2024-09-15T00:11:25.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.196339+0000", 2024-09-15T00:11:25.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6738000000000001e-05 2024-09-15T00:11:25.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.196346+0000", 2024-09-15T00:11:25.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.861e-06 2024-09-15T00:11:25.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.196412+0000", 2024-09-15T00:11:25.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6000999999999994e-05 2024-09-15T00:11:25.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.196436+0000", 2024-09-15T00:11:25.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3757999999999999e-05 2024-09-15T00:11:25.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.197224+0000", 2024-09-15T00:11:25.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00078823599999999999 2024-09-15T00:11:25.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.197256+0000", 2024-09-15T00:11:25.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2041e-05 2024-09-15T00:11:25.321 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.321 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.321 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.321 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:591 3.8 e29/23 3:10e98333:::rbd_data.11397bff333d.0000000000000019:head v 29'7, mlcod=27'5)", 2024-09-15T00:11:25.321 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.199769+0000", 2024-09-15T00:11:25.321 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.055382512000000002, 2024-09-15T00:11:25.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090941399999999999, 2024-09-15T00:11:25.322 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.322 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.322 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.322 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.199769+0000", 2024-09-15T00:11:25.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.199769+0000", 2024-09-15T00:11:25.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.199769+0000", 2024-09-15T00:11:25.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3099999999999996e-07 2024-09-15T00:11:25.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.199813+0000", 2024-09-15T00:11:25.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3788999999999997e-05 2024-09-15T00:11:25.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.199825+0000", 2024-09-15T00:11:25.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2003e-05 2024-09-15T00:11:25.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.199832+0000", 2024-09-15T00:11:25.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2119999999999997e-06 2024-09-15T00:11:25.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.199878+0000", 2024-09-15T00:11:25.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5178e-05 2024-09-15T00:11:25.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.199892+0000", 2024-09-15T00:11:25.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.438e-05 2024-09-15T00:11:25.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.200629+0000", 2024-09-15T00:11:25.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000736737 2024-09-15T00:11:25.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.200678+0000", 2024-09-15T00:11:25.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9283999999999998e-05 2024-09-15T00:11:25.326 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.326 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.326 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.326 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:604 3.4 3:255b0913:::rbd_data.11397bff333d.0000000000000039:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3538944~24696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:25.326 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.213414+0000", 2024-09-15T00:11:25.326 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.041736623, 2024-09-15T00:11:25.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014540779999999999, 2024-09-15T00:11:25.326 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.326 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.326 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.326 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:25.326 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:25.327 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:25.327 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 604 2024-09-15T00:11:25.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.327 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.213414+0000", 2024-09-15T00:11:25.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.213414+0000", 2024-09-15T00:11:25.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.213416+0000", 2024-09-15T00:11:25.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.725e-06 2024-09-15T00:11:25.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.213435+0000", 2024-09-15T00:11:25.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8664999999999998e-05 2024-09-15T00:11:25.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.213451+0000", 2024-09-15T00:11:25.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6614000000000001e-05 2024-09-15T00:11:25.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.213459+0000", 2024-09-15T00:11:25.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7400000000000004e-06 2024-09-15T00:11:25.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.213509+0000", 2024-09-15T00:11:25.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9484000000000003e-05 2024-09-15T00:11:25.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.213623+0000", 2024-09-15T00:11:25.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114806 2024-09-15T00:11:25.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:25.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.213724+0000", 2024-09-15T00:11:25.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000100076 2024-09-15T00:11:25.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:25.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.214456+0000", 2024-09-15T00:11:25.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00073228799999999999 2024-09-15T00:11:25.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:25.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.214788+0000", 2024-09-15T00:11:25.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033252299999999999 2024-09-15T00:11:25.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.214823+0000", 2024-09-15T00:11:25.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4594000000000001e-05 2024-09-15T00:11:25.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.214869+0000", 2024-09-15T00:11:25.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5562999999999997e-05 2024-09-15T00:11:25.333 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.333 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.333 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.333 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:610 3.4 3:255b0913:::rbd_data.11397bff333d.0000000000000039:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:25.333 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.228156+0000", 2024-09-15T00:11:25.333 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.026995371000000001, 2024-09-15T00:11:25.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037229099999999998, 2024-09-15T00:11:25.333 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.333 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.333 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.333 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:25.333 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:25.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:25.334 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 610 2024-09-15T00:11:25.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.334 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228156+0000", 2024-09-15T00:11:25.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228156+0000", 2024-09-15T00:11:25.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228157+0000", 2024-09-15T00:11:25.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6169999999999999e-06 2024-09-15T00:11:25.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228167+0000", 2024-09-15T00:11:25.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5950000000000005e-06 2024-09-15T00:11:25.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228184+0000", 2024-09-15T00:11:25.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7289e-05 2024-09-15T00:11:25.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228194+0000", 2024-09-15T00:11:25.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5170000000000005e-06 2024-09-15T00:11:25.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228256+0000", 2024-09-15T00:11:25.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2786999999999994e-05 2024-09-15T00:11:25.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228316+0000", 2024-09-15T00:11:25.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9103e-05 2024-09-15T00:11:25.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:25.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228398+0000", 2024-09-15T00:11:25.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2855999999999994e-05 2024-09-15T00:11:25.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:25.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228976+0000", 2024-09-15T00:11:25.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00057762099999999995 2024-09-15T00:11:25.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:25.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231781+0000", 2024-09-15T00:11:25.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028045549999999998 2024-09-15T00:11:25.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231837+0000", 2024-09-15T00:11:25.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5986999999999998e-05 2024-09-15T00:11:25.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231879+0000", 2024-09-15T00:11:25.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1983000000000001e-05 2024-09-15T00:11:25.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.338 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.339 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:608 3.8 e29/23 3:17096be3:::rbd_data.11397bff333d.0000000000000034:head v 29'8, mlcod=29'6)", 2024-09-15T00:11:25.339 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.228521+0000", 2024-09-15T00:11:25.339 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.026629940000000001, 2024-09-15T00:11:25.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003218304, 2024-09-15T00:11:25.339 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.339 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.339 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.339 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228521+0000", 2024-09-15T00:11:25.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228521+0000", 2024-09-15T00:11:25.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228522+0000", 2024-09-15T00:11:25.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.164e-06 2024-09-15T00:11:25.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228530+0000", 2024-09-15T00:11:25.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9170000000000006e-06 2024-09-15T00:11:25.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228542+0000", 2024-09-15T00:11:25.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.151e-05 2024-09-15T00:11:25.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228548+0000", 2024-09-15T00:11:25.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-06 2024-09-15T00:11:25.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228643+0000", 2024-09-15T00:11:25.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4216999999999996e-05 2024-09-15T00:11:25.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228660+0000", 2024-09-15T00:11:25.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6969e-05 2024-09-15T00:11:25.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231698+0000", 2024-09-15T00:11:25.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030388329999999999 2024-09-15T00:11:25.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231739+0000", 2024-09-15T00:11:25.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0924000000000003e-05 2024-09-15T00:11:25.343 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.343 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.344 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.344 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:612 3.8 e29/23 3:10e98333:::rbd_data.11397bff333d.0000000000000019:head v 29'9, mlcod=29'7)", 2024-09-15T00:11:25.344 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.233716+0000", 2024-09-15T00:11:25.344 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.021435075000000001, 2024-09-15T00:11:25.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00075137399999999997, 2024-09-15T00:11:25.344 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.344 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.345 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.345 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.233716+0000", 2024-09-15T00:11:25.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.233716+0000", 2024-09-15T00:11:25.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.233717+0000", 2024-09-15T00:11:25.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3900000000000004e-07 2024-09-15T00:11:25.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.233723+0000", 2024-09-15T00:11:25.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3419999999999996e-06 2024-09-15T00:11:25.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.233736+0000", 2024-09-15T00:11:25.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3139000000000001e-05 2024-09-15T00:11:25.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.233743+0000", 2024-09-15T00:11:25.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1709999999999996e-06 2024-09-15T00:11:25.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.233795+0000", 2024-09-15T00:11:25.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1932999999999999e-05 2024-09-15T00:11:25.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.233810+0000", 2024-09-15T00:11:25.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4552000000000001e-05 2024-09-15T00:11:25.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.234427+0000", 2024-09-15T00:11:25.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000617177 2024-09-15T00:11:25.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.234467+0000", 2024-09-15T00:11:25.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0221e-05 2024-09-15T00:11:25.350 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.351 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.351 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.351 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:625 3.8 e29/23 3:10d3220b:::rbd_data.11397bff333d.0000000000000011:head v 29'10, mlcod=29'8)", 2024-09-15T00:11:25.351 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.241514+0000", 2024-09-15T00:11:25.351 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.013637285000000001, 2024-09-15T00:11:25.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00080800299999999995, 2024-09-15T00:11:25.351 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.352 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.352 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.352 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.241514+0000", 2024-09-15T00:11:25.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.241514+0000", 2024-09-15T00:11:25.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.241515+0000", 2024-09-15T00:11:25.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.175e-06 2024-09-15T00:11:25.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.241542+0000", 2024-09-15T00:11:25.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7138999999999999e-05 2024-09-15T00:11:25.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.241558+0000", 2024-09-15T00:11:25.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5651e-05 2024-09-15T00:11:25.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.241565+0000", 2024-09-15T00:11:25.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6550000000000004e-06 2024-09-15T00:11:25.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.241620+0000", 2024-09-15T00:11:25.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5081000000000003e-05 2024-09-15T00:11:25.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.241643+0000", 2024-09-15T00:11:25.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2472000000000001e-05 2024-09-15T00:11:25.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.242279+0000", 2024-09-15T00:11:25.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00063640799999999998 2024-09-15T00:11:25.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.242322+0000", 2024-09-15T00:11:25.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2422000000000001e-05 2024-09-15T00:11:25.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.357 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.357 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.357 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:25.358 INFO:teuthology.orchestra.run.smithi171.stdout:108 write 0x1999f36 thru 0x19a9dd4 (0xfe9f bytes) 2024-09-15T00:11:25.358 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:25.359 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:25.361 INFO:teuthology.orchestra.run.smithi171.stdout:110 write 0xc14a011 thru 0xc154461 (0xa451 bytes) 2024-09-15T00:11:25.361 INFO:teuthology.orchestra.run.smithi171.stdout:111 write 0x38c36c7 thru 0x38d2611 (0xef4b bytes) 2024-09-15T00:11:25.361 INFO:teuthology.orchestra.run.smithi171.stdout:112 read 0xb3e324e thru 0xb3eb4dd (0x8290 bytes) 2024-09-15T00:11:25.361 INFO:teuthology.orchestra.run.smithi171.stdout:113 punch from 0x965b7d4 to 0x96637f5, (0x8021 bytes) 2024-09-15T00:11:25.361 INFO:teuthology.orchestra.run.smithi171.stdout:114 trunc from 0xc154462 to 0x2bea66b 2024-09-15T00:11:25.361 INFO:teuthology.orchestra.run.smithi171.stdout:115 trunc from 0x2bea66b to 0x1ef230e 2024-09-15T00:11:25.361 INFO:teuthology.orchestra.run.smithi171.stdout:116 writesame 0xb702af3 thru 0xb70a684 data_size 0x1(0x7b92 bytes) 2024-09-15T00:11:25.361 INFO:teuthology.orchestra.run.smithi171.stdout:truncating to largest ever: 0xec2a633 2024-09-15T00:11:25.361 INFO:teuthology.orchestra.run.smithi171.stdout:118 trunc from 0xb70a685 to 0xec2a633 2024-09-15T00:11:25.361 INFO:teuthology.orchestra.run.smithi171.stdout:119 write 0xcf99ead thru 0xcf9d759 (0x38ad bytes) 2024-09-15T00:11:25.361 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:25.361 INFO:teuthology.orchestra.run.smithi171.stdout:121 writesame 0xd3f9a96 thru 0xd404012 data_size 0x25(0xa57d bytes) 2024-09-15T00:11:25.362 INFO:teuthology.orchestra.run.smithi171.stdout:123 write 0xe2968a0 thru 0xe29ca34 (0x6195 bytes) 2024-09-15T00:11:25.362 INFO:teuthology.orchestra.run.smithi171.stdout:124 write 0xd64708 thru 0xd6a94c (0x6245 bytes) 2024-09-15T00:11:25.362 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:25.356+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:25.364 INFO:teuthology.orchestra.run.smithi171.stdout:125 writesame 0x8164ebe thru 0x816de5d data_size 0x30(0x8fa0 bytes) 2024-09-15T00:11:25.367 INFO:teuthology.orchestra.run.smithi171.stdout:126 write 0x5a76278 thru 0x5a83d38 (0xdac1 bytes) 2024-09-15T00:11:25.372 INFO:teuthology.orchestra.run.smithi171.stdout:127 write 0xd688c12 thru 0xd6973da (0xe7c9 bytes) 2024-09-15T00:11:25.374 INFO:teuthology.orchestra.run.smithi171.stdout:128 read 0x17a3431 thru 0x17a7dea (0x49ba bytes) 2024-09-15T00:11:25.375 INFO:teuthology.orchestra.run.smithi171.stdout:130 writesame 0xa750117 thru 0xa750dd4 data_size 0x7(0xcbe bytes) 2024-09-15T00:11:25.377 INFO:teuthology.orchestra.run.smithi171.stdout:131 writesame 0x2a02554 thru 0x2a0b6ea data_size 0x2f(0x9197 bytes) 2024-09-15T00:11:25.380 INFO:teuthology.orchestra.run.smithi171.stdout:134 write 0xd82f1e6 thru 0xd83a72f (0xb54a bytes) 2024-09-15T00:11:25.386 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:25.402 INFO:teuthology.orchestra.run.smithi171.stdout:136 trunc from 0xec2a633 to 0xfc4284 2024-09-15T00:11:25.432 INFO:teuthology.orchestra.run.smithi171.stdout:137 write 0x9ccf757 thru 0x9cd9fb8 (0xa862 bytes) 2024-09-15T00:11:25.438 INFO:teuthology.orchestra.run.smithi171.stdout:138 punch from 0x7b598eb to 0x7b5a165, (0x87a bytes) 2024-09-15T00:11:25.438 INFO:teuthology.orchestra.run.smithi171.stdout:139 punch from 0x8fc604a to 0x8fcbd2e, (0x5ce4 bytes) 2024-09-15T00:11:25.438 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:25.444 INFO:teuthology.orchestra.run.smithi171.stdout:141 trunc from 0x9cd9fb9 to 0x54fa134 2024-09-15T00:11:25.452 INFO:teuthology.orchestra.run.smithi171.stdout:142 read 0xcac2e thru 0xd1577 (0x694a bytes) 2024-09-15T00:11:25.453 INFO:teuthology.orchestra.run.smithi171.stdout:143 punch from 0x2a634d8 to 0x2a68047, (0x4b6f bytes) 2024-09-15T00:11:25.453 INFO:teuthology.orchestra.run.smithi171.stdout:144 punch from 0x428a686 to 0x428f00c, (0x4986 bytes) 2024-09-15T00:11:25.453 INFO:teuthology.orchestra.run.smithi171.stdout:146 punch from 0x2418029 to 0x2424b53, (0xcb2a bytes) 2024-09-15T00:11:25.453 INFO:teuthology.orchestra.run.smithi171.stdout:147 read 0x5418ace thru 0x54197d1 (0xd04 bytes) 2024-09-15T00:11:25.460 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:25.457+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:25.462 INFO:teuthology.orchestra.run.smithi171.stdout:148 trunc from 0x54fa134 to 0xc9b46e5 2024-09-15T00:11:25.467 INFO:teuthology.orchestra.run.smithi171.stdout:150 read 0x8e56d5d thru 0x8e62a1f (0xbcc3 bytes) 2024-09-15T00:11:25.467 INFO:teuthology.orchestra.run.smithi171.stdout:151 write 0x360056e thru 0x360c918 (0xc3ab bytes) 2024-09-15T00:11:25.472 INFO:teuthology.orchestra.run.smithi171.stdout:153 write 0x79c71dd thru 0x79d045a (0x927e bytes) 2024-09-15T00:11:25.476 INFO:teuthology.orchestra.run.smithi171.stdout:155 read 0x62db6ee thru 0x62df2e2 (0x3bf5 bytes) 2024-09-15T00:11:25.479 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:25.479 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:25.479 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:25.479 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:25.489 INFO:teuthology.orchestra.run.smithi171.stdout:156 trunc from 0xc9b46e5 to 0x24ad694 2024-09-15T00:11:25.493 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:25.499 INFO:teuthology.orchestra.run.smithi171.stdout:159 read 0xe511a4 thru 0xe5c301 (0xb15e bytes) 2024-09-15T00:11:25.499 INFO:teuthology.orchestra.run.smithi171.stdout:161 trunc from 0x24ad694 to 0x2f72138 2024-09-15T00:11:25.525 INFO:teuthology.orchestra.run.smithi171.stdout:163 writesame 0xd2930ba thru 0xd29f6f3 data_size 0x1(0xc63a bytes) 2024-09-15T00:11:25.525 INFO:teuthology.orchestra.run.smithi171.stdout:164 read 0x57ef39a thru 0x57faecd (0xbb34 bytes) 2024-09-15T00:11:25.525 INFO:teuthology.orchestra.run.smithi171.stdout:166 write 0x7dadf29 thru 0x7db7ffd (0xa0d5 bytes) 2024-09-15T00:11:25.534 INFO:teuthology.orchestra.run.smithi171.stdout:167 punch from 0x8dba051 to 0x8dc6838, (0xc7e7 bytes) 2024-09-15T00:11:25.534 INFO:teuthology.orchestra.run.smithi171.stdout:168 write 0x2c19726 thru 0x2c1f92a (0x6205 bytes) 2024-09-15T00:11:25.541 INFO:teuthology.orchestra.run.smithi171.stdout:172 read 0x2b60a8e thru 0x2b67c50 (0x71c3 bytes) 2024-09-15T00:11:25.541 INFO:teuthology.orchestra.run.smithi171.stdout:174 write 0x33d9bf2 thru 0x33e6f55 (0xd364 bytes) 2024-09-15T00:11:25.548 INFO:teuthology.orchestra.run.smithi171.stdout:175 writesame 0x495ddd9 thru 0x495fd0d data_size 0x1(0x1f35 bytes) 2024-09-15T00:11:25.553 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:25.554 INFO:teuthology.orchestra.run.smithi171.stdout:177 trunc from 0xd29f6f4 to 0xca7c376 2024-09-15T00:11:25.560 INFO:teuthology.orchestra.run.smithi171.stdout:178 writesame 0x7daa4cc thru 0x7db6ce3 data_size 0x4c(0xc818 bytes) 2024-09-15T00:11:25.560 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:25.559+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:25.562 INFO:teuthology.orchestra.run.smithi171.stdout:179 read 0x17f8056 thru 0x18041e3 (0xc18e bytes) 2024-09-15T00:11:25.562 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:25.565 INFO:teuthology.orchestra.run.smithi171.stdout:181 trunc from 0xca7c376 to 0x9f4918f 2024-09-15T00:11:25.572 INFO:teuthology.orchestra.run.smithi171.stdout:183 writesame 0xe1ad66 thru 0xe2aac8 data_size 0x1(0xfd63 bytes) 2024-09-15T00:11:25.608 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:25.608 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:25.616 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:25.691 INFO:teuthology.orchestra.run.smithi171.stdout:184 trunc from 0x9f4918f to 0xb188eb6 2024-09-15T00:11:25.691 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:25.688+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:25.696 INFO:teuthology.orchestra.run.smithi171.stdout:185 writesame 0x9f40661 thru 0x9f441fe data_size 0xd(0x3b9e bytes) 2024-09-15T00:11:25.703 INFO:teuthology.orchestra.run.smithi171.stdout:188 read 0x821ae2f thru 0x822892c (0xdafe bytes) 2024-09-15T00:11:25.703 INFO:teuthology.orchestra.run.smithi171.stdout:189 punch from 0xa8a1ea7 to 0xa8ab85a, (0x99b3 bytes) 2024-09-15T00:11:25.714 INFO:teuthology.orchestra.run.smithi171.stdout:190 trunc from 0xb188eb6 to 0x1558009 2024-09-15T00:11:25.731 INFO:teuthology.orchestra.run.smithi171.stdout:191 read 0x1165033 thru 0x1174cc8 (0xfc96 bytes) 2024-09-15T00:11:25.750 INFO:teuthology.orchestra.run.smithi171.stdout:192 write 0xb7468c0 thru 0xb754c64 (0xe3a5 bytes) 2024-09-15T00:11:25.756 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:25.756 INFO:teuthology.orchestra.run.smithi171.stdout:195 read 0x77c890f thru 0x77cb1ed (0x28df bytes) 2024-09-15T00:11:25.757 INFO:teuthology.orchestra.run.smithi171.stdout:198 trunc from 0xb754c65 to 0xc5ddead 2024-09-15T00:11:25.761 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:25.760+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:25.764 INFO:teuthology.orchestra.run.smithi171.stdout:199 punch from 0x287327a to 0x287924c, (0x5fd2 bytes) 2024-09-15T00:11:25.767 INFO:teuthology.orchestra.run.smithi171.stdout:201 trunc from 0xc5ddead to 0x9b4b16b 2024-09-15T00:11:25.774 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:25.774 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:25.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:25.774 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:25.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.774 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:462 3.0 e29/23 3:04479396:::rbd_id.image_client.0-clone2:head v 29'14, mlcod=27'12)", 2024-09-15T00:11:25.774 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:25.774 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.75536025100000004, 2024-09-15T00:11:25.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030963029999999999, 2024-09-15T00:11:25.774 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.775 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.775 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.775 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:25.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:25.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:25.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:11:25.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016208+0000", 2024-09-15T00:11:25.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3669999999999997e-06 2024-09-15T00:11:25.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016250+0000", 2024-09-15T00:11:25.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1845999999999999e-05 2024-09-15T00:11:25.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016260+0000", 2024-09-15T00:11:25.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9750000000000002e-06 2024-09-15T00:11:25.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016342+0000", 2024-09-15T00:11:25.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1595999999999996e-05 2024-09-15T00:11:25.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016363+0000", 2024-09-15T00:11:25.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1073e-05 2024-09-15T00:11:25.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019246+0000", 2024-09-15T00:11:25.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028835240000000002 2024-09-15T00:11:25.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019296+0000", 2024-09-15T00:11:25.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9533999999999997e-05 2024-09-15T00:11:25.778 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.778 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.778 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.778 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:463 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'9, mlcod=27'7)", 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.75074724699999995, 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040015709999999998, 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:25.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:25.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7599999999999999e-07 2024-09-15T00:11:25.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020818+0000", 2024-09-15T00:11:25.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6609999999999999e-06 2024-09-15T00:11:25.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020826+0000", 2024-09-15T00:11:25.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4770000000000003e-06 2024-09-15T00:11:25.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020833+0000", 2024-09-15T00:11:25.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0809999999999996e-06 2024-09-15T00:11:25.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020878+0000", 2024-09-15T00:11:25.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4752000000000001e-05 2024-09-15T00:11:25.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020892+0000", 2024-09-15T00:11:25.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3899e-05 2024-09-15T00:11:25.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024767+0000", 2024-09-15T00:11:25.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038752259999999998 2024-09-15T00:11:25.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024814+0000", 2024-09-15T00:11:25.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7299000000000002e-05 2024-09-15T00:11:25.782 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.782 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.782 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.783 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:464 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'10, mlcod=27'8)", 2024-09-15T00:11:25.783 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:25.783 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.745827405, 2024-09-15T00:11:25.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048109290000000002, 2024-09-15T00:11:25.783 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.783 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.783 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.783 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:25.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:25.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:25.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-15T00:11:25.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025743+0000", 2024-09-15T00:11:25.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5789999999999993e-06 2024-09-15T00:11:25.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025752+0000", 2024-09-15T00:11:25.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8440000000000004e-06 2024-09-15T00:11:25.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025757+0000", 2024-09-15T00:11:25.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2610000000000001e-06 2024-09-15T00:11:25.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025796+0000", 2024-09-15T00:11:25.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9428999999999998e-05 2024-09-15T00:11:25.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025810+0000", 2024-09-15T00:11:25.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3129000000000001e-05 2024-09-15T00:11:25.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030514+0000", 2024-09-15T00:11:25.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047040099999999998 2024-09-15T00:11:25.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030544+0000", 2024-09-15T00:11:25.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9966000000000001e-05 2024-09-15T00:11:25.787 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.787 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.787 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.787 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:465 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'11, mlcod=29'9)", 2024-09-15T00:11:25.787 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.031516+0000", 2024-09-15T00:11:25.787 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.74004369800000003, 2024-09-15T00:11:25.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018997930000000001, 2024-09-15T00:11:25.787 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.787 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.788 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.788 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.031516+0000", 2024-09-15T00:11:25.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.031516+0000", 2024-09-15T00:11:25.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.031517+0000", 2024-09-15T00:11:25.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2900000000000003e-07 2024-09-15T00:11:25.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.031524+0000", 2024-09-15T00:11:25.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5689999999999999e-06 2024-09-15T00:11:25.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.031531+0000", 2024-09-15T00:11:25.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8589999999999999e-06 2024-09-15T00:11:25.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.031538+0000", 2024-09-15T00:11:25.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5799999999999997e-06 2024-09-15T00:11:25.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.031591+0000", 2024-09-15T00:11:25.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3239999999999998e-05 2024-09-15T00:11:25.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.031612+0000", 2024-09-15T00:11:25.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0992000000000002e-05 2024-09-15T00:11:25.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.033374+0000", 2024-09-15T00:11:25.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017619650000000001 2024-09-15T00:11:25.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.033416+0000", 2024-09-15T00:11:25.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1959000000000003e-05 2024-09-15T00:11:25.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.791 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.792 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:478 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'12, mlcod=29'10)", 2024-09-15T00:11:25.792 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.040495+0000", 2024-09-15T00:11:25.792 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.73106495299999996, 2024-09-15T00:11:25.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001856468, 2024-09-15T00:11:25.792 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.792 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.792 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.792 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.040495+0000", 2024-09-15T00:11:25.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.040495+0000", 2024-09-15T00:11:25.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.040496+0000", 2024-09-15T00:11:25.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7400000000000002e-07 2024-09-15T00:11:25.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.040503+0000", 2024-09-15T00:11:25.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9619999999999996e-06 2024-09-15T00:11:25.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.040514+0000", 2024-09-15T00:11:25.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1094000000000001e-05 2024-09-15T00:11:25.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.040521+0000", 2024-09-15T00:11:25.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8050000000000001e-06 2024-09-15T00:11:25.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.040565+0000", 2024-09-15T00:11:25.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.392e-05 2024-09-15T00:11:25.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.040580+0000", 2024-09-15T00:11:25.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5420000000000001e-05 2024-09-15T00:11:25.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.042309+0000", 2024-09-15T00:11:25.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001728776 2024-09-15T00:11:25.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.042351+0000", 2024-09-15T00:11:25.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2716999999999998e-05 2024-09-15T00:11:25.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.796 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.796 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:479 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'13, mlcod=29'11)", 2024-09-15T00:11:25.797 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.042962+0000", 2024-09-15T00:11:25.797 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.72859809399999997, 2024-09-15T00:11:25.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020572609999999999, 2024-09-15T00:11:25.797 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.797 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.797 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.797 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.042962+0000", 2024-09-15T00:11:25.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.042962+0000", 2024-09-15T00:11:25.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.042962+0000", 2024-09-15T00:11:25.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:11:25.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.042971+0000", 2024-09-15T00:11:25.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6149999999999997e-06 2024-09-15T00:11:25.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.042980+0000", 2024-09-15T00:11:25.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:11:25.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.042985+0000", 2024-09-15T00:11:25.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9400000000000001e-06 2024-09-15T00:11:25.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.043022+0000", 2024-09-15T00:11:25.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6851999999999999e-05 2024-09-15T00:11:25.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.043036+0000", 2024-09-15T00:11:25.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3865e-05 2024-09-15T00:11:25.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.044989+0000", 2024-09-15T00:11:25.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019534349999999999 2024-09-15T00:11:25.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.045019+0000", 2024-09-15T00:11:25.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9901999999999999e-05 2024-09-15T00:11:25.801 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.801 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.801 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.801 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:576 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'21, mlcod=29'18)", 2024-09-15T00:11:25.801 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.180692+0000", 2024-09-15T00:11:25.801 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.59086797899999999, 2024-09-15T00:11:25.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023380139999999998, 2024-09-15T00:11:25.801 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.801 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.801 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.801 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.180692+0000", 2024-09-15T00:11:25.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.180692+0000", 2024-09-15T00:11:25.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.180693+0000", 2024-09-15T00:11:25.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4900000000000005e-07 2024-09-15T00:11:25.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.180699+0000", 2024-09-15T00:11:25.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.691e-06 2024-09-15T00:11:25.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.180712+0000", 2024-09-15T00:11:25.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2078e-05 2024-09-15T00:11:25.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.180718+0000", 2024-09-15T00:11:25.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9310000000000002e-06 2024-09-15T00:11:25.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.180754+0000", 2024-09-15T00:11:25.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5268000000000002e-05 2024-09-15T00:11:25.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.180774+0000", 2024-09-15T00:11:25.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9970999999999999e-05 2024-09-15T00:11:25.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.183003+0000", 2024-09-15T00:11:25.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022296769999999998 2024-09-15T00:11:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.183030+0000", 2024-09-15T00:11:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6648999999999998e-05 2024-09-15T00:11:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:580 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'23, mlcod=29'21)", 2024-09-15T00:11:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.189811+0000", 2024-09-15T00:11:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.58174896200000004, 2024-09-15T00:11:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017833739999999999, 2024-09-15T00:11:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.806 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.806 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.806 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.189811+0000", 2024-09-15T00:11:25.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.189811+0000", 2024-09-15T00:11:25.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.189812+0000", 2024-09-15T00:11:25.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9699999999999996e-07 2024-09-15T00:11:25.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.189816+0000", 2024-09-15T00:11:25.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6890000000000003e-06 2024-09-15T00:11:25.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.189826+0000", 2024-09-15T00:11:25.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7340000000000003e-06 2024-09-15T00:11:25.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.189831+0000", 2024-09-15T00:11:25.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8189999999999999e-06 2024-09-15T00:11:25.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.189865+0000", 2024-09-15T00:11:25.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3652000000000002e-05 2024-09-15T00:11:25.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.189878+0000", 2024-09-15T00:11:25.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3010000000000001e-05 2024-09-15T00:11:25.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.191566+0000", 2024-09-15T00:11:25.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001688677 2024-09-15T00:11:25.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.191594+0000", 2024-09-15T00:11:25.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8195999999999999e-05 2024-09-15T00:11:25.809 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.809 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.809 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.809 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:601 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'28, mlcod=29'26)", 2024-09-15T00:11:25.810 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.209655+0000", 2024-09-15T00:11:25.810 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.561905459, 2024-09-15T00:11:25.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011027859999999999, 2024-09-15T00:11:25.810 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.810 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.810 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.810 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.209655+0000", 2024-09-15T00:11:25.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.209655+0000", 2024-09-15T00:11:25.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.209655+0000", 2024-09-15T00:11:25.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4499999999999996e-07 2024-09-15T00:11:25.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.209661+0000", 2024-09-15T00:11:25.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6999999999999996e-06 2024-09-15T00:11:25.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.209672+0000", 2024-09-15T00:11:25.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1079e-05 2024-09-15T00:11:25.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.209678+0000", 2024-09-15T00:11:25.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7610000000000004e-06 2024-09-15T00:11:25.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.209721+0000", 2024-09-15T00:11:25.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2716000000000002e-05 2024-09-15T00:11:25.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.209737+0000", 2024-09-15T00:11:25.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6549999999999999e-05 2024-09-15T00:11:25.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210721+0000", 2024-09-15T00:11:25.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098353099999999995 2024-09-15T00:11:25.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210757+0000", 2024-09-15T00:11:25.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6704e-05 2024-09-15T00:11:25.814 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.814 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.814 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.814 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:602 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'29, mlcod=29'26)", 2024-09-15T00:11:25.814 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.210317+0000", 2024-09-15T00:11:25.814 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.56124332099999996, 2024-09-15T00:11:25.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002781604, 2024-09-15T00:11:25.814 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210317+0000", 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210317+0000", 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210317+0000", 2024-09-15T00:11:25.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3200000000000005e-07 2024-09-15T00:11:25.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210322+0000", 2024-09-15T00:11:25.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0039999999999999e-06 2024-09-15T00:11:25.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210332+0000", 2024-09-15T00:11:25.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3659999999999998e-06 2024-09-15T00:11:25.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210337+0000", 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4269999999999999e-06 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210387+0000", 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0201999999999998e-05 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210408+0000", 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0602e-05 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.213053+0000", 2024-09-15T00:11:25.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002645072 2024-09-15T00:11:25.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.213098+0000", 2024-09-15T00:11:25.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5299000000000001e-05 2024-09-15T00:11:25.818 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.818 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.818 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.818 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:610 3.4 e29/23 3:255b0913:::rbd_data.11397bff333d.0000000000000039:head v 29'2, mlcod=0'0)", 2024-09-15T00:11:25.818 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:11:25.819 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.54308380999999994, 2024-09-15T00:11:25.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003163014, 2024-09-15T00:11:25.819 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.819 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.819 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.819 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:11:25.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:11:25.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228477+0000", 2024-09-15T00:11:25.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1100000000000004e-07 2024-09-15T00:11:25.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228483+0000", 2024-09-15T00:11:25.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3720000000000001e-06 2024-09-15T00:11:25.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228497+0000", 2024-09-15T00:11:25.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3944000000000001e-05 2024-09-15T00:11:25.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228505+0000", 2024-09-15T00:11:25.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8140000000000001e-06 2024-09-15T00:11:25.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228549+0000", 2024-09-15T00:11:25.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4116000000000002e-05 2024-09-15T00:11:25.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228568+0000", 2024-09-15T00:11:25.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8473000000000001e-05 2024-09-15T00:11:25.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231599+0000", 2024-09-15T00:11:25.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030315590000000001 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231639+0000", 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9824999999999999e-05 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:696 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'58, mlcod=29'56)", 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.344511+0000", 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.42704911499999998, 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00109706, 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.344511+0000", 2024-09-15T00:11:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.344511+0000", 2024-09-15T00:11:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.344513+0000", 2024-09-15T00:11:25.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3269999999999999e-06 2024-09-15T00:11:25.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.344530+0000", 2024-09-15T00:11:25.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6753e-05 2024-09-15T00:11:25.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.344562+0000", 2024-09-15T00:11:25.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1708999999999997e-05 2024-09-15T00:11:25.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.344578+0000", 2024-09-15T00:11:25.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6396999999999999e-05 2024-09-15T00:11:25.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.344706+0000", 2024-09-15T00:11:25.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000127634 2024-09-15T00:11:25.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.344748+0000", 2024-09-15T00:11:25.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2722999999999997e-05 2024-09-15T00:11:25.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.345572+0000", 2024-09-15T00:11:25.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00082390399999999998 2024-09-15T00:11:25.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.345608+0000", 2024-09-15T00:11:25.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5612999999999999e-05 2024-09-15T00:11:25.827 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.827 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.828 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.828 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:712 3.4 e29/23 3:255b0913:::rbd_data.11397bff333d.0000000000000039:head v 29'3, mlcod=29'1)", 2024-09-15T00:11:25.828 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.357509+0000", 2024-09-15T00:11:25.828 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.41405122999999999, 2024-09-15T00:11:25.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001725264, 2024-09-15T00:11:25.828 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.828 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.828 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.828 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.357509+0000", 2024-09-15T00:11:25.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.357509+0000", 2024-09-15T00:11:25.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.357509+0000", 2024-09-15T00:11:25.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3399999999999998e-07 2024-09-15T00:11:25.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.357523+0000", 2024-09-15T00:11:25.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3454e-05 2024-09-15T00:11:25.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.357533+0000", 2024-09-15T00:11:25.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9680000000000001e-06 2024-09-15T00:11:25.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.357538+0000", 2024-09-15T00:11:25.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9089999999999999e-06 2024-09-15T00:11:25.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.357588+0000", 2024-09-15T00:11:25.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0444e-05 2024-09-15T00:11:25.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.357610+0000", 2024-09-15T00:11:25.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1438e-05 2024-09-15T00:11:25.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.359201+0000", 2024-09-15T00:11:25.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001590983 2024-09-15T00:11:25.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.359234+0000", 2024-09-15T00:11:25.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3334000000000003e-05 2024-09-15T00:11:25.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.832 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.832 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:714 3.9 e29/23 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head v 29'65, mlcod=29'63)", 2024-09-15T00:11:25.832 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.361414+0000", 2024-09-15T00:11:25.833 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.410146285, 2024-09-15T00:11:25.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001335188, 2024-09-15T00:11:25.833 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.833 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.833 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.833 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.361414+0000", 2024-09-15T00:11:25.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.361414+0000", 2024-09-15T00:11:25.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.361414+0000", 2024-09-15T00:11:25.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:11:25.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.361435+0000", 2024-09-15T00:11:25.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0633999999999999e-05 2024-09-15T00:11:25.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.361442+0000", 2024-09-15T00:11:25.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1690000000000003e-06 2024-09-15T00:11:25.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.361446+0000", 2024-09-15T00:11:25.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3239999999999996e-06 2024-09-15T00:11:25.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.361477+0000", 2024-09-15T00:11:25.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1151e-05 2024-09-15T00:11:25.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.361490+0000", 2024-09-15T00:11:25.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2459999999999999e-05 2024-09-15T00:11:25.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.362704+0000", 2024-09-15T00:11:25.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012142819999999999 2024-09-15T00:11:25.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.362749+0000", 2024-09-15T00:11:25.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4739e-05 2024-09-15T00:11:25.838 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.838 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.838 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.839 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:757 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'79, mlcod=29'77)", 2024-09-15T00:11:25.839 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.426753+0000", 2024-09-15T00:11:25.839 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.34480681400000002, 2024-09-15T00:11:25.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00202973, 2024-09-15T00:11:25.839 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.839 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.839 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.839 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.426753+0000", 2024-09-15T00:11:25.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.426753+0000", 2024-09-15T00:11:25.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.426754+0000", 2024-09-15T00:11:25.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.82e-07 2024-09-15T00:11:25.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.426758+0000", 2024-09-15T00:11:25.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9210000000000002e-06 2024-09-15T00:11:25.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.426775+0000", 2024-09-15T00:11:25.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7668000000000001e-05 2024-09-15T00:11:25.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.426781+0000", 2024-09-15T00:11:25.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0610000000000001e-06 2024-09-15T00:11:25.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.426817+0000", 2024-09-15T00:11:25.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5831999999999998e-05 2024-09-15T00:11:25.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.426830+0000", 2024-09-15T00:11:25.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2306e-05 2024-09-15T00:11:25.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.428752+0000", 2024-09-15T00:11:25.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001922661 2024-09-15T00:11:25.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.428783+0000", 2024-09-15T00:11:25.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0698999999999999e-05 2024-09-15T00:11:25.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.844 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.844 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:766 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'81, mlcod=29'78)", 2024-09-15T00:11:25.844 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:11:25.844 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.33853473299999998, 2024-09-15T00:11:25.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029025880000000002, 2024-09-15T00:11:25.844 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.844 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.844 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.844 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:11:25.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:11:25.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433026+0000", 2024-09-15T00:11:25.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:11:25.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433030+0000", 2024-09-15T00:11:25.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6280000000000004e-06 2024-09-15T00:11:25.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433038+0000", 2024-09-15T00:11:25.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.072e-06 2024-09-15T00:11:25.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433045+0000", 2024-09-15T00:11:25.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1510000000000001e-06 2024-09-15T00:11:25.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433101+0000", 2024-09-15T00:11:25.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6336999999999999e-05 2024-09-15T00:11:25.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433113+0000", 2024-09-15T00:11:25.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1639e-05 2024-09-15T00:11:25.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435897+0000", 2024-09-15T00:11:25.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027845779999999998 2024-09-15T00:11:25.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435928+0000", 2024-09-15T00:11:25.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0689999999999999e-05 2024-09-15T00:11:25.848 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.848 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.848 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.848 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:799 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'91, mlcod=29'89)", 2024-09-15T00:11:25.848 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.489569+0000", 2024-09-15T00:11:25.849 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.28199101900000001, 2024-09-15T00:11:25.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010500380000000001, 2024-09-15T00:11:25.849 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.849 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.849 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.849 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.489569+0000", 2024-09-15T00:11:25.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.489569+0000", 2024-09-15T00:11:25.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.489571+0000", 2024-09-15T00:11:25.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.136e-06 2024-09-15T00:11:25.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.489588+0000", 2024-09-15T00:11:25.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6889e-05 2024-09-15T00:11:25.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.489609+0000", 2024-09-15T00:11:25.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0760000000000001e-05 2024-09-15T00:11:25.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.489619+0000", 2024-09-15T00:11:25.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0301e-05 2024-09-15T00:11:25.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.489699+0000", 2024-09-15T00:11:25.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9609000000000003e-05 2024-09-15T00:11:25.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.489725+0000", 2024-09-15T00:11:25.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5939e-05 2024-09-15T00:11:25.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.490582+0000", 2024-09-15T00:11:25.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00085781099999999999 2024-09-15T00:11:25.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.490619+0000", 2024-09-15T00:11:25.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6593e-05 2024-09-15T00:11:25.856 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.856 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.856 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.857 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:832 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'100, mlcod=29'97)", 2024-09-15T00:11:25.857 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:25.857 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.246289233, 2024-09-15T00:11:25.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005498925, 2024-09-15T00:11:25.857 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.857 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.857 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.857 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:25.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:25.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:25.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7999999999999995e-07 2024-09-15T00:11:25.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525276+0000", 2024-09-15T00:11:25.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6739999999999996e-06 2024-09-15T00:11:25.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525287+0000", 2024-09-15T00:11:25.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0906e-05 2024-09-15T00:11:25.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525293+0000", 2024-09-15T00:11:25.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4409999999999998e-06 2024-09-15T00:11:25.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525329+0000", 2024-09-15T00:11:25.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5237000000000002e-05 2024-09-15T00:11:25.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525340+0000", 2024-09-15T00:11:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1782e-05 2024-09-15T00:11:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530710+0000", 2024-09-15T00:11:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053695460000000002 2024-09-15T00:11:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530770+0000", 2024-09-15T00:11:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9658999999999999e-05 2024-09-15T00:11:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.862 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.862 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:863 3.9 e29/23 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head v 29'109, mlcod=29'107)", 2024-09-15T00:11:25.862 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.580409+0000", 2024-09-15T00:11:25.862 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.19115072899999999, 2024-09-15T00:11:25.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020411610000000001, 2024-09-15T00:11:25.862 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.862 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.862 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.863 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.580409+0000", 2024-09-15T00:11:25.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.580409+0000", 2024-09-15T00:11:25.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.580411+0000", 2024-09-15T00:11:25.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3799999999999999e-06 2024-09-15T00:11:25.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.581361+0000", 2024-09-15T00:11:25.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00094994299999999997 2024-09-15T00:11:25.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.581376+0000", 2024-09-15T00:11:25.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5545000000000001e-05 2024-09-15T00:11:25.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.581401+0000", 2024-09-15T00:11:25.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4905e-05 2024-09-15T00:11:25.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.581483+0000", 2024-09-15T00:11:25.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1774999999999995e-05 2024-09-15T00:11:25.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.581502+0000", 2024-09-15T00:11:25.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8853000000000001e-05 2024-09-15T00:11:25.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.582413+0000", 2024-09-15T00:11:25.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091141300000000002 2024-09-15T00:11:25.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.582450+0000", 2024-09-15T00:11:25.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7347e-05 2024-09-15T00:11:25.867 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.867 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.867 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.867 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:876 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'114, mlcod=29'111)", 2024-09-15T00:11:25.867 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:25.867 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.075250233, 2024-09-15T00:11:25.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028465830000000002, 2024-09-15T00:11:25.868 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:25.868 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:25.868 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:25.868 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:25.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:25.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:25.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:25.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:25.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:25.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:25.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:25.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2399999999999998e-07 2024-09-15T00:11:25.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:25.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696316+0000", 2024-09-15T00:11:25.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2800000000000003e-06 2024-09-15T00:11:25.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:25.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696326+0000", 2024-09-15T00:11:25.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0546e-05 2024-09-15T00:11:25.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:25.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696333+0000", 2024-09-15T00:11:25.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7660000000000001e-06 2024-09-15T00:11:25.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:25.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696369+0000", 2024-09-15T00:11:25.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5933000000000003e-05 2024-09-15T00:11:25.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:25.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696381+0000", 2024-09-15T00:11:25.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2417e-05 2024-09-15T00:11:25.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:25.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699124+0000", 2024-09-15T00:11:25.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027428000000000001 2024-09-15T00:11:25.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:25.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:25.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:25.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699156+0000", 2024-09-15T00:11:25.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2217e-05 2024-09-15T00:11:25.872 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.872 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.872 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.872 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:25.872 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:25.873 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:25.873 INFO:teuthology.orchestra.run.smithi171.stdout:202 punch from 0x386a2af to 0x386c6c3, (0x2414 bytes) 2024-09-15T00:11:25.875 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:25.876 INFO:teuthology.orchestra.run.smithi171.stdout:203 trunc from 0x9b4b16b to 0x3199df7 2024-09-15T00:11:25.876 INFO:teuthology.orchestra.run.smithi171.stdout:204 writesame 0xb4a0c97 thru 0xb4aadcc data_size 0x5(0xa136 bytes) 2024-09-15T00:11:25.876 INFO:teuthology.orchestra.run.smithi171.stdout:205 punch from 0x74788dd to 0x7483e0e, (0xb531 bytes) 2024-09-15T00:11:25.876 INFO:teuthology.orchestra.run.smithi171.stdout:207 trunc from 0xb4aadcd to 0xdc5a6f8 2024-09-15T00:11:25.876 INFO:teuthology.orchestra.run.smithi171.stdout:210 write 0x13ca93 thru 0x141784 (0x4cf2 bytes) 2024-09-15T00:11:25.876 INFO:teuthology.orchestra.run.smithi171.stdout:211 writesame 0x39fcc6c thru 0x3a060b9 data_size 0x29(0x944e bytes) 2024-09-15T00:11:25.876 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:25.877 INFO:teuthology.orchestra.run.smithi171.stdout:213 trunc from 0xdc5a6f8 to 0x197e6a7 2024-09-15T00:11:25.877 INFO:teuthology.orchestra.run.smithi171.stdout:214 write 0x4936cae thru 0x493fa25 (0x8d78 bytes) 2024-09-15T00:11:25.877 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:25.871+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:25.877 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:25.885 INFO:teuthology.orchestra.run.smithi171.stdout:216 trunc from 0x493fa26 to 0xc124af8 2024-09-15T00:11:25.890 INFO:teuthology.orchestra.run.smithi171.stdout:217 writesame 0xa042899 thru 0xa0482be data_size 0xb(0x5a26 bytes) 2024-09-15T00:11:25.896 INFO:teuthology.orchestra.run.smithi171.stdout:219 trunc from 0xc124af8 to 0xa22ee9f 2024-09-15T00:11:25.909 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:25.909 INFO:teuthology.orchestra.run.smithi171.stdout:222 read 0x88296f1 thru 0x8838072 (0xe982 bytes) 2024-09-15T00:11:25.917 INFO:teuthology.orchestra.run.smithi171.stdout:223 write 0xc8929a7 thru 0xc8a16fb (0xed55 bytes) 2024-09-15T00:11:25.928 INFO:teuthology.orchestra.run.smithi171.stdout:224 write 0xdce5895 thru 0xdcf069f (0xae0b bytes) 2024-09-15T00:11:25.936 INFO:teuthology.orchestra.run.smithi171.stdout:225 trunc from 0xdcf06a0 to 0x9ee6f26 2024-09-15T00:11:25.946 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:25.947 INFO:teuthology.orchestra.run.smithi171.stdout:231 read 0x75f18d4 thru 0x75fc1a3 (0xa8d0 bytes) 2024-09-15T00:11:25.947 INFO:teuthology.orchestra.run.smithi171.stdout:232 read 0x8c9afc1 thru 0x8ca59fe (0xaa3e bytes) 2024-09-15T00:11:25.947 INFO:teuthology.orchestra.run.smithi171.stdout:233 writesame 0x6d7a61b thru 0x6d8333b data_size 0x1(0x8d21 bytes) 2024-09-15T00:11:25.975 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:25.972+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:25.988 INFO:teuthology.orchestra.run.smithi171.stdout:234 read 0x9553bfe thru 0x955e869 (0xac6c bytes) 2024-09-15T00:11:25.988 INFO:teuthology.orchestra.run.smithi171.stdout:235 write 0x4534f47 thru 0x4537b4b (0x2c05 bytes) 2024-09-15T00:11:25.990 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:25.990 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:25.990 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:25.990 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:25.991 INFO:teuthology.orchestra.run.smithi171.stdout:236 punch from 0x507e982 to 0x5080834, (0x1eb2 bytes) 2024-09-15T00:11:26.003 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:26.003 INFO:teuthology.orchestra.run.smithi171.stdout:237 write 0xc324728 thru 0xc32f25f (0xab38 bytes) 2024-09-15T00:11:26.008 INFO:teuthology.orchestra.run.smithi171.stdout:238 trunc from 0xc32f260 to 0xa58029b 2024-09-15T00:11:26.017 INFO:teuthology.orchestra.run.smithi171.stdout:240 read 0x875f884 thru 0x8767a1a (0x8197 bytes) 2024-09-15T00:11:26.017 INFO:teuthology.orchestra.run.smithi171.stdout:241 punch from 0x2126b7c to 0x2136459, (0xf8dd bytes) 2024-09-15T00:11:26.023 INFO:teuthology.orchestra.run.smithi171.stdout:242 write 0xb28dc28 thru 0xb2939ca (0x5da3 bytes) 2024-09-15T00:11:26.027 INFO:teuthology.orchestra.run.smithi171.stdout:245 read 0x765302c thru 0x765d48d (0xa462 bytes) 2024-09-15T00:11:26.028 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.028 INFO:teuthology.orchestra.run.smithi171.stdout:247 read 0x3eba3fd thru 0x3ec55ea (0xb1ee bytes) 2024-09-15T00:11:26.028 INFO:teuthology.orchestra.run.smithi171.stdout:248 write 0x8b09be5 thru 0x8b10d2d (0x7149 bytes) 2024-09-15T00:11:26.032 INFO:teuthology.orchestra.run.smithi171.stdout:249 write 0x8e1d3d thru 0x8e9852 (0x7b16 bytes) 2024-09-15T00:11:26.034 INFO:teuthology.orchestra.run.smithi171.stdout:252 write 0x5295b24 thru 0x52978b4 (0x1d91 bytes) 2024-09-15T00:11:26.044 INFO:teuthology.orchestra.run.smithi171.stdout:253 writesame 0xc922289 thru 0xc92759d data_size 0x1(0x5315 bytes) 2024-09-15T00:11:26.045 INFO:teuthology.orchestra.run.smithi171.stdout:254 writesame 0x19b47aa thru 0x19b7eb0 data_size 0x1(0x3707 bytes) 2024-09-15T00:11:26.064 INFO:teuthology.orchestra.run.smithi171.stdout:255 writesame 0x1cdf801 thru 0x1ce5192 data_size 0x5(0x5992 bytes) 2024-09-15T00:11:26.069 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.074 INFO:teuthology.orchestra.run.smithi171.stdout:257 write 0xc987215 thru 0xc98fa1b (0x8807 bytes) 2024-09-15T00:11:26.076 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:26.074+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:26.077 INFO:teuthology.orchestra.run.smithi171.stdout:259 writesame 0x3805c06 thru 0x38105fe data_size 0x1(0xa9f9 bytes) 2024-09-15T00:11:26.117 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:26.118 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:26.126 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:26.160 INFO:teuthology.orchestra.run.smithi171.stdout:260 write 0x4ee8426 thru 0x4ef442f (0xc00a bytes) 2024-09-15T00:11:26.165 INFO:teuthology.orchestra.run.smithi171.stdout:261 read 0x482dbd6 thru 0x48348bf (0x6cea bytes) 2024-09-15T00:11:26.165 INFO:teuthology.orchestra.run.smithi171.stdout:262 read 0x35df51 thru 0x3689a2 (0xaa52 bytes) 2024-09-15T00:11:26.170 INFO:teuthology.orchestra.run.smithi171.stdout:263 writesame 0xca19c4a thru 0xca20c9f data_size 0x3(0x7056 bytes) 2024-09-15T00:11:26.172 INFO:teuthology.orchestra.run.smithi171.stdout:264 punch from 0xec3e73 to 0xecb66d, (0x77fa bytes) 2024-09-15T00:11:26.172 INFO:teuthology.orchestra.run.smithi171.stdout:266 read 0x7b5cfcb thru 0x7b664b2 (0x94e8 bytes) 2024-09-15T00:11:26.177 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:26.175+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:26.179 INFO:teuthology.orchestra.run.smithi171.stdout:267 writesame 0xe3bfb33 thru 0xe3c6a19 data_size 0x1d(0x6ee7 bytes) 2024-09-15T00:11:26.179 INFO:teuthology.orchestra.run.smithi171.stdout:268 writesame 0xcc39210 thru 0xcc41e95 data_size 0x1(0x8c86 bytes) 2024-09-15T00:11:26.217 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.217 INFO:teuthology.orchestra.run.smithi171.stdout:271 punch from 0xb4436e6 to 0xb44c6f8, (0x9012 bytes) 2024-09-15T00:11:26.217 INFO:teuthology.orchestra.run.smithi171.stdout:273 writesame 0x6f2f0c4 thru 0x6f38593 data_size 0x8(0x94d0 bytes) 2024-09-15T00:11:26.220 INFO:teuthology.orchestra.run.smithi171.stdout:276 punch from 0x3b70af5 to 0x3b7143b, (0x946 bytes) 2024-09-15T00:11:26.220 INFO:teuthology.orchestra.run.smithi171.stdout:277 write 0x88ea139 thru 0x88f8552 (0xe41a bytes) 2024-09-15T00:11:26.222 INFO:teuthology.orchestra.run.smithi171.stdout:280 writesame 0xbd642a1 thru 0xbd70ae2 data_size 0x1(0xc842 bytes) 2024-09-15T00:11:26.267 INFO:teuthology.orchestra.run.smithi171.stdout:281 write 0x2197f46 thru 0x21a0479 (0x8534 bytes) 2024-09-15T00:11:26.270 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:26.270 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:26.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:26.270 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:26.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.271 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:13 3.7 3:e26ec244:::rbd_object_map.11395e6383ec:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:11:26.271 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:21.610080+0000", 2024-09-15T00:11:26.271 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6576229700000003, 2024-09-15T00:11:26.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022503800000000002, 2024-09-15T00:11:26.271 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.271 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.271 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.271 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:26.271 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:26.271 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.271 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13 2024-09-15T00:11:26.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.271 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.610080+0000", 2024-09-15T00:11:26.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.610080+0000", 2024-09-15T00:11:26.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.610082+0000", 2024-09-15T00:11:26.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.319e-06 2024-09-15T00:11:26.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.610088+0000", 2024-09-15T00:11:26.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5050000000000004e-06 2024-09-15T00:11:26.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.610096+0000", 2024-09-15T00:11:26.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0669999999999992e-06 2024-09-15T00:11:26.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.610103+0000", 2024-09-15T00:11:26.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.793e-06 2024-09-15T00:11:26.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.610166+0000", 2024-09-15T00:11:26.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3075000000000002e-05 2024-09-15T00:11:26.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.610309+0000", 2024-09-15T00:11:26.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014297999999999999 2024-09-15T00:11:26.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:26.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.610537+0000", 2024-09-15T00:11:26.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022796200000000001 2024-09-15T00:11:26.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.611677+0000", 2024-09-15T00:11:26.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011395929999999999 2024-09-15T00:11:26.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:26.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.612265+0000", 2024-09-15T00:11:26.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00058858800000000002 2024-09-15T00:11:26.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.612298+0000", 2024-09-15T00:11:26.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2620999999999998e-05 2024-09-15T00:11:26.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.612331+0000", 2024-09-15T00:11:26.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2876999999999997e-05 2024-09-15T00:11:26.276 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.276 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.276 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.277 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:429 3.15 e28/23 3:abbd1ee2:::rbd_object_map.1139feddb981:head v 28'42, mlcod=27'40)", 2024-09-15T00:11:26.277 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:24.627946+0000", 2024-09-15T00:11:26.277 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.6397574580000001, 2024-09-15T00:11:26.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0032042500000000001, 2024-09-15T00:11:26.277 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.277 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.277 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.277 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.627946+0000", 2024-09-15T00:11:26.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.627946+0000", 2024-09-15T00:11:26.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.627947+0000", 2024-09-15T00:11:26.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7999999999999993e-07 2024-09-15T00:11:26.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.627955+0000", 2024-09-15T00:11:26.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8480000000000001e-06 2024-09-15T00:11:26.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.627965+0000", 2024-09-15T00:11:26.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0159999999999999e-05 2024-09-15T00:11:26.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.627972+0000", 2024-09-15T00:11:26.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9110000000000004e-06 2024-09-15T00:11:26.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.628013+0000", 2024-09-15T00:11:26.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1170000000000001e-05 2024-09-15T00:11:26.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.628030+0000", 2024-09-15T00:11:26.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7405999999999999e-05 2024-09-15T00:11:26.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.631080+0000", 2024-09-15T00:11:26.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030496130000000001 2024-09-15T00:11:26.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:24.631150+0000", 2024-09-15T00:11:26.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0161999999999997e-05 2024-09-15T00:11:26.281 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.281 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.281 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.281 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1 29/28 [PushOp(3:8737ea14:::rbd_data.11395e6383ec.000000000000000f:head, version: 26'1, data_included: [781998~64596], data_size: 64596, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:8737ea14:::rbd_data.11395e6383ec.000000000000000f:head@26'1, size: 846594, copy_subset: [(0, 846594)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 846594, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:11:26.281 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.010873+0000", 2024-09-15T00:11:26.281 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.256830729, 2024-09-15T00:11:26.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023959820000000001, 2024-09-15T00:11:26.282 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.282 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.282 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:26.282 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.010873+0000", 2024-09-15T00:11:26.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.010873+0000", 2024-09-15T00:11:26.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.010873+0000", 2024-09-15T00:11:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5700000000000002e-07 2024-09-15T00:11:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.011015+0000", 2024-09-15T00:11:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014167199999999999 2024-09-15T00:11:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.011035+0000", 2024-09-15T00:11:26.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0075e-05 2024-09-15T00:11:26.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.011049+0000", 2024-09-15T00:11:26.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4256999999999999e-05 2024-09-15T00:11:26.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.011131+0000", 2024-09-15T00:11:26.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2083999999999995e-05 2024-09-15T00:11:26.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.011145+0000", 2024-09-15T00:11:26.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4098e-05 2024-09-15T00:11:26.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.013269+0000", 2024-09-15T00:11:26.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002123139 2024-09-15T00:11:26.285 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.285 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.285 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.285 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:608 3.8 3:17096be3:::rbd_data.11397bff333d.0000000000000034:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.285 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.228194+0000", 2024-09-15T00:11:26.286 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.0395092749999999, 2024-09-15T00:11:26.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037361920000000002, 2024-09-15T00:11:26.286 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.286 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.286 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.286 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:26.286 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:26.286 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.286 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 608 2024-09-15T00:11:26.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.286 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228194+0000", 2024-09-15T00:11:26.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228194+0000", 2024-09-15T00:11:26.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228194+0000", 2024-09-15T00:11:26.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.15e-07 2024-09-15T00:11:26.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228199+0000", 2024-09-15T00:11:26.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:11:26.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228206+0000", 2024-09-15T00:11:26.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9689999999999997e-06 2024-09-15T00:11:26.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228209+0000", 2024-09-15T00:11:26.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0699999999999998e-06 2024-09-15T00:11:26.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228248+0000", 2024-09-15T00:11:26.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9084000000000001e-05 2024-09-15T00:11:26.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228339+0000", 2024-09-15T00:11:26.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1731999999999995e-05 2024-09-15T00:11:26.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:26.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228427+0000", 2024-09-15T00:11:26.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-15T00:11:26.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:26.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231732+0000", 2024-09-15T00:11:26.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003304848 2024-09-15T00:11:26.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231857+0000", 2024-09-15T00:11:26.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000124733 2024-09-15T00:11:26.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231894+0000", 2024-09-15T00:11:26.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7561000000000002e-05 2024-09-15T00:11:26.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231930+0000", 2024-09-15T00:11:26.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6078000000000003e-05 2024-09-15T00:11:26.291 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.291 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.291 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.291 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:643 3.2 3:4001636a:::rbd_data.11397bff333d.0000000000000007:head [set-alloc-hint object_size 4194304 write_size 4194304,write 827190~24778] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.291 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.261644+0000", 2024-09-15T00:11:26.291 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.006059458, 2024-09-15T00:11:26.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00222529, 2024-09-15T00:11:26.292 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.292 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.292 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.292 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:26.292 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:26.292 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.292 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 643 2024-09-15T00:11:26.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.292 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.261644+0000", 2024-09-15T00:11:26.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.261644+0000", 2024-09-15T00:11:26.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.261645+0000", 2024-09-15T00:11:26.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.579e-06 2024-09-15T00:11:26.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.261664+0000", 2024-09-15T00:11:26.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8936999999999998e-05 2024-09-15T00:11:26.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.261681+0000", 2024-09-15T00:11:26.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6504999999999999e-05 2024-09-15T00:11:26.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.261689+0000", 2024-09-15T00:11:26.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4589999999999998e-06 2024-09-15T00:11:26.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.261740+0000", 2024-09-15T00:11:26.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0689000000000002e-05 2024-09-15T00:11:26.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.261846+0000", 2024-09-15T00:11:26.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000106419 2024-09-15T00:11:26.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:26.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.261956+0000", 2024-09-15T00:11:26.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000109145 2024-09-15T00:11:26.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:26.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.263607+0000", 2024-09-15T00:11:26.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001651408 2024-09-15T00:11:26.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.263790+0000", 2024-09-15T00:11:26.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183265 2024-09-15T00:11:26.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.263826+0000", 2024-09-15T00:11:26.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5537999999999998e-05 2024-09-15T00:11:26.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.263869+0000", 2024-09-15T00:11:26.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3346000000000001e-05 2024-09-15T00:11:26.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.297 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.297 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:659 3.12 e29/28 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head v 29'1, mlcod=0'0)", 2024-09-15T00:11:26.297 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.287100+0000", 2024-09-15T00:11:26.297 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.98060314699999995, 2024-09-15T00:11:26.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028354029999999998, 2024-09-15T00:11:26.298 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.298 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.298 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.298 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.287100+0000", 2024-09-15T00:11:26.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.287100+0000", 2024-09-15T00:11:26.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.287101+0000", 2024-09-15T00:11:26.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2160000000000001e-06 2024-09-15T00:11:26.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.287197+0000", 2024-09-15T00:11:26.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5692000000000005e-05 2024-09-15T00:11:26.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.287212+0000", 2024-09-15T00:11:26.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.454e-05 2024-09-15T00:11:26.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.287222+0000", 2024-09-15T00:11:26.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0332e-05 2024-09-15T00:11:26.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.287275+0000", 2024-09-15T00:11:26.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2775000000000002e-05 2024-09-15T00:11:26.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.287289+0000", 2024-09-15T00:11:26.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4793e-05 2024-09-15T00:11:26.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.289904+0000", 2024-09-15T00:11:26.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002614767 2024-09-15T00:11:26.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.289936+0000", 2024-09-15T00:11:26.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1288000000000003e-05 2024-09-15T00:11:26.301 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.301 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.301 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.302 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:720 3.c e29/23 3:361d30af:::rbd_data.11397bff333d.0000000000000017:head v 29'1, mlcod=0'0)", 2024-09-15T00:11:26.302 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.369037+0000", 2024-09-15T00:11:26.302 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.898666666, 2024-09-15T00:11:26.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023259980000000001, 2024-09-15T00:11:26.302 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.302 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.302 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.302 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.369037+0000", 2024-09-15T00:11:26.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.369037+0000", 2024-09-15T00:11:26.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.369037+0000", 2024-09-15T00:11:26.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1900000000000001e-07 2024-09-15T00:11:26.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.369068+0000", 2024-09-15T00:11:26.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0833000000000001e-05 2024-09-15T00:11:26.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.369081+0000", 2024-09-15T00:11:26.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2636000000000001e-05 2024-09-15T00:11:26.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.369089+0000", 2024-09-15T00:11:26.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7209999999999994e-06 2024-09-15T00:11:26.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.369136+0000", 2024-09-15T00:11:26.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7341e-05 2024-09-15T00:11:26.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.369154+0000", 2024-09-15T00:11:26.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8178000000000001e-05 2024-09-15T00:11:26.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.371325+0000", 2024-09-15T00:11:26.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021708909999999999 2024-09-15T00:11:26.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.371363+0000", 2024-09-15T00:11:26.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7579000000000001e-05 2024-09-15T00:11:26.306 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.306 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.306 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.306 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:721 3.8 3:17096be3:::rbd_data.11397bff333d.0000000000000034:head [write 3443730~29678] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.306 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.371826+0000", 2024-09-15T00:11:26.306 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.89587687699999996, 2024-09-15T00:11:26.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022732189999999999, 2024-09-15T00:11:26.306 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.307 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.307 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.307 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:26.307 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:26.307 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.307 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 721 2024-09-15T00:11:26.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.307 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.371826+0000", 2024-09-15T00:11:26.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.371826+0000", 2024-09-15T00:11:26.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.371828+0000", 2024-09-15T00:11:26.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6589999999999999e-06 2024-09-15T00:11:26.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.371845+0000", 2024-09-15T00:11:26.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6546e-05 2024-09-15T00:11:26.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.371856+0000", 2024-09-15T00:11:26.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1089e-05 2024-09-15T00:11:26.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.371861+0000", 2024-09-15T00:11:26.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3469999999999998e-06 2024-09-15T00:11:26.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.371915+0000", 2024-09-15T00:11:26.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3584000000000001e-05 2024-09-15T00:11:26.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.371978+0000", 2024-09-15T00:11:26.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3200000000000005e-05 2024-09-15T00:11:26.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:26.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.372071+0000", 2024-09-15T00:11:26.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3356999999999994e-05 2024-09-15T00:11:26.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:26.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.373647+0000", 2024-09-15T00:11:26.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001575391 2024-09-15T00:11:26.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.374027+0000", 2024-09-15T00:11:26.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00038021300000000001 2024-09-15T00:11:26.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.374061+0000", 2024-09-15T00:11:26.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3787e-05 2024-09-15T00:11:26.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.374100+0000", 2024-09-15T00:11:26.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9045999999999999e-05 2024-09-15T00:11:26.312 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.312 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.312 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.312 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:730 3.1 e29/28 3:800323e2:::rbd_data.11397bff333d.0000000000000037:head v 29'12, mlcod=27'10)", 2024-09-15T00:11:26.312 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.382605+0000", 2024-09-15T00:11:26.312 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88509834099999996, 2024-09-15T00:11:26.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026183199999999999, 2024-09-15T00:11:26.312 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.312 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.313 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.313 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.382605+0000", 2024-09-15T00:11:26.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.382605+0000", 2024-09-15T00:11:26.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.382606+0000", 2024-09-15T00:11:26.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8199999999999999e-07 2024-09-15T00:11:26.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.382655+0000", 2024-09-15T00:11:26.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8893000000000001e-05 2024-09-15T00:11:26.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.382665+0000", 2024-09-15T00:11:26.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0835e-05 2024-09-15T00:11:26.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.382671+0000", 2024-09-15T00:11:26.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1070000000000004e-06 2024-09-15T00:11:26.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.382729+0000", 2024-09-15T00:11:26.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8671000000000001e-05 2024-09-15T00:11:26.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.382748+0000", 2024-09-15T00:11:26.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9198000000000002e-05 2024-09-15T00:11:26.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.385190+0000", 2024-09-15T00:11:26.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024411659999999998 2024-09-15T00:11:26.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.385223+0000", 2024-09-15T00:11:26.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3667999999999997e-05 2024-09-15T00:11:26.316 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.316 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.316 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.317 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:740 3.8 3:17096be3:::rbd_data.11397bff333d.0000000000000034:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.317 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.403781+0000", 2024-09-15T00:11:26.317 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.863922313, 2024-09-15T00:11:26.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020980740000000001, 2024-09-15T00:11:26.317 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.317 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.317 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.317 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:26.317 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:26.317 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.317 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 740 2024-09-15T00:11:26.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.317 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.403781+0000", 2024-09-15T00:11:26.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.403781+0000", 2024-09-15T00:11:26.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.403781+0000", 2024-09-15T00:11:26.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-15T00:11:26.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.403785+0000", 2024-09-15T00:11:26.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-15T00:11:26.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.403794+0000", 2024-09-15T00:11:26.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3880000000000006e-06 2024-09-15T00:11:26.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.403800+0000", 2024-09-15T00:11:26.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8619999999999999e-06 2024-09-15T00:11:26.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.403850+0000", 2024-09-15T00:11:26.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0263999999999999e-05 2024-09-15T00:11:26.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.403926+0000", 2024-09-15T00:11:26.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5852999999999994e-05 2024-09-15T00:11:26.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:26.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.404057+0000", 2024-09-15T00:11:26.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013048899999999999 2024-09-15T00:11:26.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:26.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.404734+0000", 2024-09-15T00:11:26.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00067781099999999995 2024-09-15T00:11:26.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.405781+0000", 2024-09-15T00:11:26.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001046302 2024-09-15T00:11:26.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.405835+0000", 2024-09-15T00:11:26.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4738999999999999e-05 2024-09-15T00:11:26.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.405879+0000", 2024-09-15T00:11:26.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3680000000000002e-05 2024-09-15T00:11:26.322 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.322 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.322 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.323 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:767 3.1e 3:7f04344d:::rbd_data.11397bff333d.0000000000000026:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2553687~2217] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.323 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.433417+0000", 2024-09-15T00:11:26.323 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.83428662899999995, 2024-09-15T00:11:26.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027302419999999999, 2024-09-15T00:11:26.323 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.323 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.323 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.323 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:26.324 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:26.324 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.324 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 767 2024-09-15T00:11:26.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.324 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433417+0000", 2024-09-15T00:11:26.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433417+0000", 2024-09-15T00:11:26.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433419+0000", 2024-09-15T00:11:26.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.261e-06 2024-09-15T00:11:26.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433429+0000", 2024-09-15T00:11:26.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9289999999999993e-06 2024-09-15T00:11:26.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433446+0000", 2024-09-15T00:11:26.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6765999999999998e-05 2024-09-15T00:11:26.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433455+0000", 2024-09-15T00:11:26.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7480000000000005e-06 2024-09-15T00:11:26.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433524+0000", 2024-09-15T00:11:26.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8717e-05 2024-09-15T00:11:26.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433673+0000", 2024-09-15T00:11:26.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000148765 2024-09-15T00:11:26.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:26.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433833+0000", 2024-09-15T00:11:26.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000159803 2024-09-15T00:11:26.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.434632+0000", 2024-09-15T00:11:26.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00079891100000000002 2024-09-15T00:11:26.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:26.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.436066+0000", 2024-09-15T00:11:26.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014343310000000001 2024-09-15T00:11:26.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.436100+0000", 2024-09-15T00:11:26.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4125000000000003e-05 2024-09-15T00:11:26.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.436147+0000", 2024-09-15T00:11:26.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6885999999999998e-05 2024-09-15T00:11:26.329 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.329 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.329 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.329 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:840 3.1e 3:7bc72bfd:::rbd_data.11397bff333d.000000000000000d:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2006002~25614] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.329 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.544464+0000", 2024-09-15T00:11:26.329 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.72323892400000001, 2024-09-15T00:11:26.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002815672, 2024-09-15T00:11:26.329 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.329 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.329 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:26.330 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:26.330 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.330 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 840 2024-09-15T00:11:26.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.330 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.544464+0000", 2024-09-15T00:11:26.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.544464+0000", 2024-09-15T00:11:26.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.544466+0000", 2024-09-15T00:11:26.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.751e-06 2024-09-15T00:11:26.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.544481+0000", 2024-09-15T00:11:26.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.455e-05 2024-09-15T00:11:26.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.544495+0000", 2024-09-15T00:11:26.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3861e-05 2024-09-15T00:11:26.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.544503+0000", 2024-09-15T00:11:26.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4109999999999996e-06 2024-09-15T00:11:26.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.544553+0000", 2024-09-15T00:11:26.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0016999999999999e-05 2024-09-15T00:11:26.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.544667+0000", 2024-09-15T00:11:26.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114436 2024-09-15T00:11:26.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:26.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.544780+0000", 2024-09-15T00:11:26.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000112289 2024-09-15T00:11:26.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:26.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.546299+0000", 2024-09-15T00:11:26.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00151884 2024-09-15T00:11:26.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.547206+0000", 2024-09-15T00:11:26.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090756300000000001 2024-09-15T00:11:26.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.547238+0000", 2024-09-15T00:11:26.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2376000000000003e-05 2024-09-15T00:11:26.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.547280+0000", 2024-09-15T00:11:26.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1578000000000001e-05 2024-09-15T00:11:26.334 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.334 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.334 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:877 3.1e 3:7f04344d:::rbd_data.11397bff333d.0000000000000026:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.699414+0000", 2024-09-15T00:11:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.56828920100000002, 2024-09-15T00:11:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027027739999999998, 2024-09-15T00:11:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 877 2024-09-15T00:11:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.336 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699414+0000", 2024-09-15T00:11:26.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699414+0000", 2024-09-15T00:11:26.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699416+0000", 2024-09-15T00:11:26.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2340000000000001e-06 2024-09-15T00:11:26.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699426+0000", 2024-09-15T00:11:26.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0025e-05 2024-09-15T00:11:26.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699447+0000", 2024-09-15T00:11:26.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0191e-05 2024-09-15T00:11:26.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699457+0000", 2024-09-15T00:11:26.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0217000000000001e-05 2024-09-15T00:11:26.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699522+0000", 2024-09-15T00:11:26.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5432999999999995e-05 2024-09-15T00:11:26.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699612+0000", 2024-09-15T00:11:26.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9635999999999994e-05 2024-09-15T00:11:26.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:26.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699881+0000", 2024-09-15T00:11:26.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026897800000000001 2024-09-15T00:11:26.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.701728+0000", 2024-09-15T00:11:26.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018473859999999999 2024-09-15T00:11:26.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:26.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.702046+0000", 2024-09-15T00:11:26.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00031826400000000001 2024-09-15T00:11:26.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.702079+0000", 2024-09-15T00:11:26.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2298000000000001e-05 2024-09-15T00:11:26.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.702117+0000", 2024-09-15T00:11:26.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8111999999999997e-05 2024-09-15T00:11:26.340 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.341 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.341 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.341 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:881 3.1e 3:7bc72bfd:::rbd_data.11397bff333d.000000000000000d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.341 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.715960+0000", 2024-09-15T00:11:26.341 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.55174332199999998, 2024-09-15T00:11:26.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021610230000000002, 2024-09-15T00:11:26.341 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.341 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.341 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.341 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:26.341 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:26.341 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.342 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 881 2024-09-15T00:11:26.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.342 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.715960+0000", 2024-09-15T00:11:26.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.715960+0000", 2024-09-15T00:11:26.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.715962+0000", 2024-09-15T00:11:26.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.261e-06 2024-09-15T00:11:26.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.715975+0000", 2024-09-15T00:11:26.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2798e-05 2024-09-15T00:11:26.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.715997+0000", 2024-09-15T00:11:26.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1523e-05 2024-09-15T00:11:26.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.716007+0000", 2024-09-15T00:11:26.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0094e-05 2024-09-15T00:11:26.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.716063+0000", 2024-09-15T00:11:26.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6789999999999997e-05 2024-09-15T00:11:26.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.716130+0000", 2024-09-15T00:11:26.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6229999999999994e-05 2024-09-15T00:11:26.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:26.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.716212+0000", 2024-09-15T00:11:26.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2508000000000004e-05 2024-09-15T00:11:26.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:26.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.716966+0000", 2024-09-15T00:11:26.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00075398999999999996 2024-09-15T00:11:26.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.718016+0000", 2024-09-15T00:11:26.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010493970000000001 2024-09-15T00:11:26.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.718072+0000", 2024-09-15T00:11:26.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6529e-05 2024-09-15T00:11:26.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.718121+0000", 2024-09-15T00:11:26.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8903000000000003e-05 2024-09-15T00:11:26.346 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.346 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.346 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.347 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:885 3.1e 3:7f04344d:::rbd_data.11397bff333d.0000000000000026:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.347 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.716031+0000", 2024-09-15T00:11:26.347 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.55167272300000003, 2024-09-15T00:11:26.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048205679999999999, 2024-09-15T00:11:26.347 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.347 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.347 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.347 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:26.347 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:26.347 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.347 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 885 2024-09-15T00:11:26.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.348 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.716031+0000", 2024-09-15T00:11:26.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.716031+0000", 2024-09-15T00:11:26.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.716031+0000", 2024-09-15T00:11:26.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9699999999999996e-07 2024-09-15T00:11:26.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.716035+0000", 2024-09-15T00:11:26.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7009999999999998e-06 2024-09-15T00:11:26.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.716043+0000", 2024-09-15T00:11:26.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2099999999999993e-06 2024-09-15T00:11:26.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.716049+0000", 2024-09-15T00:11:26.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:11:26.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.716449+0000", 2024-09-15T00:11:26.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040031099999999998 2024-09-15T00:11:26.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.716533+0000", 2024-09-15T00:11:26.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4291999999999999e-05 2024-09-15T00:11:26.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.716651+0000", 2024-09-15T00:11:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000117587 2024-09-15T00:11:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.718347+0000", 2024-09-15T00:11:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001695839 2024-09-15T00:11:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.720795+0000", 2024-09-15T00:11:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024485119999999999 2024-09-15T00:11:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.720823+0000", 2024-09-15T00:11:26.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7676999999999999e-05 2024-09-15T00:11:26.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.720851+0000", 2024-09-15T00:11:26.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8160000000000001e-05 2024-09-15T00:11:26.352 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.352 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.352 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.353 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1077 3.8 3:1631f623:::rbd_data.1139feddb981.0000000000000009:4 [sparse-read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.353 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.819381+0000", 2024-09-15T00:11:26.353 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.44832217200000002, 2024-09-15T00:11:26.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019626230000000001, 2024-09-15T00:11:26.353 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.353 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.353 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:26.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:26.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:26.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.353 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1077 2024-09-15T00:11:26.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.353 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.819381+0000", 2024-09-15T00:11:26.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.819381+0000", 2024-09-15T00:11:26.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.819382+0000", 2024-09-15T00:11:26.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5499999999999998e-07 2024-09-15T00:11:26.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.819384+0000", 2024-09-15T00:11:26.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:11:26.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.819390+0000", 2024-09-15T00:11:26.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1240000000000002e-06 2024-09-15T00:11:26.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.819396+0000", 2024-09-15T00:11:26.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8329999999999999e-06 2024-09-15T00:11:26.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.821171+0000", 2024-09-15T00:11:26.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00177494 2024-09-15T00:11:26.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.821237+0000", 2024-09-15T00:11:26.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6354999999999997e-05 2024-09-15T00:11:26.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.821344+0000", 2024-09-15T00:11:26.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010679 2024-09-15T00:11:26.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.357 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.357 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1096 3.17 e29/28 3:e891fddf:::rbd_data.11397bff333d.000000000000000f:head v 29'8, mlcod=29'6)", 2024-09-15T00:11:26.357 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.851602+0000", 2024-09-15T00:11:26.357 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.416101691, 2024-09-15T00:11:26.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0035217780000000001, 2024-09-15T00:11:26.357 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.357 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.357 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.357 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.851602+0000", 2024-09-15T00:11:26.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.851602+0000", 2024-09-15T00:11:26.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.851603+0000", 2024-09-15T00:11:26.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1909999999999999e-06 2024-09-15T00:11:26.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.851613+0000", 2024-09-15T00:11:26.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8949999999999993e-06 2024-09-15T00:11:26.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.851632+0000", 2024-09-15T00:11:26.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8853000000000001e-05 2024-09-15T00:11:26.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.851638+0000", 2024-09-15T00:11:26.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3080000000000004e-06 2024-09-15T00:11:26.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.851696+0000", 2024-09-15T00:11:26.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.838e-05 2024-09-15T00:11:26.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.851719+0000", 2024-09-15T00:11:26.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2430999999999999e-05 2024-09-15T00:11:26.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.855009+0000", 2024-09-15T00:11:26.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003290496 2024-09-15T00:11:26.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.855123+0000", 2024-09-15T00:11:26.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114224 2024-09-15T00:11:26.361 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.361 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.361 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.361 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1240 3.2 3:4001636a:::rbd_data.11397bff333d.0000000000000007:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:26.361 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.045955+0000", 2024-09-15T00:11:26.362 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.22174802099999999, 2024-09-15T00:11:26.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017913958000000001, 2024-09-15T00:11:26.362 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.362 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.362 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.362 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:26.362 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:26.362 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.362 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1240 2024-09-15T00:11:26.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.362 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045955+0000", 2024-09-15T00:11:26.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045955+0000", 2024-09-15T00:11:26.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045957+0000", 2024-09-15T00:11:26.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6640000000000001e-06 2024-09-15T00:11:26.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045964+0000", 2024-09-15T00:11:26.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3629999999999996e-06 2024-09-15T00:11:26.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045979+0000", 2024-09-15T00:11:26.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4436e-05 2024-09-15T00:11:26.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045989+0000", 2024-09-15T00:11:26.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0307e-05 2024-09-15T00:11:26.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.046042+0000", 2024-09-15T00:11:26.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3134999999999999e-05 2024-09-15T00:11:26.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.046106+0000", 2024-09-15T00:11:26.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3807999999999996e-05 2024-09-15T00:11:26.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:26.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.047043+0000", 2024-09-15T00:11:26.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000937165 2024-09-15T00:11:26.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063721+0000", 2024-09-15T00:11:26.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016678193000000001 2024-09-15T00:11:26.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:26.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063743+0000", 2024-09-15T00:11:26.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1163999999999999e-05 2024-09-15T00:11:26.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063810+0000", 2024-09-15T00:11:26.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7317000000000006e-05 2024-09-15T00:11:26.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063869+0000", 2024-09-15T00:11:26.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9406e-05 2024-09-15T00:11:26.367 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.367 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.367 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.367 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:1240 3.2 e30/23 ondisk, result = 0)", 2024-09-15T00:11:26.367 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.050169+0000", 2024-09-15T00:11:26.367 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.21753477900000001, 2024-09-15T00:11:26.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013562852, 2024-09-15T00:11:26.367 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.368 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.368 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:26.368 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.050169+0000", 2024-09-15T00:11:26.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.050169+0000", 2024-09-15T00:11:26.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.050169+0000", 2024-09-15T00:11:26.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8999999999999995e-07 2024-09-15T00:11:26.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.050173+0000", 2024-09-15T00:11:26.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8369999999999999e-06 2024-09-15T00:11:26.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.050183+0000", 2024-09-15T00:11:26.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2890000000000003e-06 2024-09-15T00:11:26.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.050191+0000", 2024-09-15T00:11:26.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0439999999999996e-06 2024-09-15T00:11:26.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063701+0000", 2024-09-15T00:11:26.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013510810999999999 2024-09-15T00:11:26.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063713+0000", 2024-09-15T00:11:26.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2012e-05 2024-09-15T00:11:26.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063731+0000", 2024-09-15T00:11:26.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7969e-05 2024-09-15T00:11:26.371 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.371 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.371 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.371 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1261 3.19 e30/23 3:9bb69e89:::rbd_data.11397bff333d.0000000000000012:head v 30'5, mlcod=30'3)", 2024-09-15T00:11:26.371 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.162191+0000", 2024-09-15T00:11:26.371 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.105511851, 2024-09-15T00:11:26.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020755470000000001, 2024-09-15T00:11:26.372 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:26.372 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:26.372 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.372 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:26.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:26.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.162191+0000", 2024-09-15T00:11:26.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:26.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.162191+0000", 2024-09-15T00:11:26.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:26.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:26.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.162193+0000", 2024-09-15T00:11:26.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0640000000000001e-06 2024-09-15T00:11:26.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:26.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.162209+0000", 2024-09-15T00:11:26.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6056999999999999e-05 2024-09-15T00:11:26.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:26.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.162218+0000", 2024-09-15T00:11:26.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7720000000000006e-06 2024-09-15T00:11:26.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.162224+0000", 2024-09-15T00:11:26.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2900000000000002e-06 2024-09-15T00:11:26.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:26.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.162273+0000", 2024-09-15T00:11:26.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9851000000000001e-05 2024-09-15T00:11:26.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:26.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.162288+0000", 2024-09-15T00:11:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.433e-05 2024-09-15T00:11:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.164228+0000", 2024-09-15T00:11:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019401450000000001 2024-09-15T00:11:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.164267+0000", 2024-09-15T00:11:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9038000000000001e-05 2024-09-15T00:11:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:26.376 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:26.376 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:26.377 INFO:teuthology.orchestra.run.smithi171.stdout:282 read 0xc1f1736 thru 0xc1f54c4 (0x3d8f bytes) 2024-09-15T00:11:26.377 INFO:teuthology.orchestra.run.smithi171.stdout:283 read 0x381d1bb thru 0x38236a9 (0x64ef bytes) 2024-09-15T00:11:26.377 INFO:teuthology.orchestra.run.smithi171.stdout:285 punch from 0xc987fb5 to 0xc98f565, (0x75b0 bytes) 2024-09-15T00:11:26.377 INFO:teuthology.orchestra.run.smithi171.stdout:287 punch from 0xb91a1b3 to 0xb928eff, (0xed4c bytes) 2024-09-15T00:11:26.377 INFO:teuthology.orchestra.run.smithi171.stdout:288 read 0xd571436 thru 0xd57c853 (0xb41e bytes) 2024-09-15T00:11:26.378 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:26.379 INFO:teuthology.orchestra.run.smithi171.stdout:289 write 0xee07ebd thru 0xee1008b (0x81cf bytes) 2024-09-15T00:11:26.379 INFO:teuthology.orchestra.run.smithi171.stdout:292 writesame 0x210f88b thru 0x211626a data_size 0x2c(0x69e0 bytes) 2024-09-15T00:11:26.380 INFO:teuthology.orchestra.run.smithi171.stdout:295 punch from 0xa298db4 to 0xa2a4352, (0xb59e bytes) 2024-09-15T00:11:26.380 INFO:teuthology.orchestra.run.smithi171.stdout:296 write 0x4e9838b thru 0x4ea3464 (0xb0da bytes) 2024-09-15T00:11:26.380 INFO:teuthology.orchestra.run.smithi171.stdout:297 trunc from 0xee1008c to 0x4a14615 2024-09-15T00:11:26.380 INFO:teuthology.orchestra.run.smithi171.stdout:298 trunc from 0x4a14615 to 0x20b0402 2024-09-15T00:11:26.380 INFO:teuthology.orchestra.run.smithi171.stdout:299 trunc from 0x20b0402 to 0x38b1051 2024-09-15T00:11:26.380 INFO:teuthology.orchestra.run.smithi171.stdout:300 write 0x510acd6 thru 0x510ceab (0x21d6 bytes) 2024-09-15T00:11:26.380 INFO:teuthology.orchestra.run.smithi171.stdout:302 punch from 0x3566959 to 0x356818a, (0x1831 bytes) 2024-09-15T00:11:26.380 INFO:teuthology.orchestra.run.smithi171.stdout:304 write 0x48387a8 thru 0x4843f69 (0xb7c2 bytes) 2024-09-15T00:11:26.380 INFO:teuthology.orchestra.run.smithi171.stdout:305 trunc from 0x510ceac to 0x2c63f40 2024-09-15T00:11:26.380 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:26.374+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:26.389 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.396 INFO:teuthology.orchestra.run.smithi171.stdout:307 write 0x3a95a77 thru 0x3aa5465 (0xf9ef bytes) 2024-09-15T00:11:26.400 INFO:teuthology.orchestra.run.smithi171.stdout:308 write 0x1ebb262 thru 0x1ec1f8e (0x6d2d bytes) 2024-09-15T00:11:26.402 INFO:teuthology.orchestra.run.smithi171.stdout:310 write 0x3070303 thru 0x3079128 (0x8e26 bytes) 2024-09-15T00:11:26.406 INFO:teuthology.orchestra.run.smithi171.stdout:313 read 0x2b6c24c thru 0x2b73025 (0x6dda bytes) 2024-09-15T00:11:26.406 INFO:teuthology.orchestra.run.smithi171.stdout:314 read 0x1dd0b9a thru 0x1dd32c7 (0x272e bytes) 2024-09-15T00:11:26.419 INFO:teuthology.orchestra.run.smithi171.stdout:315 write 0x935206b thru 0x93576b8 (0x564e bytes) 2024-09-15T00:11:26.425 INFO:teuthology.orchestra.run.smithi171.stdout:316 trunc from 0x93576b9 to 0x62ea836 2024-09-15T00:11:26.438 INFO:teuthology.orchestra.run.smithi171.stdout:317 write 0x1c400 thru 0x2918d (0xcd8e bytes) 2024-09-15T00:11:26.441 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.441 INFO:teuthology.orchestra.run.smithi171.stdout:320 read 0x4c383a3 thru 0x4c3acc6 (0x2924 bytes) 2024-09-15T00:11:26.441 INFO:teuthology.orchestra.run.smithi171.stdout:321 writesame 0x3e00f5b thru 0x3e037b8 data_size 0x1(0x285e bytes) 2024-09-15T00:11:26.454 INFO:teuthology.orchestra.run.smithi171.stdout:324 trunc from 0x62ea836 to 0x11c0369 2024-09-15T00:11:26.468 INFO:teuthology.orchestra.run.smithi171.stdout:326 read 0x943c5b thru 0x951596 (0xd93c bytes) 2024-09-15T00:11:26.468 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.469 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.469 INFO:teuthology.orchestra.run.smithi171.stdout:329 read 0x1c962c thru 0x1cebec (0x55c1 bytes) 2024-09-15T00:11:26.478 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:26.475+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:26.482 INFO:teuthology.orchestra.run.smithi171.stdout:330 write 0x763ea04 thru 0x76453e4 (0x69e1 bytes) 2024-09-15T00:11:26.486 INFO:teuthology.orchestra.run.smithi171.stdout:332 writesame 0x1465e04 thru 0x1469303 data_size 0x4(0x3500 bytes) 2024-09-15T00:11:26.491 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.491 INFO:teuthology.orchestra.run.smithi171.stdout:334 write 0x4a14e4 thru 0x4aa05b (0x8b78 bytes) 2024-09-15T00:11:26.492 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:26.492 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:26.492 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:26.493 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:26.494 INFO:teuthology.orchestra.run.smithi171.stdout:335 write 0x343b2a9 thru 0x34479e3 (0xc73b bytes) 2024-09-15T00:11:26.502 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:26.508 INFO:teuthology.orchestra.run.smithi171.stdout:336 trunc from 0x76453e5 to 0x38cf8a 2024-09-15T00:11:26.534 INFO:teuthology.orchestra.run.smithi171.stdout:338 writesame 0x4b591d0 thru 0x4b64af8 data_size 0x1(0xb929 bytes) 2024-09-15T00:11:26.534 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.534 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.534 INFO:teuthology.orchestra.run.smithi171.stdout:343 punch from 0xdbf46d to 0xdc3822, (0x43b5 bytes) 2024-09-15T00:11:26.534 INFO:teuthology.orchestra.run.smithi171.stdout:344 write 0x12ad887 thru 0x12b0fd2 (0x374c bytes) 2024-09-15T00:11:26.548 INFO:teuthology.orchestra.run.smithi171.stdout:345 write 0x74c0496 thru 0x74c5436 (0x4fa1 bytes) 2024-09-15T00:11:26.552 INFO:teuthology.orchestra.run.smithi171.stdout:346 write 0x44b3197 thru 0x44bfbd1 (0xca3b bytes) 2024-09-15T00:11:26.557 INFO:teuthology.orchestra.run.smithi171.stdout:347 trunc from 0x74c5437 to 0x7e84fb6 2024-09-15T00:11:26.563 INFO:teuthology.orchestra.run.smithi171.stdout:348 write 0x2ab856e thru 0x2ac77ed (0xf280 bytes) 2024-09-15T00:11:26.568 INFO:teuthology.orchestra.run.smithi171.stdout:350 write 0x1316381 thru 0x13168b8 (0x538 bytes) 2024-09-15T00:11:26.569 INFO:teuthology.orchestra.run.smithi171.stdout:351 read 0x3e2d9f4 thru 0x3e39926 (0xbf33 bytes) 2024-09-15T00:11:26.570 INFO:teuthology.orchestra.run.smithi171.stdout:352 read 0x5c6f6e7 thru 0x5c785d2 (0x8eec bytes) 2024-09-15T00:11:26.570 INFO:teuthology.orchestra.run.smithi171.stdout:353 writesame 0x3104b74 thru 0x3105eb4 data_size 0x3(0x1341 bytes) 2024-09-15T00:11:26.573 INFO:teuthology.orchestra.run.smithi171.stdout:355 read 0x186b4ba thru 0x187b24d (0xfd94 bytes) 2024-09-15T00:11:26.573 INFO:teuthology.orchestra.run.smithi171.stdout:356 read 0x199f62f thru 0x19a928d (0x9c5f bytes) 2024-09-15T00:11:26.573 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.579 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:26.577+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:26.586 INFO:teuthology.orchestra.run.smithi171.stdout:358 write 0xd8ac777 thru 0xd8af039 (0x28c3 bytes) 2024-09-15T00:11:26.589 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.591 INFO:teuthology.orchestra.run.smithi171.stdout:361 trunc from 0xd8af03a to 0xba22e9d 2024-09-15T00:11:26.603 INFO:teuthology.orchestra.run.smithi171.stdout:362 trunc from 0xba22e9d to 0xc6c3769 2024-09-15T00:11:26.609 INFO:teuthology.orchestra.run.smithi171.stdout:364 read 0x337e53d thru 0x338bc0f (0xd6d3 bytes) 2024-09-15T00:11:26.613 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:26.613 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:26.613 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:26.613 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:26.613 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:26.622 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:26.625 INFO:teuthology.orchestra.run.smithi171.stdout:366 trunc from 0xc6c3769 to 0x4f11ce 2024-09-15T00:11:26.645 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:11:26.656 INFO:teuthology.orchestra.run.smithi171.stdout:367 trunc from 0x4f11ce to 0xd4968f7 2024-09-15T00:11:26.663 INFO:teuthology.orchestra.run.smithi171.stdout:368 read 0x52e965f thru 0x52f3d9e (0xa740 bytes) 2024-09-15T00:11:26.663 INFO:teuthology.orchestra.run.smithi171.stdout:369 read 0x8dccc56 thru 0x8ddb379 (0xe724 bytes) 2024-09-15T00:11:26.663 INFO:teuthology.orchestra.run.smithi171.stdout:370 writesame 0x88f3115 thru 0x88f6cdd data_size 0x1(0x3bc9 bytes) 2024-09-15T00:11:26.693 INFO:teuthology.orchestra.run.smithi171.stdout:371 trunc from 0xd4968f7 to 0x591361f 2024-09-15T00:11:26.706 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:26.705+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:26.707 INFO:teuthology.orchestra.run.smithi171.stdout:373 write 0x40e4fb8 thru 0x40edbd7 (0x8c20 bytes) 2024-09-15T00:11:26.711 INFO:teuthology.orchestra.run.smithi171.stdout:374 read 0x3f5abd3 thru 0x3f67f83 (0xd3b1 bytes) 2024-09-15T00:11:26.711 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.711 INFO:teuthology.orchestra.run.smithi171.stdout:376 punch from 0x26693c7 to 0x2673c16, (0xa84f bytes) 2024-09-15T00:11:26.711 INFO:teuthology.orchestra.run.smithi171.stdout:377 writesame 0xa6743e thru 0xa6d3f4 data_size 0x1(0x5fb7 bytes) 2024-09-15T00:11:26.748 INFO:teuthology.orchestra.run.smithi171.stdout:378 writesame 0xc8c771d thru 0xc8cf367 data_size 0x1(0x7c4b bytes) 2024-09-15T00:11:26.748 INFO:teuthology.orchestra.run.smithi171.stdout:379 read 0x572edd7 thru 0x5733d85 (0x4faf bytes) 2024-09-15T00:11:26.762 INFO:teuthology.orchestra.run.smithi171.stdout:383 trunc from 0xc8cf368 to 0x8db83a 2024-09-15T00:11:26.773 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:26.773 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:26.773 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:26.773 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:26.773 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.774 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:463 3.9 3:942069cf:::rbd_header.11397bff333d:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.774 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.020178+0000", 2024-09-15T00:11:26.774 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.7503391349999999, 2024-09-15T00:11:26.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005152022, 2024-09-15T00:11:26.774 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.774 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.774 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.774 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:26.774 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:26.774 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.775 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 463 2024-09-15T00:11:26.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.775 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.775 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.775 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.775 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020178+0000", 2024-09-15T00:11:26.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.775 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.775 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.775 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020178+0000", 2024-09-15T00:11:26.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020179+0000", 2024-09-15T00:11:26.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.733e-06 2024-09-15T00:11:26.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020189+0000", 2024-09-15T00:11:26.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2529999999999993e-06 2024-09-15T00:11:26.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020214+0000", 2024-09-15T00:11:26.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5599999999999999e-05 2024-09-15T00:11:26.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.777 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020234+0000", 2024-09-15T00:11:26.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9755999999999999e-05 2024-09-15T00:11:26.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.777 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020351+0000", 2024-09-15T00:11:26.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000116725 2024-09-15T00:11:26.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020522+0000", 2024-09-15T00:11:26.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000171807 2024-09-15T00:11:26.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:26.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020674+0000", 2024-09-15T00:11:26.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015147000000000001 2024-09-15T00:11:26.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:26.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.021195+0000", 2024-09-15T00:11:26.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052059600000000001 2024-09-15T00:11:26.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.779 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.779 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.779 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025172+0000", 2024-09-15T00:11:26.779 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039778729999999998 2024-09-15T00:11:26.779 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.779 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.779 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.779 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025259+0000", 2024-09-15T00:11:26.779 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6551000000000005e-05 2024-09-15T00:11:26.779 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.779 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.779 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.779 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025330+0000", 2024-09-15T00:11:26.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0658e-05 2024-09-15T00:11:26.780 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.780 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.780 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.780 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:464 3.9 3:942069cf:::rbd_header.11397bff333d:head [call rbd.set_stripe_unit_count in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.780 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.025416+0000", 2024-09-15T00:11:26.780 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.745101051, 2024-09-15T00:11:26.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055912979999999998, 2024-09-15T00:11:26.780 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.780 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.780 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.780 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:26.781 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:26.781 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.781 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 464 2024-09-15T00:11:26.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.781 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025416+0000", 2024-09-15T00:11:26.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025416+0000", 2024-09-15T00:11:26.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.782 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.782 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025416+0000", 2024-09-15T00:11:26.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1500000000000004e-07 2024-09-15T00:11:26.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.782 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.782 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025420+0000", 2024-09-15T00:11:26.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.473e-06 2024-09-15T00:11:26.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.782 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025427+0000", 2024-09-15T00:11:26.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4270000000000002e-06 2024-09-15T00:11:26.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025434+0000", 2024-09-15T00:11:26.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9670000000000004e-06 2024-09-15T00:11:26.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025484+0000", 2024-09-15T00:11:26.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0084999999999999e-05 2024-09-15T00:11:26.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025530+0000", 2024-09-15T00:11:26.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5286e-05 2024-09-15T00:11:26.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:26.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025666+0000", 2024-09-15T00:11:26.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000136067 2024-09-15T00:11:26.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:26.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.026078+0000", 2024-09-15T00:11:26.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00041209200000000002 2024-09-15T00:11:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.030916+0000", 2024-09-15T00:11:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048382060000000003 2024-09-15T00:11:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.030967+0000", 2024-09-15T00:11:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0540000000000001e-05 2024-09-15T00:11:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.031007+0000", 2024-09-15T00:11:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0439999999999999e-05 2024-09-15T00:11:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:603 3.1d e29/23 3:ba2ec8e4:::rbd_data.11397bff333d.0000000000000038:head v 29'1, mlcod=0'0)", 2024-09-15T00:11:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.211359+0000", 2024-09-15T00:11:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5591577079999999, 2024-09-15T00:11:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045527620000000001, 2024-09-15T00:11:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211359+0000", 2024-09-15T00:11:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211359+0000", 2024-09-15T00:11:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211359+0000", 2024-09-15T00:11:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7300000000000001e-07 2024-09-15T00:11:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211380+0000", 2024-09-15T00:11:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0125000000000001e-05 2024-09-15T00:11:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211387+0000", 2024-09-15T00:11:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9789999999999996e-06 2024-09-15T00:11:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211390+0000", 2024-09-15T00:11:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.574e-06 2024-09-15T00:11:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211434+0000", 2024-09-15T00:11:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3865000000000001e-05 2024-09-15T00:11:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211448+0000", 2024-09-15T00:11:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4246999999999999e-05 2024-09-15T00:11:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.215881+0000", 2024-09-15T00:11:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0044323569999999996 2024-09-15T00:11:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.215912+0000", 2024-09-15T00:11:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1142000000000001e-05 2024-09-15T00:11:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:766 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.431810+0000", 2024-09-15T00:11:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.3387065040000001, 2024-09-15T00:11:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042690779999999999, 2024-09-15T00:11:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 766 2024-09-15T00:11:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.431810+0000", 2024-09-15T00:11:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.431810+0000", 2024-09-15T00:11:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.431811+0000", 2024-09-15T00:11:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0199999999999998e-07 2024-09-15T00:11:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.431813+0000", 2024-09-15T00:11:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0710000000000002e-06 2024-09-15T00:11:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.431817+0000", 2024-09-15T00:11:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0099999999999997e-06 2024-09-15T00:11:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.431820+0000", 2024-09-15T00:11:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.931e-06 2024-09-15T00:11:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.432650+0000", 2024-09-15T00:11:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00083020700000000002 2024-09-15T00:11:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.432703+0000", 2024-09-15T00:11:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3480999999999998e-05 2024-09-15T00:11:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.432938+0000", 2024-09-15T00:11:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00023470500000000001 2024-09-15T00:11:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.433473+0000", 2024-09-15T00:11:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053553200000000002 2024-09-15T00:11:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.436024+0000", 2024-09-15T00:11:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0025509640000000002 2024-09-15T00:11:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.436055+0000", 2024-09-15T00:11:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0725999999999997e-05 2024-09-15T00:11:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.436079+0000", 2024-09-15T00:11:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4148999999999999e-05 2024-09-15T00:11:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:832 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.524286+0000", 2024-09-15T00:11:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.246230543, 2024-09-15T00:11:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0067154600000000004, 2024-09-15T00:11:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 832 2024-09-15T00:11:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524286+0000", 2024-09-15T00:11:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524286+0000", 2024-09-15T00:11:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524287+0000", 2024-09-15T00:11:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7899999999999999e-07 2024-09-15T00:11:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524289+0000", 2024-09-15T00:11:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2010000000000002e-06 2024-09-15T00:11:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524295+0000", 2024-09-15T00:11:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0979999999999999e-06 2024-09-15T00:11:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524300+0000", 2024-09-15T00:11:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6970000000000001e-06 2024-09-15T00:11:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524888+0000", 2024-09-15T00:11:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00058825000000000001 2024-09-15T00:11:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524941+0000", 2024-09-15T00:11:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3529000000000002e-05 2024-09-15T00:11:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525192+0000", 2024-09-15T00:11:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025088299999999998 2024-09-15T00:11:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.528065+0000", 2024-09-15T00:11:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002873085 2024-09-15T00:11:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.530935+0000", 2024-09-15T00:11:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028699950000000002 2024-09-15T00:11:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.530968+0000", 2024-09-15T00:11:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.235e-05 2024-09-15T00:11:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.531002+0000", 2024-09-15T00:11:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3992999999999998e-05 2024-09-15T00:11:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:833 3.1a 3:58b59828:::rbd_data.11397bff333d.000000000000001e:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3006249~8407] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.525900+0000", 2024-09-15T00:11:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.244616714, 2024-09-15T00:11:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052167660000000003, 2024-09-15T00:11:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 833 2024-09-15T00:11:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525900+0000", 2024-09-15T00:11:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525900+0000", 2024-09-15T00:11:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525901+0000", 2024-09-15T00:11:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1799999999999995e-07 2024-09-15T00:11:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525909+0000", 2024-09-15T00:11:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4130000000000006e-06 2024-09-15T00:11:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525916+0000", 2024-09-15T00:11:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2350000000000002e-06 2024-09-15T00:11:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525920+0000", 2024-09-15T00:11:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0820000000000001e-06 2024-09-15T00:11:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525957+0000", 2024-09-15T00:11:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6257000000000002e-05 2024-09-15T00:11:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525998+0000", 2024-09-15T00:11:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1662000000000002e-05 2024-09-15T00:11:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.526076+0000", 2024-09-15T00:11:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7985000000000006e-05 2024-09-15T00:11:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.528223+0000", 2024-09-15T00:11:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021471870000000001 2024-09-15T00:11:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.531054+0000", 2024-09-15T00:11:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00283037 2024-09-15T00:11:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.531089+0000", 2024-09-15T00:11:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5364999999999998e-05 2024-09-15T00:11:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.531117+0000", 2024-09-15T00:11:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7691999999999998e-05 2024-09-15T00:11:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:839 3.13 e29/23 3:cd30f0f1:::rbd_data.11397bff333d.000000000000000c:head v 29'5, mlcod=29'3)", 2024-09-15T00:11:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.542374+0000", 2024-09-15T00:11:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.2281426959999999, 2024-09-15T00:11:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0046782610000000004, 2024-09-15T00:11:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.542374+0000", 2024-09-15T00:11:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.542374+0000", 2024-09-15T00:11:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.542375+0000", 2024-09-15T00:11:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.99e-07 2024-09-15T00:11:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.542389+0000", 2024-09-15T00:11:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4283e-05 2024-09-15T00:11:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.542397+0000", 2024-09-15T00:11:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8909999999999995e-06 2024-09-15T00:11:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.542401+0000", 2024-09-15T00:11:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2490000000000003e-06 2024-09-15T00:11:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.542438+0000", 2024-09-15T00:11:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7287999999999999e-05 2024-09-15T00:11:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.542451+0000", 2024-09-15T00:11:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2109999999999999e-05 2024-09-15T00:11:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.547016+0000", 2024-09-15T00:11:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045655089999999997 2024-09-15T00:11:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.547052+0000", 2024-09-15T00:11:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6232000000000001e-05 2024-09-15T00:11:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:11:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.1986023809999999, 2024-09-15T00:11:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:11:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:11:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:11:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:11:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:11:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:11:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:11:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:11:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:11:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:11:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:11:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:11:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:11:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:11:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:11:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:11:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:11:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:11:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:11:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:11:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:11:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:11:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:11:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:11:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:11:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:11:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.1985676889999999, 2024-09-15T00:11:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:11:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:11:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:11:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:11:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:11:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:11:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:11:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:11:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:11:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:11:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:11:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:11:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:11:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:11:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:11:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:11:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:11:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:11:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:11:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:11:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:11:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:11:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:11:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:863 3.9 e29/23 ondisk, result = 0)", 2024-09-15T00:11:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.585151+0000", 2024-09-15T00:11:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.185365453, 2024-09-15T00:11:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.063949342000000006, 2024-09-15T00:11:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585151+0000", 2024-09-15T00:11:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585151+0000", 2024-09-15T00:11:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585152+0000", 2024-09-15T00:11:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.046e-06 2024-09-15T00:11:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585156+0000", 2024-09-15T00:11:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6339999999999999e-06 2024-09-15T00:11:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585185+0000", 2024-09-15T00:11:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8646e-05 2024-09-15T00:11:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585193+0000", 2024-09-15T00:11:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5699999999999993e-06 2024-09-15T00:11:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649017+0000", 2024-09-15T00:11:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.063823787000000007 2024-09-15T00:11:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649023+0000", 2024-09-15T00:11:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.728e-06 2024-09-15T00:11:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649101+0000", 2024-09-15T00:11:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7930999999999996e-05 2024-09-15T00:11:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:919 3.6 e29/28 3:6173e148:::rbd_data.11397bff333d.000000000000002c:head v 29'4, mlcod=29'2)", 2024-09-15T00:11:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.769074+0000", 2024-09-15T00:11:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.001443028, 2024-09-15T00:11:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042833840000000003, 2024-09-15T00:11:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.769074+0000", 2024-09-15T00:11:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.769074+0000", 2024-09-15T00:11:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.769074+0000", 2024-09-15T00:11:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6000000000000003e-07 2024-09-15T00:11:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.769081+0000", 2024-09-15T00:11:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9700000000000002e-06 2024-09-15T00:11:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.769091+0000", 2024-09-15T00:11:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6670000000000008e-06 2024-09-15T00:11:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.769097+0000", 2024-09-15T00:11:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2829999999999998e-06 2024-09-15T00:11:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.769141+0000", 2024-09-15T00:11:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3646000000000002e-05 2024-09-15T00:11:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.769161+0000", 2024-09-15T00:11:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0463999999999999e-05 2024-09-15T00:11:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.773323+0000", 2024-09-15T00:11:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041615690000000004 2024-09-15T00:11:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.773357+0000", 2024-09-15T00:11:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4025e-05 2024-09-15T00:11:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:1129 3.1d e29/23 3:ba4e0201:::rbd_data.11397bff333d.0000000000000028:head v 29'4, mlcod=29'2)", 2024-09-15T00:11:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.896838+0000", 2024-09-15T00:11:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.87367844299999997, 2024-09-15T00:11:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0059020139999999997, 2024-09-15T00:11:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896838+0000", 2024-09-15T00:11:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896838+0000", 2024-09-15T00:11:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896839+0000", 2024-09-15T00:11:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3900000000000004e-07 2024-09-15T00:11:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896844+0000", 2024-09-15T00:11:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6990000000000002e-06 2024-09-15T00:11:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896854+0000", 2024-09-15T00:11:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9790000000000005e-06 2024-09-15T00:11:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896858+0000", 2024-09-15T00:11:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5850000000000001e-06 2024-09-15T00:11:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896932+0000", 2024-09-15T00:11:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4286e-05 2024-09-15T00:11:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896954+0000", 2024-09-15T00:11:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1421999999999999e-05 2024-09-15T00:11:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.902706+0000", 2024-09-15T00:11:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057519570000000003 2024-09-15T00:11:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.902740+0000", 2024-09-15T00:11:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4446999999999997e-05 2024-09-15T00:11:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1184 3.1a 3:5b582af0:::rbd_data.11397bff333d.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.948047+0000", 2024-09-15T00:11:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.82246966399999999, 2024-09-15T00:11:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.039602018000000003, 2024-09-15T00:11:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1184 2024-09-15T00:11:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948047+0000", 2024-09-15T00:11:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948047+0000", 2024-09-15T00:11:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948048+0000", 2024-09-15T00:11:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3499999999999996e-07 2024-09-15T00:11:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948050+0000", 2024-09-15T00:11:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7659999999999999e-06 2024-09-15T00:11:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948057+0000", 2024-09-15T00:11:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1369999999999996e-06 2024-09-15T00:11:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948062+0000", 2024-09-15T00:11:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0160000000000002e-06 2024-09-15T00:11:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948091+0000", 2024-09-15T00:11:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9556999999999998e-05 2024-09-15T00:11:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948172+0000", 2024-09-15T00:11:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1180999999999994e-05 2024-09-15T00:11:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.949974+0000", 2024-09-15T00:11:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018014299999999999 2024-09-15T00:11:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987559+0000", 2024-09-15T00:11:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037585358999999999 2024-09-15T00:11:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987575+0000", 2024-09-15T00:11:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6101000000000001e-05 2024-09-15T00:11:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987611+0000", 2024-09-15T00:11:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6016999999999998e-05 2024-09-15T00:11:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987649+0000", 2024-09-15T00:11:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7919000000000001e-05 2024-09-15T00:11:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1184 3.1a e29/23 ondisk, result = 0)", 2024-09-15T00:11:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.955158+0000", 2024-09-15T00:11:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.81535862699999995, 2024-09-15T00:11:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032410688, 2024-09-15T00:11:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955158+0000", 2024-09-15T00:11:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955158+0000", 2024-09-15T00:11:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955159+0000", 2024-09-15T00:11:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0399999999999996e-07 2024-09-15T00:11:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955161+0000", 2024-09-15T00:11:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4710000000000001e-06 2024-09-15T00:11:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955172+0000", 2024-09-15T00:11:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1059e-05 2024-09-15T00:11:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955177+0000", 2024-09-15T00:11:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1649999999999997e-06 2024-09-15T00:11:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987541+0000", 2024-09-15T00:11:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032363268000000001 2024-09-15T00:11:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987551+0000", 2024-09-15T00:11:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0699e-05 2024-09-15T00:11:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987569+0000", 2024-09-15T00:11:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7422e-05 2024-09-15T00:11:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1297 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.266395+0000", 2024-09-15T00:11:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.50412152200000004, 2024-09-15T00:11:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004382146, 2024-09-15T00:11:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1297 2024-09-15T00:11:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.266395+0000", 2024-09-15T00:11:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.266395+0000", 2024-09-15T00:11:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.266396+0000", 2024-09-15T00:11:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2500000000000001e-07 2024-09-15T00:11:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.266398+0000", 2024-09-15T00:11:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.531e-06 2024-09-15T00:11:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.266403+0000", 2024-09-15T00:11:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8199999999999996e-06 2024-09-15T00:11:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.266406+0000", 2024-09-15T00:11:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0809999999999998e-06 2024-09-15T00:11:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.267385+0000", 2024-09-15T00:11:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00097872900000000001 2024-09-15T00:11:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.267474+0000", 2024-09-15T00:11:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9476999999999997e-05 2024-09-15T00:11:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.267829+0000", 2024-09-15T00:11:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0003546 2024-09-15T00:11:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.268388+0000", 2024-09-15T00:11:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00055896799999999996 2024-09-15T00:11:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.270722+0000", 2024-09-15T00:11:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0023345950000000001 2024-09-15T00:11:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.270754+0000", 2024-09-15T00:11:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1254000000000003e-05 2024-09-15T00:11:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.270777+0000", 2024-09-15T00:11:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3765999999999999e-05 2024-09-15T00:11:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1373 3.9 3:942069cf:::rbd_header.11397bff333d:head [watch reconnect cookie 139649721243872 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.353073+0000", 2024-09-15T00:11:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.41744358399999998, 2024-09-15T00:11:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0044103579999999996, 2024-09-15T00:11:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1373 2024-09-15T00:11:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.353073+0000", 2024-09-15T00:11:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.353073+0000", 2024-09-15T00:11:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.353074+0000", 2024-09-15T00:11:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999996e-07 2024-09-15T00:11:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.353076+0000", 2024-09-15T00:11:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5550000000000001e-06 2024-09-15T00:11:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.353083+0000", 2024-09-15T00:11:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.781e-06 2024-09-15T00:11:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.353089+0000", 2024-09-15T00:11:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.614e-06 2024-09-15T00:11:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.353561+0000", 2024-09-15T00:11:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00047195499999999998 2024-09-15T00:11:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.353608+0000", 2024-09-15T00:11:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7916e-05 2024-09-15T00:11:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.357483+0000", 2024-09-15T00:11:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038750400000000002 2024-09-15T00:11:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1374 3.9 3:942069cf:::rbd_header.11397bff333d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.354861+0000", 2024-09-15T00:11:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.41565606100000002, 2024-09-15T00:11:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055109729999999997, 2024-09-15T00:11:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1374 2024-09-15T00:11:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354861+0000", 2024-09-15T00:11:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354861+0000", 2024-09-15T00:11:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354861+0000", 2024-09-15T00:11:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:11:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354865+0000", 2024-09-15T00:11:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2030000000000001e-06 2024-09-15T00:11:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354871+0000", 2024-09-15T00:11:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8170000000000001e-06 2024-09-15T00:11:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354884+0000", 2024-09-15T00:11:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2468e-05 2024-09-15T00:11:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354922+0000", 2024-09-15T00:11:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8380000000000002e-05 2024-09-15T00:11:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354971+0000", 2024-09-15T00:11:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9051999999999997e-05 2024-09-15T00:11:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.355080+0000", 2024-09-15T00:11:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000108421 2024-09-15T00:11:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.355454+0000", 2024-09-15T00:11:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00037466600000000001 2024-09-15T00:11:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.360227+0000", 2024-09-15T00:11:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047726740000000002 2024-09-15T00:11:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.360310+0000", 2024-09-15T00:11:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3465000000000001e-05 2024-09-15T00:11:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.360372+0000", 2024-09-15T00:11:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1155999999999995e-05 2024-09-15T00:11:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1385 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.366685+0000", 2024-09-15T00:11:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.40383126499999999, 2024-09-15T00:11:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0044400179999999996, 2024-09-15T00:11:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1385 2024-09-15T00:11:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.366685+0000", 2024-09-15T00:11:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.366685+0000", 2024-09-15T00:11:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.366686+0000", 2024-09-15T00:11:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8000000000000001e-07 2024-09-15T00:11:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.366689+0000", 2024-09-15T00:11:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8550000000000002e-06 2024-09-15T00:11:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.366696+0000", 2024-09-15T00:11:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2180000000000002e-06 2024-09-15T00:11:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.366701+0000", 2024-09-15T00:11:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0649999999999998e-06 2024-09-15T00:11:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.367283+0000", 2024-09-15T00:11:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000582479 2024-09-15T00:11:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.367332+0000", 2024-09-15T00:11:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.897e-05 2024-09-15T00:11:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.367547+0000", 2024-09-15T00:11:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021422600000000001 2024-09-15T00:11:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.368131+0000", 2024-09-15T00:11:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00058418399999999996 2024-09-15T00:11:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.370977+0000", 2024-09-15T00:11:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028461670000000001 2024-09-15T00:11:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.371064+0000", 2024-09-15T00:11:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7435999999999994e-05 2024-09-15T00:11:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.371125+0000", 2024-09-15T00:11:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1037999999999994e-05 2024-09-15T00:11:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1628 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.712543+0000", 2024-09-15T00:11:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.057973681999999999, 2024-09-15T00:11:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.018404876000000001, 2024-09-15T00:11:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1628 2024-09-15T00:11:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712543+0000", 2024-09-15T00:11:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712543+0000", 2024-09-15T00:11:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712544+0000", 2024-09-15T00:11:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6899999999999996e-07 2024-09-15T00:11:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712548+0000", 2024-09-15T00:11:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1239999999999998e-06 2024-09-15T00:11:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712558+0000", 2024-09-15T00:11:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0387000000000001e-05 2024-09-15T00:11:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712565+0000", 2024-09-15T00:11:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.545e-06 2024-09-15T00:11:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712603+0000", 2024-09-15T00:11:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7818000000000003e-05 2024-09-15T00:11:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712644+0000", 2024-09-15T00:11:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1743999999999999e-05 2024-09-15T00:11:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.714418+0000", 2024-09-15T00:11:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0017731699999999999 2024-09-15T00:11:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730862+0000", 2024-09-15T00:11:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016444317999999999 2024-09-15T00:11:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730878+0000", 2024-09-15T00:11:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6269e-05 2024-09-15T00:11:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730915+0000", 2024-09-15T00:11:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6365000000000002e-05 2024-09-15T00:11:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730948+0000", 2024-09-15T00:11:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3367000000000001e-05 2024-09-15T00:11:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.878 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1628 3.9 e30/23 ondisk, result = 0)", 2024-09-15T00:11:26.878 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.720479+0000", 2024-09-15T00:11:26.878 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.050037757000000002, 2024-09-15T00:11:26.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010388134, 2024-09-15T00:11:26.878 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:26.878 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:26.878 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:26.878 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:26.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:26.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720479+0000", 2024-09-15T00:11:26.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:26.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720479+0000", 2024-09-15T00:11:26.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:26.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:26.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720480+0000", 2024-09-15T00:11:26.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2299999999999999e-07 2024-09-15T00:11:26.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:26.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720484+0000", 2024-09-15T00:11:26.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-06 2024-09-15T00:11:26.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:26.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720499+0000", 2024-09-15T00:11:26.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4783999999999999e-05 2024-09-15T00:11:26.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:26.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720507+0000", 2024-09-15T00:11:26.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9370000000000004e-06 2024-09-15T00:11:26.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:26.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730848+0000", 2024-09-15T00:11:26.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010341408999999999 2024-09-15T00:11:26.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:26.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730856+0000", 2024-09-15T00:11:26.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0679999999999997e-06 2024-09-15T00:11:26.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:26.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:26.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:26.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730867+0000", 2024-09-15T00:11:26.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0733e-05 2024-09-15T00:11:26.881 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.881 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.882 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.882 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:26.882 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:26.882 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:26.882 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:11:26.885 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:26.885 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:26.882+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:26.886 INFO:teuthology.orchestra.run.smithi171.stdout:384 read 0x233b60 thru 0x23db2f (0x9fd0 bytes) 2024-09-15T00:11:26.886 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.886 INFO:teuthology.orchestra.run.smithi171.stdout:387 write 0x339ec2a thru 0x33aab19 (0xbef0 bytes) 2024-09-15T00:11:26.886 INFO:teuthology.orchestra.run.smithi171.stdout:389 trunc from 0x33aab1a to 0xa329e59 2024-09-15T00:11:26.886 INFO:teuthology.orchestra.run.smithi171.stdout:390 writesame 0x8a2452c thru 0x8a2e63f data_size 0x7a(0xa114 bytes) 2024-09-15T00:11:26.887 INFO:teuthology.orchestra.run.smithi171.stdout:392 write 0x1609d8b thru 0x16174bc (0xd732 bytes) 2024-09-15T00:11:26.887 INFO:teuthology.orchestra.run.smithi171.stdout:393 write 0x27cd572 thru 0x27dbad9 (0xe568 bytes) 2024-09-15T00:11:26.887 INFO:teuthology.orchestra.run.smithi171.stdout:394 write 0x6cbb802 thru 0x6cc91ec (0xd9eb bytes) 2024-09-15T00:11:26.887 INFO:teuthology.orchestra.run.smithi171.stdout:395 writesame 0x8374266 thru 0x837e31e data_size 0x41(0xa0b9 bytes) 2024-09-15T00:11:26.887 INFO:teuthology.orchestra.run.smithi171.stdout:396 punch from 0x4b160e5 to 0x4b1d7e8, (0x7703 bytes) 2024-09-15T00:11:26.887 INFO:teuthology.orchestra.run.smithi171.stdout:397 writesame 0x604c49a thru 0x6051d1c data_size 0x53(0x5883 bytes) 2024-09-15T00:11:26.887 INFO:teuthology.orchestra.run.smithi171.stdout:398 write 0x800a93e thru 0x800d59c (0x2c5f bytes) 2024-09-15T00:11:26.887 INFO:teuthology.orchestra.run.smithi171.stdout:399 trunc from 0xa329e59 to 0xaaa7f9a 2024-09-15T00:11:26.887 INFO:teuthology.orchestra.run.smithi171.stdout:400 punch from 0x4a42709 to 0x4a4a233, (0x7b2a bytes) 2024-09-15T00:11:26.887 INFO:teuthology.orchestra.run.smithi171.stdout:401 write 0xb27968d thru 0xb27c6e7 (0x305b bytes) 2024-09-15T00:11:26.887 INFO:teuthology.orchestra.run.smithi171.stdout:402 read 0x577f142 thru 0x5781103 (0x1fc2 bytes) 2024-09-15T00:11:26.887 INFO:teuthology.orchestra.run.smithi171.stdout:403 write 0x35d7e21 thru 0x35e3126 (0xb306 bytes) 2024-09-15T00:11:26.887 INFO:teuthology.orchestra.run.smithi171.stdout:405 writesame 0x63cd926 thru 0x63ce05a data_size 0xf(0x735 bytes) 2024-09-15T00:11:26.887 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.888 INFO:teuthology.orchestra.run.smithi171.stdout:408 trunc from 0xb27c6e8 to 0x234c6ed 2024-09-15T00:11:26.897 INFO:teuthology.orchestra.run.smithi171.stdout:410 write 0x1949de4 thru 0x19519eb (0x7c08 bytes) 2024-09-15T00:11:26.918 INFO:teuthology.orchestra.run.smithi171.stdout:411 write 0x8f40f0f thru 0x8f4efb7 (0xe0a9 bytes) 2024-09-15T00:11:26.932 INFO:teuthology.orchestra.run.smithi171.stdout:412 writesame 0xd224a04 thru 0xd231bb0 data_size 0x1(0xd1ad bytes) 2024-09-15T00:11:26.932 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:26.933 INFO:teuthology.orchestra.run.smithi171.stdout:417 write 0x3204d55 thru 0x320809c (0x3348 bytes) 2024-09-15T00:11:26.935 INFO:teuthology.orchestra.run.smithi171.stdout:420 write 0x5802d43 thru 0x5804b50 (0x1e0e bytes) 2024-09-15T00:11:26.941 INFO:teuthology.orchestra.run.smithi171.stdout:422 trunc from 0xd231bb1 to 0x9fab2e7 2024-09-15T00:11:26.948 INFO:teuthology.orchestra.run.smithi171.stdout:423 read 0x2855536 thru 0x2859e4c (0x4917 bytes) 2024-09-15T00:11:26.957 INFO:teuthology.orchestra.run.smithi171.stdout:424 write 0xc1ac524 thru 0xc1b2db5 (0x6892 bytes) 2024-09-15T00:11:26.965 INFO:teuthology.orchestra.run.smithi171.stdout:425 write 0xa9d0fe6 thru 0xa9e0cc3 (0xfcde bytes) 2024-09-15T00:11:26.971 INFO:teuthology.orchestra.run.smithi171.stdout:429 punch from 0x41359f0 to 0x413bf72, (0x6582 bytes) 2024-09-15T00:11:26.971 INFO:teuthology.orchestra.run.smithi171.stdout:432 punch from 0xae5a29f to 0xae5c980, (0x26e1 bytes) 2024-09-15T00:11:26.972 INFO:teuthology.orchestra.run.smithi171.stdout:433 read 0xb975146 thru 0xb984dfe (0xfcb9 bytes) 2024-09-15T00:11:26.972 INFO:teuthology.orchestra.run.smithi171.stdout:434 write 0x73a80df thru 0x73ae422 (0x6344 bytes) 2024-09-15T00:11:26.975 INFO:teuthology.orchestra.run.smithi171.stdout:435 writesame 0x7ae3547 thru 0x7aebd56 data_size 0x38(0x8810 bytes) 2024-09-15T00:11:26.978 INFO:teuthology.orchestra.run.smithi171.stdout:436 write 0x83cb592 thru 0x83d2b3c (0x75ab bytes) 2024-09-15T00:11:26.983 INFO:teuthology.orchestra.run.smithi171.stdout:438 write 0xaed1e67 thru 0xaed879a (0x6934 bytes) 2024-09-15T00:11:26.984 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:26.982+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:26.986 INFO:teuthology.orchestra.run.smithi171.stdout:441 write 0xe1103f thru 0xe120c5 (0x1087 bytes) 2024-09-15T00:11:26.989 INFO:teuthology.orchestra.run.smithi171.stdout:442 punch from 0x552edbf to 0x553a55d, (0xb79e bytes) 2024-09-15T00:11:26.989 INFO:teuthology.orchestra.run.smithi171.stdout:443 write 0x44f0038 thru 0x44f83cf (0x8398 bytes) 2024-09-15T00:11:26.992 INFO:teuthology.orchestra.run.smithi171.stdout:444 read 0x1c2be1b thru 0x1c2fc38 (0x3e1e bytes) 2024-09-15T00:11:26.993 INFO:teuthology.orchestra.run.smithi171.stdout:445 read 0x409559f thru 0x40a4916 (0xf378 bytes) 2024-09-15T00:11:26.993 INFO:teuthology.orchestra.run.smithi171.stdout:447 writesame 0x66b3065 thru 0x66b54e2 data_size 0x1b(0x247e bytes) 2024-09-15T00:11:26.996 INFO:teuthology.orchestra.run.smithi171.stdout:449 punch from 0x2616a5a to 0x2620256, (0x97fc bytes) 2024-09-15T00:11:27.004 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is unset 2024-09-15T00:11:27.007 INFO:teuthology.orchestra.run.smithi171.stdout:451 write 0xc453297 thru 0xc4586c5 (0x542f bytes) 2024-09-15T00:11:27.010 INFO:teuthology.orchestra.run.smithi171.stdout:452 read 0x1ab0872 thru 0x1ab284c (0x1fdb bytes) 2024-09-15T00:11:27.011 INFO:teuthology.orchestra.run.smithi171.stdout:453 write 0x4e2292 thru 0x4e852c (0x629b bytes) 2024-09-15T00:11:27.013 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:27.013 INFO:teuthology.orchestra.run.smithi171.stdout:455 punch from 0x1e8c19d to 0x1e8f0f0, (0x2f53 bytes) 2024-09-15T00:11:27.013 INFO:teuthology.orchestra.run.smithi171.stdout:456 read 0x2d5688e thru 0x2d5b77f (0x4ef2 bytes) 2024-09-15T00:11:27.013 INFO:teuthology.orchestra.run.smithi171.stdout:459 punch from 0x93a09b5 to 0x93ad486, (0xcad1 bytes) 2024-09-15T00:11:27.025 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:27.026 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:27.027 INFO:teuthology.orchestra.run.smithi171.stdout:460 trunc from 0xc4586c6 to 0xa9fa18 2024-09-15T00:11:27.042 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:27.042 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:27.042 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:27.042 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:27.045 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:27.045 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:27.049 INFO:teuthology.orchestra.run.smithi171.stdout:462 write 0xdfa37d thru 0xe02b83 (0x8807 bytes) 2024-09-15T00:11:27.051 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:27.057 INFO:teuthology.orchestra.run.smithi171.stdout:463 write 0x1ccdf60 thru 0x1cdb835 (0xd8d6 bytes) 2024-09-15T00:11:27.062 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:27.062 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:27.080 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:27.080 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:27.081 INFO:teuthology.orchestra.run.smithi171.stdout:465 write 0xcc5f9ff thru 0xcc6ba75 (0xc077 bytes) 2024-09-15T00:11:27.084 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:27.082+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:27.085 INFO:teuthology.orchestra.run.smithi171.stdout:466 punch from 0x9d8d4ff to 0x9d94dbe, (0x78bf bytes) 2024-09-15T00:11:27.085 INFO:teuthology.orchestra.run.smithi171.stdout:467 write 0x60bc2dd thru 0x60be72b (0x244f bytes) 2024-09-15T00:11:27.091 INFO:teuthology.orchestra.run.smithi171.stdout:468 trunc from 0xcc6ba76 to 0x91722d6 2024-09-15T00:11:27.097 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:27.097 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:27.104 INFO:teuthology.orchestra.run.smithi171.stdout:469 writesame 0x8ef8c51 thru 0x8eff2a5 data_size 0x17(0x6655 bytes) 2024-09-15T00:11:27.115 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:27.115 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:27.118 INFO:teuthology.orchestra.run.smithi171.stdout:470 trunc from 0x91722d6 to 0xcbf70d 2024-09-15T00:11:27.134 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:27.135 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:27.147 INFO:teuthology.orchestra.run.smithi171.stdout:471 writesame 0x7f91b95 thru 0x7f9ce09 data_size 0x1(0xb275 bytes) 2024-09-15T00:11:27.147 INFO:teuthology.orchestra.run.smithi171.stdout:472 punch from 0x5afc532 to 0x5b09203, (0xccd1 bytes) 2024-09-15T00:11:27.153 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:27.154 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:27.156 INFO:teuthology.orchestra.run.smithi171.stdout:473 writesame 0x8910aff thru 0x891fb4c data_size 0x3(0xf04e bytes) 2024-09-15T00:11:27.156 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:27.156 INFO:teuthology.orchestra.run.smithi171.stdout:475 read 0x616a400 thru 0x6174840 (0xa441 bytes) 2024-09-15T00:11:27.159 INFO:teuthology.orchestra.run.smithi171.stdout:477 trunc from 0x891fb4d to 0x5ec2bf7 2024-09-15T00:11:27.165 INFO:teuthology.orchestra.run.smithi171.stdout:480 punch from 0x45ca792 to 0x45d5915, (0xb183 bytes) 2024-09-15T00:11:27.173 INFO:teuthology.orchestra.run.smithi171.stdout:481 trunc from 0x5ec2bf7 to 0x61f27 2024-09-15T00:11:27.185 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:27.183+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:27.202 INFO:teuthology.orchestra.run.smithi171.stdout:482 writesame 0xb006f9e thru 0xb00beae data_size 0x27(0x4f11 bytes) 2024-09-15T00:11:27.202 INFO:teuthology.orchestra.run.smithi171.stdout:484 read 0x55ee06b thru 0x55f579d (0x7733 bytes) 2024-09-15T00:11:27.202 INFO:teuthology.orchestra.run.smithi171.stdout:485 read 0x1be1b61 thru 0x1be4319 (0x27b9 bytes) 2024-09-15T00:11:27.202 INFO:teuthology.orchestra.run.smithi171.stdout:487 clone 3 order 19 su 65536 sc 11 2024-09-15T00:11:27.204 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:27.204 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:27.213 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:27.298 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:27.296+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:27.358 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:27.358 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:27.367 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:11:27.408 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:27.406+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:27.487 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:27.485+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:27.511 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:27.511 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:27.520 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:11:27.589 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:27.587+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:27.623 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:27.623 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:27.632 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:11:27.690 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:27.688+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:27.774 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:27.774 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:27.782 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:11:27.826 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:27.824+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:27.885 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:27.885 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:27.891 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:27.890+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:27.894 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:11:27.992 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:27.990+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:28.041 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:28.042 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:28.051 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:11:28.070 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone2 from 0xb00beaf (overlap 0x61f27) to 0x44901 2024-09-15T00:11:28.094 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:28.092+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:28.118 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-15T00:11:28.132 INFO:teuthology.orchestra.run.smithi171.stdout:488 trunc from 0xb00beaf to 0xccc054a 2024-09-15T00:11:28.140 INFO:teuthology.orchestra.run.smithi171.stdout:truncating to largest ever: 0xedb222b 2024-09-15T00:11:28.142 INFO:teuthology.orchestra.run.smithi171.stdout:489 trunc from 0xccc054a to 0xedb222b 2024-09-15T00:11:28.147 INFO:teuthology.orchestra.run.smithi171.stdout:490 read 0x42c33e thru 0x42e842 (0x2505 bytes) 2024-09-15T00:11:28.148 INFO:teuthology.orchestra.run.smithi171.stdout:492 write 0x5633a28 thru 0x563529d (0x1876 bytes) 2024-09-15T00:11:28.151 INFO:teuthology.orchestra.run.smithi171.stdout:495 read 0x9bed07c thru 0x9bf067c (0x3601 bytes) 2024-09-15T00:11:28.158 INFO:teuthology.orchestra.run.smithi171.stdout:496 trunc from 0xedb222b to 0x95ed5d4 2024-09-15T00:11:28.182 INFO:teuthology.orchestra.run.smithi171.stdout:497 write 0x63b7c6 thru 0x63c9fd (0x1238 bytes) 2024-09-15T00:11:28.185 INFO:teuthology.orchestra.run.smithi171.stdout:500 punch from 0x525468b to 0x5261d21, (0xd696 bytes) 2024-09-15T00:11:28.185 INFO:teuthology.orchestra.run.smithi171.stdout:501 read 0x3d830d6 thru 0x3d91230 (0xe15b bytes) 2024-09-15T00:11:28.185 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:28.185 INFO:teuthology.orchestra.run.smithi171.stdout:504 read 0x777ccfe thru 0x7787e10 (0xb113 bytes) 2024-09-15T00:11:28.186 INFO:teuthology.orchestra.run.smithi171.stdout:505 read 0x39d524c thru 0x39e099b (0xb750 bytes) 2024-09-15T00:11:28.186 INFO:teuthology.orchestra.run.smithi171.stdout:511 write 0x3313253 thru 0x331ed0a (0xbab8 bytes) 2024-09-15T00:11:28.190 INFO:teuthology.orchestra.run.smithi171.stdout:512 write 0x7cd6c47 thru 0x7ce48de (0xdc98 bytes) 2024-09-15T00:11:28.194 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:28.192+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:28.198 INFO:teuthology.orchestra.run.smithi171.stdout:513 punch from 0x11a0e93 to 0x11b0277, (0xf3e4 bytes) 2024-09-15T00:11:28.198 INFO:teuthology.orchestra.run.smithi171.stdout:514 read 0x85aeceb thru 0x85b2bc7 (0x3edd bytes) 2024-09-15T00:11:28.202 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:28.202 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:28.205 INFO:teuthology.orchestra.run.smithi171.stdout:515 trunc from 0x95ed5d4 to 0x350101a 2024-09-15T00:11:28.212 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:28.256 INFO:teuthology.orchestra.run.smithi171.stdout:517 write 0x945afad thru 0x946233b (0x738f bytes) 2024-09-15T00:11:28.258 INFO:teuthology.orchestra.run.smithi171.stdout:519 punch from 0x625cf54 to 0x626b18f, (0xe23b bytes) 2024-09-15T00:11:28.258 INFO:teuthology.orchestra.run.smithi171.stdout:520 clone 4 order 21 su 8192 sc 14 2024-09-15T00:11:28.295 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:28.292+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:28.322 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:28.322 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:28.331 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:11:28.415 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:28.413+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:28.475 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:28.476 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:28.485 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:11:28.497 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:28.494+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:28.598 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:28.596+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:28.629 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:28.629 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:28.638 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:28.699 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:28.697+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:28.782 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:28.782 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:28.790 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:28.800 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:28.798+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:28.894 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:28.894 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:28.900 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:28.898+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:28.902 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:29.001 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:28.998+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:29.005 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:29.005 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:29.016 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:29.023 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:11:29.080 INFO:teuthology.orchestra.run.smithi171.stdout:flattening image image_client.0-clone3 2024-09-15T00:11:29.102 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:29.099+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:29.131 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:29.131 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:29.141 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:29.168 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:11:29.225 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:29.223+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:29.243 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-15T00:11:29.247 INFO:teuthology.orchestra.run.smithi171.stdout:521 read 0x6e57b3a thru 0x6e5ddda (0x62a1 bytes) 2024-09-15T00:11:29.247 INFO:teuthology.orchestra.run.smithi171.stdout:522 read 0x69b7c72 thru 0x69c37a9 (0xbb38 bytes) 2024-09-15T00:11:29.250 INFO:teuthology.orchestra.run.smithi171.stdout:523 trunc from 0x946233c to 0x718ead1 2024-09-15T00:11:29.279 INFO:teuthology.orchestra.run.smithi171.stdout:524 read 0x19139ad thru 0x192037b (0xc9cf bytes) 2024-09-15T00:11:29.290 INFO:teuthology.orchestra.run.smithi171.stdout:525 write 0xb188758 thru 0xb18b245 (0x2aee bytes) 2024-09-15T00:11:29.293 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:29.293 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:29.293 INFO:teuthology.orchestra.run.smithi171.stdout:529 write 0x51e2ce5 thru 0x51f1e3d (0xf159 bytes) 2024-09-15T00:11:29.302 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:29.300+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:29.307 INFO:teuthology.orchestra.run.smithi171.stdout:530 read 0x6da5375 thru 0x6db29e8 (0xd674 bytes) 2024-09-15T00:11:29.308 INFO:teuthology.orchestra.run.smithi171.stdout:531 read 0x5e1a325 thru 0x5e29361 (0xf03d bytes) 2024-09-15T00:11:29.308 INFO:teuthology.orchestra.run.smithi171.stdout:532 read 0x13675a3 thru 0x136a738 (0x3196 bytes) 2024-09-15T00:11:29.308 INFO:teuthology.orchestra.run.smithi171.stdout:533 write 0x7bc2e88 thru 0x7bc7523 (0x469c bytes) 2024-09-15T00:11:29.313 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:29.313 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:29.315 INFO:teuthology.orchestra.run.smithi171.stdout:534 trunc from 0xb18b246 to 0x70f089f 2024-09-15T00:11:29.325 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:29.344 INFO:teuthology.orchestra.run.smithi171.stdout:535 write 0xcf09386 thru 0xcf0f6e8 (0x6363 bytes) 2024-09-15T00:11:29.349 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:29.349 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:29.352 INFO:teuthology.orchestra.run.smithi171.stdout:536 punch from 0x69beaa9 to 0x69c6429, (0x7980 bytes) 2024-09-15T00:11:29.353 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:29.353 INFO:teuthology.orchestra.run.smithi171.stdout:540 read 0x668eb1a thru 0x6692b63 (0x404a bytes) 2024-09-15T00:11:29.353 INFO:teuthology.orchestra.run.smithi171.stdout:542 read 0x9bac61a thru 0x9bac8f2 (0x2d9 bytes) 2024-09-15T00:11:29.353 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:29.353 INFO:teuthology.orchestra.run.smithi171.stdout:547 write 0x7ec3cef thru 0x7ed239a (0xe6ac bytes) 2024-09-15T00:11:29.356 INFO:teuthology.orchestra.run.smithi171.stdout:549 read 0x959311a thru 0x95994a4 (0x638b bytes) 2024-09-15T00:11:29.356 INFO:teuthology.orchestra.run.smithi171.stdout:551 punch from 0x70eb268 to 0x70f4880, (0x9618 bytes) 2024-09-15T00:11:29.356 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:29.357 INFO:teuthology.orchestra.run.smithi171.stdout:553 writesame 0x7aaeb51 thru 0x7ab5eb6 data_size 0x1(0x7366 bytes) 2024-09-15T00:11:29.373 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:29.373 INFO:teuthology.orchestra.run.smithi171.stdout:556 writesame 0xb87b018 thru 0xb88799d data_size 0x6e(0xc986 bytes) 2024-09-15T00:11:29.383 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:29.383 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:29.394 INFO:teuthology.orchestra.run.smithi171.stdout:557 trunc from 0xcf0f6e9 to 0x17f8605 2024-09-15T00:11:29.403 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:29.403 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:29.404 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:29.401+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:29.427 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:29.427 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:29.450 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:29.450 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:29.466 INFO:teuthology.orchestra.run.smithi171.stdout:558 write 0x19abf78 thru 0x19b0d12 (0x4d9b bytes) 2024-09-15T00:11:29.471 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:29.471 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:29.476 INFO:teuthology.orchestra.run.smithi171.stdout:559 trunc from 0x19b0d13 to 0x7956266 2024-09-15T00:11:29.481 INFO:teuthology.orchestra.run.smithi171.stdout:561 read 0x59ebbf8 thru 0x59ece98 (0x12a1 bytes) 2024-09-15T00:11:29.481 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:29.482 INFO:teuthology.orchestra.run.smithi171.stdout:564 writesame 0x7a268f thru 0x7a3ad8 data_size 0xe(0x144a bytes) 2024-09-15T00:11:29.482 INFO:teuthology.orchestra.run.smithi171.stdout:566 punch from 0x693cc53 to 0x6943780, (0x6b2d bytes) 2024-09-15T00:11:29.491 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:29.491 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:29.496 INFO:teuthology.orchestra.run.smithi171.stdout:567 write 0xe54774a thru 0xe556fc6 (0xf87d bytes) 2024-09-15T00:11:29.504 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:29.501+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:29.509 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:29.514 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:29.514 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:29.517 INFO:teuthology.orchestra.run.smithi171.stdout:569 trunc from 0xe556fc7 to 0x7b58fdf 2024-09-15T00:11:29.521 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:29.521 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:29.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:29.521 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:29.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.521 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:462 3.0 e29/23 3:04479396:::rbd_id.image_client.0-clone2:head v 29'14, mlcod=27'12)", 2024-09-15T00:11:29.522 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:29.522 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5017356150000003, 2024-09-15T00:11:29.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030963029999999999, 2024-09-15T00:11:29.522 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.522 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.522 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.522 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:29.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:29.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:29.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:11:29.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016208+0000", 2024-09-15T00:11:29.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3669999999999997e-06 2024-09-15T00:11:29.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016250+0000", 2024-09-15T00:11:29.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1845999999999999e-05 2024-09-15T00:11:29.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016260+0000", 2024-09-15T00:11:29.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9750000000000002e-06 2024-09-15T00:11:29.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016342+0000", 2024-09-15T00:11:29.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1595999999999996e-05 2024-09-15T00:11:29.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016363+0000", 2024-09-15T00:11:29.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1073e-05 2024-09-15T00:11:29.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019246+0000", 2024-09-15T00:11:29.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028835240000000002 2024-09-15T00:11:29.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019296+0000", 2024-09-15T00:11:29.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9533999999999997e-05 2024-09-15T00:11:29.526 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.526 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.526 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.526 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:463 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'9, mlcod=27'7)", 2024-09-15T00:11:29.526 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:29.526 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.497122611, 2024-09-15T00:11:29.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040015709999999998, 2024-09-15T00:11:29.526 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.526 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.526 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.526 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:29.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:29.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:29.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7599999999999999e-07 2024-09-15T00:11:29.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020818+0000", 2024-09-15T00:11:29.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6609999999999999e-06 2024-09-15T00:11:29.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020826+0000", 2024-09-15T00:11:29.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4770000000000003e-06 2024-09-15T00:11:29.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020833+0000", 2024-09-15T00:11:29.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0809999999999996e-06 2024-09-15T00:11:29.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020878+0000", 2024-09-15T00:11:29.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4752000000000001e-05 2024-09-15T00:11:29.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020892+0000", 2024-09-15T00:11:29.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3899e-05 2024-09-15T00:11:29.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024767+0000", 2024-09-15T00:11:29.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038752259999999998 2024-09-15T00:11:29.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024814+0000", 2024-09-15T00:11:29.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7299000000000002e-05 2024-09-15T00:11:29.531 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.532 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.532 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.532 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:464 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'10, mlcod=27'8)", 2024-09-15T00:11:29.532 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:29.532 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.4922027690000004, 2024-09-15T00:11:29.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048109290000000002, 2024-09-15T00:11:29.532 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.532 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.532 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.532 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:29.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:29.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:29.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-15T00:11:29.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025743+0000", 2024-09-15T00:11:29.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5789999999999993e-06 2024-09-15T00:11:29.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025752+0000", 2024-09-15T00:11:29.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8440000000000004e-06 2024-09-15T00:11:29.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025757+0000", 2024-09-15T00:11:29.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2610000000000001e-06 2024-09-15T00:11:29.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025796+0000", 2024-09-15T00:11:29.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9428999999999998e-05 2024-09-15T00:11:29.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025810+0000", 2024-09-15T00:11:29.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3129000000000001e-05 2024-09-15T00:11:29.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030514+0000", 2024-09-15T00:11:29.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047040099999999998 2024-09-15T00:11:29.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030544+0000", 2024-09-15T00:11:29.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9966000000000001e-05 2024-09-15T00:11:29.536 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.536 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.536 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.536 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:602 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'29, mlcod=29'26)", 2024-09-15T00:11:29.536 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.210317+0000", 2024-09-15T00:11:29.536 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3076186849999996, 2024-09-15T00:11:29.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002781604, 2024-09-15T00:11:29.537 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.537 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.537 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.537 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210317+0000", 2024-09-15T00:11:29.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210317+0000", 2024-09-15T00:11:29.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210317+0000", 2024-09-15T00:11:29.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3200000000000005e-07 2024-09-15T00:11:29.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210322+0000", 2024-09-15T00:11:29.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0039999999999999e-06 2024-09-15T00:11:29.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210332+0000", 2024-09-15T00:11:29.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3659999999999998e-06 2024-09-15T00:11:29.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210337+0000", 2024-09-15T00:11:29.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4269999999999999e-06 2024-09-15T00:11:29.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210387+0000", 2024-09-15T00:11:29.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0201999999999998e-05 2024-09-15T00:11:29.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.210408+0000", 2024-09-15T00:11:29.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0602e-05 2024-09-15T00:11:29.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.213053+0000", 2024-09-15T00:11:29.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002645072 2024-09-15T00:11:29.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.213098+0000", 2024-09-15T00:11:29.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5299000000000001e-05 2024-09-15T00:11:29.541 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.541 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.541 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.541 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:610 3.4 e29/23 3:255b0913:::rbd_data.11397bff333d.0000000000000039:head v 29'2, mlcod=0'0)", 2024-09-15T00:11:29.541 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:11:29.541 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.2894591740000001, 2024-09-15T00:11:29.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003163014, 2024-09-15T00:11:29.541 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.541 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.541 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.541 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:11:29.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:11:29.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228477+0000", 2024-09-15T00:11:29.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1100000000000004e-07 2024-09-15T00:11:29.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228483+0000", 2024-09-15T00:11:29.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3720000000000001e-06 2024-09-15T00:11:29.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228497+0000", 2024-09-15T00:11:29.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3944000000000001e-05 2024-09-15T00:11:29.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228505+0000", 2024-09-15T00:11:29.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8140000000000001e-06 2024-09-15T00:11:29.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228549+0000", 2024-09-15T00:11:29.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4116000000000002e-05 2024-09-15T00:11:29.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228568+0000", 2024-09-15T00:11:29.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8473000000000001e-05 2024-09-15T00:11:29.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231599+0000", 2024-09-15T00:11:29.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030315590000000001 2024-09-15T00:11:29.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231639+0000", 2024-09-15T00:11:29.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9824999999999999e-05 2024-09-15T00:11:29.545 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.545 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.545 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.545 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:766 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'81, mlcod=29'78)", 2024-09-15T00:11:29.545 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:11:29.545 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.0849100969999999, 2024-09-15T00:11:29.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029025880000000002, 2024-09-15T00:11:29.546 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.546 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.546 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.546 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:11:29.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:11:29.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433026+0000", 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433030+0000", 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6280000000000004e-06 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433038+0000", 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.072e-06 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433045+0000", 2024-09-15T00:11:29.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1510000000000001e-06 2024-09-15T00:11:29.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433101+0000", 2024-09-15T00:11:29.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6336999999999999e-05 2024-09-15T00:11:29.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433113+0000", 2024-09-15T00:11:29.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1639e-05 2024-09-15T00:11:29.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435897+0000", 2024-09-15T00:11:29.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027845779999999998 2024-09-15T00:11:29.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435928+0000", 2024-09-15T00:11:29.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0689999999999999e-05 2024-09-15T00:11:29.549 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.549 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.549 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.549 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:832 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'100, mlcod=29'97)", 2024-09-15T00:11:29.549 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:29.550 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.9926645970000001, 2024-09-15T00:11:29.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005498925, 2024-09-15T00:11:29.550 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.550 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.550 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.550 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:29.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:29.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:29.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7999999999999995e-07 2024-09-15T00:11:29.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525276+0000", 2024-09-15T00:11:29.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6739999999999996e-06 2024-09-15T00:11:29.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525287+0000", 2024-09-15T00:11:29.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0906e-05 2024-09-15T00:11:29.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525293+0000", 2024-09-15T00:11:29.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4409999999999998e-06 2024-09-15T00:11:29.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525329+0000", 2024-09-15T00:11:29.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5237000000000002e-05 2024-09-15T00:11:29.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525340+0000", 2024-09-15T00:11:29.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1782e-05 2024-09-15T00:11:29.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530710+0000", 2024-09-15T00:11:29.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053695460000000002 2024-09-15T00:11:29.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530770+0000", 2024-09-15T00:11:29.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9658999999999999e-05 2024-09-15T00:11:29.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.553 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.554 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:876 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'114, mlcod=29'111)", 2024-09-15T00:11:29.554 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:29.554 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.8216255970000002, 2024-09-15T00:11:29.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028465830000000002, 2024-09-15T00:11:29.554 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.554 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.554 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.554 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:29.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:29.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:29.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2399999999999998e-07 2024-09-15T00:11:29.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696316+0000", 2024-09-15T00:11:29.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2800000000000003e-06 2024-09-15T00:11:29.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696326+0000", 2024-09-15T00:11:29.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0546e-05 2024-09-15T00:11:29.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696333+0000", 2024-09-15T00:11:29.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7660000000000001e-06 2024-09-15T00:11:29.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696369+0000", 2024-09-15T00:11:29.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5933000000000003e-05 2024-09-15T00:11:29.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696381+0000", 2024-09-15T00:11:29.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2417e-05 2024-09-15T00:11:29.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699124+0000", 2024-09-15T00:11:29.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027428000000000001 2024-09-15T00:11:29.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699156+0000", 2024-09-15T00:11:29.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2217e-05 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1313 3.1f e30/28 3:fa9cde43:::rbd_data.11397bff333d.000000000000003a:head v 30'3, mlcod=26'1)", 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.2337859600000001, 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044064359999999997, 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:11:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:11:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284150+0000", 2024-09-15T00:11:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3799999999999999e-07 2024-09-15T00:11:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284228+0000", 2024-09-15T00:11:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:11:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284235+0000", 2024-09-15T00:11:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:11:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284239+0000", 2024-09-15T00:11:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3449999999999999e-06 2024-09-15T00:11:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284282+0000", 2024-09-15T00:11:29.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2506000000000003e-05 2024-09-15T00:11:29.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284297+0000", 2024-09-15T00:11:29.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5523999999999999e-05 2024-09-15T00:11:29.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288517+0000", 2024-09-15T00:11:29.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042194019999999997 2024-09-15T00:11:29.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288556+0000", 2024-09-15T00:11:29.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8952000000000002e-05 2024-09-15T00:11:29.562 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.562 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.562 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.562 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1321 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'7, mlcod=29'5)", 2024-09-15T00:11:29.562 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:29.562 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.2096927009999998, 2024-09-15T00:11:29.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003131254, 2024-09-15T00:11:29.562 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.562 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.562 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.562 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308250+0000", 2024-09-15T00:11:29.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6109999999999999e-06 2024-09-15T00:11:29.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308264+0000", 2024-09-15T00:11:29.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4473e-05 2024-09-15T00:11:29.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308272+0000", 2024-09-15T00:11:29.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3540000000000002e-06 2024-09-15T00:11:29.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308340+0000", 2024-09-15T00:11:29.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8305000000000004e-05 2024-09-15T00:11:29.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308361+0000", 2024-09-15T00:11:29.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0409999999999999e-05 2024-09-15T00:11:29.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311332+0000", 2024-09-15T00:11:29.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029715950000000001 2024-09-15T00:11:29.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311374+0000", 2024-09-15T00:11:29.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1786000000000003e-05 2024-09-15T00:11:29.566 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.566 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.566 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.566 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.9 epoch 30/23 rep_tid 277 entries 30'217 (0'0) error 3:942069cf:::rbd_header.11397bff333d:head by client.4409.0:1373 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'214)", 2024-09-15T00:11:29.566 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:11:29.567 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.1641401039999999, 2024-09-15T00:11:29.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034617960000000001, 2024-09-15T00:11:29.567 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.567 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.567 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:11:29.567 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:11:29.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:11:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353796+0000", 2024-09-15T00:11:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6699999999999999e-07 2024-09-15T00:11:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353803+0000", 2024-09-15T00:11:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.832e-06 2024-09-15T00:11:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353814+0000", 2024-09-15T00:11:29.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1815e-05 2024-09-15T00:11:29.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353822+0000", 2024-09-15T00:11:29.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7440000000000007e-06 2024-09-15T00:11:29.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353910+0000", 2024-09-15T00:11:29.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7491000000000006e-05 2024-09-15T00:11:29.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.357257+0000", 2024-09-15T00:11:29.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033474469999999999 2024-09-15T00:11:29.570 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.570 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.570 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.570 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1374 3.9 e30/23 3:942069cf:::rbd_header.11397bff333d:head v 30'218, mlcod=30'215)", 2024-09-15T00:11:29.570 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:11:29.570 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.1627404619999999, 2024-09-15T00:11:29.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047299830000000001, 2024-09-15T00:11:29.570 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.570 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.570 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355196+0000", 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3900000000000004e-07 2024-09-15T00:11:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355200+0000", 2024-09-15T00:11:29.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5700000000000003e-06 2024-09-15T00:11:29.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355210+0000", 2024-09-15T00:11:29.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6050000000000004e-06 2024-09-15T00:11:29.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355215+0000", 2024-09-15T00:11:29.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.933e-06 2024-09-15T00:11:29.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355249+0000", 2024-09-15T00:11:29.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4110999999999999e-05 2024-09-15T00:11:29.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355260+0000", 2024-09-15T00:11:29.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1235e-05 2024-09-15T00:11:29.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359895+0000", 2024-09-15T00:11:29.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046345550000000003 2024-09-15T00:11:29.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359925+0000", 2024-09-15T00:11:29.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0335e-05 2024-09-15T00:11:29.574 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.574 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.574 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1385 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'221, mlcod=30'218)", 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.1502839269999998, 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030443950000000001, 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:11:29.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367652+0000", 2024-09-15T00:11:29.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:11:29.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367657+0000", 2024-09-15T00:11:29.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5410000000000002e-06 2024-09-15T00:11:29.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367666+0000", 2024-09-15T00:11:29.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9280000000000005e-06 2024-09-15T00:11:29.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367671+0000", 2024-09-15T00:11:29.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7870000000000001e-06 2024-09-15T00:11:29.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367707+0000", 2024-09-15T00:11:29.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6102000000000001e-05 2024-09-15T00:11:29.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367719+0000", 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1571e-05 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370666+0000", 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029474750000000002 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370696+0000", 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9402000000000001e-05 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.578 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.579 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1662 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'300, mlcod=30'297)", 2024-09-15T00:11:29.579 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:11:29.579 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.7299192899999998, 2024-09-15T00:11:29.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031296090000000002, 2024-09-15T00:11:29.579 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.579 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.579 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.579 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:11:29.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:11:29.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788017+0000", 2024-09-15T00:11:29.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8999999999999996e-07 2024-09-15T00:11:29.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788021+0000", 2024-09-15T00:11:29.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4970000000000003e-06 2024-09-15T00:11:29.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788027+0000", 2024-09-15T00:11:29.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1840000000000004e-06 2024-09-15T00:11:29.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788032+0000", 2024-09-15T00:11:29.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.135e-06 2024-09-15T00:11:29.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788070+0000", 2024-09-15T00:11:29.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7073999999999998e-05 2024-09-15T00:11:29.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788081+0000", 2024-09-15T00:11:29.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0987e-05 2024-09-15T00:11:29.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791116+0000", 2024-09-15T00:11:29.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030355209999999998 2024-09-15T00:11:29.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791146+0000", 2024-09-15T00:11:29.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9621e-05 2024-09-15T00:11:29.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.583 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.583 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1677 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'305, mlcod=30'302)", 2024-09-15T00:11:29.583 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:11:29.583 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.7061909279999998, 2024-09-15T00:11:29.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031364320000000002, 2024-09-15T00:11:29.583 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.583 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.583 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.584 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:11:29.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:11:29.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811745+0000", 2024-09-15T00:11:29.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:11:29.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811749+0000", 2024-09-15T00:11:29.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9369999999999997e-06 2024-09-15T00:11:29.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811756+0000", 2024-09-15T00:11:29.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:11:29.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811765+0000", 2024-09-15T00:11:29.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7930000000000003e-06 2024-09-15T00:11:29.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811797+0000", 2024-09-15T00:11:29.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2203999999999999e-05 2024-09-15T00:11:29.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811810+0000", 2024-09-15T00:11:29.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2359e-05 2024-09-15T00:11:29.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814852+0000", 2024-09-15T00:11:29.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003042276 2024-09-15T00:11:29.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814881+0000", 2024-09-15T00:11:29.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9037999999999999e-05 2024-09-15T00:11:29.588 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.588 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.588 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.588 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1681 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'307, mlcod=30'304)", 2024-09-15T00:11:29.588 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.819082+0000", 2024-09-15T00:11:29.588 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.6988530769999999, 2024-09-15T00:11:29.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027510070000000002, 2024-09-15T00:11:29.588 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.588 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.588 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.588 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.819082+0000", 2024-09-15T00:11:29.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.819082+0000", 2024-09-15T00:11:29.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.819083+0000", 2024-09-15T00:11:29.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4099999999999999e-07 2024-09-15T00:11:29.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.819087+0000", 2024-09-15T00:11:29.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8279999999999996e-06 2024-09-15T00:11:29.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.819093+0000", 2024-09-15T00:11:29.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6279999999999999e-06 2024-09-15T00:11:29.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.819097+0000", 2024-09-15T00:11:29.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6610000000000002e-06 2024-09-15T00:11:29.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.819125+0000", 2024-09-15T00:11:29.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7719999999999999e-05 2024-09-15T00:11:29.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.819135+0000", 2024-09-15T00:11:29.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0261999999999999e-05 2024-09-15T00:11:29.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.821803+0000", 2024-09-15T00:11:29.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026685179999999999 2024-09-15T00:11:29.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.821833+0000", 2024-09-15T00:11:29.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9949000000000001e-05 2024-09-15T00:11:29.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.592 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.592 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1724 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'9, mlcod=30'7)", 2024-09-15T00:11:29.592 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:11:29.592 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.637265244, 2024-09-15T00:11:29.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003115769, 2024-09-15T00:11:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:11:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:11:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880671+0000", 2024-09-15T00:11:29.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4900000000000005e-07 2024-09-15T00:11:29.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880678+0000", 2024-09-15T00:11:29.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0979999999999996e-06 2024-09-15T00:11:29.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880691+0000", 2024-09-15T00:11:29.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3288999999999999e-05 2024-09-15T00:11:29.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880698+0000", 2024-09-15T00:11:29.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9240000000000001e-06 2024-09-15T00:11:29.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880757+0000", 2024-09-15T00:11:29.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8866000000000002e-05 2024-09-15T00:11:29.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880790+0000", 2024-09-15T00:11:29.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2926999999999998e-05 2024-09-15T00:11:29.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883738+0000", 2024-09-15T00:11:29.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002948418 2024-09-15T00:11:29.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883786+0000", 2024-09-15T00:11:29.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7398000000000003e-05 2024-09-15T00:11:29.597 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.597 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.597 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.597 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1743 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'324, mlcod=30'321)", 2024-09-15T00:11:29.597 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.898053+0000", 2024-09-15T00:11:29.597 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.6198826020000001, 2024-09-15T00:11:29.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027925620000000002, 2024-09-15T00:11:29.597 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.597 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.598 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.598 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.898053+0000", 2024-09-15T00:11:29.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.898053+0000", 2024-09-15T00:11:29.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.898053+0000", 2024-09-15T00:11:29.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:11:29.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.898062+0000", 2024-09-15T00:11:29.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4819999999999994e-06 2024-09-15T00:11:29.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.898072+0000", 2024-09-15T00:11:29.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0319000000000001e-05 2024-09-15T00:11:29.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.898078+0000", 2024-09-15T00:11:29.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9549999999999997e-06 2024-09-15T00:11:29.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.898125+0000", 2024-09-15T00:11:29.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6408e-05 2024-09-15T00:11:29.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.898147+0000", 2024-09-15T00:11:29.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2569999999999999e-05 2024-09-15T00:11:29.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.900807+0000", 2024-09-15T00:11:29.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026593430000000002 2024-09-15T00:11:29.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.900845+0000", 2024-09-15T00:11:29.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8764999999999999e-05 2024-09-15T00:11:29.601 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.601 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.601 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.601 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1789 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'337, mlcod=30'334)", 2024-09-15T00:11:29.602 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.958666+0000", 2024-09-15T00:11:29.602 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.5592696909999999, 2024-09-15T00:11:29.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027363140000000001, 2024-09-15T00:11:29.602 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.602 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.602 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.602 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.958666+0000", 2024-09-15T00:11:29.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.958666+0000", 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.958666+0000", 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.958670+0000", 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.958679+0000", 2024-09-15T00:11:29.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.242e-06 2024-09-15T00:11:29.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.958684+0000", 2024-09-15T00:11:29.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9849999999999997e-06 2024-09-15T00:11:29.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.958730+0000", 2024-09-15T00:11:29.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.651e-05 2024-09-15T00:11:29.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.958742+0000", 2024-09-15T00:11:29.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1943999999999999e-05 2024-09-15T00:11:29.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.961359+0000", 2024-09-15T00:11:29.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002616967 2024-09-15T00:11:29.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.961402+0000", 2024-09-15T00:11:29.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2964999999999999e-05 2024-09-15T00:11:29.605 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.605 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.606 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.606 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1790 3.9 e30/23 3:9676e01b:::rbd_data.11397bff333d.0000000000000030:head v 30'338, mlcod=30'335)", 2024-09-15T00:11:29.606 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:11:29.606 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.5583828739999999, 2024-09-15T00:11:29.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044496980000000002, 2024-09-15T00:11:29.606 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.606 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.606 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:29.607 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:11:29.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:11:29.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959553+0000", 2024-09-15T00:11:29.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3600000000000004e-07 2024-09-15T00:11:29.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959569+0000", 2024-09-15T00:11:29.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6152999999999999e-05 2024-09-15T00:11:29.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959577+0000", 2024-09-15T00:11:29.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5179999999999998e-06 2024-09-15T00:11:29.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959581+0000", 2024-09-15T00:11:29.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3669999999999999e-06 2024-09-15T00:11:29.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959619+0000", 2024-09-15T00:11:29.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8209e-05 2024-09-15T00:11:29.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959632+0000", 2024-09-15T00:11:29.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2405000000000001e-05 2024-09-15T00:11:29.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:29.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.963954+0000", 2024-09-15T00:11:29.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043226660000000002 2024-09-15T00:11:29.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:29.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.964002+0000", 2024-09-15T00:11:29.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7843999999999998e-05 2024-09-15T00:11:29.611 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.611 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.611 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.611 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.611 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.612 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:29.614 INFO:teuthology.orchestra.run.smithi171.stdout:570 read 0x781229f thru 0x78172bd (0x501f bytes) 2024-09-15T00:11:29.614 INFO:teuthology.orchestra.run.smithi171.stdout:572 read 0x56a54d0 thru 0x56afadf (0xa610 bytes) 2024-09-15T00:11:29.614 INFO:teuthology.orchestra.run.smithi171.stdout:573 read 0x5a8b6d2 thru 0x5a9aee8 (0xf817 bytes) 2024-09-15T00:11:29.614 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:29.614 INFO:teuthology.orchestra.run.smithi171.stdout:575 trunc from 0x7b58fdf to 0x6e8f9bd 2024-09-15T00:11:29.615 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:29.616 INFO:teuthology.orchestra.run.smithi171.stdout:577 punch from 0x1afcdff to 0x1b01fa0, (0x51a1 bytes) 2024-09-15T00:11:29.616 INFO:teuthology.orchestra.run.smithi171.stdout:truncating to largest ever: 0xeddc440 2024-09-15T00:11:29.616 INFO:teuthology.orchestra.run.smithi171.stdout:579 trunc from 0x6e8f9bd to 0xeddc440 2024-09-15T00:11:29.616 INFO:teuthology.orchestra.run.smithi171.stdout:580 read 0x343df88 thru 0x344c9c2 (0xea3b bytes) 2024-09-15T00:11:29.617 INFO:teuthology.orchestra.run.smithi171.stdout:581 trunc from 0xeddc440 to 0x522559 2024-09-15T00:11:29.617 INFO:teuthology.orchestra.run.smithi171.stdout:583 write 0x50fb557 thru 0x510186a (0x6314 bytes) 2024-09-15T00:11:29.617 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:29.617 INFO:teuthology.orchestra.run.smithi171.stdout:586 punch from 0x377bf7f to 0x378baf7, (0xfb78 bytes) 2024-09-15T00:11:29.617 INFO:teuthology.orchestra.run.smithi171.stdout:587 writesame 0xcfee127 thru 0xcff4473 data_size 0x1(0x634d bytes) 2024-09-15T00:11:29.617 INFO:teuthology.orchestra.run.smithi171.stdout:590 read 0x2e761b2 thru 0x2e7f249 (0x9098 bytes) 2024-09-15T00:11:29.617 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:29.610+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:29.618 INFO:teuthology.orchestra.run.smithi171.stdout:591 write 0xd052075 thru 0xd060eab (0xee37 bytes) 2024-09-15T00:11:29.632 INFO:teuthology.orchestra.run.smithi171.stdout:593 trunc from 0xd060eac to 0xe58008c 2024-09-15T00:11:29.639 INFO:teuthology.orchestra.run.smithi171.stdout:596 punch from 0x8707fed to 0x87096c7, (0x16da bytes) 2024-09-15T00:11:29.639 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:29.639 INFO:teuthology.orchestra.run.smithi171.stdout:600 read 0xaac91da thru 0xaad330f (0xa136 bytes) 2024-09-15T00:11:29.642 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T00:11:29.642 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:11:29.642 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-15T00:11:29.642 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T00:11:29.642 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T00:11:29.642 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-15T00:11:29.653 INFO:teuthology.orchestra.run.smithi171.stdout:601 trunc from 0xe58008c to 0x2a08f1b 2024-09-15T00:11:29.672 INFO:teuthology.orchestra.run.smithi171.stdout:604 read 0x20d1c79 thru 0x20d8c6b (0x6ff3 bytes) 2024-09-15T00:11:29.672 INFO:teuthology.orchestra.run.smithi171.stdout:607 read 0x22398d thru 0x22cd51 (0x93c5 bytes) 2024-09-15T00:11:29.686 INFO:teuthology.orchestra.run.smithi171.stdout:608 write 0x8f3d8c8 thru 0x8f4794f (0xa088 bytes) 2024-09-15T00:11:29.701 INFO:teuthology.orchestra.run.smithi171.stdout:609 writesame 0x1159191 thru 0x115ceb1 data_size 0x1(0x3d21 bytes) 2024-09-15T00:11:29.714 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:29.711+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:29.718 INFO:teuthology.orchestra.run.smithi171.stdout:611 read 0x6e593fa thru 0x6e62b88 (0x978f bytes) 2024-09-15T00:11:29.726 INFO:teuthology.orchestra.run.smithi171.stdout:612 writesame 0xa3abbc1 thru 0xa3b1e33 data_size 0x1f(0x6273 bytes) 2024-09-15T00:11:29.727 INFO:teuthology.orchestra.run.smithi171.stdout:613 writesame 0x7191ecb thru 0x719ec24 data_size 0x23(0xcd5a bytes) 2024-09-15T00:11:29.735 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:29.735 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:29.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.735 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:3244 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:29.735 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.726943+0000", 2024-09-15T00:11:29.735 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0068070930000000002, 2024-09-15T00:11:29.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0068330559999999997, 2024-09-15T00:11:29.735 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.735 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.735 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-15T00:11:29.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:29.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:29.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:29.736 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 3244 2024-09-15T00:11:29.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.736 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.726943+0000", 2024-09-15T00:11:29.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.726943+0000", 2024-09-15T00:11:29.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.726944+0000", 2024-09-15T00:11:29.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6300000000000005e-07 2024-09-15T00:11:29.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.726947+0000", 2024-09-15T00:11:29.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8959999999999999e-06 2024-09-15T00:11:29.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.726954+0000", 2024-09-15T00:11:29.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3499999999999999e-06 2024-09-15T00:11:29.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.726960+0000", 2024-09-15T00:11:29.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4580000000000001e-06 2024-09-15T00:11:29.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.730498+0000", 2024-09-15T00:11:29.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0035385149999999999 2024-09-15T00:11:29.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.730549+0000", 2024-09-15T00:11:29.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0793e-05 2024-09-15T00:11:29.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:29.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.730757+0000", 2024-09-15T00:11:29.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000208044 2024-09-15T00:11:29.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:29.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733154+0000", 2024-09-15T00:11:29.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023972310000000001 2024-09-15T00:11:29.740 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.740 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.740 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.740 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:3245 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:29.740 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.726975+0000", 2024-09-15T00:11:29.741 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0067756910000000004, 2024-09-15T00:11:29.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0068595050000000001, 2024-09-15T00:11:29.741 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.741 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.741 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-15T00:11:29.741 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:29.741 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:29.741 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:29.741 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 3245 2024-09-15T00:11:29.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.741 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.726975+0000", 2024-09-15T00:11:29.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.726975+0000", 2024-09-15T00:11:29.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.726975+0000", 2024-09-15T00:11:29.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6000000000000004e-07 2024-09-15T00:11:29.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.726979+0000", 2024-09-15T00:11:29.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1939999999999998e-06 2024-09-15T00:11:29.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.726986+0000", 2024-09-15T00:11:29.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3150000000000003e-06 2024-09-15T00:11:29.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.726992+0000", 2024-09-15T00:11:29.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:11:29.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.731041+0000", 2024-09-15T00:11:29.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040497140000000003 2024-09-15T00:11:29.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.731091+0000", 2024-09-15T00:11:29.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9925000000000001e-05 2024-09-15T00:11:29.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:29.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.731291+0000", 2024-09-15T00:11:29.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019981 2024-09-15T00:11:29.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:29.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733278+0000", 2024-09-15T00:11:29.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019873 2024-09-15T00:11:29.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.745 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.745 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:3249 3.7 3:e5a887af:::rbd_data.1139b531b717.000000000000003f:head [set-alloc-hint object_size 2097152 write_size 2097152,write 114688~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:29.745 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.732112+0000", 2024-09-15T00:11:29.745 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.001638217, 2024-09-15T00:11:29.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018044389999999999, 2024-09-15T00:11:29.745 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.745 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.746 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-15T00:11:29.746 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:29.746 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:29.746 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:29.746 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 3249 2024-09-15T00:11:29.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.746 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.732112+0000", 2024-09-15T00:11:29.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.732112+0000", 2024-09-15T00:11:29.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.732113+0000", 2024-09-15T00:11:29.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1100000000000005e-07 2024-09-15T00:11:29.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.732122+0000", 2024-09-15T00:11:29.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5550000000000003e-06 2024-09-15T00:11:29.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.732130+0000", 2024-09-15T00:11:29.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4859999999999997e-06 2024-09-15T00:11:29.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.732135+0000", 2024-09-15T00:11:29.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7679999999999999e-06 2024-09-15T00:11:29.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.732177+0000", 2024-09-15T00:11:29.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1749999999999998e-05 2024-09-15T00:11:29.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.732224+0000", 2024-09-15T00:11:29.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7536e-05 2024-09-15T00:11:29.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:29.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.732305+0000", 2024-09-15T00:11:29.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0331e-05 2024-09-15T00:11:29.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.749 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.750 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:3250 3.8 3:16971325:::rbd_data.1139b531b717.0000000000000040:head [set-alloc-hint object_size 2097152 write_size 2097152,write 114688~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:29.750 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.733060+0000", 2024-09-15T00:11:29.750 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00069105100000000004, 2024-09-15T00:11:29.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00093563000000000001, 2024-09-15T00:11:29.750 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.750 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.750 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-15T00:11:29.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:29.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:29.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:29.750 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 3250 2024-09-15T00:11:29.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.750 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733060+0000", 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733060+0000", 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733060+0000", 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6899999999999996e-07 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733071+0000", 2024-09-15T00:11:29.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0427e-05 2024-09-15T00:11:29.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733082+0000", 2024-09-15T00:11:29.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0868e-05 2024-09-15T00:11:29.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733089+0000", 2024-09-15T00:11:29.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6599999999999995e-06 2024-09-15T00:11:29.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733152+0000", 2024-09-15T00:11:29.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2904000000000006e-05 2024-09-15T00:11:29.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733212+0000", 2024-09-15T00:11:29.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9932999999999997e-05 2024-09-15T00:11:29.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733297+0000", 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5141000000000004e-05 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:3251 3.7 3:e041e23a:::rbd_data.1139b531b717.0000000000000041:head [set-alloc-hint object_size 2097152 write_size 2097152,write 114688~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.733292+0000", 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00045826100000000001, 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00077795800000000003, 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for sub ops", 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:29.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:29.755 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:29.755 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 3251 2024-09-15T00:11:29.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.755 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:29.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:29.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733292+0000", 2024-09-15T00:11:29.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:29.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733292+0000", 2024-09-15T00:11:29.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:29.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:29.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733293+0000", 2024-09-15T00:11:29.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0599999999999999e-07 2024-09-15T00:11:29.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:29.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733301+0000", 2024-09-15T00:11:29.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8979999999999996e-06 2024-09-15T00:11:29.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:29.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733311+0000", 2024-09-15T00:11:29.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9000000000000001e-06 2024-09-15T00:11:29.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:29.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733319+0000", 2024-09-15T00:11:29.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0749999999999998e-06 2024-09-15T00:11:29.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:29.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733363+0000", 2024-09-15T00:11:29.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3835999999999998e-05 2024-09-15T00:11:29.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:29.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733436+0000", 2024-09-15T00:11:29.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2741e-05 2024-09-15T00:11:29.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:29.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:29.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:29.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.733579+0000", 2024-09-15T00:11:29.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014314199999999999 2024-09-15T00:11:29.758 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.758 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:29.758 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.758 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:29.758 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:11:29.758 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 5 2024-09-15T00:11:29.759 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:29.759 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:29.759 INFO:teuthology.orchestra.run.smithi184.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-15T00:11:29.759 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:29.760 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:29.760 INFO:teuthology.orchestra.run.smithi171.stdout:618 write 0x375a5c0 thru 0x3764a71 (0xa4b2 bytes) 2024-09-15T00:11:29.761 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:29.761 INFO:teuthology.orchestra.run.smithi171.stdout:619 trunc from 0xa3b1e34 to 0xe91ff13 2024-09-15T00:11:29.765 INFO:teuthology.orchestra.run.smithi171.stdout:620 trunc from 0xe91ff13 to 0xce5372b 2024-09-15T00:11:29.770 INFO:teuthology.orchestra.run.smithi171.stdout:621 write 0x93e9d8d thru 0x93edb33 (0x3da7 bytes) 2024-09-15T00:11:29.779 INFO:teuthology.orchestra.run.smithi171.stdout:622 read 0x6d2c79b thru 0x6d2c803 (0x69 bytes) 2024-09-15T00:11:29.779 INFO:teuthology.orchestra.run.smithi171.stdout:623 punch from 0x382923 to 0x38957c, (0x6c59 bytes) 2024-09-15T00:11:29.779 INFO:teuthology.orchestra.run.smithi171.stdout:624 writesame 0x7569a73 thru 0x756c70f data_size 0x2f(0x2c9d bytes) 2024-09-15T00:11:29.785 INFO:teuthology.orchestra.run.smithi171.stdout:628 trunc from 0xce5372b to 0x89c39a2 2024-09-15T00:11:29.798 INFO:teuthology.orchestra.run.smithi171.stdout:630 punch from 0xb123ff to 0xb1b486, (0x9087 bytes) 2024-09-15T00:11:29.804 INFO:teuthology.orchestra.run.smithi171.stdout:632 write 0x8da2b4d thru 0x8da7eef (0x53a3 bytes) 2024-09-15T00:11:29.808 INFO:teuthology.orchestra.run.smithi171.stdout:633 write 0x69bcce5 thru 0x69c3c1e (0x6f3a bytes) 2024-09-15T00:11:29.814 INFO:teuthology.orchestra.run.smithi171.stdout:634 read 0x5e6103c thru 0x5e67feb (0x6fb0 bytes) 2024-09-15T00:11:29.814 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:29.814 INFO:teuthology.orchestra.run.smithi171.stdout:638 trunc from 0x8da7ef0 to 0x8ed2f19 2024-09-15T00:11:29.815 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:29.812+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:29.819 INFO:teuthology.orchestra.run.smithi171.stdout:639 read 0x1bd2097 thru 0x1bdaf30 (0x8e9a bytes) 2024-09-15T00:11:29.819 INFO:teuthology.orchestra.run.smithi171.stdout:642 punch from 0x4486e33 to 0x448c728, (0x58f5 bytes) 2024-09-15T00:11:29.820 INFO:teuthology.orchestra.run.smithi171.stdout:643 punch from 0x6bb4d96 to 0x6bc1484, (0xc6ee bytes) 2024-09-15T00:11:29.820 INFO:teuthology.orchestra.run.smithi171.stdout:644 read 0xb7e702 thru 0xb869bf (0x82be bytes) 2024-09-15T00:11:29.820 INFO:teuthology.orchestra.run.smithi171.stdout:645 read 0x77ebf07 thru 0x77ee859 (0x2953 bytes) 2024-09-15T00:11:29.827 INFO:teuthology.orchestra.run.smithi171.stdout:646 trunc from 0x8ed2f19 to 0xd54f978 2024-09-15T00:11:29.831 INFO:teuthology.orchestra.run.smithi171.stdout:648 punch from 0x9cab799 to 0x9cb57b0, (0xa017 bytes) 2024-09-15T00:11:29.832 INFO:teuthology.orchestra.run.smithi171.stdout:649 write 0x440cf85 thru 0x441517c (0x81f8 bytes) 2024-09-15T00:11:29.858 INFO:teuthology.orchestra.run.smithi171.stdout:650 trunc from 0xd54f978 to 0x750715 2024-09-15T00:11:29.893 INFO:teuthology.orchestra.run.smithi171.stdout:651 write 0x8b08cae thru 0x8b159eb (0xcd3e bytes) 2024-09-15T00:11:29.900 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:29.900 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:29.900 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:29.900 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:29.900 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:29.907 INFO:teuthology.orchestra.run.smithi171.stdout:653 read 0x40c19be thru 0x40c806c (0x66af bytes) 2024-09-15T00:11:29.910 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:29.912 INFO:teuthology.orchestra.run.smithi171.stdout:656 trunc from 0x8b159ec to 0xc77a49b 2024-09-15T00:11:29.915 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:29.912+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:29.917 INFO:teuthology.orchestra.run.smithi171.stdout:657 write 0x7d7faf thru 0x7df824 (0x7876 bytes) 2024-09-15T00:11:29.921 INFO:teuthology.orchestra.run.smithi171.stdout:659 punch from 0xbbaecea to 0xbbb8bdc, (0x9ef2 bytes) 2024-09-15T00:11:29.927 INFO:teuthology.orchestra.run.smithi171.stdout:660 trunc from 0xc77a49b to 0x7b446a2 2024-09-15T00:11:29.947 INFO:teuthology.orchestra.run.smithi171.stdout:661 writesame 0xc043b2c thru 0xc0483d4 data_size 0x13(0x48a9 bytes) 2024-09-15T00:11:29.947 INFO:teuthology.orchestra.run.smithi171.stdout:662 write 0x62029b5 thru 0x62112e3 (0xe92f bytes) 2024-09-15T00:11:29.967 INFO:teuthology.orchestra.run.smithi171.stdout:663 trunc from 0xc0483d5 to 0x62e7dd2 2024-09-15T00:11:29.992 INFO:teuthology.orchestra.run.smithi171.stdout:664 write 0x7ada2a9 thru 0x7adfe48 (0x5ba0 bytes) 2024-09-15T00:11:30.005 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is set 2024-09-15T00:11:30.009 INFO:teuthology.orchestra.run.smithi171.stdout:665 read 0x3c2fd6f thru 0x3c3c456 (0xc6e8 bytes) 2024-09-15T00:11:30.015 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:30.013+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:30.017 INFO:teuthology.orchestra.run.smithi171.stdout:666 writesame 0x8a1156b thru 0x8a17e20 data_size 0xd(0x68b6 bytes) 2024-09-15T00:11:30.018 INFO:teuthology.orchestra.run.smithi171.stdout:667 read 0x35493f7 thru 0x354ea48 (0x5652 bytes) 2024-09-15T00:11:30.028 INFO:teuthology.orchestra.run.smithi171.stdout:668 write 0xcaa796b thru 0xcaaf32e (0x79c4 bytes) 2024-09-15T00:11:30.038 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1240 3.2 3:4001636a:::rbd_data.11397bff333d.0000000000000007:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.045955+0000", 2024-09-15T00:11:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.988574973, 2024-09-15T00:11:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017913958000000001, 2024-09-15T00:11:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.038 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1240 2024-09-15T00:11:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045955+0000", 2024-09-15T00:11:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045955+0000", 2024-09-15T00:11:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045957+0000", 2024-09-15T00:11:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6640000000000001e-06 2024-09-15T00:11:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045964+0000", 2024-09-15T00:11:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3629999999999996e-06 2024-09-15T00:11:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045979+0000", 2024-09-15T00:11:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4436e-05 2024-09-15T00:11:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045989+0000", 2024-09-15T00:11:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0307e-05 2024-09-15T00:11:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.046042+0000", 2024-09-15T00:11:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3134999999999999e-05 2024-09-15T00:11:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.046106+0000", 2024-09-15T00:11:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3807999999999996e-05 2024-09-15T00:11:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:30.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.047043+0000", 2024-09-15T00:11:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000937165 2024-09-15T00:11:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063721+0000", 2024-09-15T00:11:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016678193000000001 2024-09-15T00:11:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063743+0000", 2024-09-15T00:11:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1163999999999999e-05 2024-09-15T00:11:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063810+0000", 2024-09-15T00:11:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7317000000000006e-05 2024-09-15T00:11:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063869+0000", 2024-09-15T00:11:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9406e-05 2024-09-15T00:11:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:1240 3.2 e30/23 ondisk, result = 0)", 2024-09-15T00:11:30.044 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.050169+0000", 2024-09-15T00:11:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.9843617309999999, 2024-09-15T00:11:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013562852, 2024-09-15T00:11:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.050169+0000", 2024-09-15T00:11:30.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.050169+0000", 2024-09-15T00:11:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.050169+0000", 2024-09-15T00:11:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8999999999999995e-07 2024-09-15T00:11:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.050173+0000", 2024-09-15T00:11:30.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8369999999999999e-06 2024-09-15T00:11:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.050183+0000", 2024-09-15T00:11:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2890000000000003e-06 2024-09-15T00:11:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.050191+0000", 2024-09-15T00:11:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0439999999999996e-06 2024-09-15T00:11:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063701+0000", 2024-09-15T00:11:30.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013510810999999999 2024-09-15T00:11:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063713+0000", 2024-09-15T00:11:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2012e-05 2024-09-15T00:11:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063731+0000", 2024-09-15T00:11:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7969e-05 2024-09-15T00:11:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:2973 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.427444+0000", 2024-09-15T00:11:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.60708620300000005, 2024-09-15T00:11:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017040047999999999, 2024-09-15T00:11:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 2973 2024-09-15T00:11:30.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.427444+0000", 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.427444+0000", 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.427445+0000", 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4399999999999998e-07 2024-09-15T00:11:30.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.427450+0000", 2024-09-15T00:11:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0100000000000003e-06 2024-09-15T00:11:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.427458+0000", 2024-09-15T00:11:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0290000000000005e-06 2024-09-15T00:11:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.427469+0000", 2024-09-15T00:11:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0936e-05 2024-09-15T00:11:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.442566+0000", 2024-09-15T00:11:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015097202000000001 2024-09-15T00:11:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.442610+0000", 2024-09-15T00:11:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4274999999999999e-05 2024-09-15T00:11:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.442837+0000", 2024-09-15T00:11:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022660000000000001 2024-09-15T00:11:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.444393+0000", 2024-09-15T00:11:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001555931 2024-09-15T00:11:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.444404+0000", 2024-09-15T00:11:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.064e-05 2024-09-15T00:11:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.444448+0000", 2024-09-15T00:11:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4116999999999998e-05 2024-09-15T00:11:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.444484+0000", 2024-09-15T00:11:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6363999999999999e-05 2024-09-15T00:11:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:2974 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.429244+0000", 2024-09-15T00:11:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.60528637500000004, 2024-09-15T00:11:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017046233000000001, 2024-09-15T00:11:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 2974 2024-09-15T00:11:30.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.429244+0000", 2024-09-15T00:11:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.429244+0000", 2024-09-15T00:11:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.429245+0000", 2024-09-15T00:11:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0079999999999999e-06 2024-09-15T00:11:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.429253+0000", 2024-09-15T00:11:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1249999999999993e-06 2024-09-15T00:11:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.429267+0000", 2024-09-15T00:11:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4124000000000001e-05 2024-09-15T00:11:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.429275+0000", 2024-09-15T00:11:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0299999999999994e-06 2024-09-15T00:11:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.443095+0000", 2024-09-15T00:11:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013819829 2024-09-15T00:11:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.443145+0000", 2024-09-15T00:11:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0417000000000002e-05 2024-09-15T00:11:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.443435+0000", 2024-09-15T00:11:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00028970500000000002 2024-09-15T00:11:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.444541+0000", 2024-09-15T00:11:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011055500000000001 2024-09-15T00:11:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.446234+0000", 2024-09-15T00:11:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001692966 2024-09-15T00:11:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.446260+0000", 2024-09-15T00:11:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6628999999999999e-05 2024-09-15T00:11:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.446290+0000", 2024-09-15T00:11:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9850000000000001e-05 2024-09-15T00:11:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.061 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:2975 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.430726+0000", 2024-09-15T00:11:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.60380391600000005, 2024-09-15T00:11:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017566919, 2024-09-15T00:11:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 2975 2024-09-15T00:11:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.430726+0000", 2024-09-15T00:11:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.430726+0000", 2024-09-15T00:11:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.430727+0000", 2024-09-15T00:11:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9000000000000005e-07 2024-09-15T00:11:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.430735+0000", 2024-09-15T00:11:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6190000000000002e-06 2024-09-15T00:11:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.430747+0000", 2024-09-15T00:11:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2140999999999999e-05 2024-09-15T00:11:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.430755+0000", 2024-09-15T00:11:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4030000000000007e-06 2024-09-15T00:11:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.443711+0000", 2024-09-15T00:11:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012955599 2024-09-15T00:11:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.443764+0000", 2024-09-15T00:11:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3090000000000002e-05 2024-09-15T00:11:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.443992+0000", 2024-09-15T00:11:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022779199999999999 2024-09-15T00:11:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.446299+0000", 2024-09-15T00:11:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023075130000000002 2024-09-15T00:11:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.448194+0000", 2024-09-15T00:11:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018949839999999999 2024-09-15T00:11:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.448250+0000", 2024-09-15T00:11:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5132e-05 2024-09-15T00:11:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.448293+0000", 2024-09-15T00:11:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3655999999999997e-05 2024-09-15T00:11:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.067 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:2976 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.432292+0000", 2024-09-15T00:11:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.60223870800000001, 2024-09-15T00:11:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017183189000000001, 2024-09-15T00:11:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 2976 2024-09-15T00:11:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.432292+0000", 2024-09-15T00:11:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.432292+0000", 2024-09-15T00:11:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.432293+0000", 2024-09-15T00:11:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9999999999999995e-07 2024-09-15T00:11:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.432298+0000", 2024-09-15T00:11:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2469999999999999e-06 2024-09-15T00:11:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.432310+0000", 2024-09-15T00:11:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1739999999999999e-05 2024-09-15T00:11:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.432319+0000", 2024-09-15T00:11:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4860000000000003e-06 2024-09-15T00:11:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.444578+0000", 2024-09-15T00:11:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012258903 2024-09-15T00:11:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.444629+0000", 2024-09-15T00:11:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1495000000000001e-05 2024-09-15T00:11:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.444856+0000", 2024-09-15T00:11:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000226529 2024-09-15T00:11:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.448142+0000", 2024-09-15T00:11:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0032857429999999998 2024-09-15T00:11:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.449381+0000", 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012391360000000001 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.449434+0000", 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3378999999999998e-05 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.449475+0000", 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0531000000000002e-05 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:2977 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.433850+0000", 2024-09-15T00:11:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.60067975299999998, 2024-09-15T00:11:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015744360999999998, 2024-09-15T00:11:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 2977 2024-09-15T00:11:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.433850+0000", 2024-09-15T00:11:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.433850+0000", 2024-09-15T00:11:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.433852+0000", 2024-09-15T00:11:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.048e-06 2024-09-15T00:11:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.433857+0000", 2024-09-15T00:11:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.13e-06 2024-09-15T00:11:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.433870+0000", 2024-09-15T00:11:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3158e-05 2024-09-15T00:11:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.433877+0000", 2024-09-15T00:11:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.435e-06 2024-09-15T00:11:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.445176+0000", 2024-09-15T00:11:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011298842999999999 2024-09-15T00:11:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.445254+0000", 2024-09-15T00:11:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8239999999999996e-05 2024-09-15T00:11:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.445441+0000", 2024-09-15T00:11:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000187076 2024-09-15T00:11:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.448884+0000", 2024-09-15T00:11:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034426629999999999 2024-09-15T00:11:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.449507+0000", 2024-09-15T00:11:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000623154 2024-09-15T00:11:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.449557+0000", 2024-09-15T00:11:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9308000000000003e-05 2024-09-15T00:11:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.449595+0000", 2024-09-15T00:11:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8306000000000002e-05 2024-09-15T00:11:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:2978 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.436266+0000", 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.598264146, 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01347112, 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 2978 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.436266+0000", 2024-09-15T00:11:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.436266+0000", 2024-09-15T00:11:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.436267+0000", 2024-09-15T00:11:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3179999999999999e-06 2024-09-15T00:11:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.436273+0000", 2024-09-15T00:11:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-15T00:11:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.436286+0000", 2024-09-15T00:11:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.31e-05 2024-09-15T00:11:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.436294+0000", 2024-09-15T00:11:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5140000000000004e-06 2024-09-15T00:11:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.445749+0000", 2024-09-15T00:11:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094554029999999994 2024-09-15T00:11:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.445808+0000", 2024-09-15T00:11:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9076999999999997e-05 2024-09-15T00:11:30.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.446014+0000", 2024-09-15T00:11:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020557799999999999 2024-09-15T00:11:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.448889+0000", 2024-09-15T00:11:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028750389999999998 2024-09-15T00:11:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.449645+0000", 2024-09-15T00:11:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00075636300000000002 2024-09-15T00:11:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.449700+0000", 2024-09-15T00:11:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4731999999999997e-05 2024-09-15T00:11:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.449737+0000", 2024-09-15T00:11:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7496000000000001e-05 2024-09-15T00:11:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:2981 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.439220+0000", 2024-09-15T00:11:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.59531032900000003, 2024-09-15T00:11:30.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012993394, 2024-09-15T00:11:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 2981 2024-09-15T00:11:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.439220+0000", 2024-09-15T00:11:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.439220+0000", 2024-09-15T00:11:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.439221+0000", 2024-09-15T00:11:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6099999999999999e-07 2024-09-15T00:11:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.439226+0000", 2024-09-15T00:11:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1850000000000003e-06 2024-09-15T00:11:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.439234+0000", 2024-09-15T00:11:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9929999999999995e-06 2024-09-15T00:11:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.439241+0000", 2024-09-15T00:11:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5819999999999998e-06 2024-09-15T00:11:30.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.446344+0000", 2024-09-15T00:11:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0071035860000000003 2024-09-15T00:11:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.446424+0000", 2024-09-15T00:11:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0141000000000004e-05 2024-09-15T00:11:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.446751+0000", 2024-09-15T00:11:30.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000327101 2024-09-15T00:11:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.448891+0000", 2024-09-15T00:11:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002139644 2024-09-15T00:11:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.452142+0000", 2024-09-15T00:11:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0032508849999999998 2024-09-15T00:11:30.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.452187+0000", 2024-09-15T00:11:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5084999999999999e-05 2024-09-15T00:11:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.452213+0000", 2024-09-15T00:11:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6231e-05 2024-09-15T00:11:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.096 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:2982 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.440716+0000", 2024-09-15T00:11:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.59381421999999995, 2024-09-15T00:11:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011616052999999999, 2024-09-15T00:11:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.097 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 2982 2024-09-15T00:11:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.440716+0000", 2024-09-15T00:11:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.440716+0000", 2024-09-15T00:11:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.440717+0000", 2024-09-15T00:11:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-06 2024-09-15T00:11:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.440723+0000", 2024-09-15T00:11:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:11:30.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.440735+0000", 2024-09-15T00:11:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2311e-05 2024-09-15T00:11:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.440743+0000", 2024-09-15T00:11:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3799999999999996e-06 2024-09-15T00:11:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.447261+0000", 2024-09-15T00:11:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0065184580000000004 2024-09-15T00:11:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.447343+0000", 2024-09-15T00:11:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1681999999999995e-05 2024-09-15T00:11:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.447857+0000", 2024-09-15T00:11:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000514398 2024-09-15T00:11:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.448895+0000", 2024-09-15T00:11:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001037514 2024-09-15T00:11:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.452260+0000", 2024-09-15T00:11:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033654240000000001 2024-09-15T00:11:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.452300+0000", 2024-09-15T00:11:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9478999999999999e-05 2024-09-15T00:11:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.452332+0000", 2024-09-15T00:11:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2425999999999998e-05 2024-09-15T00:11:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.111 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:2983 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.111 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.442108+0000", 2024-09-15T00:11:30.111 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.59242176199999996, 2024-09-15T00:11:30.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010327489, 2024-09-15T00:11:30.111 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.112 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.112 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.112 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.112 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.112 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.113 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 2983 2024-09-15T00:11:30.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.113 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.442108+0000", 2024-09-15T00:11:30.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.442108+0000", 2024-09-15T00:11:30.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.442109+0000", 2024-09-15T00:11:30.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9699999999999994e-07 2024-09-15T00:11:30.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.442115+0000", 2024-09-15T00:11:30.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2939999999999996e-06 2024-09-15T00:11:30.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.442129+0000", 2024-09-15T00:11:30.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4089e-05 2024-09-15T00:11:30.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.442137+0000", 2024-09-15T00:11:30.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9480000000000008e-06 2024-09-15T00:11:30.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.448317+0000", 2024-09-15T00:11:30.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061804770000000002 2024-09-15T00:11:30.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.448381+0000", 2024-09-15T00:11:30.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3681000000000002e-05 2024-09-15T00:11:30.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.448592+0000", 2024-09-15T00:11:30.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000211373 2024-09-15T00:11:30.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.449157+0000", 2024-09-15T00:11:30.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00056419300000000005 2024-09-15T00:11:30.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.452369+0000", 2024-09-15T00:11:30.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0032125550000000002 2024-09-15T00:11:30.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.452401+0000", 2024-09-15T00:11:30.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2082000000000002e-05 2024-09-15T00:11:30.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.452436+0000", 2024-09-15T00:11:30.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4799999999999999e-05 2024-09-15T00:11:30.120 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.120 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.120 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.120 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:3037 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.120 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.495857+0000", 2024-09-15T00:11:30.120 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.53867347300000001, 2024-09-15T00:11:30.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.009743191, 2024-09-15T00:11:30.120 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.120 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.121 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.121 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.121 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.121 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.121 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 3037 2024-09-15T00:11:30.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.121 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495857+0000", 2024-09-15T00:11:30.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495857+0000", 2024-09-15T00:11:30.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495857+0000", 2024-09-15T00:11:30.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7300000000000002e-07 2024-09-15T00:11:30.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495860+0000", 2024-09-15T00:11:30.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4569999999999999e-06 2024-09-15T00:11:30.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495865+0000", 2024-09-15T00:11:30.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9429999999999999e-06 2024-09-15T00:11:30.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495868+0000", 2024-09-15T00:11:30.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9900000000000002e-06 2024-09-15T00:11:30.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.496980+0000", 2024-09-15T00:11:30.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011121219999999999 2024-09-15T00:11:30.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.497071+0000", 2024-09-15T00:11:30.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1096999999999999e-05 2024-09-15T00:11:30.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.497483+0000", 2024-09-15T00:11:30.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00041239899999999998 2024-09-15T00:11:30.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.499970+0000", 2024-09-15T00:11:30.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024863709999999998 2024-09-15T00:11:30.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.505508+0000", 2024-09-15T00:11:30.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055387689999999998 2024-09-15T00:11:30.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.505557+0000", 2024-09-15T00:11:30.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8217999999999998e-05 2024-09-15T00:11:30.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.505600+0000", 2024-09-15T00:11:30.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3451999999999997e-05 2024-09-15T00:11:30.126 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.126 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.126 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.126 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:3038 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.126 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.495872+0000", 2024-09-15T00:11:30.126 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.53865861400000004, 2024-09-15T00:11:30.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0099062899999999999, 2024-09-15T00:11:30.126 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.127 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.127 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.127 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 3038 2024-09-15T00:11:30.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.127 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495872+0000", 2024-09-15T00:11:30.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495872+0000", 2024-09-15T00:11:30.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495872+0000", 2024-09-15T00:11:30.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1699999999999999e-07 2024-09-15T00:11:30.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495874+0000", 2024-09-15T00:11:30.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.153e-06 2024-09-15T00:11:30.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495877+0000", 2024-09-15T00:11:30.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2940000000000001e-06 2024-09-15T00:11:30.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495880+0000", 2024-09-15T00:11:30.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.452e-06 2024-09-15T00:11:30.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.497896+0000", 2024-09-15T00:11:30.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020159039999999998 2024-09-15T00:11:30.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.497967+0000", 2024-09-15T00:11:30.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1452000000000006e-05 2024-09-15T00:11:30.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.498332+0000", 2024-09-15T00:11:30.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00036526100000000003 2024-09-15T00:11:30.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.500718+0000", 2024-09-15T00:11:30.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023850389999999998 2024-09-15T00:11:30.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.505665+0000", 2024-09-15T00:11:30.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049474389999999997 2024-09-15T00:11:30.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.505734+0000", 2024-09-15T00:11:30.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9543999999999997e-05 2024-09-15T00:11:30.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.505778+0000", 2024-09-15T00:11:30.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3435e-05 2024-09-15T00:11:30.132 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.132 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.132 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.132 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:3039 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.132 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.495884+0000", 2024-09-15T00:11:30.132 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.53864674700000004, 2024-09-15T00:11:30.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010023007, 2024-09-15T00:11:30.132 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.132 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.132 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.132 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.133 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.133 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.133 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 3039 2024-09-15T00:11:30.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.133 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495884+0000", 2024-09-15T00:11:30.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495884+0000", 2024-09-15T00:11:30.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495884+0000", 2024-09-15T00:11:30.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1299999999999999e-07 2024-09-15T00:11:30.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495885+0000", 2024-09-15T00:11:30.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.621e-06 2024-09-15T00:11:30.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495889+0000", 2024-09-15T00:11:30.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1839999999999999e-06 2024-09-15T00:11:30.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495891+0000", 2024-09-15T00:11:30.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:11:30.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.498891+0000", 2024-09-15T00:11:30.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030001020000000002 2024-09-15T00:11:30.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.498971+0000", 2024-09-15T00:11:30.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9949000000000003e-05 2024-09-15T00:11:30.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.499159+0000", 2024-09-15T00:11:30.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018784800000000001 2024-09-15T00:11:30.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.500723+0000", 2024-09-15T00:11:30.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015640929999999999 2024-09-15T00:11:30.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.505827+0000", 2024-09-15T00:11:30.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051035430000000003 2024-09-15T00:11:30.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.505870+0000", 2024-09-15T00:11:30.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3862000000000001e-05 2024-09-15T00:11:30.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.505907+0000", 2024-09-15T00:11:30.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6028000000000001e-05 2024-09-15T00:11:30.137 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.137 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.137 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.138 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:3040 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.138 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.495894+0000", 2024-09-15T00:11:30.138 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.53863628399999997, 2024-09-15T00:11:30.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010111561999999999, 2024-09-15T00:11:30.138 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.138 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.138 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.138 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 3040 2024-09-15T00:11:30.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495894+0000", 2024-09-15T00:11:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495894+0000", 2024-09-15T00:11:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495894+0000", 2024-09-15T00:11:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-15T00:11:30.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495896+0000", 2024-09-15T00:11:30.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.486e-06 2024-09-15T00:11:30.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495899+0000", 2024-09-15T00:11:30.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1829999999999998e-06 2024-09-15T00:11:30.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495901+0000", 2024-09-15T00:11:30.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6199999999999999e-06 2024-09-15T00:11:30.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.499444+0000", 2024-09-15T00:11:30.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0035421530000000001 2024-09-15T00:11:30.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.499491+0000", 2024-09-15T00:11:30.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7083999999999998e-05 2024-09-15T00:11:30.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.499690+0000", 2024-09-15T00:11:30.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000198915 2024-09-15T00:11:30.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.501560+0000", 2024-09-15T00:11:30.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018703210000000001 2024-09-15T00:11:30.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.505945+0000", 2024-09-15T00:11:30.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043848100000000003 2024-09-15T00:11:30.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.505972+0000", 2024-09-15T00:11:30.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7123000000000001e-05 2024-09-15T00:11:30.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.506006+0000", 2024-09-15T00:11:30.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3658999999999997e-05 2024-09-15T00:11:30.143 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.143 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.144 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:3041 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.144 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.495911+0000", 2024-09-15T00:11:30.144 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.538619026, 2024-09-15T00:11:30.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010203853000000001, 2024-09-15T00:11:30.144 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.144 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.144 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.144 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.144 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.144 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.144 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 3041 2024-09-15T00:11:30.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.145 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495911+0000", 2024-09-15T00:11:30.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495911+0000", 2024-09-15T00:11:30.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495912+0000", 2024-09-15T00:11:30.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9900000000000002e-07 2024-09-15T00:11:30.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495916+0000", 2024-09-15T00:11:30.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4510000000000002e-06 2024-09-15T00:11:30.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495920+0000", 2024-09-15T00:11:30.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9770000000000002e-06 2024-09-15T00:11:30.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495923+0000", 2024-09-15T00:11:30.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0649999999999999e-06 2024-09-15T00:11:30.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.500006+0000", 2024-09-15T00:11:30.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040829730000000002 2024-09-15T00:11:30.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.500077+0000", 2024-09-15T00:11:30.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0864000000000005e-05 2024-09-15T00:11:30.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.500357+0000", 2024-09-15T00:11:30.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00028055499999999999 2024-09-15T00:11:30.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.501569+0000", 2024-09-15T00:11:30.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001211869 2024-09-15T00:11:30.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.506042+0000", 2024-09-15T00:11:30.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004472551 2024-09-15T00:11:30.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.506080+0000", 2024-09-15T00:11:30.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8516000000000002e-05 2024-09-15T00:11:30.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.506115+0000", 2024-09-15T00:11:30.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4733000000000001e-05 2024-09-15T00:11:30.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.149 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.150 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:3042 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.150 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.495927+0000", 2024-09-15T00:11:30.150 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.53860333999999999, 2024-09-15T00:11:30.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010536079, 2024-09-15T00:11:30.150 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.150 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.150 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.150 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.150 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.150 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.150 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 3042 2024-09-15T00:11:30.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.151 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495927+0000", 2024-09-15T00:11:30.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495927+0000", 2024-09-15T00:11:30.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495927+0000", 2024-09-15T00:11:30.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-15T00:11:30.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495929+0000", 2024-09-15T00:11:30.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7799999999999999e-06 2024-09-15T00:11:30.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495932+0000", 2024-09-15T00:11:30.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0299999999999998e-06 2024-09-15T00:11:30.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495934+0000", 2024-09-15T00:11:30.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4439999999999998e-06 2024-09-15T00:11:30.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.500755+0000", 2024-09-15T00:11:30.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048207149999999997 2024-09-15T00:11:30.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.500806+0000", 2024-09-15T00:11:30.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.083e-05 2024-09-15T00:11:30.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.501171+0000", 2024-09-15T00:11:30.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00036550200000000003 2024-09-15T00:11:30.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.502214+0000", 2024-09-15T00:11:30.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001042231 2024-09-15T00:11:30.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.506374+0000", 2024-09-15T00:11:30.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041599610000000002 2024-09-15T00:11:30.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.506424+0000", 2024-09-15T00:11:30.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0016999999999999e-05 2024-09-15T00:11:30.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.506463+0000", 2024-09-15T00:11:30.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9347000000000001e-05 2024-09-15T00:11:30.155 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.155 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.155 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.156 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:3043 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.156 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.495937+0000", 2024-09-15T00:11:30.156 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.53859301800000003, 2024-09-15T00:11:30.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010675179, 2024-09-15T00:11:30.156 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.156 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.156 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.156 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.156 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.156 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.156 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 3043 2024-09-15T00:11:30.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.156 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495937+0000", 2024-09-15T00:11:30.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495937+0000", 2024-09-15T00:11:30.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495937+0000", 2024-09-15T00:11:30.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1299999999999999e-07 2024-09-15T00:11:30.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495939+0000", 2024-09-15T00:11:30.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:11:30.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495942+0000", 2024-09-15T00:11:30.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8760000000000001e-06 2024-09-15T00:11:30.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.495944+0000", 2024-09-15T00:11:30.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5330000000000001e-06 2024-09-15T00:11:30.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.501597+0000", 2024-09-15T00:11:30.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056523149999999998 2024-09-15T00:11:30.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.501679+0000", 2024-09-15T00:11:30.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2089999999999995e-05 2024-09-15T00:11:30.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.501945+0000", 2024-09-15T00:11:30.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026595299999999998 2024-09-15T00:11:30.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.502433+0000", 2024-09-15T00:11:30.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00048801299999999998 2024-09-15T00:11:30.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.506504+0000", 2024-09-15T00:11:30.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004071262 2024-09-15T00:11:30.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.506573+0000", 2024-09-15T00:11:30.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8484000000000004e-05 2024-09-15T00:11:30.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.506612+0000", 2024-09-15T00:11:30.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9820999999999997e-05 2024-09-15T00:11:30.161 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.161 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.161 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.161 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:3506 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.161 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.946568+0000", 2024-09-15T00:11:30.161 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.087962234, 2024-09-15T00:11:30.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010276761000000001, 2024-09-15T00:11:30.162 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.162 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.162 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.162 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.162 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.162 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.162 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 3506 2024-09-15T00:11:30.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.162 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.946568+0000", 2024-09-15T00:11:30.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.946568+0000", 2024-09-15T00:11:30.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.946568+0000", 2024-09-15T00:11:30.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8299999999999997e-07 2024-09-15T00:11:30.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.946571+0000", 2024-09-15T00:11:30.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8550000000000002e-06 2024-09-15T00:11:30.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.946580+0000", 2024-09-15T00:11:30.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4200000000000007e-06 2024-09-15T00:11:30.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.946589+0000", 2024-09-15T00:11:30.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3899999999999999e-06 2024-09-15T00:11:30.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.950486+0000", 2024-09-15T00:11:30.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003897218 2024-09-15T00:11:30.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.950568+0000", 2024-09-15T00:11:30.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2071999999999996e-05 2024-09-15T00:11:30.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.950941+0000", 2024-09-15T00:11:30.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00037279300000000001 2024-09-15T00:11:30.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.955872+0000", 2024-09-15T00:11:30.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004930912 2024-09-15T00:11:30.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.956736+0000", 2024-09-15T00:11:30.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00086403000000000001 2024-09-15T00:11:30.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.956796+0000", 2024-09-15T00:11:30.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0263999999999998e-05 2024-09-15T00:11:30.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.956845+0000", 2024-09-15T00:11:30.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8324e-05 2024-09-15T00:11:30.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.167 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.167 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:3507 3.1e 3:792e48de:::rbd_object_map.1139b531b717:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.167 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.946599+0000", 2024-09-15T00:11:30.167 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.087930777000000002, 2024-09-15T00:11:30.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010367711, 2024-09-15T00:11:30.167 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:30.167 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:30.167 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.167 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:30.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:30.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.168 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 3507 2024-09-15T00:11:30.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.168 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:30.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:30.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.946599+0000", 2024-09-15T00:11:30.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:30.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.946599+0000", 2024-09-15T00:11:30.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:30.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:30.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.946600+0000", 2024-09-15T00:11:30.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5400000000000001e-07 2024-09-15T00:11:30.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:30.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.946603+0000", 2024-09-15T00:11:30.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.072e-06 2024-09-15T00:11:30.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:30.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.946610+0000", 2024-09-15T00:11:30.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.289e-06 2024-09-15T00:11:30.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.946615+0000", 2024-09-15T00:11:30.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8849999999999998e-06 2024-09-15T00:11:30.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:30.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.951347+0000", 2024-09-15T00:11:30.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047314000000000002 2024-09-15T00:11:30.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:30.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.951396+0000", 2024-09-15T00:11:30.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9048000000000002e-05 2024-09-15T00:11:30.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.951595+0000", 2024-09-15T00:11:30.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019946499999999999 2024-09-15T00:11:30.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.955912+0000", 2024-09-15T00:11:30.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043171529999999998 2024-09-15T00:11:30.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:30.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.956856+0000", 2024-09-15T00:11:30.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00094370200000000004 2024-09-15T00:11:30.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:30.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.956907+0000", 2024-09-15T00:11:30.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1159999999999998e-05 2024-09-15T00:11:30.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:30.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:30.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:30.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.956967+0000", 2024-09-15T00:11:30.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9982999999999998e-05 2024-09-15T00:11:30.172 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.172 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.173 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.173 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:30.173 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:30.173 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:30.174 INFO:teuthology.orchestra.run.smithi171.stdout:669 writesame 0xda75849 thru 0xda7643f data_size 0x1(0xbf7 bytes) 2024-09-15T00:11:30.174 INFO:teuthology.orchestra.run.smithi171.stdout:672 punch from 0xac5ae42 to 0xac64890, (0x9a4e bytes) 2024-09-15T00:11:30.174 INFO:teuthology.orchestra.run.smithi171.stdout:674 writesame 0x63e02f2 thru 0x63ed79a data_size 0x45(0xd4a9 bytes) 2024-09-15T00:11:30.175 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:30.176 INFO:teuthology.orchestra.run.smithi171.stdout:678 punch from 0xc27fd6b to 0xc28f4ac, (0xf741 bytes) 2024-09-15T00:11:30.176 INFO:teuthology.orchestra.run.smithi171.stdout:679 read 0x7fc2b26 thru 0x7fc6fc8 (0x44a3 bytes) 2024-09-15T00:11:30.176 INFO:teuthology.orchestra.run.smithi171.stdout:680 write 0x682d0f1 thru 0x682f8bd (0x27cd bytes) 2024-09-15T00:11:30.176 INFO:teuthology.orchestra.run.smithi171.stdout:681 write 0x5d7e309 thru 0x5d7e6a1 (0x399 bytes) 2024-09-15T00:11:30.176 INFO:teuthology.orchestra.run.smithi171.stdout:682 clone 5 order 23 su 4194304 sc 6 2024-09-15T00:11:30.177 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:30.171+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:30.275 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:30.273+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:30.284 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:30.284 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:30.284 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:30.284 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:30.293 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:30.377 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:30.375+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:30.404 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:30.478 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:30.475+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:30.510 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:463 3.9 3:942069cf:::rbd_header.11397bff333d:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.020178+0000", 2024-09-15T00:11:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.4870050319999999, 2024-09-15T00:11:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005152022, 2024-09-15T00:11:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 463 2024-09-15T00:11:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020178+0000", 2024-09-15T00:11:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020178+0000", 2024-09-15T00:11:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020179+0000", 2024-09-15T00:11:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.733e-06 2024-09-15T00:11:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020189+0000", 2024-09-15T00:11:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2529999999999993e-06 2024-09-15T00:11:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020214+0000", 2024-09-15T00:11:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5599999999999999e-05 2024-09-15T00:11:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020234+0000", 2024-09-15T00:11:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9755999999999999e-05 2024-09-15T00:11:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020351+0000", 2024-09-15T00:11:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000116725 2024-09-15T00:11:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020522+0000", 2024-09-15T00:11:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000171807 2024-09-15T00:11:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.020674+0000", 2024-09-15T00:11:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015147000000000001 2024-09-15T00:11:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.021195+0000", 2024-09-15T00:11:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052059600000000001 2024-09-15T00:11:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025172+0000", 2024-09-15T00:11:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039778729999999998 2024-09-15T00:11:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025259+0000", 2024-09-15T00:11:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6551000000000005e-05 2024-09-15T00:11:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025330+0000", 2024-09-15T00:11:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0658e-05 2024-09-15T00:11:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:464 3.9 3:942069cf:::rbd_header.11397bff333d:head [call rbd.set_stripe_unit_count in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.025416+0000", 2024-09-15T00:11:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.4817669479999998, 2024-09-15T00:11:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055912979999999998, 2024-09-15T00:11:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 464 2024-09-15T00:11:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025416+0000", 2024-09-15T00:11:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025416+0000", 2024-09-15T00:11:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025416+0000", 2024-09-15T00:11:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1500000000000004e-07 2024-09-15T00:11:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025420+0000", 2024-09-15T00:11:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.473e-06 2024-09-15T00:11:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025427+0000", 2024-09-15T00:11:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4270000000000002e-06 2024-09-15T00:11:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025434+0000", 2024-09-15T00:11:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9670000000000004e-06 2024-09-15T00:11:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025484+0000", 2024-09-15T00:11:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0084999999999999e-05 2024-09-15T00:11:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025530+0000", 2024-09-15T00:11:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5286e-05 2024-09-15T00:11:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025666+0000", 2024-09-15T00:11:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000136067 2024-09-15T00:11:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.026078+0000", 2024-09-15T00:11:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00041209200000000002 2024-09-15T00:11:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.030916+0000", 2024-09-15T00:11:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048382060000000003 2024-09-15T00:11:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.030967+0000", 2024-09-15T00:11:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0540000000000001e-05 2024-09-15T00:11:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.031007+0000", 2024-09-15T00:11:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0439999999999999e-05 2024-09-15T00:11:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:832 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.524286+0000", 2024-09-15T00:11:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.9828964400000002, 2024-09-15T00:11:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0067154600000000004, 2024-09-15T00:11:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 832 2024-09-15T00:11:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524286+0000", 2024-09-15T00:11:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524286+0000", 2024-09-15T00:11:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524287+0000", 2024-09-15T00:11:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7899999999999999e-07 2024-09-15T00:11:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524289+0000", 2024-09-15T00:11:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2010000000000002e-06 2024-09-15T00:11:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524295+0000", 2024-09-15T00:11:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0979999999999999e-06 2024-09-15T00:11:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524300+0000", 2024-09-15T00:11:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6970000000000001e-06 2024-09-15T00:11:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524888+0000", 2024-09-15T00:11:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00058825000000000001 2024-09-15T00:11:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524941+0000", 2024-09-15T00:11:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3529000000000002e-05 2024-09-15T00:11:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525192+0000", 2024-09-15T00:11:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025088299999999998 2024-09-15T00:11:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.528065+0000", 2024-09-15T00:11:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002873085 2024-09-15T00:11:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.530935+0000", 2024-09-15T00:11:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028699950000000002 2024-09-15T00:11:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.530968+0000", 2024-09-15T00:11:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.235e-05 2024-09-15T00:11:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.531002+0000", 2024-09-15T00:11:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3992999999999998e-05 2024-09-15T00:11:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:833 3.1a 3:58b59828:::rbd_data.11397bff333d.000000000000001e:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3006249~8407] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.525900+0000", 2024-09-15T00:11:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.9812826110000001, 2024-09-15T00:11:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052167660000000003, 2024-09-15T00:11:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 833 2024-09-15T00:11:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525900+0000", 2024-09-15T00:11:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525900+0000", 2024-09-15T00:11:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525901+0000", 2024-09-15T00:11:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1799999999999995e-07 2024-09-15T00:11:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525909+0000", 2024-09-15T00:11:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4130000000000006e-06 2024-09-15T00:11:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525916+0000", 2024-09-15T00:11:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2350000000000002e-06 2024-09-15T00:11:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525920+0000", 2024-09-15T00:11:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0820000000000001e-06 2024-09-15T00:11:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525957+0000", 2024-09-15T00:11:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6257000000000002e-05 2024-09-15T00:11:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525998+0000", 2024-09-15T00:11:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1662000000000002e-05 2024-09-15T00:11:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.526076+0000", 2024-09-15T00:11:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7985000000000006e-05 2024-09-15T00:11:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.528223+0000", 2024-09-15T00:11:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021471870000000001 2024-09-15T00:11:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.531054+0000", 2024-09-15T00:11:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00283037 2024-09-15T00:11:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.531089+0000", 2024-09-15T00:11:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5364999999999998e-05 2024-09-15T00:11:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.531117+0000", 2024-09-15T00:11:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7691999999999998e-05 2024-09-15T00:11:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:11:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.9352682779999997, 2024-09-15T00:11:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:11:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:11:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:11:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:11:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:11:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:11:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:11:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:11:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:11:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:11:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:11:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:11:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:11:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:11:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:11:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:11:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:11:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:11:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:11:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:11:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:11:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:11:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:11:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:11:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:11:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:11:30.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.540 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.540 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:30.540 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:30.540 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.9352335859999998, 2024-09-15T00:11:30.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:11:30.540 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.540 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.540 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.540 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:30.540 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:30.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.541 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:11:30.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.541 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:30.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:30.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:30.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:11:30.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:11:30.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:11:30.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:11:30.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:11:30.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:11:30.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:11:30.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:11:30.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:11:30.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:11:30.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:11:30.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:30.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:11:30.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:11:30.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:30.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:11:30.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:11:30.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:11:30.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:11:30.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:11:30.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:11:30.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:11:30.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:11:30.545 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.545 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.546 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.546 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:863 3.9 e29/23 ondisk, result = 0)", 2024-09-15T00:11:30.546 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.585151+0000", 2024-09-15T00:11:30.546 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.9220313500000001, 2024-09-15T00:11:30.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.063949342000000006, 2024-09-15T00:11:30.546 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.546 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.546 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:30.546 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585151+0000", 2024-09-15T00:11:30.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585151+0000", 2024-09-15T00:11:30.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585152+0000", 2024-09-15T00:11:30.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.046e-06 2024-09-15T00:11:30.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585156+0000", 2024-09-15T00:11:30.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6339999999999999e-06 2024-09-15T00:11:30.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585185+0000", 2024-09-15T00:11:30.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8646e-05 2024-09-15T00:11:30.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585193+0000", 2024-09-15T00:11:30.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5699999999999993e-06 2024-09-15T00:11:30.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649017+0000", 2024-09-15T00:11:30.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.063823787000000007 2024-09-15T00:11:30.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649023+0000", 2024-09-15T00:11:30.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.728e-06 2024-09-15T00:11:30.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649101+0000", 2024-09-15T00:11:30.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7930999999999996e-05 2024-09-15T00:11:30.551 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.551 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.551 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.552 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:1129 3.1d e29/23 3:ba4e0201:::rbd_data.11397bff333d.0000000000000028:head v 29'4, mlcod=29'2)", 2024-09-15T00:11:30.552 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.896838+0000", 2024-09-15T00:11:30.552 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.6103443400000002, 2024-09-15T00:11:30.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0059020139999999997, 2024-09-15T00:11:30.552 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.552 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.552 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.552 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896838+0000", 2024-09-15T00:11:30.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896838+0000", 2024-09-15T00:11:30.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896839+0000", 2024-09-15T00:11:30.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3900000000000004e-07 2024-09-15T00:11:30.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896844+0000", 2024-09-15T00:11:30.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6990000000000002e-06 2024-09-15T00:11:30.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896854+0000", 2024-09-15T00:11:30.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9790000000000005e-06 2024-09-15T00:11:30.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896858+0000", 2024-09-15T00:11:30.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5850000000000001e-06 2024-09-15T00:11:30.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896932+0000", 2024-09-15T00:11:30.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4286e-05 2024-09-15T00:11:30.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896954+0000", 2024-09-15T00:11:30.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1421999999999999e-05 2024-09-15T00:11:30.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.902706+0000", 2024-09-15T00:11:30.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057519570000000003 2024-09-15T00:11:30.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.902740+0000", 2024-09-15T00:11:30.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4446999999999997e-05 2024-09-15T00:11:30.556 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.556 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.556 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.556 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1184 3.1a 3:5b582af0:::rbd_data.11397bff333d.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:30.556 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.948047+0000", 2024-09-15T00:11:30.556 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.5591355609999997, 2024-09-15T00:11:30.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.039602018000000003, 2024-09-15T00:11:30.557 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.557 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.557 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.557 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:30.557 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:30.557 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.557 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1184 2024-09-15T00:11:30.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.557 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948047+0000", 2024-09-15T00:11:30.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948047+0000", 2024-09-15T00:11:30.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948048+0000", 2024-09-15T00:11:30.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3499999999999996e-07 2024-09-15T00:11:30.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948050+0000", 2024-09-15T00:11:30.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7659999999999999e-06 2024-09-15T00:11:30.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948057+0000", 2024-09-15T00:11:30.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1369999999999996e-06 2024-09-15T00:11:30.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948062+0000", 2024-09-15T00:11:30.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0160000000000002e-06 2024-09-15T00:11:30.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948091+0000", 2024-09-15T00:11:30.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9556999999999998e-05 2024-09-15T00:11:30.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948172+0000", 2024-09-15T00:11:30.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1180999999999994e-05 2024-09-15T00:11:30.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:30.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.949974+0000", 2024-09-15T00:11:30.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018014299999999999 2024-09-15T00:11:30.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987559+0000", 2024-09-15T00:11:30.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037585358999999999 2024-09-15T00:11:30.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:30.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987575+0000", 2024-09-15T00:11:30.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6101000000000001e-05 2024-09-15T00:11:30.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987611+0000", 2024-09-15T00:11:30.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6016999999999998e-05 2024-09-15T00:11:30.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987649+0000", 2024-09-15T00:11:30.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7919000000000001e-05 2024-09-15T00:11:30.562 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.562 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.562 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.562 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1184 3.1a e29/23 ondisk, result = 0)", 2024-09-15T00:11:30.562 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.955158+0000", 2024-09-15T00:11:30.562 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.5520245240000001, 2024-09-15T00:11:30.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032410688, 2024-09-15T00:11:30.563 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.563 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.563 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:30.563 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955158+0000", 2024-09-15T00:11:30.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955158+0000", 2024-09-15T00:11:30.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955159+0000", 2024-09-15T00:11:30.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0399999999999996e-07 2024-09-15T00:11:30.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955161+0000", 2024-09-15T00:11:30.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4710000000000001e-06 2024-09-15T00:11:30.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955172+0000", 2024-09-15T00:11:30.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1059e-05 2024-09-15T00:11:30.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955177+0000", 2024-09-15T00:11:30.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1649999999999997e-06 2024-09-15T00:11:30.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987541+0000", 2024-09-15T00:11:30.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032363268000000001 2024-09-15T00:11:30.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987551+0000", 2024-09-15T00:11:30.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0699e-05 2024-09-15T00:11:30.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987569+0000", 2024-09-15T00:11:30.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7422e-05 2024-09-15T00:11:30.566 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.566 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.566 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.566 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1374 3.9 3:942069cf:::rbd_header.11397bff333d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:30.566 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.354861+0000", 2024-09-15T00:11:30.567 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1523219579999999, 2024-09-15T00:11:30.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055109729999999997, 2024-09-15T00:11:30.567 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.567 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.567 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.567 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:30.567 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:30.567 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.567 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1374 2024-09-15T00:11:30.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.567 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354861+0000", 2024-09-15T00:11:30.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354861+0000", 2024-09-15T00:11:30.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354861+0000", 2024-09-15T00:11:30.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:11:30.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354865+0000", 2024-09-15T00:11:30.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2030000000000001e-06 2024-09-15T00:11:30.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354871+0000", 2024-09-15T00:11:30.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8170000000000001e-06 2024-09-15T00:11:30.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354884+0000", 2024-09-15T00:11:30.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2468e-05 2024-09-15T00:11:30.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354922+0000", 2024-09-15T00:11:30.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8380000000000002e-05 2024-09-15T00:11:30.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354971+0000", 2024-09-15T00:11:30.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9051999999999997e-05 2024-09-15T00:11:30.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:30.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.355080+0000", 2024-09-15T00:11:30.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000108421 2024-09-15T00:11:30.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:30.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.355454+0000", 2024-09-15T00:11:30.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00037466600000000001 2024-09-15T00:11:30.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.360227+0000", 2024-09-15T00:11:30.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047726740000000002 2024-09-15T00:11:30.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.360310+0000", 2024-09-15T00:11:30.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3465000000000001e-05 2024-09-15T00:11:30.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.360372+0000", 2024-09-15T00:11:30.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1155999999999995e-05 2024-09-15T00:11:30.572 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.572 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.572 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.572 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1628 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:30.573 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.712543+0000", 2024-09-15T00:11:30.573 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.794639579, 2024-09-15T00:11:30.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.018404876000000001, 2024-09-15T00:11:30.573 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.573 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.573 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.573 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:30.573 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:30.573 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.573 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1628 2024-09-15T00:11:30.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.573 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712543+0000", 2024-09-15T00:11:30.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712543+0000", 2024-09-15T00:11:30.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712544+0000", 2024-09-15T00:11:30.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6899999999999996e-07 2024-09-15T00:11:30.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712548+0000", 2024-09-15T00:11:30.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1239999999999998e-06 2024-09-15T00:11:30.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712558+0000", 2024-09-15T00:11:30.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0387000000000001e-05 2024-09-15T00:11:30.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712565+0000", 2024-09-15T00:11:30.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.545e-06 2024-09-15T00:11:30.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712603+0000", 2024-09-15T00:11:30.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7818000000000003e-05 2024-09-15T00:11:30.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712644+0000", 2024-09-15T00:11:30.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1743999999999999e-05 2024-09-15T00:11:30.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:30.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.714418+0000", 2024-09-15T00:11:30.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0017731699999999999 2024-09-15T00:11:30.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730862+0000", 2024-09-15T00:11:30.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016444317999999999 2024-09-15T00:11:30.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:30.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730878+0000", 2024-09-15T00:11:30.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6269e-05 2024-09-15T00:11:30.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730915+0000", 2024-09-15T00:11:30.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6365000000000002e-05 2024-09-15T00:11:30.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730948+0000", 2024-09-15T00:11:30.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3367000000000001e-05 2024-09-15T00:11:30.578 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.578 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.578 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.578 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1628 3.9 e30/23 ondisk, result = 0)", 2024-09-15T00:11:30.579 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.720479+0000", 2024-09-15T00:11:30.579 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.7867036540000001, 2024-09-15T00:11:30.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010388134, 2024-09-15T00:11:30.579 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.579 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.579 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:30.579 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720479+0000", 2024-09-15T00:11:30.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720479+0000", 2024-09-15T00:11:30.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720480+0000", 2024-09-15T00:11:30.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2299999999999999e-07 2024-09-15T00:11:30.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720484+0000", 2024-09-15T00:11:30.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-06 2024-09-15T00:11:30.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720499+0000", 2024-09-15T00:11:30.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4783999999999999e-05 2024-09-15T00:11:30.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720507+0000", 2024-09-15T00:11:30.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9370000000000004e-06 2024-09-15T00:11:30.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730848+0000", 2024-09-15T00:11:30.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010341408999999999 2024-09-15T00:11:30.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730856+0000", 2024-09-15T00:11:30.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0679999999999997e-06 2024-09-15T00:11:30.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730867+0000", 2024-09-15T00:11:30.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0733e-05 2024-09-15T00:11:30.583 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.583 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.583 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.583 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1691 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:30.583 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.834973+0000", 2024-09-15T00:11:30.583 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.6722097590000002, 2024-09-15T00:11:30.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005043273, 2024-09-15T00:11:30.584 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.584 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.584 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.584 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:30.584 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:30.584 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.584 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1691 2024-09-15T00:11:30.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.584 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.834973+0000", 2024-09-15T00:11:30.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.834973+0000", 2024-09-15T00:11:30.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.834973+0000", 2024-09-15T00:11:30.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1699999999999999e-07 2024-09-15T00:11:30.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.834975+0000", 2024-09-15T00:11:30.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2460000000000002e-06 2024-09-15T00:11:30.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.834981+0000", 2024-09-15T00:11:30.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1789999999999999e-06 2024-09-15T00:11:30.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.834984+0000", 2024-09-15T00:11:30.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.269e-06 2024-09-15T00:11:30.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.835651+0000", 2024-09-15T00:11:30.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00066688399999999997 2024-09-15T00:11:30.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.835733+0000", 2024-09-15T00:11:30.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2541999999999997e-05 2024-09-15T00:11:30.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:30.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.836094+0000", 2024-09-15T00:11:30.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00036069800000000001 2024-09-15T00:11:30.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.839109+0000", 2024-09-15T00:11:30.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003014727 2024-09-15T00:11:30.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:30.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.839928+0000", 2024-09-15T00:11:30.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081909400000000003 2024-09-15T00:11:30.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.839977+0000", 2024-09-15T00:11:30.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9394000000000002e-05 2024-09-15T00:11:30.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.840016+0000", 2024-09-15T00:11:30.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8923e-05 2024-09-15T00:11:30.589 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1790 3.9 3:9676e01b:::rbd_data.11397bff333d.0000000000000030:head [set-alloc-hint object_size 4194304 write_size 4194304,write 902436~15068] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.959193+0000", 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.5479894490000001, 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049582680000000001, 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1790 2024-09-15T00:11:30.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.959193+0000", 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.959193+0000", 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.959194+0000", 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:11:30.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.959220+0000", 2024-09-15T00:11:30.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6171000000000001e-05 2024-09-15T00:11:30.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.959228+0000", 2024-09-15T00:11:30.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1929999999999993e-06 2024-09-15T00:11:30.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.959233+0000", 2024-09-15T00:11:30.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6439999999999999e-06 2024-09-15T00:11:30.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.959275+0000", 2024-09-15T00:11:30.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2218999999999997e-05 2024-09-15T00:11:30.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.959345+0000", 2024-09-15T00:11:30.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9647999999999995e-05 2024-09-15T00:11:30.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:30.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.959479+0000", 2024-09-15T00:11:30.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000134525 2024-09-15T00:11:30.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:30.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.960134+0000", 2024-09-15T00:11:30.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00065476900000000003 2024-09-15T00:11:30.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.964092+0000", 2024-09-15T00:11:30.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003957871 2024-09-15T00:11:30.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.964124+0000", 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2533000000000002e-05 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.964151+0000", 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7100000000000001e-05 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1837 3.1a 3:58b59828:::rbd_data.11397bff333d.000000000000001e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:27.031352+0000", 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.4758306110000001, 2024-09-15T00:11:30.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053302460000000003, 2024-09-15T00:11:30.596 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.596 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.596 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.596 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:30.596 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:30.596 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.596 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1837 2024-09-15T00:11:30.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.596 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031352+0000", 2024-09-15T00:11:30.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031352+0000", 2024-09-15T00:11:30.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031353+0000", 2024-09-15T00:11:30.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8999999999999995e-07 2024-09-15T00:11:30.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031357+0000", 2024-09-15T00:11:30.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2570000000000001e-06 2024-09-15T00:11:30.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031367+0000", 2024-09-15T00:11:30.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9510000000000001e-06 2024-09-15T00:11:30.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031374+0000", 2024-09-15T00:11:30.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5169999999999996e-06 2024-09-15T00:11:30.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031433+0000", 2024-09-15T00:11:30.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9262000000000003e-05 2024-09-15T00:11:30.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031519+0000", 2024-09-15T00:11:30.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6428000000000007e-05 2024-09-15T00:11:30.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:30.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031624+0000", 2024-09-15T00:11:30.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104435 2024-09-15T00:11:30.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.032914+0000", 2024-09-15T00:11:30.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001290795 2024-09-15T00:11:30.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:30.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.036387+0000", 2024-09-15T00:11:30.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034723079999999999 2024-09-15T00:11:30.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.036646+0000", 2024-09-15T00:11:30.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025974800000000002 2024-09-15T00:11:30.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.036682+0000", 2024-09-15T00:11:30.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5754999999999999e-05 2024-09-15T00:11:30.601 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.601 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.601 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.601 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1839 3.3 3:c0e2a618:::rbd_data.11397bff333d.0000000000000021:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-15T00:11:30.601 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:27.031436+0000", 2024-09-15T00:11:30.602 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.4757462939999999, 2024-09-15T00:11:30.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050196119999999997, 2024-09-15T00:11:30.602 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.602 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.602 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.602 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:30.602 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:30.602 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.602 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1839 2024-09-15T00:11:30.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.602 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031436+0000", 2024-09-15T00:11:30.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031436+0000", 2024-09-15T00:11:30.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031437+0000", 2024-09-15T00:11:30.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1900000000000002e-07 2024-09-15T00:11:30.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031440+0000", 2024-09-15T00:11:30.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0860000000000002e-06 2024-09-15T00:11:30.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031448+0000", 2024-09-15T00:11:30.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4850000000000003e-06 2024-09-15T00:11:30.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031453+0000", 2024-09-15T00:11:30.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:11:30.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031510+0000", 2024-09-15T00:11:30.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6282000000000001e-05 2024-09-15T00:11:30.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031582+0000", 2024-09-15T00:11:30.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2364e-05 2024-09-15T00:11:30.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031679+0000", 2024-09-15T00:11:30.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6538000000000003e-05 2024-09-15T00:11:30.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.033551+0000", 2024-09-15T00:11:30.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001872177 2024-09-15T00:11:30.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:30.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.036359+0000", 2024-09-15T00:11:30.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002808474 2024-09-15T00:11:30.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.036413+0000", 2024-09-15T00:11:30.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3881000000000001e-05 2024-09-15T00:11:30.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.036456+0000", 2024-09-15T00:11:30.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.278e-05 2024-09-15T00:11:30.607 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.607 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.607 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.607 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3179 3.13 e33/23 3:c84732bf:::rbd_data.1139b531b717.0000000000000024:head v 33'34, mlcod=33'32)", 2024-09-15T00:11:30.607 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.658340+0000", 2024-09-15T00:11:30.607 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.84884247300000004, 2024-09-15T00:11:30.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053288930000000003, 2024-09-15T00:11:30.608 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.608 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.608 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.608 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658340+0000", 2024-09-15T00:11:30.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658340+0000", 2024-09-15T00:11:30.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658341+0000", 2024-09-15T00:11:30.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7999999999999995e-07 2024-09-15T00:11:30.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658347+0000", 2024-09-15T00:11:30.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6840000000000001e-06 2024-09-15T00:11:30.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658356+0000", 2024-09-15T00:11:30.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9799999999999993e-06 2024-09-15T00:11:30.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658363+0000", 2024-09-15T00:11:30.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3330000000000002e-06 2024-09-15T00:11:30.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658818+0000", 2024-09-15T00:11:30.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045501500000000002 2024-09-15T00:11:30.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658838+0000", 2024-09-15T00:11:30.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0021e-05 2024-09-15T00:11:30.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663629+0000", 2024-09-15T00:11:30.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047910089999999997 2024-09-15T00:11:30.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663669+0000", 2024-09-15T00:11:30.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0170999999999999e-05 2024-09-15T00:11:30.611 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.611 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.611 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.612 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3223 3.1 3:81046149:::rbd_data.1139b531b717.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:30.612 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.692223+0000", 2024-09-15T00:11:30.612 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.81495968299999999, 2024-09-15T00:11:30.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055865869999999996, 2024-09-15T00:11:30.612 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.612 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.612 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.612 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:30.612 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:30.612 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.612 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3223 2024-09-15T00:11:30.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.613 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692223+0000", 2024-09-15T00:11:30.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692223+0000", 2024-09-15T00:11:30.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692225+0000", 2024-09-15T00:11:30.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.075e-06 2024-09-15T00:11:30.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692237+0000", 2024-09-15T00:11:30.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1623e-05 2024-09-15T00:11:30.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692250+0000", 2024-09-15T00:11:30.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3808000000000001e-05 2024-09-15T00:11:30.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692257+0000", 2024-09-15T00:11:30.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4200000000000004e-06 2024-09-15T00:11:30.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692316+0000", 2024-09-15T00:11:30.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9537999999999998e-05 2024-09-15T00:11:30.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692403+0000", 2024-09-15T00:11:30.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6509999999999997e-05 2024-09-15T00:11:30.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:30.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692492+0000", 2024-09-15T00:11:30.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9320999999999994e-05 2024-09-15T00:11:30.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:30.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.693044+0000", 2024-09-15T00:11:30.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00055210800000000005 2024-09-15T00:11:30.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.697746+0000", 2024-09-15T00:11:30.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047013949999999997 2024-09-15T00:11:30.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.697780+0000", 2024-09-15T00:11:30.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4755000000000002e-05 2024-09-15T00:11:30.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.697810+0000", 2024-09-15T00:11:30.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9034e-05 2024-09-15T00:11:30.617 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.617 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.617 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.618 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3556 3.3 3:c5ee06a1:::rbd_data.1139b531b717.000000000000003d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 811008~7753] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T00:11:30.618 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:30.000120+0000", 2024-09-15T00:11:30.618 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.50706308300000003, 2024-09-15T00:11:30.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0087124330000000003, 2024-09-15T00:11:30.618 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:30.618 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:30.618 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:30.618 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:30.618 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:30.618 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:30.618 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3556 2024-09-15T00:11:30.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.618 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:30.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:30.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000120+0000", 2024-09-15T00:11:30.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:30.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000120+0000", 2024-09-15T00:11:30.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:30.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:30.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000120+0000", 2024-09-15T00:11:30.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7899999999999998e-07 2024-09-15T00:11:30.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:30.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000128+0000", 2024-09-15T00:11:30.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1319999999999994e-06 2024-09-15T00:11:30.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:30.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000136+0000", 2024-09-15T00:11:30.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0790000000000001e-06 2024-09-15T00:11:30.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:30.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000141+0000", 2024-09-15T00:11:30.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6890000000000003e-06 2024-09-15T00:11:30.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000186+0000", 2024-09-15T00:11:30.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4406999999999997e-05 2024-09-15T00:11:30.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-15T00:11:30.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000190+0000", 2024-09-15T00:11:30.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3540000000000002e-06 2024-09-15T00:11:30.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:30.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.003443+0000", 2024-09-15T00:11:30.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032532749999999999 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.003517+0000", 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4256000000000002e-05 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.003644+0000", 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000126854 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.006305+0000", 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026606289999999999 2024-09-15T00:11:30.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:30.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.008768+0000", 2024-09-15T00:11:30.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0024626299999999999 2024-09-15T00:11:30.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:30.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.008797+0000", 2024-09-15T00:11:30.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9524999999999999e-05 2024-09-15T00:11:30.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:30.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:30.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:30.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.008832+0000", 2024-09-15T00:11:30.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4924e-05 2024-09-15T00:11:30.624 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.624 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.624 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.624 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:30.624 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:30.624 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:30.626 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:30.626 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:30.623+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:30.727 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:30.724+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:30.770 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:30.771 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:30.771 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:30.771 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:30.780 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:30.828 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:30.826+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:30.923 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:30.924 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:30.924 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:30.924 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:30.924 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:30.928 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:30.926+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:30.932 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:31.029 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:31.027+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:31.041 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:31.041 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:31.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:31.041 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:31.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.041 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4287.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T00:11:31.041 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:09.531630+0000", 2024-09-15T00:11:31.041 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 21.506170162, 2024-09-15T00:11:31.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0063638770000000004, 2024-09-15T00:11:31.042 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.042 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.042 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4287", 2024-09-15T00:11:31.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/605273324", 2024-09-15T00:11:31.042 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1 2024-09-15T00:11:31.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.042 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531630+0000", 2024-09-15T00:11:31.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531630+0000", 2024-09-15T00:11:31.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531649+0000", 2024-09-15T00:11:31.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8944e-05 2024-09-15T00:11:31.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531655+0000", 2024-09-15T00:11:31.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9569999999999996e-06 2024-09-15T00:11:31.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531670+0000", 2024-09-15T00:11:31.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4202999999999999e-05 2024-09-15T00:11:31.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531682+0000", 2024-09-15T00:11:31.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2449000000000001e-05 2024-09-15T00:11:31.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531834+0000", 2024-09-15T00:11:31.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000152132 2024-09-15T00:11:31.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:11:31.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531841+0000", 2024-09-15T00:11:31.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8990000000000004e-06 2024-09-15T00:11:31.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.536655+0000", 2024-09-15T00:11:31.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048135749999999996 2024-09-15T00:11:31.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.536861+0000", 2024-09-15T00:11:31.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000206467 2024-09-15T00:11:31.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:31.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.537049+0000", 2024-09-15T00:11:31.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018749500000000001 2024-09-15T00:11:31.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.537642+0000", 2024-09-15T00:11:31.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00059315599999999996 2024-09-15T00:11:31.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.537927+0000", 2024-09-15T00:11:31.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000285548 2024-09-15T00:11:31.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.537965+0000", 2024-09-15T00:11:31.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7826e-05 2024-09-15T00:11:31.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.537993+0000", 2024-09-15T00:11:31.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8226e-05 2024-09-15T00:11:31.048 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.048 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.048 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.048 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4287.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T00:11:31.048 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:11.540720+0000", 2024-09-15T00:11:31.048 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.497079597999999, 2024-09-15T00:11:31.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0054958009999999998, 2024-09-15T00:11:31.048 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.048 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.049 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.049 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.049 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4287", 2024-09-15T00:11:31.049 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/605273324", 2024-09-15T00:11:31.049 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 7 2024-09-15T00:11:31.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.049 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540720+0000", 2024-09-15T00:11:31.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540720+0000", 2024-09-15T00:11:31.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540722+0000", 2024-09-15T00:11:31.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3499999999999999e-06 2024-09-15T00:11:31.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540731+0000", 2024-09-15T00:11:31.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2029999999999992e-06 2024-09-15T00:11:31.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540747+0000", 2024-09-15T00:11:31.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6104999999999999e-05 2024-09-15T00:11:31.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540764+0000", 2024-09-15T00:11:31.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7303e-05 2024-09-15T00:11:31.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540843+0000", 2024-09-15T00:11:31.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8905000000000004e-05 2024-09-15T00:11:31.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-15T00:11:31.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540853+0000", 2024-09-15T00:11:31.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0003999999999999e-05 2024-09-15T00:11:31.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.542542+0000", 2024-09-15T00:11:31.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001689383 2024-09-15T00:11:31.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.542624+0000", 2024-09-15T00:11:31.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1185000000000003e-05 2024-09-15T00:11:31.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:31.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.542758+0000", 2024-09-15T00:11:31.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013473399999999999 2024-09-15T00:11:31.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.544732+0000", 2024-09-15T00:11:31.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0019736240000000002 2024-09-15T00:11:31.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.546145+0000", 2024-09-15T00:11:31.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001413037 2024-09-15T00:11:31.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.546183+0000", 2024-09-15T00:11:31.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8105000000000002e-05 2024-09-15T00:11:31.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.546216+0000", 2024-09-15T00:11:31.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2863e-05 2024-09-15T00:11:31.055 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.055 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.055 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.055 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:11:31.055 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:21.598005+0000", 2024-09-15T00:11:31.055 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.4397944299999992, 2024-09-15T00:11:31.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053046270000000001, 2024-09-15T00:11:31.055 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.055 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.055 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.056 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.056 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.056 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.056 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 10 2024-09-15T00:11:31.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.056 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598005+0000", 2024-09-15T00:11:31.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598005+0000", 2024-09-15T00:11:31.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598007+0000", 2024-09-15T00:11:31.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6750000000000001e-06 2024-09-15T00:11:31.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598015+0000", 2024-09-15T00:11:31.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8099999999999998e-06 2024-09-15T00:11:31.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598029+0000", 2024-09-15T00:11:31.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3756e-05 2024-09-15T00:11:31.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598037+0000", 2024-09-15T00:11:31.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7299999999999994e-06 2024-09-15T00:11:31.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598096+0000", 2024-09-15T00:11:31.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8229000000000001e-05 2024-09-15T00:11:31.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-15T00:11:31.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598101+0000", 2024-09-15T00:11:31.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.558e-06 2024-09-15T00:11:31.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.601647+0000", 2024-09-15T00:11:31.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035457779999999999 2024-09-15T00:11:31.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.601788+0000", 2024-09-15T00:11:31.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014144400000000001 2024-09-15T00:11:31.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:31.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.602008+0000", 2024-09-15T00:11:31.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000219488 2024-09-15T00:11:31.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.602766+0000", 2024-09-15T00:11:31.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00075815400000000003 2024-09-15T00:11:31.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.603180+0000", 2024-09-15T00:11:31.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000414039 2024-09-15T00:11:31.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.603236+0000", 2024-09-15T00:11:31.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5773999999999998e-05 2024-09-15T00:11:31.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.603310+0000", 2024-09-15T00:11:31.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4191999999999997e-05 2024-09-15T00:11:31.062 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.062 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.062 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.062 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:603 3.1d 3:ba2ec8e4:::rbd_data.11397bff333d.0000000000000038:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3578962~25518] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:31.062 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.210996+0000", 2024-09-15T00:11:31.062 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.8268038830000002, 2024-09-15T00:11:31.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050683109999999998, 2024-09-15T00:11:31.062 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.062 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.062 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.062 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.062 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.063 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.063 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 603 2024-09-15T00:11:31.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.063 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.210996+0000", 2024-09-15T00:11:31.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.210996+0000", 2024-09-15T00:11:31.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.210998+0000", 2024-09-15T00:11:31.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.689e-06 2024-09-15T00:11:31.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211052+0000", 2024-09-15T00:11:31.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4664999999999999e-05 2024-09-15T00:11:31.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211064+0000", 2024-09-15T00:11:31.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1874e-05 2024-09-15T00:11:31.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211070+0000", 2024-09-15T00:11:31.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9170000000000003e-06 2024-09-15T00:11:31.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211121+0000", 2024-09-15T00:11:31.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1013000000000001e-05 2024-09-15T00:11:31.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211211+0000", 2024-09-15T00:11:31.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0446999999999996e-05 2024-09-15T00:11:31.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:31.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211292+0000", 2024-09-15T00:11:31.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0655999999999995e-05 2024-09-15T00:11:31.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.213176+0000", 2024-09-15T00:11:31.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001884286 2024-09-15T00:11:31.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.216010+0000", 2024-09-15T00:11:31.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028331720000000001 2024-09-15T00:11:31.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.216038+0000", 2024-09-15T00:11:31.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8062999999999999e-05 2024-09-15T00:11:31.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.216064+0000", 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6529e-05 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:865 3.5 3:a614403b:::rbd_data.11397bff333d.0000000000000003:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.650001+0000", 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3877989250000002, 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.03879726, 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.068 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.069 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 865 2024-09-15T00:11:31.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.069 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650001+0000", 2024-09-15T00:11:31.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650001+0000", 2024-09-15T00:11:31.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650002+0000", 2024-09-15T00:11:31.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.19e-06 2024-09-15T00:11:31.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650008+0000", 2024-09-15T00:11:31.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1380000000000004e-06 2024-09-15T00:11:31.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650019+0000", 2024-09-15T00:11:31.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1323e-05 2024-09-15T00:11:31.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650025+0000", 2024-09-15T00:11:31.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8710000000000002e-06 2024-09-15T00:11:31.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650080+0000", 2024-09-15T00:11:31.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4573000000000001e-05 2024-09-15T00:11:31.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650177+0000", 2024-09-15T00:11:31.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7460999999999994e-05 2024-09-15T00:11:31.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:31.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.651728+0000", 2024-09-15T00:11:31.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015505709999999999 2024-09-15T00:11:31.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.688714+0000", 2024-09-15T00:11:31.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.036986342999999998 2024-09-15T00:11:31.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.688727+0000", 2024-09-15T00:11:31.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2448e-05 2024-09-15T00:11:31.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.688762+0000", 2024-09-15T00:11:31.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4906000000000001e-05 2024-09-15T00:11:31.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.688798+0000", 2024-09-15T00:11:31.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6436000000000001e-05 2024-09-15T00:11:31.073 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.073 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.073 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.073 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:865 3.5 e29/23 ondisk, result = 0)", 2024-09-15T00:11:31.074 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.656357+0000", 2024-09-15T00:11:31.074 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.381443151, 2024-09-15T00:11:31.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032363683999999997, 2024-09-15T00:11:31.074 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.074 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.074 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:31.074 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.656357+0000", 2024-09-15T00:11:31.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.656357+0000", 2024-09-15T00:11:31.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.656357+0000", 2024-09-15T00:11:31.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5099999999999999e-07 2024-09-15T00:11:31.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.656360+0000", 2024-09-15T00:11:31.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1159999999999999e-06 2024-09-15T00:11:31.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.656371+0000", 2024-09-15T00:11:31.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0737000000000001e-05 2024-09-15T00:11:31.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.656378+0000", 2024-09-15T00:11:31.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:11:31.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.688701+0000", 2024-09-15T00:11:31.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032323172999999997 2024-09-15T00:11:31.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.688709+0000", 2024-09-15T00:11:31.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6279999999999996e-06 2024-09-15T00:11:31.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.688720+0000", 2024-09-15T00:11:31.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1559999999999999e-05 2024-09-15T00:11:31.077 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.078 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.078 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.078 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1129 3.1d 3:ba4e0201:::rbd_data.11397bff333d.0000000000000028:head [truncate 1175199] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:31.078 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.896284+0000", 2024-09-15T00:11:31.078 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.1415162250000002, 2024-09-15T00:11:31.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0066267089999999997, 2024-09-15T00:11:31.078 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.078 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.078 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.078 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.078 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.079 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.079 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1129 2024-09-15T00:11:31.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.079 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896284+0000", 2024-09-15T00:11:31.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896284+0000", 2024-09-15T00:11:31.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896285+0000", 2024-09-15T00:11:31.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.59e-06 2024-09-15T00:11:31.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896296+0000", 2024-09-15T00:11:31.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1287000000000001e-05 2024-09-15T00:11:31.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896324+0000", 2024-09-15T00:11:31.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7946e-05 2024-09-15T00:11:31.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896340+0000", 2024-09-15T00:11:31.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.525e-05 2024-09-15T00:11:31.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896456+0000", 2024-09-15T00:11:31.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000116118 2024-09-15T00:11:31.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896584+0000", 2024-09-15T00:11:31.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000127904 2024-09-15T00:11:31.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:31.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896751+0000", 2024-09-15T00:11:31.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016762 2024-09-15T00:11:31.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.897191+0000", 2024-09-15T00:11:31.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043940199999999998 2024-09-15T00:11:31.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.902847+0000", 2024-09-15T00:11:31.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0056558959999999997 2024-09-15T00:11:31.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.902885+0000", 2024-09-15T00:11:31.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8198000000000003e-05 2024-09-15T00:11:31.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.902910+0000", 2024-09-15T00:11:31.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5497999999999999e-05 2024-09-15T00:11:31.083 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.084 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.084 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.084 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1313 3.1f 3:fa9cde43:::rbd_data.11397bff333d.000000000000003a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3178173~33091] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:31.084 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.283783+0000", 2024-09-15T00:11:31.084 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.7540167200000001, 2024-09-15T00:11:31.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049414089999999999, 2024-09-15T00:11:31.084 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.084 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.084 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.084 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.084 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.084 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.084 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1313 2024-09-15T00:11:31.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.085 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283783+0000", 2024-09-15T00:11:31.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283783+0000", 2024-09-15T00:11:31.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283784+0000", 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8100000000000002e-07 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283806+0000", 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1970000000000001e-05 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283816+0000", 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0641e-05 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283821+0000", 2024-09-15T00:11:31.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0039999999999999e-06 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283860+0000", 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9054000000000003e-05 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283950+0000", 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9907000000000005e-05 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.284036+0000", 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5774999999999998e-05 2024-09-15T00:11:31.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.284885+0000", 2024-09-15T00:11:31.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00084878899999999997 2024-09-15T00:11:31.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.288668+0000", 2024-09-15T00:11:31.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0037833369999999999 2024-09-15T00:11:31.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.288700+0000", 2024-09-15T00:11:31.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1962000000000003e-05 2024-09-15T00:11:31.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.288724+0000", 2024-09-15T00:11:31.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4189000000000002e-05 2024-09-15T00:11:31.089 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.089 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.089 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.089 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1844 3.1c 3:3bb1cbbe:::rbd_data.11397bff333d.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-15T00:11:31.089 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:27.032353+0000", 2024-09-15T00:11:31.089 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.0054469539999999, 2024-09-15T00:11:31.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043890450000000003, 2024-09-15T00:11:31.089 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.089 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.090 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.090 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.090 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.090 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.090 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1844 2024-09-15T00:11:31.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.090 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.032353+0000", 2024-09-15T00:11:31.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.032353+0000", 2024-09-15T00:11:31.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.032354+0000", 2024-09-15T00:11:31.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9999999999999996e-07 2024-09-15T00:11:31.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.032359+0000", 2024-09-15T00:11:31.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2920000000000003e-06 2024-09-15T00:11:31.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.032369+0000", 2024-09-15T00:11:31.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0478e-05 2024-09-15T00:11:31.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.032381+0000", 2024-09-15T00:11:31.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1783999999999999e-05 2024-09-15T00:11:31.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.032436+0000", 2024-09-15T00:11:31.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5179999999999997e-05 2024-09-15T00:11:31.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.032521+0000", 2024-09-15T00:11:31.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4973000000000001e-05 2024-09-15T00:11:31.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:31.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.032621+0000", 2024-09-15T00:11:31.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9233000000000003e-05 2024-09-15T00:11:31.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.036428+0000", 2024-09-15T00:11:31.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038077219999999999 2024-09-15T00:11:31.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.036638+0000", 2024-09-15T00:11:31.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020958899999999999 2024-09-15T00:11:31.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.036695+0000", 2024-09-15T00:11:31.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7627999999999998e-05 2024-09-15T00:11:31.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.036742+0000", 2024-09-15T00:11:31.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6366000000000003e-05 2024-09-15T00:11:31.095 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.095 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.095 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.095 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3160 3.1d 3:bf735e48:::rbd_data.1139b531b717.0000000000000064:head [set-alloc-hint object_size 2097152 write_size 2097152,write 925696~3756] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:31.095 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.624419+0000", 2024-09-15T00:11:31.095 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.413380978, 2024-09-15T00:11:31.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050107880000000004, 2024-09-15T00:11:31.095 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.096 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.096 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.096 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.096 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.096 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.096 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3160 2024-09-15T00:11:31.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.096 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624419+0000", 2024-09-15T00:11:31.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624419+0000", 2024-09-15T00:11:31.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624420+0000", 2024-09-15T00:11:31.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3790000000000001e-06 2024-09-15T00:11:31.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624428+0000", 2024-09-15T00:11:31.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1289999999999996e-06 2024-09-15T00:11:31.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624440+0000", 2024-09-15T00:11:31.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1252e-05 2024-09-15T00:11:31.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624445+0000", 2024-09-15T00:11:31.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5249999999999996e-06 2024-09-15T00:11:31.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624492+0000", 2024-09-15T00:11:31.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6517999999999997e-05 2024-09-15T00:11:31.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624605+0000", 2024-09-15T00:11:31.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113588 2024-09-15T00:11:31.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:31.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624748+0000", 2024-09-15T00:11:31.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014249 2024-09-15T00:11:31.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.626822+0000", 2024-09-15T00:11:31.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020740310000000001 2024-09-15T00:11:31.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.629341+0000", 2024-09-15T00:11:31.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0025189880000000002 2024-09-15T00:11:31.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.629384+0000", 2024-09-15T00:11:31.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3788000000000002e-05 2024-09-15T00:11:31.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.629430+0000", 2024-09-15T00:11:31.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5099999999999998e-05 2024-09-15T00:11:31.101 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.101 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.101 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.101 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3163 3.17 3:ef86f319:::rbd_data.1139b531b717.000000000000006d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 917504~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:31.101 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.624764+0000", 2024-09-15T00:11:31.101 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.4130360739999999, 2024-09-15T00:11:31.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0046892560000000002, 2024-09-15T00:11:31.102 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.102 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.102 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.102 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.102 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.102 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.102 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3163 2024-09-15T00:11:31.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.102 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624764+0000", 2024-09-15T00:11:31.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624764+0000", 2024-09-15T00:11:31.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624764+0000", 2024-09-15T00:11:31.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1699999999999998e-07 2024-09-15T00:11:31.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624776+0000", 2024-09-15T00:11:31.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1547e-05 2024-09-15T00:11:31.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624785+0000", 2024-09-15T00:11:31.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.346e-06 2024-09-15T00:11:31.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624794+0000", 2024-09-15T00:11:31.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0790000000000007e-06 2024-09-15T00:11:31.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624844+0000", 2024-09-15T00:11:31.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9917999999999999e-05 2024-09-15T00:11:31.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624973+0000", 2024-09-15T00:11:31.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000128687 2024-09-15T00:11:31.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:31.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.625105+0000", 2024-09-15T00:11:31.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013188699999999999 2024-09-15T00:11:31.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.626188+0000", 2024-09-15T00:11:31.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001082981 2024-09-15T00:11:31.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.629367+0000", 2024-09-15T00:11:31.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031788039999999999 2024-09-15T00:11:31.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.629413+0000", 2024-09-15T00:11:31.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6792999999999997e-05 2024-09-15T00:11:31.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.629453+0000", 2024-09-15T00:11:31.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9597000000000001e-05 2024-09-15T00:11:31.107 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.107 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.107 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.107 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3164 3.5 3:a3ee9f36:::rbd_data.1139b531b717.000000000000006e:head [set-alloc-hint object_size 2097152 write_size 2097152,write 917504~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:31.107 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.624969+0000", 2024-09-15T00:11:31.107 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.4128310230000001, 2024-09-15T00:11:31.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045719209999999996, 2024-09-15T00:11:31.107 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.108 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.108 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.108 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.108 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.108 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.108 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3164 2024-09-15T00:11:31.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.108 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624969+0000", 2024-09-15T00:11:31.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624969+0000", 2024-09-15T00:11:31.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624969+0000", 2024-09-15T00:11:31.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3200000000000004e-07 2024-09-15T00:11:31.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624979+0000", 2024-09-15T00:11:31.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3179999999999996e-06 2024-09-15T00:11:31.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624988+0000", 2024-09-15T00:11:31.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8939999999999999e-06 2024-09-15T00:11:31.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624994+0000", 2024-09-15T00:11:31.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0349999999999998e-06 2024-09-15T00:11:31.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.625083+0000", 2024-09-15T00:11:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9084000000000003e-05 2024-09-15T00:11:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.625150+0000", 2024-09-15T00:11:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6748000000000005e-05 2024-09-15T00:11:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.625279+0000", 2024-09-15T00:11:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129286 2024-09-15T00:11:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.626275+0000", 2024-09-15T00:11:31.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00099648800000000002 2024-09-15T00:11:31.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.629441+0000", 2024-09-15T00:11:31.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031656169999999999 2024-09-15T00:11:31.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.629501+0000", 2024-09-15T00:11:31.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0061e-05 2024-09-15T00:11:31.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.629541+0000", 2024-09-15T00:11:31.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9657999999999999e-05 2024-09-15T00:11:31.113 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.113 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.113 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.113 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3165 3.17 3:ebd45212:::rbd_data.1139b531b717.000000000000006f:head [set-alloc-hint object_size 2097152 write_size 2097152,write 917504~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:31.113 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.625052+0000", 2024-09-15T00:11:31.113 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.4127475010000001, 2024-09-15T00:11:31.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045077329999999999, 2024-09-15T00:11:31.113 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.114 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.114 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.114 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.114 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.114 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.114 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3165 2024-09-15T00:11:31.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.114 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.625052+0000", 2024-09-15T00:11:31.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.625052+0000", 2024-09-15T00:11:31.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.625053+0000", 2024-09-15T00:11:31.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9999999999999997e-07 2024-09-15T00:11:31.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.625062+0000", 2024-09-15T00:11:31.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7050000000000005e-06 2024-09-15T00:11:31.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.625070+0000", 2024-09-15T00:11:31.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8809999999999996e-06 2024-09-15T00:11:31.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.625075+0000", 2024-09-15T00:11:31.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.801e-06 2024-09-15T00:11:31.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.625755+0000", 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00067918999999999998 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.625812+0000", 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7759e-05 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.625900+0000", 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7874999999999995e-05 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.626712+0000", 2024-09-15T00:11:31.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081220700000000001 2024-09-15T00:11:31.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.629467+0000", 2024-09-15T00:11:31.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002754926 2024-09-15T00:11:31.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.629518+0000", 2024-09-15T00:11:31.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0528999999999997e-05 2024-09-15T00:11:31.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.629560+0000", 2024-09-15T00:11:31.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2160000000000003e-05 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3183 3.d e33/23 3:b2b2ff6e:::rbd_data.1139b531b717.000000000000006b:head v 33'19, mlcod=33'17)", 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.658700+0000", 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.3791001270000001, 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051160099999999998, 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658700+0000", 2024-09-15T00:11:31.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658700+0000", 2024-09-15T00:11:31.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658700+0000", 2024-09-15T00:11:31.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7400000000000003e-07 2024-09-15T00:11:31.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658707+0000", 2024-09-15T00:11:31.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4760000000000003e-06 2024-09-15T00:11:31.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658725+0000", 2024-09-15T00:11:31.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8539999999999999e-05 2024-09-15T00:11:31.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658732+0000", 2024-09-15T00:11:31.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1590000000000004e-06 2024-09-15T00:11:31.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658837+0000", 2024-09-15T00:11:31.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104941 2024-09-15T00:11:31.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658857+0000", 2024-09-15T00:11:31.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9179999999999999e-05 2024-09-15T00:11:31.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663771+0000", 2024-09-15T00:11:31.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049141660000000002 2024-09-15T00:11:31.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663816+0000", 2024-09-15T00:11:31.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4873999999999998e-05 2024-09-15T00:11:31.123 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.123 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.123 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.123 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3184 3.d e33/23 3:b57e2e8d:::rbd_data.1139b531b717.000000000000006c:head v 33'20, mlcod=33'17)", 2024-09-15T00:11:31.123 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.659024+0000", 2024-09-15T00:11:31.123 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.378775978, 2024-09-15T00:11:31.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048185770000000001, 2024-09-15T00:11:31.124 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.124 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.124 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.124 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659024+0000", 2024-09-15T00:11:31.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659024+0000", 2024-09-15T00:11:31.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659024+0000", 2024-09-15T00:11:31.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8599999999999998e-07 2024-09-15T00:11:31.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659030+0000", 2024-09-15T00:11:31.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3430000000000003e-06 2024-09-15T00:11:31.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659036+0000", 2024-09-15T00:11:31.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4849999999999997e-06 2024-09-15T00:11:31.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659040+0000", 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.206e-06 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659080+0000", 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9217000000000001e-05 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659097+0000", 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6969999999999998e-05 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663819+0000", 2024-09-15T00:11:31.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047219289999999997 2024-09-15T00:11:31.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663842+0000", 2024-09-15T00:11:31.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3740999999999999e-05 2024-09-15T00:11:31.127 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.127 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.127 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.127 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3186 3.5 3:a3ee9f36:::rbd_data.1139b531b717.000000000000006e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:31.127 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.659078+0000", 2024-09-15T00:11:31.128 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.3787217789999999, 2024-09-15T00:11:31.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004869852, 2024-09-15T00:11:31.128 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.128 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.128 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.128 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.128 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.128 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.128 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3186 2024-09-15T00:11:31.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.128 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659078+0000", 2024-09-15T00:11:31.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659078+0000", 2024-09-15T00:11:31.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659079+0000", 2024-09-15T00:11:31.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4499999999999996e-07 2024-09-15T00:11:31.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659083+0000", 2024-09-15T00:11:31.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8689999999999997e-06 2024-09-15T00:11:31.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659091+0000", 2024-09-15T00:11:31.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0639999999999994e-06 2024-09-15T00:11:31.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659104+0000", 2024-09-15T00:11:31.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3049e-05 2024-09-15T00:11:31.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659149+0000", 2024-09-15T00:11:31.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5108000000000002e-05 2024-09-15T00:11:31.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659228+0000", 2024-09-15T00:11:31.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.941e-05 2024-09-15T00:11:31.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:31.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659312+0000", 2024-09-15T00:11:31.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3829e-05 2024-09-15T00:11:31.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.661416+0000", 2024-09-15T00:11:31.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021034080000000002 2024-09-15T00:11:31.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663849+0000", 2024-09-15T00:11:31.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0024330340000000002 2024-09-15T00:11:31.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663904+0000", 2024-09-15T00:11:31.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5535999999999998e-05 2024-09-15T00:11:31.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663948+0000", 2024-09-15T00:11:31.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3699999999999998e-05 2024-09-15T00:11:31.133 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.133 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.133 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.133 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3221 3.17 3:eb942860:::rbd_data.1139b531b717.0000000000000050:head [set-alloc-hint object_size 2097152 write_size 2097152,write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:31.133 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.689861+0000", 2024-09-15T00:11:31.133 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.3479388970000001, 2024-09-15T00:11:31.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005121919, 2024-09-15T00:11:31.133 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.134 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.134 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.134 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.134 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.134 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.134 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3221 2024-09-15T00:11:31.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.134 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.689861+0000", 2024-09-15T00:11:31.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.689861+0000", 2024-09-15T00:11:31.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.689862+0000", 2024-09-15T00:11:31.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9999999999999997e-07 2024-09-15T00:11:31.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.689872+0000", 2024-09-15T00:11:31.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0739e-05 2024-09-15T00:11:31.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.689889+0000", 2024-09-15T00:11:31.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6948000000000001e-05 2024-09-15T00:11:31.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.689894+0000", 2024-09-15T00:11:31.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7709999999999997e-06 2024-09-15T00:11:31.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.689943+0000", 2024-09-15T00:11:31.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8916999999999999e-05 2024-09-15T00:11:31.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.690038+0000", 2024-09-15T00:11:31.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5320000000000002e-05 2024-09-15T00:11:31.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:31.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.690150+0000", 2024-09-15T00:11:31.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000111716 2024-09-15T00:11:31.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.690683+0000", 2024-09-15T00:11:31.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053344999999999998 2024-09-15T00:11:31.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.694907+0000", 2024-09-15T00:11:31.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042237389999999998 2024-09-15T00:11:31.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.694940+0000", 2024-09-15T00:11:31.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3046999999999997e-05 2024-09-15T00:11:31.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.694983+0000", 2024-09-15T00:11:31.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2571999999999998e-05 2024-09-15T00:11:31.139 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.139 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.139 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.139 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3422 3.16 3:683c2b90:::rbd_data.1139b531b717.0000000000000007:head [stat,truncate 548864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:31.139 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.868521+0000", 2024-09-15T00:11:31.139 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.169279231, 2024-09-15T00:11:31.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051108050000000004, 2024-09-15T00:11:31.140 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.140 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.140 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.140 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.140 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.140 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.140 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3422 2024-09-15T00:11:31.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.140 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868521+0000", 2024-09-15T00:11:31.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868521+0000", 2024-09-15T00:11:31.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868521+0000", 2024-09-15T00:11:31.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9400000000000004e-07 2024-09-15T00:11:31.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868532+0000", 2024-09-15T00:11:31.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1e-05 2024-09-15T00:11:31.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868544+0000", 2024-09-15T00:11:31.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1673e-05 2024-09-15T00:11:31.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868551+0000", 2024-09-15T00:11:31.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9469999999999997e-06 2024-09-15T00:11:31.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868598+0000", 2024-09-15T00:11:31.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7228999999999998e-05 2024-09-15T00:11:31.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868665+0000", 2024-09-15T00:11:31.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6334999999999994e-05 2024-09-15T00:11:31.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:31.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868775+0000", 2024-09-15T00:11:31.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110649 2024-09-15T00:11:31.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.869761+0000", 2024-09-15T00:11:31.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00098550000000000005 2024-09-15T00:11:31.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.873536+0000", 2024-09-15T00:11:31.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0037752739999999999 2024-09-15T00:11:31.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.873589+0000", 2024-09-15T00:11:31.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3019000000000002e-05 2024-09-15T00:11:31.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.873631+0000", 2024-09-15T00:11:31.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2284999999999999e-05 2024-09-15T00:11:31.145 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.145 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.145 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.145 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3509 3.17 3:e9bf9e8f:::rbd_data.1139b531b717.000000000000002c:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1048576~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:31.145 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.951418+0000", 2024-09-15T00:11:31.145 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.0863815699999999, 2024-09-15T00:11:31.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053473030000000003, 2024-09-15T00:11:31.145 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.145 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.146 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.146 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.146 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.146 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.146 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3509 2024-09-15T00:11:31.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.146 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951418+0000", 2024-09-15T00:11:31.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951418+0000", 2024-09-15T00:11:31.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951420+0000", 2024-09-15T00:11:31.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4929999999999999e-06 2024-09-15T00:11:31.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951429+0000", 2024-09-15T00:11:31.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2879999999999998e-06 2024-09-15T00:11:31.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951439+0000", 2024-09-15T00:11:31.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0205e-05 2024-09-15T00:11:31.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951444+0000", 2024-09-15T00:11:31.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0139999999999998e-06 2024-09-15T00:11:31.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951487+0000", 2024-09-15T00:11:31.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2364999999999998e-05 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951567+0000", 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0248999999999997e-05 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951657+0000", 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9717999999999997e-05 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955025+0000", 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003368149 2024-09-15T00:11:31.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.956667+0000", 2024-09-15T00:11:31.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0016422640000000001 2024-09-15T00:11:31.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.956722+0000", 2024-09-15T00:11:31.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4926999999999998e-05 2024-09-15T00:11:31.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.956765+0000", 2024-09-15T00:11:31.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3631000000000003e-05 2024-09-15T00:11:31.150 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.151 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.151 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.151 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3584 3.17 3:ef86f319:::rbd_data.1139b531b717.000000000000006d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 498027~1685] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T00:11:31.151 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:30.032469+0000", 2024-09-15T00:11:31.151 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.005330641, 2024-09-15T00:11:31.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0056325170000000001, 2024-09-15T00:11:31.151 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.151 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.151 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.151 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.151 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.151 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.151 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3584 2024-09-15T00:11:31.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.152 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032469+0000", 2024-09-15T00:11:31.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032469+0000", 2024-09-15T00:11:31.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032470+0000", 2024-09-15T00:11:31.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3710000000000001e-06 2024-09-15T00:11:31.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032479+0000", 2024-09-15T00:11:31.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5839999999999995e-06 2024-09-15T00:11:31.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032492+0000", 2024-09-15T00:11:31.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3125000000000001e-05 2024-09-15T00:11:31.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032499+0000", 2024-09-15T00:11:31.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:11:31.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032553+0000", 2024-09-15T00:11:31.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3761000000000002e-05 2024-09-15T00:11:31.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-15T00:11:31.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032557+0000", 2024-09-15T00:11:31.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6659999999999999e-06 2024-09-15T00:11:31.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.035250+0000", 2024-09-15T00:11:31.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026922890000000001 2024-09-15T00:11:31.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.035319+0000", 2024-09-15T00:11:31.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8919999999999997e-05 2024-09-15T00:11:31.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:31.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.035441+0000", 2024-09-15T00:11:31.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012218799999999999 2024-09-15T00:11:31.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.036328+0000", 2024-09-15T00:11:31.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00088673199999999999 2024-09-15T00:11:31.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.038036+0000", 2024-09-15T00:11:31.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00170856 2024-09-15T00:11:31.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.038073+0000", 2024-09-15T00:11:31.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7324999999999999e-05 2024-09-15T00:11:31.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.038102+0000", 2024-09-15T00:11:31.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8194000000000001e-05 2024-09-15T00:11:31.157 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.157 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.157 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.157 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.157 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.157 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:31.161 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone4 from 0xda76440 (overlap 0x522559) to 0x39808b 2024-09-15T00:11:31.161 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-15T00:11:31.161 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:31.158+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:31.162 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:11:31.262 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:31.259+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:31.304 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:31.305 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:31.305 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:31.305 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:31.313 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:11:31.316 INFO:teuthology.orchestra.run.smithi171.stdout:683 writesame 0x154d7a8 thru 0x1553071 data_size 0x5(0x58ca bytes) 2024-09-15T00:11:31.330 INFO:teuthology.orchestra.run.smithi171.stdout:684 write 0xd23eef6 thru 0xd242777 (0x3882 bytes) 2024-09-15T00:11:31.334 INFO:teuthology.orchestra.run.smithi171.stdout:685 write 0x7559748 thru 0x75685ed (0xeea6 bytes) 2024-09-15T00:11:31.340 INFO:teuthology.orchestra.run.smithi171.stdout:687 write 0x15144e2 thru 0x151d204 (0x8d23 bytes) 2024-09-15T00:11:31.343 INFO:teuthology.orchestra.run.smithi171.stdout:688 writesame 0x8237cff thru 0x824449f data_size 0x5(0xc7a1 bytes) 2024-09-15T00:11:31.364 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:31.360+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:31.382 INFO:teuthology.orchestra.run.smithi171.stdout:689 trunc from 0xda76440 to 0x11c7c14 2024-09-15T00:11:31.417 INFO:teuthology.orchestra.run.smithi171.stdout:690 writesame 0x8dd40ad thru 0x8dd736a data_size 0xf(0x32be bytes) 2024-09-15T00:11:31.417 INFO:teuthology.orchestra.run.smithi171.stdout:692 writesame 0x203d65b thru 0x2044159 data_size 0x31(0x6aff bytes) 2024-09-15T00:11:31.419 INFO:teuthology.orchestra.run.smithi171.stdout:694 punch from 0x4fe8a5e to 0x4fec3ed, (0x398f bytes) 2024-09-15T00:11:31.423 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:31.423 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:31.426 INFO:teuthology.orchestra.run.smithi171.stdout:695 write 0xa3aadbf thru 0xa3ad5cd (0x280f bytes) 2024-09-15T00:11:31.429 INFO:teuthology.orchestra.run.smithi171.stdout:696 read 0xd921c8 thru 0xd9c094 (0x9ecd bytes) 2024-09-15T00:11:31.433 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:11:31.438 INFO:teuthology.orchestra.run.smithi171.stdout:697 write 0xcd41b4e thru 0xcd4a5c4 (0x8a77 bytes) 2024-09-15T00:11:31.441 INFO:teuthology.orchestra.run.smithi171.stdout:698 punch from 0xc9fc16e to 0xc9fdb2b, (0x19bd bytes) 2024-09-15T00:11:31.442 INFO:teuthology.orchestra.run.smithi171.stdout:699 read 0x2bf5576 thru 0x2c007d1 (0xb25c bytes) 2024-09-15T00:11:31.442 INFO:teuthology.orchestra.run.smithi171.stdout:700 write 0x15fcde7 thru 0x1609ba3 (0xcdbd bytes) 2024-09-15T00:11:31.446 INFO:teuthology.orchestra.run.smithi171.stdout:701 write 0x7d5e3c3 thru 0x7d62832 (0x4470 bytes) 2024-09-15T00:11:31.449 INFO:teuthology.orchestra.run.smithi171.stdout:702 write 0x617f6b1 thru 0x618c5b2 (0xcf02 bytes) 2024-09-15T00:11:31.454 INFO:teuthology.orchestra.run.smithi171.stdout:704 writesame 0x454c3a6 thru 0x45578e3 data_size 0x72(0xb53e bytes) 2024-09-15T00:11:31.457 INFO:teuthology.orchestra.run.smithi171.stdout:705 punch from 0x780e17f to 0x78137d3, (0x5654 bytes) 2024-09-15T00:11:31.457 INFO:teuthology.orchestra.run.smithi171.stdout:706 clone 6 order 21 su 8192 sc 12 2024-09-15T00:11:31.477 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:31.474+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:31.537 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:31.538 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:31.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:31.538 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:31.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.538 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1089 3.a 3:53d7c8e3:::rbd_data.11397bff333d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 641683~13677] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:31.538 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.824794+0000", 2024-09-15T00:11:31.538 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.7103973290000001, 2024-09-15T00:11:31.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00445219, 2024-09-15T00:11:31.538 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.538 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.538 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.539 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.539 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.539 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.539 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1089 2024-09-15T00:11:31.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.539 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.824794+0000", 2024-09-15T00:11:31.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.824794+0000", 2024-09-15T00:11:31.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.824795+0000", 2024-09-15T00:11:31.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3600000000000002e-07 2024-09-15T00:11:31.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.824924+0000", 2024-09-15T00:11:31.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012916100000000001 2024-09-15T00:11:31.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.824935+0000", 2024-09-15T00:11:31.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0341e-05 2024-09-15T00:11:31.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.824942+0000", 2024-09-15T00:11:31.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8469999999999998e-06 2024-09-15T00:11:31.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.824994+0000", 2024-09-15T00:11:31.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2377999999999999e-05 2024-09-15T00:11:31.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.825087+0000", 2024-09-15T00:11:31.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2609999999999996e-05 2024-09-15T00:11:31.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:31.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.825247+0000", 2024-09-15T00:11:31.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016063500000000001 2024-09-15T00:11:31.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.826931+0000", 2024-09-15T00:11:31.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0016839050000000001 2024-09-15T00:11:31.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.829183+0000", 2024-09-15T00:11:31.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0022522269999999999 2024-09-15T00:11:31.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.829220+0000", 2024-09-15T00:11:31.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6910999999999999e-05 2024-09-15T00:11:31.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.829247+0000", 2024-09-15T00:11:31.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6339e-05 2024-09-15T00:11:31.544 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.544 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.544 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.544 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3037 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'86, mlcod=33'82)", 2024-09-15T00:11:31.544 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.497583+0000", 2024-09-15T00:11:31.544 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.0376084570000002, 2024-09-15T00:11:31.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077497779999999997, 2024-09-15T00:11:31.544 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.544 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.544 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.544 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497583+0000", 2024-09-15T00:11:31.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497583+0000", 2024-09-15T00:11:31.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497584+0000", 2024-09-15T00:11:31.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6599999999999996e-07 2024-09-15T00:11:31.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497588+0000", 2024-09-15T00:11:31.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3529999999999997e-06 2024-09-15T00:11:31.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497598+0000", 2024-09-15T00:11:31.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4839999999999993e-06 2024-09-15T00:11:31.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497602+0000", 2024-09-15T00:11:31.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:11:31.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497643+0000", 2024-09-15T00:11:31.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0326e-05 2024-09-15T00:11:31.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497662+0000", 2024-09-15T00:11:31.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9590999999999999e-05 2024-09-15T00:11:31.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505303+0000", 2024-09-15T00:11:31.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076408919999999998 2024-09-15T00:11:31.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505333+0000", 2024-09-15T00:11:31.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0071999999999999e-05 2024-09-15T00:11:31.548 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.548 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.548 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.548 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3038 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'87, mlcod=33'82)", 2024-09-15T00:11:31.548 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:31.549 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.0367639450000001, 2024-09-15T00:11:31.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069253919999999998, 2024-09-15T00:11:31.549 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.549 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.549 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.549 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:31.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:31.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:31.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6899999999999997e-07 2024-09-15T00:11:31.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498435+0000", 2024-09-15T00:11:31.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1330000000000004e-06 2024-09-15T00:11:31.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498444+0000", 2024-09-15T00:11:31.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6749999999999997e-06 2024-09-15T00:11:31.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498449+0000", 2024-09-15T00:11:31.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5449999999999997e-06 2024-09-15T00:11:31.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498485+0000", 2024-09-15T00:11:31.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5737e-05 2024-09-15T00:11:31.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498501+0000", 2024-09-15T00:11:31.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6637e-05 2024-09-15T00:11:31.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505334+0000", 2024-09-15T00:11:31.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0068333100000000004 2024-09-15T00:11:31.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505353+0000", 2024-09-15T00:11:31.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8785999999999999e-05 2024-09-15T00:11:31.552 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.552 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.553 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.553 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3039 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'88, mlcod=33'83)", 2024-09-15T00:11:31.553 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:31.553 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.0359733109999998, 2024-09-15T00:11:31.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0061577309999999996, 2024-09-15T00:11:31.553 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.553 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.553 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.553 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:31.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:31.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:31.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3099999999999997e-07 2024-09-15T00:11:31.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499224+0000", 2024-09-15T00:11:31.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.916e-06 2024-09-15T00:11:31.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499233+0000", 2024-09-15T00:11:31.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7949999999999996e-06 2024-09-15T00:11:31.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499239+0000", 2024-09-15T00:11:31.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0090000000000004e-06 2024-09-15T00:11:31.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499292+0000", 2024-09-15T00:11:31.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3049e-05 2024-09-15T00:11:31.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499312+0000", 2024-09-15T00:11:31.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0574e-05 2024-09-15T00:11:31.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505355+0000", 2024-09-15T00:11:31.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060423930000000001 2024-09-15T00:11:31.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505376+0000", 2024-09-15T00:11:31.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1364000000000001e-05 2024-09-15T00:11:31.557 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.557 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.557 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.557 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3040 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'89, mlcod=33'83)", 2024-09-15T00:11:31.557 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.499752+0000", 2024-09-15T00:11:31.557 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.0354395200000002, 2024-09-15T00:11:31.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0059059300000000002, 2024-09-15T00:11:31.558 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.558 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.558 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.558 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499752+0000", 2024-09-15T00:11:31.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499752+0000", 2024-09-15T00:11:31.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499753+0000", 2024-09-15T00:11:31.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1399999999999997e-07 2024-09-15T00:11:31.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499763+0000", 2024-09-15T00:11:31.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0365e-05 2024-09-15T00:11:31.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499774+0000", 2024-09-15T00:11:31.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0920999999999999e-05 2024-09-15T00:11:31.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499780+0000", 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.02e-06 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499829+0000", 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9066000000000001e-05 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499849+0000", 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9596999999999999e-05 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505633+0000", 2024-09-15T00:11:31.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057841189999999999 2024-09-15T00:11:31.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505658+0000", 2024-09-15T00:11:31.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5228e-05 2024-09-15T00:11:31.561 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.561 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.561 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.561 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3041 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'90, mlcod=33'83)", 2024-09-15T00:11:31.562 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.500417+0000", 2024-09-15T00:11:31.562 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.034774707, 2024-09-15T00:11:31.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052558600000000002, 2024-09-15T00:11:31.562 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.562 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.562 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.562 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500417+0000", 2024-09-15T00:11:31.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500417+0000", 2024-09-15T00:11:31.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500418+0000", 2024-09-15T00:11:31.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6199999999999998e-07 2024-09-15T00:11:31.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500422+0000", 2024-09-15T00:11:31.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1169999999999997e-06 2024-09-15T00:11:31.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500429+0000", 2024-09-15T00:11:31.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6620000000000005e-06 2024-09-15T00:11:31.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500435+0000", 2024-09-15T00:11:31.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6130000000000003e-06 2024-09-15T00:11:31.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500471+0000", 2024-09-15T00:11:31.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5899999999999998e-05 2024-09-15T00:11:31.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500484+0000", 2024-09-15T00:11:31.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3502999999999999e-05 2024-09-15T00:11:31.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505660+0000", 2024-09-15T00:11:31.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051752930000000001 2024-09-15T00:11:31.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505673+0000", 2024-09-15T00:11:31.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3210000000000001e-05 2024-09-15T00:11:31.565 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.566 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.566 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.566 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3102 3.a 3:566166ea:::rbd_data.1139b531b717.000000000000000b:head [stat,truncate 385024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:31.566 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:31.566 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.9653321190000002, 2024-09-15T00:11:31.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060108410000000003, 2024-09-15T00:11:31.566 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.566 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.566 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.566 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.566 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.566 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.567 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3102 2024-09-15T00:11:31.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.568 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:31.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:31.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:31.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:11:31.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569863+0000", 2024-09-15T00:11:31.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.819e-06 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569871+0000", 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6690000000000006e-06 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569882+0000", 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1195999999999999e-05 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.570647+0000", 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00076547900000000001 2024-09-15T00:11:31.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.570734+0000", 2024-09-15T00:11:31.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7087000000000002e-05 2024-09-15T00:11:31.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:31.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.570863+0000", 2024-09-15T00:11:31.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012899799999999999 2024-09-15T00:11:31.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.573483+0000", 2024-09-15T00:11:31.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026192799999999999 2024-09-15T00:11:31.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.575796+0000", 2024-09-15T00:11:31.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0023136570000000002 2024-09-15T00:11:31.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.575827+0000", 2024-09-15T00:11:31.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0468e-05 2024-09-15T00:11:31.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.575871+0000", 2024-09-15T00:11:31.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3656999999999999e-05 2024-09-15T00:11:31.573 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.573 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.573 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.573 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3152 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'108, mlcod=33'104)", 2024-09-15T00:11:31.573 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.619166+0000", 2024-09-15T00:11:31.573 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.9160259040000001, 2024-09-15T00:11:31.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049522180000000004, 2024-09-15T00:11:31.573 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.573 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.573 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.574 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619166+0000", 2024-09-15T00:11:31.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619166+0000", 2024-09-15T00:11:31.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619166+0000", 2024-09-15T00:11:31.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:11:31.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619170+0000", 2024-09-15T00:11:31.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2990000000000001e-06 2024-09-15T00:11:31.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619177+0000", 2024-09-15T00:11:31.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7260000000000002e-06 2024-09-15T00:11:31.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619181+0000", 2024-09-15T00:11:31.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9759999999999997e-06 2024-09-15T00:11:31.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619215+0000", 2024-09-15T00:11:31.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4066000000000002e-05 2024-09-15T00:11:31.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619230+0000", 2024-09-15T00:11:31.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4392999999999999e-05 2024-09-15T00:11:31.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624081+0000", 2024-09-15T00:11:31.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048513560000000002 2024-09-15T00:11:31.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624118+0000", 2024-09-15T00:11:31.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6973e-05 2024-09-15T00:11:31.577 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.577 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.577 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.578 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3227 3.a 3:53de29a7:::rbd_data.1139b531b717.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:31.578 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.701025+0000", 2024-09-15T00:11:31.578 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.834166832, 2024-09-15T00:11:31.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0046101520000000002, 2024-09-15T00:11:31.578 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.578 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.578 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.578 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.578 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.578 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.578 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3227 2024-09-15T00:11:31.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.578 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701025+0000", 2024-09-15T00:11:31.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701025+0000", 2024-09-15T00:11:31.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701027+0000", 2024-09-15T00:11:31.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.668e-06 2024-09-15T00:11:31.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701034+0000", 2024-09-15T00:11:31.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0899999999999999e-06 2024-09-15T00:11:31.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701051+0000", 2024-09-15T00:11:31.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7377999999999999e-05 2024-09-15T00:11:31.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701060+0000", 2024-09-15T00:11:31.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1109999999999997e-06 2024-09-15T00:11:31.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701132+0000", 2024-09-15T00:11:31.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1401000000000003e-05 2024-09-15T00:11:31.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701226+0000", 2024-09-15T00:11:31.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4758000000000003e-05 2024-09-15T00:11:31.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:31.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701688+0000", 2024-09-15T00:11:31.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00046189899999999999 2024-09-15T00:11:31.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.703775+0000", 2024-09-15T00:11:31.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020864210000000002 2024-09-15T00:11:31.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705553+0000", 2024-09-15T00:11:31.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001777874 2024-09-15T00:11:31.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705590+0000", 2024-09-15T00:11:31.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7512000000000003e-05 2024-09-15T00:11:31.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705635+0000", 2024-09-15T00:11:31.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5040000000000002e-05 2024-09-15T00:11:31.583 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.583 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.583 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.583 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3379 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'162, mlcod=33'158)", 2024-09-15T00:11:31.583 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.833370+0000", 2024-09-15T00:11:31.583 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.7018215570000002, 2024-09-15T00:11:31.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049772030000000004, 2024-09-15T00:11:31.583 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.584 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.584 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.584 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833370+0000", 2024-09-15T00:11:31.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833370+0000", 2024-09-15T00:11:31.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833371+0000", 2024-09-15T00:11:31.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.23e-07 2024-09-15T00:11:31.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833375+0000", 2024-09-15T00:11:31.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0389999999999998e-06 2024-09-15T00:11:31.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833384+0000", 2024-09-15T00:11:31.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0110000000000006e-06 2024-09-15T00:11:31.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833388+0000", 2024-09-15T00:11:31.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2490000000000003e-06 2024-09-15T00:11:31.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833450+0000", 2024-09-15T00:11:31.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1849999999999999e-05 2024-09-15T00:11:31.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833469+0000", 2024-09-15T00:11:31.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9222999999999999e-05 2024-09-15T00:11:31.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.838319+0000", 2024-09-15T00:11:31.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048498860000000003 2024-09-15T00:11:31.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.838347+0000", 2024-09-15T00:11:31.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8221999999999998e-05 2024-09-15T00:11:31.587 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.587 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.587 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.588 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3451 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'173, mlcod=33'168)", 2024-09-15T00:11:31.588 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.896135+0000", 2024-09-15T00:11:31.588 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.6390563440000001, 2024-09-15T00:11:31.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045560890000000001, 2024-09-15T00:11:31.588 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.588 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.588 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.588 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.896135+0000", 2024-09-15T00:11:31.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.896135+0000", 2024-09-15T00:11:31.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.896136+0000", 2024-09-15T00:11:31.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:11:31.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.896140+0000", 2024-09-15T00:11:31.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8120000000000001e-06 2024-09-15T00:11:31.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.896147+0000", 2024-09-15T00:11:31.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6410000000000002e-06 2024-09-15T00:11:31.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.896153+0000", 2024-09-15T00:11:31.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6389999999999997e-06 2024-09-15T00:11:31.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.896216+0000", 2024-09-15T00:11:31.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2650000000000005e-05 2024-09-15T00:11:31.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.896227+0000", 2024-09-15T00:11:31.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1569999999999999e-05 2024-09-15T00:11:31.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.900653+0000", 2024-09-15T00:11:31.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0044257109999999997 2024-09-15T00:11:31.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.900692+0000", 2024-09-15T00:11:31.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8705999999999998e-05 2024-09-15T00:11:31.592 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.592 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.592 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.592 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3502 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'181, mlcod=33'177)", 2024-09-15T00:11:31.592 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:31.592 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.586876953, 2024-09-15T00:11:31.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0067634649999999998, 2024-09-15T00:11:31.592 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.593 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.593 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.593 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:31.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:31.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5199999999999997e-07 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948319+0000", 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0740000000000003e-06 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948329+0000", 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1409999999999994e-06 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948335+0000", 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6290000000000004e-06 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948398+0000", 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2422999999999995e-05 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948412+0000", 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4538999999999999e-05 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.954988+0000", 2024-09-15T00:11:31.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.006575455 2024-09-15T00:11:31.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955078+0000", 2024-09-15T00:11:31.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0651999999999999e-05 2024-09-15T00:11:31.596 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.596 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.596 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.596 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3503 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'182, mlcod=33'177)", 2024-09-15T00:11:31.596 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.948857+0000", 2024-09-15T00:11:31.596 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.586334358, 2024-09-15T00:11:31.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0062792079999999997, 2024-09-15T00:11:31.597 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.597 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.597 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.597 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948857+0000", 2024-09-15T00:11:31.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948857+0000", 2024-09-15T00:11:31.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948858+0000", 2024-09-15T00:11:31.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6899999999999997e-07 2024-09-15T00:11:31.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948862+0000", 2024-09-15T00:11:31.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2429999999999999e-06 2024-09-15T00:11:31.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948872+0000", 2024-09-15T00:11:31.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5340000000000005e-06 2024-09-15T00:11:31.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948883+0000", 2024-09-15T00:11:31.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1304999999999999e-05 2024-09-15T00:11:31.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948917+0000", 2024-09-15T00:11:31.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3510999999999998e-05 2024-09-15T00:11:31.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948929+0000", 2024-09-15T00:11:31.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2473000000000001e-05 2024-09-15T00:11:31.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955082+0000", 2024-09-15T00:11:31.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0061525010000000003 2024-09-15T00:11:31.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955137+0000", 2024-09-15T00:11:31.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5071999999999997e-05 2024-09-15T00:11:31.600 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.600 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.600 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.601 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3504 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'183, mlcod=33'178)", 2024-09-15T00:11:31.601 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:31.601 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5855789810000001, 2024-09-15T00:11:31.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058979369999999998, 2024-09-15T00:11:31.601 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.601 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.601 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.601 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:31.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:31.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:31.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8400000000000004e-07 2024-09-15T00:11:31.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949620+0000", 2024-09-15T00:11:31.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9739999999999996e-06 2024-09-15T00:11:31.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949630+0000", 2024-09-15T00:11:31.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6399999999999992e-06 2024-09-15T00:11:31.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949635+0000", 2024-09-15T00:11:31.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:11:31.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949688+0000", 2024-09-15T00:11:31.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2667000000000002e-05 2024-09-15T00:11:31.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949701+0000", 2024-09-15T00:11:31.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.36e-05 2024-09-15T00:11:31.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955477+0000", 2024-09-15T00:11:31.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057763119999999996 2024-09-15T00:11:31.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955511+0000", 2024-09-15T00:11:31.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3337000000000003e-05 2024-09-15T00:11:31.605 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.605 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.605 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.605 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3505 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'184, mlcod=33'178)", 2024-09-15T00:11:31.605 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.950258+0000", 2024-09-15T00:11:31.605 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.58493364, 2024-09-15T00:11:31.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052830350000000002, 2024-09-15T00:11:31.605 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.605 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.605 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.605 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950258+0000", 2024-09-15T00:11:31.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950258+0000", 2024-09-15T00:11:31.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950259+0000", 2024-09-15T00:11:31.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7700000000000004e-07 2024-09-15T00:11:31.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950262+0000", 2024-09-15T00:11:31.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6839999999999998e-06 2024-09-15T00:11:31.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950271+0000", 2024-09-15T00:11:31.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8249999999999994e-06 2024-09-15T00:11:31.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950276+0000", 2024-09-15T00:11:31.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7600000000000002e-06 2024-09-15T00:11:31.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950311+0000", 2024-09-15T00:11:31.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4629999999999999e-05 2024-09-15T00:11:31.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950323+0000", 2024-09-15T00:11:31.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1853e-05 2024-09-15T00:11:31.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955513+0000", 2024-09-15T00:11:31.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051903599999999998 2024-09-15T00:11:31.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955541+0000", 2024-09-15T00:11:31.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8345999999999999e-05 2024-09-15T00:11:31.609 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.609 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.609 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.609 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3506 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'185, mlcod=33'178)", 2024-09-15T00:11:31.609 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.951011+0000", 2024-09-15T00:11:31.609 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5841810670000001, 2024-09-15T00:11:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045575379999999999, 2024-09-15T00:11:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951011+0000", 2024-09-15T00:11:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951011+0000", 2024-09-15T00:11:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951011+0000", 2024-09-15T00:11:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.58e-07 2024-09-15T00:11:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951015+0000", 2024-09-15T00:11:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6009999999999999e-06 2024-09-15T00:11:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951021+0000", 2024-09-15T00:11:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6699999999999997e-06 2024-09-15T00:11:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951025+0000", 2024-09-15T00:11:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.568e-06 2024-09-15T00:11:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951061+0000", 2024-09-15T00:11:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5992999999999999e-05 2024-09-15T00:11:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951073+0000", 2024-09-15T00:11:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2442e-05 2024-09-15T00:11:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955542+0000", 2024-09-15T00:11:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0044688189999999997 2024-09-15T00:11:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955568+0000", 2024-09-15T00:11:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5987e-05 2024-09-15T00:11:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3724 3.1e e35/23 3:792e48de:::rbd_object_map.1139b531b717:head v 35'223, mlcod=35'214)", 2024-09-15T00:11:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.097406+0000", 2024-09-15T00:11:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.43778605399999998, 2024-09-15T00:11:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050324560000000003, 2024-09-15T00:11:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097406+0000", 2024-09-15T00:11:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097406+0000", 2024-09-15T00:11:31.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097406+0000", 2024-09-15T00:11:31.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5299999999999999e-07 2024-09-15T00:11:31.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097409+0000", 2024-09-15T00:11:31.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2399999999999999e-06 2024-09-15T00:11:31.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097419+0000", 2024-09-15T00:11:31.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7920000000000004e-06 2024-09-15T00:11:31.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097425+0000", 2024-09-15T00:11:31.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7610000000000004e-06 2024-09-15T00:11:31.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097460+0000", 2024-09-15T00:11:31.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4997999999999999e-05 2024-09-15T00:11:31.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097472+0000", 2024-09-15T00:11:31.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1627000000000001e-05 2024-09-15T00:11:31.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102401+0000", 2024-09-15T00:11:31.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049290480000000001 2024-09-15T00:11:31.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102438+0000", 2024-09-15T00:11:31.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7537000000000003e-05 2024-09-15T00:11:31.618 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.618 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.618 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.618 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3726 3.1e e35/23 3:792e48de:::rbd_object_map.1139b531b717:head v 35'225, mlcod=35'220)", 2024-09-15T00:11:31.618 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.099757+0000", 2024-09-15T00:11:31.618 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.43543464900000001, 2024-09-15T00:11:31.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00535652, 2024-09-15T00:11:31.618 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.618 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.618 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.618 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099757+0000", 2024-09-15T00:11:31.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099757+0000", 2024-09-15T00:11:31.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099758+0000", 2024-09-15T00:11:31.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7400000000000002e-07 2024-09-15T00:11:31.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099764+0000", 2024-09-15T00:11:31.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0469999999999999e-06 2024-09-15T00:11:31.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099774+0000", 2024-09-15T00:11:31.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0118999999999999e-05 2024-09-15T00:11:31.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099781+0000", 2024-09-15T00:11:31.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7730000000000002e-06 2024-09-15T00:11:31.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099826+0000", 2024-09-15T00:11:31.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5229999999999999e-05 2024-09-15T00:11:31.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099844+0000", 2024-09-15T00:11:31.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8345000000000001e-05 2024-09-15T00:11:31.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.105063+0000", 2024-09-15T00:11:31.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052188920000000001 2024-09-15T00:11:31.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.105114+0000", 2024-09-15T00:11:31.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0340000000000003e-05 2024-09-15T00:11:31.622 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.622 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.622 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.622 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3737 3.a 3:53de29a7:::rbd_data.1139b531b717.000000000000000a:head [stat,truncate 270336] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:31.622 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.100151+0000", 2024-09-15T00:11:31.622 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.43504083599999999, 2024-09-15T00:11:31.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076797300000000001, 2024-09-15T00:11:31.623 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.623 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.623 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.623 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.623 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.623 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.623 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3737 2024-09-15T00:11:31.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.623 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100151+0000", 2024-09-15T00:11:31.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100151+0000", 2024-09-15T00:11:31.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100153+0000", 2024-09-15T00:11:31.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6530000000000001e-06 2024-09-15T00:11:31.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100158+0000", 2024-09-15T00:11:31.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7100000000000004e-06 2024-09-15T00:11:31.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100173+0000", 2024-09-15T00:11:31.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4242e-05 2024-09-15T00:11:31.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100180+0000", 2024-09-15T00:11:31.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7680000000000008e-06 2024-09-15T00:11:31.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100248+0000", 2024-09-15T00:11:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7459000000000006e-05 2024-09-15T00:11:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100321+0000", 2024-09-15T00:11:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2854000000000004e-05 2024-09-15T00:11:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100418+0000", 2024-09-15T00:11:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7827999999999999e-05 2024-09-15T00:11:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.105090+0000", 2024-09-15T00:11:31.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0046714349999999998 2024-09-15T00:11:31.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.107770+0000", 2024-09-15T00:11:31.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026799660000000002 2024-09-15T00:11:31.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.107801+0000", 2024-09-15T00:11:31.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1158000000000002e-05 2024-09-15T00:11:31.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.107831+0000", 2024-09-15T00:11:31.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9657000000000001e-05 2024-09-15T00:11:31.628 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.628 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.628 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.628 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3738 3.a 3:566166ea:::rbd_data.1139b531b717.000000000000000b:head [stat,truncate 270336] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:31.628 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.102790+0000", 2024-09-15T00:11:31.628 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.43240141199999999, 2024-09-15T00:11:31.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053085249999999997, 2024-09-15T00:11:31.629 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:31.629 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:31.629 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:31.629 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:31.629 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:31.629 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:31.629 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3738 2024-09-15T00:11:31.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.629 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:31.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:31.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102790+0000", 2024-09-15T00:11:31.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:31.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102790+0000", 2024-09-15T00:11:31.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:31.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:31.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102791+0000", 2024-09-15T00:11:31.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1900000000000001e-07 2024-09-15T00:11:31.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:31.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102796+0000", 2024-09-15T00:11:31.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5560000000000001e-06 2024-09-15T00:11:31.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:31.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102809+0000", 2024-09-15T00:11:31.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3213e-05 2024-09-15T00:11:31.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:31.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102817+0000", 2024-09-15T00:11:31.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6090000000000003e-06 2024-09-15T00:11:31.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:31.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102859+0000", 2024-09-15T00:11:31.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2536000000000001e-05 2024-09-15T00:11:31.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:31.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102933+0000", 2024-09-15T00:11:31.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3960000000000003e-05 2024-09-15T00:11:31.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:31.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.103108+0000", 2024-09-15T00:11:31.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00017522800000000001 2024-09-15T00:11:31.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:31.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.105349+0000", 2024-09-15T00:11:31.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002240377 2024-09-15T00:11:31.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:31.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.108036+0000", 2024-09-15T00:11:31.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002686964 2024-09-15T00:11:31.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:31.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.108066+0000", 2024-09-15T00:11:31.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9779999999999999e-05 2024-09-15T00:11:31.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:31.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:31.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:31.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.108099+0000", 2024-09-15T00:11:31.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3383000000000002e-05 2024-09-15T00:11:31.634 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.634 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.634 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.634 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:31.634 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:31.634 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:31.635 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:11:31.636 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:11:31.637 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:31.633+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:31.736 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:31.733+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:31.755 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:31.756 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:31.772 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:31.772 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:31.781 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:31.781 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:31.781 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:31.781 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:31.787 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:31.787 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:31.790 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:11:31.803 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:31.803 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:31.820 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:31.820 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:31.836 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:31.834+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:31.893 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:31.893 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:31.893 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:31.893 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:31.894 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:31.902 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:11:31.937 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:31.935+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:32.022 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:11:32.041 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:32.042 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:32.042 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:32.039+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:32.053 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:05.534145+0000", 2024-09-15T00:11:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 26.516317325999999, 2024-09-15T00:11:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0080234130000000001, 2024-09-15T00:11:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 70 2024-09-15T00:11:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.534145+0000", 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.534145+0000", 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.534146+0000", 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7e-07 2024-09-15T00:11:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.534148+0000", 2024-09-15T00:11:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2879999999999999e-06 2024-09-15T00:11:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.534151+0000", 2024-09-15T00:11:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6669999999999998e-06 2024-09-15T00:11:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.534155+0000", 2024-09-15T00:11:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1329999999999999e-06 2024-09-15T00:11:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.535886+0000", 2024-09-15T00:11:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001731607 2024-09-15T00:11:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.535936+0000", 2024-09-15T00:11:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9395999999999999e-05 2024-09-15T00:11:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.536023+0000", 2024-09-15T00:11:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7540000000000006e-05 2024-09-15T00:11:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.537864+0000", 2024-09-15T00:11:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018404140000000001 2024-09-15T00:11:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.542072+0000", 2024-09-15T00:11:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042080069999999997 2024-09-15T00:11:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.542124+0000", 2024-09-15T00:11:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2244000000000003e-05 2024-09-15T00:11:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:05.542169+0000", 2024-09-15T00:11:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4746999999999997e-05 2024-09-15T00:11:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056375+0000", 2024-09-15T00:11:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 25.994087857, 2024-09-15T00:11:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0090326620000000003, 2024-09-15T00:11:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 136 2024-09-15T00:11:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056375+0000", 2024-09-15T00:11:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056375+0000", 2024-09-15T00:11:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056375+0000", 2024-09-15T00:11:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9900000000000001e-07 2024-09-15T00:11:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056379+0000", 2024-09-15T00:11:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7759999999999999e-06 2024-09-15T00:11:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056390+0000", 2024-09-15T00:11:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0720000000000001e-05 2024-09-15T00:11:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056395+0000", 2024-09-15T00:11:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.558e-06 2024-09-15T00:11:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.059634+0000", 2024-09-15T00:11:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032390890000000001 2024-09-15T00:11:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.059682+0000", 2024-09-15T00:11:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7868000000000003e-05 2024-09-15T00:11:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.059761+0000", 2024-09-15T00:11:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8863999999999996e-05 2024-09-15T00:11:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061758+0000", 2024-09-15T00:11:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00199731 2024-09-15T00:11:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065345+0000", 2024-09-15T00:11:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035862989999999998 2024-09-15T00:11:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065383+0000", 2024-09-15T00:11:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8408999999999998e-05 2024-09-15T00:11:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065407+0000", 2024-09-15T00:11:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.427e-05 2024-09-15T00:11:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056408+0000", 2024-09-15T00:11:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 25.994054389999999, 2024-09-15T00:11:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0090848679999999994, 2024-09-15T00:11:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 137 2024-09-15T00:11:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056408+0000", 2024-09-15T00:11:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056408+0000", 2024-09-15T00:11:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056409+0000", 2024-09-15T00:11:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0100000000000005e-07 2024-09-15T00:11:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056478+0000", 2024-09-15T00:11:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9580999999999997e-05 2024-09-15T00:11:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056485+0000", 2024-09-15T00:11:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3459999999999996e-06 2024-09-15T00:11:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056491+0000", 2024-09-15T00:11:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.417e-06 2024-09-15T00:11:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.060060+0000", 2024-09-15T00:11:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003569511 2024-09-15T00:11:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.060108+0000", 2024-09-15T00:11:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7788999999999999e-05 2024-09-15T00:11:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.060180+0000", 2024-09-15T00:11:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2248000000000004e-05 2024-09-15T00:11:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065304+0000", 2024-09-15T00:11:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051239550000000003 2024-09-15T00:11:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065443+0000", 2024-09-15T00:11:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000138632 2024-09-15T00:11:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065472+0000", 2024-09-15T00:11:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8459999999999999e-05 2024-09-15T00:11:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065493+0000", 2024-09-15T00:11:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1427999999999999e-05 2024-09-15T00:11:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056499+0000", 2024-09-15T00:11:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 25.993963642000001, 2024-09-15T00:11:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0090724829999999992, 2024-09-15T00:11:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 138 2024-09-15T00:11:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056499+0000", 2024-09-15T00:11:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056499+0000", 2024-09-15T00:11:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056499+0000", 2024-09-15T00:11:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:11:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056502+0000", 2024-09-15T00:11:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4739999999999999e-06 2024-09-15T00:11:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056506+0000", 2024-09-15T00:11:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2470000000000002e-06 2024-09-15T00:11:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056509+0000", 2024-09-15T00:11:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8729999999999999e-06 2024-09-15T00:11:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061051+0000", 2024-09-15T00:11:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045423410000000001 2024-09-15T00:11:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061105+0000", 2024-09-15T00:11:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4101000000000003e-05 2024-09-15T00:11:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061178+0000", 2024-09-15T00:11:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3104999999999999e-05 2024-09-15T00:11:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065310+0000", 2024-09-15T00:11:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041314990000000003 2024-09-15T00:11:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065525+0000", 2024-09-15T00:11:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000214947 2024-09-15T00:11:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065551+0000", 2024-09-15T00:11:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5955000000000001e-05 2024-09-15T00:11:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065571+0000", 2024-09-15T00:11:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0594e-05 2024-09-15T00:11:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056512+0000", 2024-09-15T00:11:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 25.993950010999999, 2024-09-15T00:11:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.009307727, 2024-09-15T00:11:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 139 2024-09-15T00:11:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056512+0000", 2024-09-15T00:11:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056512+0000", 2024-09-15T00:11:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056513+0000", 2024-09-15T00:11:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5100000000000001e-07 2024-09-15T00:11:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056515+0000", 2024-09-15T00:11:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.26e-06 2024-09-15T00:11:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056519+0000", 2024-09-15T00:11:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0300000000000004e-06 2024-09-15T00:11:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056523+0000", 2024-09-15T00:11:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2379999999999999e-06 2024-09-15T00:11:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061418+0000", 2024-09-15T00:11:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048945630000000002 2024-09-15T00:11:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061474+0000", 2024-09-15T00:11:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6688999999999999e-05 2024-09-15T00:11:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061554+0000", 2024-09-15T00:11:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9810999999999999e-05 2024-09-15T00:11:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065502+0000", 2024-09-15T00:11:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039479720000000001 2024-09-15T00:11:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065776+0000", 2024-09-15T00:11:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00027390599999999998 2024-09-15T00:11:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065801+0000", 2024-09-15T00:11:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5219999999999999e-05 2024-09-15T00:11:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065820+0000", 2024-09-15T00:11:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8787000000000002e-05 2024-09-15T00:11:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 25.993934805999999, 2024-09-15T00:11:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0096064029999999995, 2024-09-15T00:11:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 140 2024-09-15T00:11:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4499999999999998e-07 2024-09-15T00:11:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056583+0000", 2024-09-15T00:11:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5377000000000002e-05 2024-09-15T00:11:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056588+0000", 2024-09-15T00:11:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0209999999999999e-06 2024-09-15T00:11:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056591+0000", 2024-09-15T00:11:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1089999999999998e-06 2024-09-15T00:11:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061790+0000", 2024-09-15T00:11:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051988759999999998 2024-09-15T00:11:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061837+0000", 2024-09-15T00:11:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6233000000000003e-05 2024-09-15T00:11:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061927+0000", 2024-09-15T00:11:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0525999999999994e-05 2024-09-15T00:11:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065584+0000", 2024-09-15T00:11:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036566630000000001 2024-09-15T00:11:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066084+0000", 2024-09-15T00:11:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000500557 2024-09-15T00:11:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066116+0000", 2024-09-15T00:11:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1272000000000001e-05 2024-09-15T00:11:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066134+0000", 2024-09-15T00:11:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8524000000000001e-05 2024-09-15T00:11:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 25.993865423999999, 2024-09-15T00:11:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0096621280000000007, 2024-09-15T00:11:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 141 2024-09-15T00:11:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9299999999999999e-07 2024-09-15T00:11:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056600+0000", 2024-09-15T00:11:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5330000000000001e-06 2024-09-15T00:11:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056606+0000", 2024-09-15T00:11:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5030000000000002e-06 2024-09-15T00:11:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056610+0000", 2024-09-15T00:11:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2279999999999998e-06 2024-09-15T00:11:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.064980+0000", 2024-09-15T00:11:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008370097 2024-09-15T00:11:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065030+0000", 2024-09-15T00:11:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0268000000000001e-05 2024-09-15T00:11:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065102+0000", 2024-09-15T00:11:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2120999999999996e-05 2024-09-15T00:11:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066159+0000", 2024-09-15T00:11:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001056881 2024-09-15T00:11:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066217+0000", 2024-09-15T00:11:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8344999999999998e-05 2024-09-15T00:11:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066241+0000", 2024-09-15T00:11:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3773000000000001e-05 2024-09-15T00:11:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066259+0000", 2024-09-15T00:11:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8085999999999999e-05 2024-09-15T00:11:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:155 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523404+0000", 2024-09-15T00:11:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 25.527058243999999, 2024-09-15T00:11:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092202580000000003, 2024-09-15T00:11:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 155 2024-09-15T00:11:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523404+0000", 2024-09-15T00:11:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523404+0000", 2024-09-15T00:11:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523405+0000", 2024-09-15T00:11:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:11:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523523+0000", 2024-09-15T00:11:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00011842500000000001 2024-09-15T00:11:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523531+0000", 2024-09-15T00:11:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7649999999999993e-06 2024-09-15T00:11:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523544+0000", 2024-09-15T00:11:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3243e-05 2024-09-15T00:11:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.524482+0000", 2024-09-15T00:11:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00093750599999999995 2024-09-15T00:11:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.524527+0000", 2024-09-15T00:11:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5296000000000001e-05 2024-09-15T00:11:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.524600+0000", 2024-09-15T00:11:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3271999999999999e-05 2024-09-15T00:11:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527438+0000", 2024-09-15T00:11:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002837783 2024-09-15T00:11:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532545+0000", 2024-09-15T00:11:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051066690000000003 2024-09-15T00:11:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532590+0000", 2024-09-15T00:11:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4929999999999998e-05 2024-09-15T00:11:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532624+0000", 2024-09-15T00:11:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4838000000000001e-05 2024-09-15T00:11:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523563+0000", 2024-09-15T00:11:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 25.526899526000001, 2024-09-15T00:11:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0091540000000000007, 2024-09-15T00:11:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 156 2024-09-15T00:11:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523563+0000", 2024-09-15T00:11:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523563+0000", 2024-09-15T00:11:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523564+0000", 2024-09-15T00:11:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9800000000000003e-07 2024-09-15T00:11:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523572+0000", 2024-09-15T00:11:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7180000000000002e-06 2024-09-15T00:11:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523580+0000", 2024-09-15T00:11:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9799999999999998e-06 2024-09-15T00:11:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523586+0000", 2024-09-15T00:11:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.251e-06 2024-09-15T00:11:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525453+0000", 2024-09-15T00:11:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001866604 2024-09-15T00:11:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525496+0000", 2024-09-15T00:11:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3222000000000001e-05 2024-09-15T00:11:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525569+0000", 2024-09-15T00:11:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2408000000000002e-05 2024-09-15T00:11:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527444+0000", 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001875294 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532664+0000", 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052197759999999998 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532694+0000", 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0020000000000001e-05 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532717+0000", 2024-09-15T00:11:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3129000000000002e-05 2024-09-15T00:11:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 25.526864566, 2024-09-15T00:11:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092491780000000003, 2024-09-15T00:11:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 157 2024-09-15T00:11:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0500000000000004e-07 2024-09-15T00:11:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523602+0000", 2024-09-15T00:11:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5930000000000002e-06 2024-09-15T00:11:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523609+0000", 2024-09-15T00:11:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9970000000000001e-06 2024-09-15T00:11:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523614+0000", 2024-09-15T00:11:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8640000000000004e-06 2024-09-15T00:11:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525797+0000", 2024-09-15T00:11:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021830500000000002 2024-09-15T00:11:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525880+0000", 2024-09-15T00:11:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3394999999999997e-05 2024-09-15T00:11:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525997+0000", 2024-09-15T00:11:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000116971 2024-09-15T00:11:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527448+0000", 2024-09-15T00:11:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0014509690000000001 2024-09-15T00:11:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532758+0000", 2024-09-15T00:11:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053093319999999999 2024-09-15T00:11:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532804+0000", 2024-09-15T00:11:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5995000000000003e-05 2024-09-15T00:11:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532847+0000", 2024-09-15T00:11:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3507000000000002e-05 2024-09-15T00:11:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523622+0000", 2024-09-15T00:11:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 25.526840091, 2024-09-15T00:11:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0093097089999999993, 2024-09-15T00:11:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 158 2024-09-15T00:11:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523622+0000", 2024-09-15T00:11:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523622+0000", 2024-09-15T00:11:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523623+0000", 2024-09-15T00:11:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5400000000000002e-07 2024-09-15T00:11:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523689+0000", 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6434999999999997e-05 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523698+0000", 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2229999999999996e-06 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523705+0000", 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6969999999999996e-06 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526370+0000", 2024-09-15T00:11:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026647989999999998 2024-09-15T00:11:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526415+0000", 2024-09-15T00:11:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5278999999999998e-05 2024-09-15T00:11:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526484+0000", 2024-09-15T00:11:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8487999999999999e-05 2024-09-15T00:11:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.528098+0000", 2024-09-15T00:11:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0016147480000000001 2024-09-15T00:11:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532887+0000", 2024-09-15T00:11:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047889860000000003 2024-09-15T00:11:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532912+0000", 2024-09-15T00:11:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4694000000000001e-05 2024-09-15T00:11:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532932+0000", 2024-09-15T00:11:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9905999999999999e-05 2024-09-15T00:11:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 25.526731777999998, 2024-09-15T00:11:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0093142290000000003, 2024-09-15T00:11:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 159 2024-09-15T00:11:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6899999999999997e-07 2024-09-15T00:11:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523735+0000", 2024-09-15T00:11:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7359999999999999e-06 2024-09-15T00:11:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523745+0000", 2024-09-15T00:11:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0149000000000001e-05 2024-09-15T00:11:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523752+0000", 2024-09-15T00:11:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7220000000000002e-06 2024-09-15T00:11:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526967+0000", 2024-09-15T00:11:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032149280000000001 2024-09-15T00:11:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527039+0000", 2024-09-15T00:11:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2261000000000005e-05 2024-09-15T00:11:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527148+0000", 2024-09-15T00:11:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000108743 2024-09-15T00:11:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.528988+0000", 2024-09-15T00:11:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001840008 2024-09-15T00:11:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532965+0000", 2024-09-15T00:11:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039768980000000004 2024-09-15T00:11:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.533008+0000", 2024-09-15T00:11:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3673e-05 2024-09-15T00:11:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.533045+0000", 2024-09-15T00:11:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6541999999999997e-05 2024-09-15T00:11:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:11:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.971942919, 2024-09-15T00:11:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:11:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:11:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:11:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:11:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:11:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:11:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:11:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:11:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:11:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:11:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:11:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:11:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:11:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:11:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:11:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:11:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:11:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.9592023770000004, 2024-09-15T00:11:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:11:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:11:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:11:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:11:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:11:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:11:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:11:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:11:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:11:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:11:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:11:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:11:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:11:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:11:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:11:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:11:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:11:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:11:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.8268466180000003, 2024-09-15T00:11:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:11:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:11:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:11:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:11:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:11:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:11:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:11:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:11:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:11:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:11:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:11:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:11:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:11:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:11:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:11:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:11:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:11:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:11:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:11:32.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:11:32.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:11:32.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:11:32.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:11:32.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:11:32.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:11:32.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:11:32.138 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.138 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.138 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.138 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:11:32.138 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:11:32.138 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.8133849729999998, 2024-09-15T00:11:32.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:11:32.138 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.138 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.138 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:32.139 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:11:32.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:11:32.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:11:32.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:11:32.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:11:32.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:11:32.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:11:32.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:11:32.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:11:32.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:11:32.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:11:32.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:11:32.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:11:32.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:11:32.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:11:32.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:11:32.142 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.142 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.142 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.142 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1612 3.e 3:77b26f8c:::rbd_data.11397bff333d.0000000000000023:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:32.142 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.663923+0000", 2024-09-15T00:11:32.142 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3865394530000001, 2024-09-15T00:11:32.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017776304999999999, 2024-09-15T00:11:32.142 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.142 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.142 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.142 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.143 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:32.143 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:32.143 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1612 2024-09-15T00:11:32.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.143 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663923+0000", 2024-09-15T00:11:32.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663923+0000", 2024-09-15T00:11:32.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663925+0000", 2024-09-15T00:11:32.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5290000000000001e-06 2024-09-15T00:11:32.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663934+0000", 2024-09-15T00:11:32.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2040000000000003e-06 2024-09-15T00:11:32.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663948+0000", 2024-09-15T00:11:32.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3978999999999999e-05 2024-09-15T00:11:32.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663954+0000", 2024-09-15T00:11:32.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6059999999999999e-06 2024-09-15T00:11:32.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.664002+0000", 2024-09-15T00:11:32.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8127999999999997e-05 2024-09-15T00:11:32.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.664055+0000", 2024-09-15T00:11:32.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2237999999999997e-05 2024-09-15T00:11:32.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:32.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.665257+0000", 2024-09-15T00:11:32.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0012021989999999999 2024-09-15T00:11:32.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681584+0000", 2024-09-15T00:11:32.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016327264000000001 2024-09-15T00:11:32.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681620+0000", 2024-09-15T00:11:32.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6340000000000001e-05 2024-09-15T00:11:32.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681661+0000", 2024-09-15T00:11:32.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0259000000000002e-05 2024-09-15T00:11:32.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681699+0000", 2024-09-15T00:11:32.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8559000000000001e-05 2024-09-15T00:11:32.148 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.148 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.148 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.148 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1612 3.e e30/23 ondisk, result = 0)", 2024-09-15T00:11:32.148 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.668460+0000", 2024-09-15T00:11:32.148 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3820022239999998, 2024-09-15T00:11:32.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013245329, 2024-09-15T00:11:32.148 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.148 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.148 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:32.149 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668460+0000", 2024-09-15T00:11:32.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668460+0000", 2024-09-15T00:11:32.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668461+0000", 2024-09-15T00:11:32.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3600000000000002e-07 2024-09-15T00:11:32.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668467+0000", 2024-09-15T00:11:32.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6559999999999997e-06 2024-09-15T00:11:32.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668477+0000", 2024-09-15T00:11:32.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0539e-05 2024-09-15T00:11:32.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668483+0000", 2024-09-15T00:11:32.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1580000000000002e-06 2024-09-15T00:11:32.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681608+0000", 2024-09-15T00:11:32.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013125009 2024-09-15T00:11:32.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681616+0000", 2024-09-15T00:11:32.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2880000000000004e-06 2024-09-15T00:11:32.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681706+0000", 2024-09-15T00:11:32.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9843e-05 2024-09-15T00:11:32.152 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.152 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.152 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.152 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3226 3.d 3:b1632ea2:::rbd_data.1139b531b717.0000000000000009:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:32.152 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.701042+0000", 2024-09-15T00:11:32.152 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.3494208950000002, 2024-09-15T00:11:32.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016783762000000001, 2024-09-15T00:11:32.152 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.152 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.153 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:32.153 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:32.153 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:32.153 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:32.153 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3226 2024-09-15T00:11:32.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.153 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701042+0000", 2024-09-15T00:11:32.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701042+0000", 2024-09-15T00:11:32.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701042+0000", 2024-09-15T00:11:32.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:11:32.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701045+0000", 2024-09-15T00:11:32.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1379999999999999e-06 2024-09-15T00:11:32.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701053+0000", 2024-09-15T00:11:32.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6350000000000006e-06 2024-09-15T00:11:32.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701058+0000", 2024-09-15T00:11:32.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3650000000000003e-06 2024-09-15T00:11:32.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701097+0000", 2024-09-15T00:11:32.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8769000000000001e-05 2024-09-15T00:11:32.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701164+0000", 2024-09-15T00:11:32.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7143000000000004e-05 2024-09-15T00:11:32.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:32.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.702192+0000", 2024-09-15T00:11:32.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00102808 2024-09-15T00:11:32.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:32.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717734+0000", 2024-09-15T00:11:32.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015541806 2024-09-15T00:11:32.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:32.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717748+0000", 2024-09-15T00:11:32.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3893999999999999e-05 2024-09-15T00:11:32.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:32.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717786+0000", 2024-09-15T00:11:32.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7996e-05 2024-09-15T00:11:32.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717825+0000", 2024-09-15T00:11:32.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9397000000000002e-05 2024-09-15T00:11:32.158 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.158 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.158 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.158 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:3226 3.d e33/23 ondisk, result = 0)", 2024-09-15T00:11:32.158 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.705838+0000", 2024-09-15T00:11:32.158 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.3446241400000001, 2024-09-15T00:11:32.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011902983000000001, 2024-09-15T00:11:32.158 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:32.158 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:32.159 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:32.159 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:32.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:32.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705838+0000", 2024-09-15T00:11:32.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:32.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705838+0000", 2024-09-15T00:11:32.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:32.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:32.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705839+0000", 2024-09-15T00:11:32.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9099999999999991e-07 2024-09-15T00:11:32.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:32.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705846+0000", 2024-09-15T00:11:32.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.251e-06 2024-09-15T00:11:32.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:32.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705860+0000", 2024-09-15T00:11:32.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4849e-05 2024-09-15T00:11:32.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:32.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705868+0000", 2024-09-15T00:11:32.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6629999999999993e-06 2024-09-15T00:11:32.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:32.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717669+0000", 2024-09-15T00:11:32.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011800966 2024-09-15T00:11:32.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:32.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717681+0000", 2024-09-15T00:11:32.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2201000000000001e-05 2024-09-15T00:11:32.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:32.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:32.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:32.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717741+0000", 2024-09-15T00:11:32.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0062000000000002e-05 2024-09-15T00:11:32.162 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.162 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.162 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.162 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:32.162 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:32.162 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:32.163 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:32.163 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:32.164 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:32.164 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:32.160+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:32.165 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:32.166 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:32.166 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone5 from 0xcd4a5c5 (overlap 0x11c7c14) to 0x0 2024-09-15T00:11:32.166 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-15T00:11:32.166 INFO:teuthology.orchestra.run.smithi171.stdout:707 write 0x7d40e76 thru 0x7d43db8 (0x2f43 bytes) 2024-09-15T00:11:32.189 INFO:teuthology.orchestra.run.smithi171.stdout:709 trunc from 0xcd4a5c5 to 0xb45452e 2024-09-15T00:11:32.264 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:32.261+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:32.266 INFO:teuthology.orchestra.run.smithi171.stdout:710 write 0x7f171fe thru 0x7f2610d (0xef10 bytes) 2024-09-15T00:11:32.310 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:32.310 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:32.320 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:11:32.332 INFO:teuthology.orchestra.run.smithi171.stdout:711 read 0x73c7cad thru 0x73d4424 (0xc778 bytes) 2024-09-15T00:11:32.332 INFO:teuthology.orchestra.run.smithi171.stdout:713 read 0x820150c thru 0x820fb84 (0xe679 bytes) 2024-09-15T00:11:32.333 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:32.333 INFO:teuthology.orchestra.run.smithi171.stdout:715 punch from 0x1e54644 to 0x1e62dba, (0xe776 bytes) 2024-09-15T00:11:32.341 INFO:teuthology.orchestra.run.smithi171.stdout:717 write 0xdfb5049 thru 0xdfbf396 (0xa34e bytes) 2024-09-15T00:11:32.350 INFO:teuthology.orchestra.run.smithi171.stdout:719 write 0xd78692b thru 0xd78c0bd (0x5793 bytes) 2024-09-15T00:11:32.356 INFO:teuthology.orchestra.run.smithi171.stdout:723 punch from 0xab92bc7 to 0xab94160, (0x1599 bytes) 2024-09-15T00:11:32.356 INFO:teuthology.orchestra.run.smithi171.stdout:724 punch from 0x5d4d0e1 to 0x5d4e442, (0x1361 bytes) 2024-09-15T00:11:32.365 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:32.362+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:32.370 INFO:teuthology.orchestra.run.smithi171.stdout:726 trunc from 0xdfbf397 to 0x2bb8582 2024-09-15T00:11:32.465 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:32.463+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:32.475 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:32.475 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:32.487 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:11:32.511 INFO:teuthology.orchestra.run.smithi171.stdout:727 write 0xb60e7bc thru 0xb61d0f6 (0xe93b bytes) 2024-09-15T00:11:32.521 INFO:teuthology.orchestra.run.smithi171.stdout:728 write 0x5d8b21a thru 0x5d8e9cd (0x37b4 bytes) 2024-09-15T00:11:32.529 INFO:teuthology.orchestra.run.smithi171.stdout:729 read 0x831679e thru 0x831ed84 (0x85e7 bytes) 2024-09-15T00:11:32.529 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:32.540 INFO:teuthology.orchestra.run.smithi171.stdout:733 trunc from 0xb61d0f7 to 0x20efcc3 2024-09-15T00:11:32.561 INFO:teuthology.orchestra.run.smithi171.stdout:734 read 0x1599c39 thru 0x15a41d1 (0xa599 bytes) 2024-09-15T00:11:32.562 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:32.563 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:32.566 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:32.563+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:32.585 INFO:teuthology.orchestra.run.smithi171.stdout:737 write 0xea12bdc thru 0xea1947c (0x68a1 bytes) 2024-09-15T00:11:32.593 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:32.593 INFO:teuthology.orchestra.run.smithi171.stdout:742 read 0x61e38ba thru 0x61f1eb1 (0xe5f8 bytes) 2024-09-15T00:11:32.593 INFO:teuthology.orchestra.run.smithi171.stdout:743 read 0x358874 thru 0x35e72a (0x5eb7 bytes) 2024-09-15T00:11:32.594 INFO:teuthology.orchestra.run.smithi171.stdout:744 read 0xbe2a5a4 thru 0xbe2e395 (0x3df2 bytes) 2024-09-15T00:11:32.594 INFO:teuthology.orchestra.run.smithi171.stdout:745 write 0x6de2449 thru 0x6ded167 (0xad1f bytes) 2024-09-15T00:11:32.605 INFO:teuthology.orchestra.run.smithi171.stdout:746 read 0x57693b7 thru 0x5770d01 (0x794b bytes) 2024-09-15T00:11:32.608 INFO:teuthology.orchestra.run.smithi171.stdout:747 trunc from 0xea1947d to 0xce21865 2024-09-15T00:11:32.618 INFO:teuthology.orchestra.run.smithi171.stdout:748 punch from 0xb89092d to 0xb890de4, (0x4b7 bytes) 2024-09-15T00:11:32.619 INFO:teuthology.orchestra.run.smithi171.stdout:749 read 0xb6ae4ce thru 0xb6bb3dc (0xcf0f bytes) 2024-09-15T00:11:32.619 INFO:teuthology.orchestra.run.smithi171.stdout:750 read 0x3f13e28 thru 0x3f2037e (0xc557 bytes) 2024-09-15T00:11:32.619 INFO:teuthology.orchestra.run.smithi171.stdout:751 write 0xabf5bed thru 0xac027aa (0xcbbe bytes) 2024-09-15T00:11:32.632 INFO:teuthology.orchestra.run.smithi171.stdout:753 punch from 0x9f69d52 to 0x9f6ec4f, (0x4efd bytes) 2024-09-15T00:11:32.632 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:32.637 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:32.637 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:32.642 INFO:teuthology.orchestra.run.smithi171.stdout:755 trunc from 0xce21865 to 0x411492f 2024-09-15T00:11:32.650 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:32.658 INFO:teuthology.orchestra.run.smithi171.stdout:756 punch from 0x2397cf8 to 0x2399451, (0x1759 bytes) 2024-09-15T00:11:32.658 INFO:teuthology.orchestra.run.smithi171.stdout:757 read 0x3380b46 thru 0x3388561 (0x7a1c bytes) 2024-09-15T00:11:32.658 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:32.658 INFO:teuthology.orchestra.run.smithi171.stdout:760 trunc from 0x411492f to 0x3fd9713 2024-09-15T00:11:32.662 INFO:teuthology.orchestra.run.smithi171.stdout:761 write 0x19bc667 thru 0x19cbebe (0xf858 bytes) 2024-09-15T00:11:32.666 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:32.664+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:32.667 INFO:teuthology.orchestra.run.smithi171.stdout:763 punch from 0x30a9749 to 0x30b1d45, (0x85fc bytes) 2024-09-15T00:11:32.667 INFO:teuthology.orchestra.run.smithi171.stdout:764 punch from 0xf867db to 0xf8e1bd, (0x79e2 bytes) 2024-09-15T00:11:32.667 INFO:teuthology.orchestra.run.smithi171.stdout:765 clone 7 order 23 su 131072 sc 10 2024-09-15T00:11:32.762 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:32.762 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:32.767 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:32.765+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:32.771 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:32.868 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:32.866+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:32.915 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:32.915 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:32.923 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:32.969 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:32.966+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:33.027 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:33.027 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:33.030 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is set 2024-09-15T00:11:33.039 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:33.088 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:33.086+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:33.105 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone6 from 0x3fd9713 (overlap 0x20efcc3) to 0x9e18a0 2024-09-15T00:11:33.170 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:33.168+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:33.241 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:33.242 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:33.253 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:33.272 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:33.270+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:33.323 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-15T00:11:33.339 INFO:teuthology.orchestra.run.smithi171.stdout:766 write 0x96230bb thru 0x96251f7 (0x213d bytes) 2024-09-15T00:11:33.342 INFO:teuthology.orchestra.run.smithi171.stdout:767 punch from 0x3864149 to 0x386862d, (0x44e4 bytes) 2024-09-15T00:11:33.342 INFO:teuthology.orchestra.run.smithi171.stdout:768 write 0x66fbec1 thru 0x6702290 (0x63d0 bytes) 2024-09-15T00:11:33.346 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:33.346 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:33.346 INFO:teuthology.orchestra.run.smithi171.stdout:771 read 0x8ae40f5 thru 0x8ae9c47 (0x5b53 bytes) 2024-09-15T00:11:33.346 INFO:teuthology.orchestra.run.smithi171.stdout:772 read 0x4fdeaeb thru 0x4fead4b (0xc261 bytes) 2024-09-15T00:11:33.346 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:33.355 INFO:teuthology.orchestra.run.smithi171.stdout:775 write 0xc8a9502 thru 0xc8b5edc (0xc9db bytes) 2024-09-15T00:11:33.359 INFO:teuthology.orchestra.run.smithi171.stdout:776 write 0x2ba17f1 thru 0x2ba3e9d (0x26ad bytes) 2024-09-15T00:11:33.373 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:33.370+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:33.380 INFO:teuthology.orchestra.run.smithi171.stdout:778 write 0x3cac6dc thru 0x3cb6fae (0xa8d3 bytes) 2024-09-15T00:11:33.417 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:33.417 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:33.428 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:33.429 INFO:teuthology.orchestra.run.smithi171.stdout:779 trunc from 0xc8b5edd to 0x5ec49ab 2024-09-15T00:11:33.448 INFO:teuthology.orchestra.run.smithi171.stdout:780 trunc from 0x5ec49ab to 0x2425c7f 2024-09-15T00:11:33.475 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:33.472+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:33.576 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:33.573+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:33.600 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:33.600 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:33.614 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:33.637 INFO:teuthology.orchestra.run.smithi171.stdout:781 read 0x1fed831 thru 0x1ffbca2 (0xe472 bytes) 2024-09-15T00:11:33.639 INFO:teuthology.orchestra.run.smithi171.stdout:782 read 0x19e1480 thru 0x19e1e4a (0x9cb bytes) 2024-09-15T00:11:33.639 INFO:teuthology.orchestra.run.smithi171.stdout:783 read 0x132cfd8 thru 0x132dcd7 (0xd00 bytes) 2024-09-15T00:11:33.647 INFO:teuthology.orchestra.run.smithi171.stdout:784 trunc from 0x2425c7f to 0x8a3ae2c 2024-09-15T00:11:33.654 INFO:teuthology.orchestra.run.smithi171.stdout:786 read 0xeb368f thru 0xebc26c (0x8bde bytes) 2024-09-15T00:11:33.655 INFO:teuthology.orchestra.run.smithi171.stdout:787 write 0x5be4e95 thru 0x5bf47b5 (0xf921 bytes) 2024-09-15T00:11:33.661 INFO:teuthology.orchestra.run.smithi171.stdout:788 punch from 0x1f49a6c to 0x1f4c52a, (0x2abe bytes) 2024-09-15T00:11:33.661 INFO:teuthology.orchestra.run.smithi171.stdout:789 write 0x1e635f4 thru 0x1e7332d (0xfd3a bytes) 2024-09-15T00:11:33.665 INFO:teuthology.orchestra.run.smithi171.stdout:790 read 0x834f08a thru 0x835c1d1 (0xd148 bytes) 2024-09-15T00:11:33.665 INFO:teuthology.orchestra.run.smithi171.stdout:792 read 0x2b815ab thru 0x2b8e8ff (0xd355 bytes) 2024-09-15T00:11:33.666 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:33.666 INFO:teuthology.orchestra.run.smithi171.stdout:797 punch from 0x73d59b2 to 0x73d755a, (0x1ba8 bytes) 2024-09-15T00:11:33.666 INFO:teuthology.orchestra.run.smithi171.stdout:799 write 0x56cba56 thru 0x56d7dfb (0xc3a6 bytes) 2024-09-15T00:11:33.670 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:33.671 INFO:teuthology.orchestra.run.smithi171.stdout:801 read 0x2c2bb1f thru 0x2c340e7 (0x85c9 bytes) 2024-09-15T00:11:33.676 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:33.674+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:33.678 INFO:teuthology.orchestra.run.smithi171.stdout:802 write 0xa09371a thru 0xa09754d (0x3e34 bytes) 2024-09-15T00:11:33.682 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:33.682 INFO:teuthology.orchestra.run.smithi171.stdout:804 writesame 0x3326b06 thru 0x33360e4 data_size 0x1(0xf5df bytes) 2024-09-15T00:11:33.748 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:33.749 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:33.758 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:33.777 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:33.775+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:33.798 INFO:teuthology.orchestra.run.smithi171.stdout:805 punch from 0x2b33598 to 0x2b3dbf5, (0xa65d bytes) 2024-09-15T00:11:33.802 INFO:teuthology.orchestra.run.smithi171.stdout:806 trunc from 0xa09754e to 0xd4cd102 2024-09-15T00:11:33.807 INFO:teuthology.orchestra.run.smithi171.stdout:808 writesame 0xce2b34 thru 0xceafe1 data_size 0x11(0x84ae bytes) 2024-09-15T00:11:33.811 INFO:teuthology.orchestra.run.smithi171.stdout:811 punch from 0xd91f04 to 0xda1919, (0xfa15 bytes) 2024-09-15T00:11:33.811 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:33.813 INFO:teuthology.orchestra.run.smithi171.stdout:813 trunc from 0xd4cd102 to 0xb6850f5 2024-09-15T00:11:33.820 INFO:teuthology.orchestra.run.smithi171.stdout:815 write 0x39a0b7c thru 0x39ae9fe (0xde83 bytes) 2024-09-15T00:11:33.836 INFO:teuthology.orchestra.run.smithi171.stdout:816 trunc from 0xb6850f5 to 0x16eab6c 2024-09-15T00:11:33.869 INFO:teuthology.orchestra.run.smithi171.stdout:817 writesame 0xd35f736 thru 0xd360a4c data_size 0x9(0x1317 bytes) 2024-09-15T00:11:33.870 INFO:teuthology.orchestra.run.smithi171.stdout:818 read 0xcd46abe thru 0xcd4971a (0x2c5d bytes) 2024-09-15T00:11:33.870 INFO:teuthology.orchestra.run.smithi171.stdout:819 read 0xbd250a9 thru 0xbd3110f (0xc067 bytes) 2024-09-15T00:11:33.870 INFO:teuthology.orchestra.run.smithi171.stdout:821 read 0x55d304 thru 0x56acce (0xd9cb bytes) 2024-09-15T00:11:33.870 INFO:teuthology.orchestra.run.smithi171.stdout:822 write 0x5779856 thru 0x577d720 (0x3ecb bytes) 2024-09-15T00:11:33.873 INFO:teuthology.orchestra.run.smithi171.stdout:824 punch from 0xb9ea040 to 0xb9f58e7, (0xb8a7 bytes) 2024-09-15T00:11:33.873 INFO:teuthology.orchestra.run.smithi171.stdout:825 punch from 0xf27c8a to 0xf34ed6, (0xd24c bytes) 2024-09-15T00:11:33.873 INFO:teuthology.orchestra.run.smithi171.stdout:827 write 0x7e4b5bc thru 0x7e5671b (0xb160 bytes) 2024-09-15T00:11:33.878 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:33.878 INFO:teuthology.orchestra.run.smithi171.stdout:830 punch from 0x417e8fc to 0x418d843, (0xef47 bytes) 2024-09-15T00:11:33.878 INFO:teuthology.orchestra.run.smithi171.stdout:831 read 0x8368b35 thru 0x836c9f8 (0x3ec4 bytes) 2024-09-15T00:11:33.879 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:33.876+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:33.892 INFO:teuthology.orchestra.run.smithi171.stdout:832 trunc from 0xd360a4d to 0x159d797 2024-09-15T00:11:33.910 INFO:teuthology.orchestra.run.smithi171.stdout:833 read 0x1d33ed thru 0x1df26f (0xbe83 bytes) 2024-09-15T00:11:33.911 INFO:teuthology.orchestra.run.smithi171.stdout:835 punch from 0x1067b3a to 0x107407b, (0xc541 bytes) 2024-09-15T00:11:33.921 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:33.921 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:33.928 INFO:teuthology.orchestra.run.smithi171.stdout:836 writesame 0x98af886 thru 0x98be722 data_size 0x13(0xee9d bytes) 2024-09-15T00:11:33.928 INFO:teuthology.orchestra.run.smithi171.stdout:837 writesame 0x29afe75 thru 0x29afee5 data_size 0x1(0x71 bytes) 2024-09-15T00:11:33.929 INFO:teuthology.orchestra.run.smithi171.stdout:838 write 0x2d67665 thru 0x2d75e21 (0xe7bd bytes) 2024-09-15T00:11:33.931 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:33.933 INFO:teuthology.orchestra.run.smithi171.stdout:839 read 0x87f5850 thru 0x87f8bd1 (0x3382 bytes) 2024-09-15T00:11:33.941 INFO:teuthology.orchestra.run.smithi171.stdout:840 write 0xa6e9af7 thru 0xa6f2ef4 (0x93fe bytes) 2024-09-15T00:11:33.946 INFO:teuthology.orchestra.run.smithi171.stdout:841 write 0xbeb145 thru 0xbf17ec (0x66a8 bytes) 2024-09-15T00:11:33.947 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:33.957 INFO:teuthology.orchestra.run.smithi171.stdout:847 trunc from 0xa6f2ef5 to 0x2f3c867 2024-09-15T00:11:33.972 INFO:teuthology.orchestra.run.smithi171.stdout:849 write 0x276fb47 thru 0x277bc6b (0xc125 bytes) 2024-09-15T00:11:33.975 INFO:teuthology.orchestra.run.smithi171.stdout:850 read 0x2816b83 thru 0x2820367 (0x97e5 bytes) 2024-09-15T00:11:33.976 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:33.979 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:33.976+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:33.994 INFO:teuthology.orchestra.run.smithi171.stdout:852 writesame 0xbfaa4a4 thru 0xbfacf8c data_size 0xd(0x2ae9 bytes) 2024-09-15T00:11:33.994 INFO:teuthology.orchestra.run.smithi171.stdout:853 writesame 0x4203367 thru 0x42053be data_size 0xa(0x2058 bytes) 2024-09-15T00:11:33.996 INFO:teuthology.orchestra.run.smithi171.stdout:854 read 0x38be712 thru 0x38c0fb6 (0x28a5 bytes) 2024-09-15T00:11:33.996 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:33.996 INFO:teuthology.orchestra.run.smithi171.stdout:857 read 0xbf17d2d thru 0xbf256af (0xd983 bytes) 2024-09-15T00:11:33.996 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:33.996 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:33.996 INFO:teuthology.orchestra.run.smithi171.stdout:860 writesame 0x9cfbc96 thru 0x9d05a8f data_size 0x49(0x9dfa bytes) 2024-09-15T00:11:34.000 INFO:teuthology.orchestra.run.smithi171.stdout:861 read 0x19011db thru 0x1908a43 (0x7869 bytes) 2024-09-15T00:11:34.000 INFO:teuthology.orchestra.run.smithi171.stdout:862 punch from 0x5e6ef01 to 0x5e722b5, (0x33b4 bytes) 2024-09-15T00:11:34.009 INFO:teuthology.orchestra.run.smithi171.stdout:863 trunc from 0xbfacf8d to 0x474b5b7 2024-09-15T00:11:34.029 INFO:teuthology.orchestra.run.smithi171.stdout:864 trunc from 0x474b5b7 to 0x443c84 2024-09-15T00:11:34.038 INFO:teuthology.orchestra.run.smithi171.stdout:865 read 0xdb2a4 thru 0xea20c (0xef69 bytes) 2024-09-15T00:11:34.052 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:34.052 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:34.061 INFO:teuthology.orchestra.run.smithi171.stdout:867 write 0xdd28511 thru 0xdd2efdc (0x6acc bytes) 2024-09-15T00:11:34.063 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:34.064 INFO:teuthology.orchestra.run.smithi171.stdout:868 read 0xd0a3ded thru 0xd0ac309 (0x851d bytes) 2024-09-15T00:11:34.065 INFO:teuthology.orchestra.run.smithi171.stdout:870 read 0x2529666 thru 0x252cdde (0x3779 bytes) 2024-09-15T00:11:34.065 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:34.066 INFO:teuthology.orchestra.run.smithi171.stdout:872 punch from 0xd07597d to 0xd07f858, (0x9edb bytes) 2024-09-15T00:11:34.066 INFO:teuthology.orchestra.run.smithi171.stdout:873 punch from 0x8c1629 to 0x8cf674, (0xe04b bytes) 2024-09-15T00:11:34.070 INFO:teuthology.orchestra.run.smithi171.stdout:874 write 0xde07b55 thru 0xde112c3 (0x976f bytes) 2024-09-15T00:11:34.073 INFO:teuthology.orchestra.run.smithi171.stdout:875 punch from 0x2ec0ffb to 0x2ec6010, (0x5015 bytes) 2024-09-15T00:11:34.073 INFO:teuthology.orchestra.run.smithi171.stdout:880 write 0x88ed978 thru 0x88f3590 (0x5c19 bytes) 2024-09-15T00:11:34.084 INFO:teuthology.orchestra.run.smithi171.stdout:881 trunc from 0xde112c4 to 0x7421412 2024-09-15T00:11:34.101 INFO:teuthology.orchestra.run.smithi171.stdout:882 trunc from 0x7421412 to 0xb88c451 2024-09-15T00:11:34.106 INFO:teuthology.orchestra.run.smithi171.stdout:883 write 0xa40630c thru 0xa409e5b (0x3b50 bytes) 2024-09-15T00:11:34.107 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:34.105+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:34.116 INFO:teuthology.orchestra.run.smithi171.stdout:884 trunc from 0xb88c451 to 0x6ad09e3 2024-09-15T00:11:34.126 INFO:teuthology.orchestra.run.smithi171.stdout:885 read 0x3ad7aab thru 0x3adb5ac (0x3b02 bytes) 2024-09-15T00:11:34.126 INFO:teuthology.orchestra.run.smithi171.stdout:887 punch from 0x858fb to 0x9506a, (0xf76f bytes) 2024-09-15T00:11:34.131 INFO:teuthology.orchestra.run.smithi171.stdout:888 write 0x6ead9d2 thru 0x6eb0748 (0x2d77 bytes) 2024-09-15T00:11:34.146 INFO:teuthology.orchestra.run.smithi171.stdout:889 write 0xd1ec958 thru 0xd1fa600 (0xdca9 bytes) 2024-09-15T00:11:34.149 INFO:teuthology.orchestra.run.smithi171.stdout:890 write 0xbe8a1a2 thru 0xbe8a5cf (0x42e bytes) 2024-09-15T00:11:34.152 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:34.152 INFO:teuthology.orchestra.run.smithi171.stdout:894 writesame 0x458740c thru 0x4595c68 data_size 0x1(0xe85d bytes) 2024-09-15T00:11:34.166 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:11:34.172 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:34.172 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:34.180 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:34.177+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:34.181 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:34.261 INFO:teuthology.orchestra.run.smithi171.stdout:895 trunc from 0xd1fa601 to 0xea04239 2024-09-15T00:11:34.266 INFO:teuthology.orchestra.run.smithi171.stdout:897 write 0x3a4198f thru 0x3a4958d (0x7bff bytes) 2024-09-15T00:11:34.269 INFO:teuthology.orchestra.run.smithi171.stdout:898 read 0x307a576 thru 0x308a20e (0xfc99 bytes) 2024-09-15T00:11:34.269 INFO:teuthology.orchestra.run.smithi171.stdout:900 write 0x2aa9a4d thru 0x2ab3a2e (0x9fe2 bytes) 2024-09-15T00:11:34.272 INFO:teuthology.orchestra.run.smithi171.stdout:902 write 0xa03e81f thru 0xa03fa68 (0x124a bytes) 2024-09-15T00:11:34.280 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:34.278+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:34.283 INFO:teuthology.orchestra.run.smithi171.stdout:903 trunc from 0xea04239 to 0x846ac4c 2024-09-15T00:11:34.292 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:34.298 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:34.298 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:34.305 INFO:teuthology.orchestra.run.smithi171.stdout:905 writesame 0xecf85cc thru 0xed06ee1 data_size 0xc3(0xe916 bytes) 2024-09-15T00:11:34.317 INFO:teuthology.orchestra.run.smithi171.stdout:908 trunc from 0xed06ee2 to 0x4e1fb64 2024-09-15T00:11:34.318 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:34.318 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:34.329 INFO:teuthology.orchestra.run.smithi171.stdout:909 read 0x3c4630e thru 0x3c50857 (0xa54a bytes) 2024-09-15T00:11:34.331 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:34.331 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:34.337 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:34.337 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:34.340 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:34.343 INFO:teuthology.orchestra.run.smithi171.stdout:910 writesame 0xb3c0e30 thru 0xb3cb86a data_size 0x1(0xaa3b bytes) 2024-09-15T00:11:34.343 INFO:teuthology.orchestra.run.smithi171.stdout:911 write 0xad20cd8 thru 0xad25e2b (0x5154 bytes) 2024-09-15T00:11:34.347 INFO:teuthology.orchestra.run.smithi171.stdout:912 read 0x1c8bf40 thru 0x1c942e1 (0x83a2 bytes) 2024-09-15T00:11:34.348 INFO:teuthology.orchestra.run.smithi171.stdout:913 punch from 0x9005df7 to 0x9007a9a, (0x1ca3 bytes) 2024-09-15T00:11:34.355 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:34.356 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:34.357 INFO:teuthology.orchestra.run.smithi171.stdout:914 trunc from 0xb3cb86b to 0x30fd88a 2024-09-15T00:11:34.368 INFO:teuthology.orchestra.run.smithi171.stdout:915 punch from 0x1dbb6c0 to 0x1dc51cf, (0x9b0f bytes) 2024-09-15T00:11:34.368 INFO:teuthology.orchestra.run.smithi171.stdout:916 writesame 0xb1a664 thru 0xb20fdf data_size 0x56(0x697c bytes) 2024-09-15T00:11:34.370 INFO:teuthology.orchestra.run.smithi171.stdout:917 read 0x2abb656 thru 0x2abd66f (0x201a bytes) 2024-09-15T00:11:34.370 INFO:teuthology.orchestra.run.smithi171.stdout:919 read 0xc5a5b5 thru 0xc6708f (0xcadb bytes) 2024-09-15T00:11:34.371 INFO:teuthology.orchestra.run.smithi171.stdout:920 clone 8 order 21 su 262144 sc 15 2024-09-15T00:11:34.374 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:34.374 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:34.381 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:34.379+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:34.392 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:34.392 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:34.408 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:34.408 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:34.425 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:34.425 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:34.449 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:34.449 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:34.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:34.450 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:34.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.450 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:464 3.9 3:942069cf:::rbd_header.11397bff333d:head [call rbd.set_stripe_unit_count in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:34.450 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.025416+0000", 2024-09-15T00:11:34.450 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.4216817220000006, 2024-09-15T00:11:34.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055912979999999998, 2024-09-15T00:11:34.450 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.450 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.450 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.450 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.450 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.450 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.450 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 464 2024-09-15T00:11:34.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.451 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025416+0000", 2024-09-15T00:11:34.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025416+0000", 2024-09-15T00:11:34.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025416+0000", 2024-09-15T00:11:34.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1500000000000004e-07 2024-09-15T00:11:34.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025420+0000", 2024-09-15T00:11:34.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.473e-06 2024-09-15T00:11:34.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025427+0000", 2024-09-15T00:11:34.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4270000000000002e-06 2024-09-15T00:11:34.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025434+0000", 2024-09-15T00:11:34.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9670000000000004e-06 2024-09-15T00:11:34.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025484+0000", 2024-09-15T00:11:34.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0084999999999999e-05 2024-09-15T00:11:34.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025530+0000", 2024-09-15T00:11:34.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5286e-05 2024-09-15T00:11:34.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:34.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.025666+0000", 2024-09-15T00:11:34.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000136067 2024-09-15T00:11:34.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.026078+0000", 2024-09-15T00:11:34.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00041209200000000002 2024-09-15T00:11:34.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.030916+0000", 2024-09-15T00:11:34.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048382060000000003 2024-09-15T00:11:34.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.030967+0000", 2024-09-15T00:11:34.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0540000000000001e-05 2024-09-15T00:11:34.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.031007+0000", 2024-09-15T00:11:34.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0439999999999999e-05 2024-09-15T00:11:34.456 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.456 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.456 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.456 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:832 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:34.456 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.524286+0000", 2024-09-15T00:11:34.456 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9228112139999993, 2024-09-15T00:11:34.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0067154600000000004, 2024-09-15T00:11:34.456 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.456 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.457 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.457 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 832 2024-09-15T00:11:34.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.457 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524286+0000", 2024-09-15T00:11:34.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524286+0000", 2024-09-15T00:11:34.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524287+0000", 2024-09-15T00:11:34.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7899999999999999e-07 2024-09-15T00:11:34.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524289+0000", 2024-09-15T00:11:34.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2010000000000002e-06 2024-09-15T00:11:34.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524295+0000", 2024-09-15T00:11:34.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0979999999999999e-06 2024-09-15T00:11:34.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524300+0000", 2024-09-15T00:11:34.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6970000000000001e-06 2024-09-15T00:11:34.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524888+0000", 2024-09-15T00:11:34.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00058825000000000001 2024-09-15T00:11:34.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.524941+0000", 2024-09-15T00:11:34.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3529000000000002e-05 2024-09-15T00:11:34.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:34.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525192+0000", 2024-09-15T00:11:34.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025088299999999998 2024-09-15T00:11:34.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.528065+0000", 2024-09-15T00:11:34.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002873085 2024-09-15T00:11:34.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.530935+0000", 2024-09-15T00:11:34.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028699950000000002 2024-09-15T00:11:34.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.530968+0000", 2024-09-15T00:11:34.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.235e-05 2024-09-15T00:11:34.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.531002+0000", 2024-09-15T00:11:34.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3992999999999998e-05 2024-09-15T00:11:34.462 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.462 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.462 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.462 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:833 3.1a 3:58b59828:::rbd_data.11397bff333d.000000000000001e:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3006249~8407] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:34.462 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.525900+0000", 2024-09-15T00:11:34.462 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9211973849999993, 2024-09-15T00:11:34.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052167660000000003, 2024-09-15T00:11:34.462 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.462 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.462 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.462 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.462 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.463 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.463 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 833 2024-09-15T00:11:34.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.463 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525900+0000", 2024-09-15T00:11:34.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525900+0000", 2024-09-15T00:11:34.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525901+0000", 2024-09-15T00:11:34.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1799999999999995e-07 2024-09-15T00:11:34.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525909+0000", 2024-09-15T00:11:34.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4130000000000006e-06 2024-09-15T00:11:34.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525916+0000", 2024-09-15T00:11:34.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2350000000000002e-06 2024-09-15T00:11:34.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525920+0000", 2024-09-15T00:11:34.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0820000000000001e-06 2024-09-15T00:11:34.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525957+0000", 2024-09-15T00:11:34.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6257000000000002e-05 2024-09-15T00:11:34.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.525998+0000", 2024-09-15T00:11:34.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1662000000000002e-05 2024-09-15T00:11:34.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:34.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.526076+0000", 2024-09-15T00:11:34.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7985000000000006e-05 2024-09-15T00:11:34.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.528223+0000", 2024-09-15T00:11:34.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021471870000000001 2024-09-15T00:11:34.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.531054+0000", 2024-09-15T00:11:34.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00283037 2024-09-15T00:11:34.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.531089+0000", 2024-09-15T00:11:34.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5364999999999998e-05 2024-09-15T00:11:34.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.531117+0000", 2024-09-15T00:11:34.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7691999999999998e-05 2024-09-15T00:11:34.467 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.468 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.468 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.468 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:34.468 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:11:34.468 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.8751830520000006, 2024-09-15T00:11:34.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:11:34.468 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.468 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.468 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.468 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.468 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.469 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.469 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:11:34.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.469 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:11:34.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:11:34.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:11:34.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:11:34.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:11:34.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:11:34.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:11:34.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:11:34.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:11:34.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:11:34.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:11:34.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:11:34.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:11:34.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:11:34.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:34.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:11:34.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:11:34.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:11:34.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:11:34.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:11:34.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:11:34.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:11:34.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:11:34.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:11:34.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:11:34.474 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.474 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.474 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.474 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:34.474 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:34.474 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.8751483600000007, 2024-09-15T00:11:34.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:11:34.474 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.474 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.474 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.475 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.475 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.475 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.475 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:11:34.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.475 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:34.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:34.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:34.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:11:34.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:11:34.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:11:34.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:11:34.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:11:34.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:11:34.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:11:34.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:11:34.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:11:34.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:11:34.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:11:34.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:34.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:11:34.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:11:34.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:11:34.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:11:34.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:11:34.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:11:34.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:11:34.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:11:34.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:11:34.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:11:34.480 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.480 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.480 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.480 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:863 3.9 e29/23 ondisk, result = 0)", 2024-09-15T00:11:34.480 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.585151+0000", 2024-09-15T00:11:34.480 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.8619461239999993, 2024-09-15T00:11:34.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.063949342000000006, 2024-09-15T00:11:34.480 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.480 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.480 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:34.480 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585151+0000", 2024-09-15T00:11:34.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585151+0000", 2024-09-15T00:11:34.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585152+0000", 2024-09-15T00:11:34.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.046e-06 2024-09-15T00:11:34.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585156+0000", 2024-09-15T00:11:34.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6339999999999999e-06 2024-09-15T00:11:34.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585185+0000", 2024-09-15T00:11:34.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8646e-05 2024-09-15T00:11:34.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.585193+0000", 2024-09-15T00:11:34.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5699999999999993e-06 2024-09-15T00:11:34.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649017+0000", 2024-09-15T00:11:34.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.063823787000000007 2024-09-15T00:11:34.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649023+0000", 2024-09-15T00:11:34.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.728e-06 2024-09-15T00:11:34.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649101+0000", 2024-09-15T00:11:34.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7930999999999996e-05 2024-09-15T00:11:34.483 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.484 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.484 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.484 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:1129 3.1d e29/23 3:ba4e0201:::rbd_data.11397bff333d.0000000000000028:head v 29'4, mlcod=29'2)", 2024-09-15T00:11:34.484 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.896838+0000", 2024-09-15T00:11:34.484 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5502591139999993, 2024-09-15T00:11:34.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0059020139999999997, 2024-09-15T00:11:34.484 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.484 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.484 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.484 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896838+0000", 2024-09-15T00:11:34.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896838+0000", 2024-09-15T00:11:34.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896839+0000", 2024-09-15T00:11:34.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3900000000000004e-07 2024-09-15T00:11:34.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896844+0000", 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6990000000000002e-06 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896854+0000", 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9790000000000005e-06 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896858+0000", 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5850000000000001e-06 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896932+0000", 2024-09-15T00:11:34.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4286e-05 2024-09-15T00:11:34.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896954+0000", 2024-09-15T00:11:34.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1421999999999999e-05 2024-09-15T00:11:34.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.902706+0000", 2024-09-15T00:11:34.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057519570000000003 2024-09-15T00:11:34.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.902740+0000", 2024-09-15T00:11:34.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4446999999999997e-05 2024-09-15T00:11:34.488 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.488 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.488 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.488 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1184 3.1a 3:5b582af0:::rbd_data.11397bff333d.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:34.488 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.948047+0000", 2024-09-15T00:11:34.488 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4990503349999997, 2024-09-15T00:11:34.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.039602018000000003, 2024-09-15T00:11:34.488 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.489 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.489 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.489 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.489 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.489 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.489 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1184 2024-09-15T00:11:34.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.489 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948047+0000", 2024-09-15T00:11:34.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948047+0000", 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948048+0000", 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3499999999999996e-07 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948050+0000", 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7659999999999999e-06 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948057+0000", 2024-09-15T00:11:34.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1369999999999996e-06 2024-09-15T00:11:34.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948062+0000", 2024-09-15T00:11:34.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0160000000000002e-06 2024-09-15T00:11:34.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948091+0000", 2024-09-15T00:11:34.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9556999999999998e-05 2024-09-15T00:11:34.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.948172+0000", 2024-09-15T00:11:34.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1180999999999994e-05 2024-09-15T00:11:34.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:34.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.949974+0000", 2024-09-15T00:11:34.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018014299999999999 2024-09-15T00:11:34.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987559+0000", 2024-09-15T00:11:34.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037585358999999999 2024-09-15T00:11:34.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987575+0000", 2024-09-15T00:11:34.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6101000000000001e-05 2024-09-15T00:11:34.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987611+0000", 2024-09-15T00:11:34.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6016999999999998e-05 2024-09-15T00:11:34.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987649+0000", 2024-09-15T00:11:34.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7919000000000001e-05 2024-09-15T00:11:34.494 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.494 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.494 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.494 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1184 3.1a e29/23 ondisk, result = 0)", 2024-09-15T00:11:34.494 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.955158+0000", 2024-09-15T00:11:34.494 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4919392980000001, 2024-09-15T00:11:34.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032410688, 2024-09-15T00:11:34.494 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.494 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.494 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:34.494 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955158+0000", 2024-09-15T00:11:34.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955158+0000", 2024-09-15T00:11:34.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955159+0000", 2024-09-15T00:11:34.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0399999999999996e-07 2024-09-15T00:11:34.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955161+0000", 2024-09-15T00:11:34.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4710000000000001e-06 2024-09-15T00:11:34.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955172+0000", 2024-09-15T00:11:34.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1059e-05 2024-09-15T00:11:34.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.955177+0000", 2024-09-15T00:11:34.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1649999999999997e-06 2024-09-15T00:11:34.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987541+0000", 2024-09-15T00:11:34.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032363268000000001 2024-09-15T00:11:34.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987551+0000", 2024-09-15T00:11:34.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0699e-05 2024-09-15T00:11:34.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.987569+0000", 2024-09-15T00:11:34.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7422e-05 2024-09-15T00:11:34.498 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.498 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.498 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.498 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1374 3.9 3:942069cf:::rbd_header.11397bff333d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:34.498 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.354861+0000", 2024-09-15T00:11:34.498 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.0922367319999999, 2024-09-15T00:11:34.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055109729999999997, 2024-09-15T00:11:34.499 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.499 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.499 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.499 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.499 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.499 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.499 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1374 2024-09-15T00:11:34.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.499 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354861+0000", 2024-09-15T00:11:34.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354861+0000", 2024-09-15T00:11:34.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354861+0000", 2024-09-15T00:11:34.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:11:34.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354865+0000", 2024-09-15T00:11:34.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2030000000000001e-06 2024-09-15T00:11:34.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354871+0000", 2024-09-15T00:11:34.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8170000000000001e-06 2024-09-15T00:11:34.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354884+0000", 2024-09-15T00:11:34.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2468e-05 2024-09-15T00:11:34.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354922+0000", 2024-09-15T00:11:34.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8380000000000002e-05 2024-09-15T00:11:34.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.354971+0000", 2024-09-15T00:11:34.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9051999999999997e-05 2024-09-15T00:11:34.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:34.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.355080+0000", 2024-09-15T00:11:34.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000108421 2024-09-15T00:11:34.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.355454+0000", 2024-09-15T00:11:34.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00037466600000000001 2024-09-15T00:11:34.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.360227+0000", 2024-09-15T00:11:34.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047726740000000002 2024-09-15T00:11:34.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.360310+0000", 2024-09-15T00:11:34.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3465000000000001e-05 2024-09-15T00:11:34.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.360372+0000", 2024-09-15T00:11:34.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1155999999999995e-05 2024-09-15T00:11:34.504 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.504 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.505 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.505 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1628 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:34.505 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.712543+0000", 2024-09-15T00:11:34.505 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 7.734554353, 2024-09-15T00:11:34.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.018404876000000001, 2024-09-15T00:11:34.505 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.505 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.505 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.505 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.505 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.505 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.505 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1628 2024-09-15T00:11:34.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.506 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712543+0000", 2024-09-15T00:11:34.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712543+0000", 2024-09-15T00:11:34.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712544+0000", 2024-09-15T00:11:34.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6899999999999996e-07 2024-09-15T00:11:34.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712548+0000", 2024-09-15T00:11:34.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1239999999999998e-06 2024-09-15T00:11:34.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712558+0000", 2024-09-15T00:11:34.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0387000000000001e-05 2024-09-15T00:11:34.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712565+0000", 2024-09-15T00:11:34.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.545e-06 2024-09-15T00:11:34.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712603+0000", 2024-09-15T00:11:34.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7818000000000003e-05 2024-09-15T00:11:34.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.712644+0000", 2024-09-15T00:11:34.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1743999999999999e-05 2024-09-15T00:11:34.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:34.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.714418+0000", 2024-09-15T00:11:34.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0017731699999999999 2024-09-15T00:11:34.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730862+0000", 2024-09-15T00:11:34.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016444317999999999 2024-09-15T00:11:34.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730878+0000", 2024-09-15T00:11:34.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6269e-05 2024-09-15T00:11:34.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730915+0000", 2024-09-15T00:11:34.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6365000000000002e-05 2024-09-15T00:11:34.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730948+0000", 2024-09-15T00:11:34.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3367000000000001e-05 2024-09-15T00:11:34.511 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.511 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.511 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.511 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1628 3.9 e30/23 ondisk, result = 0)", 2024-09-15T00:11:34.511 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.720479+0000", 2024-09-15T00:11:34.511 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 7.7266184280000001, 2024-09-15T00:11:34.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010388134, 2024-09-15T00:11:34.511 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.511 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.512 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:34.512 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720479+0000", 2024-09-15T00:11:34.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720479+0000", 2024-09-15T00:11:34.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720480+0000", 2024-09-15T00:11:34.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2299999999999999e-07 2024-09-15T00:11:34.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720484+0000", 2024-09-15T00:11:34.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-06 2024-09-15T00:11:34.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720499+0000", 2024-09-15T00:11:34.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4783999999999999e-05 2024-09-15T00:11:34.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.720507+0000", 2024-09-15T00:11:34.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9370000000000004e-06 2024-09-15T00:11:34.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730848+0000", 2024-09-15T00:11:34.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010341408999999999 2024-09-15T00:11:34.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730856+0000", 2024-09-15T00:11:34.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0679999999999997e-06 2024-09-15T00:11:34.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.730867+0000", 2024-09-15T00:11:34.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0733e-05 2024-09-15T00:11:34.515 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.515 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.515 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.515 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1837 3.1a 3:58b59828:::rbd_data.11397bff333d.000000000000001e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-15T00:11:34.515 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:27.031352+0000", 2024-09-15T00:11:34.515 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 7.4157453850000001, 2024-09-15T00:11:34.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053302460000000003, 2024-09-15T00:11:34.515 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.516 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.516 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.516 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.516 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.516 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.516 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1837 2024-09-15T00:11:34.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.516 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031352+0000", 2024-09-15T00:11:34.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031352+0000", 2024-09-15T00:11:34.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031353+0000", 2024-09-15T00:11:34.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8999999999999995e-07 2024-09-15T00:11:34.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031357+0000", 2024-09-15T00:11:34.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2570000000000001e-06 2024-09-15T00:11:34.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031367+0000", 2024-09-15T00:11:34.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9510000000000001e-06 2024-09-15T00:11:34.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031374+0000", 2024-09-15T00:11:34.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5169999999999996e-06 2024-09-15T00:11:34.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031433+0000", 2024-09-15T00:11:34.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9262000000000003e-05 2024-09-15T00:11:34.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031519+0000", 2024-09-15T00:11:34.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6428000000000007e-05 2024-09-15T00:11:34.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:34.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.031624+0000", 2024-09-15T00:11:34.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104435 2024-09-15T00:11:34.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.032914+0000", 2024-09-15T00:11:34.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001290795 2024-09-15T00:11:34.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.036387+0000", 2024-09-15T00:11:34.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034723079999999999 2024-09-15T00:11:34.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.036646+0000", 2024-09-15T00:11:34.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025974800000000002 2024-09-15T00:11:34.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:27.036682+0000", 2024-09-15T00:11:34.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5754999999999999e-05 2024-09-15T00:11:34.521 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.521 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.521 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.521 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3179 3.13 e33/23 3:c84732bf:::rbd_data.1139b531b717.0000000000000024:head v 33'34, mlcod=33'32)", 2024-09-15T00:11:34.521 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.658340+0000", 2024-09-15T00:11:34.521 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.7887572470000004, 2024-09-15T00:11:34.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053288930000000003, 2024-09-15T00:11:34.521 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.521 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.521 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.522 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658340+0000", 2024-09-15T00:11:34.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658340+0000", 2024-09-15T00:11:34.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658341+0000", 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7999999999999995e-07 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658347+0000", 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6840000000000001e-06 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658356+0000", 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9799999999999993e-06 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658363+0000", 2024-09-15T00:11:34.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3330000000000002e-06 2024-09-15T00:11:34.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658818+0000", 2024-09-15T00:11:34.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045501500000000002 2024-09-15T00:11:34.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658838+0000", 2024-09-15T00:11:34.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0021e-05 2024-09-15T00:11:34.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663629+0000", 2024-09-15T00:11:34.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047910089999999997 2024-09-15T00:11:34.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663669+0000", 2024-09-15T00:11:34.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0170999999999999e-05 2024-09-15T00:11:34.525 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.525 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.525 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.525 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3223 3.1 3:81046149:::rbd_data.1139b531b717.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:34.525 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.692223+0000", 2024-09-15T00:11:34.525 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.7548744569999997, 2024-09-15T00:11:34.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055865869999999996, 2024-09-15T00:11:34.526 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.526 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.526 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.526 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.526 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.526 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.526 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3223 2024-09-15T00:11:34.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.526 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692223+0000", 2024-09-15T00:11:34.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692223+0000", 2024-09-15T00:11:34.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692225+0000", 2024-09-15T00:11:34.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.075e-06 2024-09-15T00:11:34.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692237+0000", 2024-09-15T00:11:34.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1623e-05 2024-09-15T00:11:34.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692250+0000", 2024-09-15T00:11:34.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3808000000000001e-05 2024-09-15T00:11:34.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692257+0000", 2024-09-15T00:11:34.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4200000000000004e-06 2024-09-15T00:11:34.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692316+0000", 2024-09-15T00:11:34.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9537999999999998e-05 2024-09-15T00:11:34.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692403+0000", 2024-09-15T00:11:34.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6509999999999997e-05 2024-09-15T00:11:34.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:34.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.692492+0000", 2024-09-15T00:11:34.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9320999999999994e-05 2024-09-15T00:11:34.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.693044+0000", 2024-09-15T00:11:34.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00055210800000000005 2024-09-15T00:11:34.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.697746+0000", 2024-09-15T00:11:34.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047013949999999997 2024-09-15T00:11:34.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.697780+0000", 2024-09-15T00:11:34.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4755000000000002e-05 2024-09-15T00:11:34.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.697810+0000", 2024-09-15T00:11:34.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9034e-05 2024-09-15T00:11:34.531 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.531 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.531 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.531 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3556 3.3 3:c5ee06a1:::rbd_data.1139b531b717.000000000000003d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 811008~7753] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T00:11:34.531 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:30.000120+0000", 2024-09-15T00:11:34.531 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.4469778570000003, 2024-09-15T00:11:34.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0087124330000000003, 2024-09-15T00:11:34.531 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.531 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.531 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.531 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.532 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.532 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.532 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3556 2024-09-15T00:11:34.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.532 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000120+0000", 2024-09-15T00:11:34.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000120+0000", 2024-09-15T00:11:34.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000120+0000", 2024-09-15T00:11:34.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7899999999999998e-07 2024-09-15T00:11:34.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000128+0000", 2024-09-15T00:11:34.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1319999999999994e-06 2024-09-15T00:11:34.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000136+0000", 2024-09-15T00:11:34.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0790000000000001e-06 2024-09-15T00:11:34.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000141+0000", 2024-09-15T00:11:34.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6890000000000003e-06 2024-09-15T00:11:34.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000186+0000", 2024-09-15T00:11:34.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4406999999999997e-05 2024-09-15T00:11:34.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-15T00:11:34.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.000190+0000", 2024-09-15T00:11:34.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3540000000000002e-06 2024-09-15T00:11:34.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.003443+0000", 2024-09-15T00:11:34.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032532749999999999 2024-09-15T00:11:34.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.003517+0000", 2024-09-15T00:11:34.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4256000000000002e-05 2024-09-15T00:11:34.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 6", 2024-09-15T00:11:34.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.003644+0000", 2024-09-15T00:11:34.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000126854 2024-09-15T00:11:34.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.006305+0000", 2024-09-15T00:11:34.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026606289999999999 2024-09-15T00:11:34.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.008768+0000", 2024-09-15T00:11:34.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0024626299999999999 2024-09-15T00:11:34.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.008797+0000", 2024-09-15T00:11:34.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9524999999999999e-05 2024-09-15T00:11:34.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.008832+0000", 2024-09-15T00:11:34.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4924e-05 2024-09-15T00:11:34.537 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.537 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.538 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.538 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3735 3.f e35/23 3:f40857ee:::rbd_data.1139b531b717.0000000000000008:head v 35'17, mlcod=34'13)", 2024-09-15T00:11:34.538 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.099634+0000", 2024-09-15T00:11:34.538 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.3474630689999998, 2024-09-15T00:11:34.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053310040000000003, 2024-09-15T00:11:34.538 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.538 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.538 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.538 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099634+0000", 2024-09-15T00:11:34.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099634+0000", 2024-09-15T00:11:34.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099635+0000", 2024-09-15T00:11:34.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2500000000000005e-07 2024-09-15T00:11:34.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099642+0000", 2024-09-15T00:11:34.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4729999999999997e-06 2024-09-15T00:11:34.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099651+0000", 2024-09-15T00:11:34.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9189999999999994e-06 2024-09-15T00:11:34.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099658+0000", 2024-09-15T00:11:34.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7070000000000003e-06 2024-09-15T00:11:34.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099705+0000", 2024-09-15T00:11:34.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6672000000000003e-05 2024-09-15T00:11:34.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099724+0000", 2024-09-15T00:11:34.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8930999999999999e-05 2024-09-15T00:11:34.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.104928+0000", 2024-09-15T00:11:34.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052039740000000001 2024-09-15T00:11:34.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.104965+0000", 2024-09-15T00:11:34.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7602999999999999e-05 2024-09-15T00:11:34.542 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.542 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.542 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.542 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3737 3.a e35/23 3:53de29a7:::rbd_data.1139b531b717.000000000000000a:head v 35'46, mlcod=33'43)", 2024-09-15T00:11:34.542 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.100480+0000", 2024-09-15T00:11:34.542 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.3466177450000001, 2024-09-15T00:11:34.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0071958660000000004, 2024-09-15T00:11:34.543 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.543 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.543 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.543 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100480+0000", 2024-09-15T00:11:34.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100480+0000", 2024-09-15T00:11:34.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100480+0000", 2024-09-15T00:11:34.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8700000000000005e-07 2024-09-15T00:11:34.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100488+0000", 2024-09-15T00:11:34.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3209999999999999e-06 2024-09-15T00:11:34.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100499+0000", 2024-09-15T00:11:34.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1148999999999999e-05 2024-09-15T00:11:34.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100506+0000", 2024-09-15T00:11:34.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3119999999999996e-06 2024-09-15T00:11:34.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100557+0000", 2024-09-15T00:11:34.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0735999999999997e-05 2024-09-15T00:11:34.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100571+0000", 2024-09-15T00:11:34.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4113e-05 2024-09-15T00:11:34.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.107635+0000", 2024-09-15T00:11:34.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0070640080000000001 2024-09-15T00:11:34.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.107676+0000", 2024-09-15T00:11:34.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0540000000000001e-05 2024-09-15T00:11:34.546 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.546 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.546 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.547 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:4155 3.9 3:907c8e4e:::rbd_data.11396389f8f2.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:34.547 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.347292+0000", 2024-09-15T00:11:34.547 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.0998054769999999, 2024-09-15T00:11:34.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.018904537999999999, 2024-09-15T00:11:34.547 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.547 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.547 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.547 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.547 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.547 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.547 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4155 2024-09-15T00:11:34.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.548 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.347292+0000", 2024-09-15T00:11:34.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.347292+0000", 2024-09-15T00:11:34.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.347293+0000", 2024-09-15T00:11:34.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2200000000000003e-07 2024-09-15T00:11:34.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.347302+0000", 2024-09-15T00:11:34.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4569999999999994e-06 2024-09-15T00:11:34.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.347313+0000", 2024-09-15T00:11:34.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.065e-05 2024-09-15T00:11:34.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.347320+0000", 2024-09-15T00:11:34.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9970000000000001e-06 2024-09-15T00:11:34.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.347393+0000", 2024-09-15T00:11:34.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3146999999999996e-05 2024-09-15T00:11:34.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.347491+0000", 2024-09-15T00:11:34.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8332000000000007e-05 2024-09-15T00:11:34.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:34.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.348397+0000", 2024-09-15T00:11:34.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000905483 2024-09-15T00:11:34.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.366056+0000", 2024-09-15T00:11:34.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017658956999999999 2024-09-15T00:11:34.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.366076+0000", 2024-09-15T00:11:34.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0026000000000001e-05 2024-09-15T00:11:34.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.366139+0000", 2024-09-15T00:11:34.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3034000000000007e-05 2024-09-15T00:11:34.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.366196+0000", 2024-09-15T00:11:34.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7732999999999998e-05 2024-09-15T00:11:34.552 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.552 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.553 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.553 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:4155 3.9 e35/23 ondisk, result = 0)", 2024-09-15T00:11:34.553 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.350500+0000", 2024-09-15T00:11:34.553 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.096597262, 2024-09-15T00:11:34.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015565334, 2024-09-15T00:11:34.553 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.553 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.553 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:34.553 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.350500+0000", 2024-09-15T00:11:34.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.350500+0000", 2024-09-15T00:11:34.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.350501+0000", 2024-09-15T00:11:34.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1900000000000001e-07 2024-09-15T00:11:34.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.350504+0000", 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2720000000000002e-06 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.350515+0000", 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1080999999999999e-05 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.350520+0000", 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7999999999999998e-06 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.366036+0000", 2024-09-15T00:11:34.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015516113999999999 2024-09-15T00:11:34.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.366047+0000", 2024-09-15T00:11:34.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0969e-05 2024-09-15T00:11:34.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.366065+0000", 2024-09-15T00:11:34.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8278999999999999e-05 2024-09-15T00:11:34.556 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.557 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.557 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.557 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.557 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.557 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:34.558 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:34.559 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:34.556+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:34.659 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:34.657+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:34.661 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:34.661 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:34.661 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:34.661 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:34.670 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:34.759 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:34.758+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:34.766 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T00:11:34.766 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:11:34.766 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [1, 2, 3, 4, 5, 6, 7] 2024-09-15T00:11:34.766 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-15T00:11:34.813 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:34.813 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:34.813 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:34.813 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:34.813 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:34.822 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:34.860 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:34.859+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:34.927 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:34.927 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:34.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:34.927 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:34.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.927 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4287.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T00:11:34.928 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:09.531630+0000", 2024-09-15T00:11:34.928 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 25.392665782000002, 2024-09-15T00:11:34.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0063638770000000004, 2024-09-15T00:11:34.928 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.928 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.928 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.928 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.928 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4287", 2024-09-15T00:11:34.928 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/605273324", 2024-09-15T00:11:34.928 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1 2024-09-15T00:11:34.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.928 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531630+0000", 2024-09-15T00:11:34.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531630+0000", 2024-09-15T00:11:34.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531649+0000", 2024-09-15T00:11:34.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8944e-05 2024-09-15T00:11:34.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531655+0000", 2024-09-15T00:11:34.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9569999999999996e-06 2024-09-15T00:11:34.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531670+0000", 2024-09-15T00:11:34.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4202999999999999e-05 2024-09-15T00:11:34.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531682+0000", 2024-09-15T00:11:34.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2449000000000001e-05 2024-09-15T00:11:34.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531834+0000", 2024-09-15T00:11:34.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000152132 2024-09-15T00:11:34.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:11:34.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.531841+0000", 2024-09-15T00:11:34.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8990000000000004e-06 2024-09-15T00:11:34.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.536655+0000", 2024-09-15T00:11:34.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048135749999999996 2024-09-15T00:11:34.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.536861+0000", 2024-09-15T00:11:34.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000206467 2024-09-15T00:11:34.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:34.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.537049+0000", 2024-09-15T00:11:34.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018749500000000001 2024-09-15T00:11:34.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.537642+0000", 2024-09-15T00:11:34.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00059315599999999996 2024-09-15T00:11:34.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.537927+0000", 2024-09-15T00:11:34.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000285548 2024-09-15T00:11:34.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.537965+0000", 2024-09-15T00:11:34.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7826e-05 2024-09-15T00:11:34.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:09.537993+0000", 2024-09-15T00:11:34.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8226e-05 2024-09-15T00:11:34.934 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.934 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.934 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.934 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4287.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T00:11:34.934 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:11.540720+0000", 2024-09-15T00:11:34.934 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.383575218000001, 2024-09-15T00:11:34.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0054958009999999998, 2024-09-15T00:11:34.934 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.934 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.934 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.934 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.934 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4287", 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/605273324", 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 7 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540720+0000", 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540720+0000", 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540722+0000", 2024-09-15T00:11:34.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3499999999999999e-06 2024-09-15T00:11:34.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540731+0000", 2024-09-15T00:11:34.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2029999999999992e-06 2024-09-15T00:11:34.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540747+0000", 2024-09-15T00:11:34.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6104999999999999e-05 2024-09-15T00:11:34.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540764+0000", 2024-09-15T00:11:34.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7303e-05 2024-09-15T00:11:34.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540843+0000", 2024-09-15T00:11:34.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8905000000000004e-05 2024-09-15T00:11:34.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-15T00:11:34.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.540853+0000", 2024-09-15T00:11:34.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0003999999999999e-05 2024-09-15T00:11:34.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.542542+0000", 2024-09-15T00:11:34.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001689383 2024-09-15T00:11:34.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.542624+0000", 2024-09-15T00:11:34.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1185000000000003e-05 2024-09-15T00:11:34.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:34.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.542758+0000", 2024-09-15T00:11:34.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013473399999999999 2024-09-15T00:11:34.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.544732+0000", 2024-09-15T00:11:34.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0019736240000000002 2024-09-15T00:11:34.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.546145+0000", 2024-09-15T00:11:34.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001413037 2024-09-15T00:11:34.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.546183+0000", 2024-09-15T00:11:34.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8105000000000002e-05 2024-09-15T00:11:34.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:11.546216+0000", 2024-09-15T00:11:34.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2863e-05 2024-09-15T00:11:34.940 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.940 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.940 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.940 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:11:34.940 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:21.598005+0000", 2024-09-15T00:11:34.940 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.326290050000001, 2024-09-15T00:11:34.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053046270000000001, 2024-09-15T00:11:34.940 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.941 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.941 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.941 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 10 2024-09-15T00:11:34.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.941 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598005+0000", 2024-09-15T00:11:34.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598005+0000", 2024-09-15T00:11:34.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598007+0000", 2024-09-15T00:11:34.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6750000000000001e-06 2024-09-15T00:11:34.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598015+0000", 2024-09-15T00:11:34.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8099999999999998e-06 2024-09-15T00:11:34.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598029+0000", 2024-09-15T00:11:34.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3756e-05 2024-09-15T00:11:34.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598037+0000", 2024-09-15T00:11:34.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7299999999999994e-06 2024-09-15T00:11:34.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598096+0000", 2024-09-15T00:11:34.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8229000000000001e-05 2024-09-15T00:11:34.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-15T00:11:34.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.598101+0000", 2024-09-15T00:11:34.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.558e-06 2024-09-15T00:11:34.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.601647+0000", 2024-09-15T00:11:34.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035457779999999999 2024-09-15T00:11:34.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.601788+0000", 2024-09-15T00:11:34.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014144400000000001 2024-09-15T00:11:34.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:34.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.602008+0000", 2024-09-15T00:11:34.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000219488 2024-09-15T00:11:34.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.602766+0000", 2024-09-15T00:11:34.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00075815400000000003 2024-09-15T00:11:34.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.603180+0000", 2024-09-15T00:11:34.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000414039 2024-09-15T00:11:34.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.603236+0000", 2024-09-15T00:11:34.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5773999999999998e-05 2024-09-15T00:11:34.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:21.603310+0000", 2024-09-15T00:11:34.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4191999999999997e-05 2024-09-15T00:11:34.947 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.947 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.947 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.947 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:603 3.1d 3:ba2ec8e4:::rbd_data.11397bff333d.0000000000000038:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3578962~25518] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:34.947 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.210996+0000", 2024-09-15T00:11:34.947 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.713299503, 2024-09-15T00:11:34.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050683109999999998, 2024-09-15T00:11:34.947 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.947 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.947 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.947 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.948 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.948 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.948 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 603 2024-09-15T00:11:34.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.948 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.210996+0000", 2024-09-15T00:11:34.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.210996+0000", 2024-09-15T00:11:34.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.210998+0000", 2024-09-15T00:11:34.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.689e-06 2024-09-15T00:11:34.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211052+0000", 2024-09-15T00:11:34.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4664999999999999e-05 2024-09-15T00:11:34.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211064+0000", 2024-09-15T00:11:34.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1874e-05 2024-09-15T00:11:34.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211070+0000", 2024-09-15T00:11:34.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9170000000000003e-06 2024-09-15T00:11:34.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211121+0000", 2024-09-15T00:11:34.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1013000000000001e-05 2024-09-15T00:11:34.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211211+0000", 2024-09-15T00:11:34.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0446999999999996e-05 2024-09-15T00:11:34.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:34.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.211292+0000", 2024-09-15T00:11:34.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0655999999999995e-05 2024-09-15T00:11:34.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.213176+0000", 2024-09-15T00:11:34.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001884286 2024-09-15T00:11:34.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.216010+0000", 2024-09-15T00:11:34.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028331720000000001 2024-09-15T00:11:34.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.216038+0000", 2024-09-15T00:11:34.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8062999999999999e-05 2024-09-15T00:11:34.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.216064+0000", 2024-09-15T00:11:34.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6529e-05 2024-09-15T00:11:34.952 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.953 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.953 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.953 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:865 3.5 3:a614403b:::rbd_data.11397bff333d.0000000000000003:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:34.953 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.650001+0000", 2024-09-15T00:11:34.953 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2742945450000001, 2024-09-15T00:11:34.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.03879726, 2024-09-15T00:11:34.953 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.953 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.953 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.953 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.953 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.953 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.954 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 865 2024-09-15T00:11:34.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.954 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650001+0000", 2024-09-15T00:11:34.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650001+0000", 2024-09-15T00:11:34.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650002+0000", 2024-09-15T00:11:34.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.19e-06 2024-09-15T00:11:34.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650008+0000", 2024-09-15T00:11:34.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1380000000000004e-06 2024-09-15T00:11:34.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650019+0000", 2024-09-15T00:11:34.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1323e-05 2024-09-15T00:11:34.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650025+0000", 2024-09-15T00:11:34.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8710000000000002e-06 2024-09-15T00:11:34.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650080+0000", 2024-09-15T00:11:34.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4573000000000001e-05 2024-09-15T00:11:34.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.650177+0000", 2024-09-15T00:11:34.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7460999999999994e-05 2024-09-15T00:11:34.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:34.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.651728+0000", 2024-09-15T00:11:34.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015505709999999999 2024-09-15T00:11:34.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.688714+0000", 2024-09-15T00:11:34.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.036986342999999998 2024-09-15T00:11:34.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.688727+0000", 2024-09-15T00:11:34.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2448e-05 2024-09-15T00:11:34.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.688762+0000", 2024-09-15T00:11:34.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4906000000000001e-05 2024-09-15T00:11:34.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.688798+0000", 2024-09-15T00:11:34.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6436000000000001e-05 2024-09-15T00:11:34.959 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.959 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.959 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.959 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:865 3.5 e29/23 ondisk, result = 0)", 2024-09-15T00:11:34.959 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.656357+0000", 2024-09-15T00:11:34.959 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2679387710000007, 2024-09-15T00:11:34.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032363683999999997, 2024-09-15T00:11:34.959 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.959 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.959 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:34.959 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.656357+0000", 2024-09-15T00:11:34.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.656357+0000", 2024-09-15T00:11:34.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.656357+0000", 2024-09-15T00:11:34.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5099999999999999e-07 2024-09-15T00:11:34.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.656360+0000", 2024-09-15T00:11:34.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1159999999999999e-06 2024-09-15T00:11:34.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.656371+0000", 2024-09-15T00:11:34.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0737000000000001e-05 2024-09-15T00:11:34.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.656378+0000", 2024-09-15T00:11:34.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:11:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.688701+0000", 2024-09-15T00:11:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032323172999999997 2024-09-15T00:11:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.688709+0000", 2024-09-15T00:11:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6279999999999996e-06 2024-09-15T00:11:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.688720+0000", 2024-09-15T00:11:34.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1559999999999999e-05 2024-09-15T00:11:34.963 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.963 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.963 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.963 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1129 3.1d 3:ba4e0201:::rbd_data.11397bff333d.0000000000000028:head [truncate 1175199] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:34.963 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.896284+0000", 2024-09-15T00:11:34.963 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.028011845, 2024-09-15T00:11:34.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0066267089999999997, 2024-09-15T00:11:34.963 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.963 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.963 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.963 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.964 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.964 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.964 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1129 2024-09-15T00:11:34.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.964 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896284+0000", 2024-09-15T00:11:34.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896284+0000", 2024-09-15T00:11:34.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896285+0000", 2024-09-15T00:11:34.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.59e-06 2024-09-15T00:11:34.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896296+0000", 2024-09-15T00:11:34.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1287000000000001e-05 2024-09-15T00:11:34.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896324+0000", 2024-09-15T00:11:34.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7946e-05 2024-09-15T00:11:34.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896340+0000", 2024-09-15T00:11:34.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.525e-05 2024-09-15T00:11:34.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896456+0000", 2024-09-15T00:11:34.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000116118 2024-09-15T00:11:34.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896584+0000", 2024-09-15T00:11:34.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000127904 2024-09-15T00:11:34.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:34.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.896751+0000", 2024-09-15T00:11:34.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016762 2024-09-15T00:11:34.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.897191+0000", 2024-09-15T00:11:34.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043940199999999998 2024-09-15T00:11:34.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.902847+0000", 2024-09-15T00:11:34.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0056558959999999997 2024-09-15T00:11:34.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.902885+0000", 2024-09-15T00:11:34.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8198000000000003e-05 2024-09-15T00:11:34.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.902910+0000", 2024-09-15T00:11:34.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5497999999999999e-05 2024-09-15T00:11:34.969 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.969 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.969 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.969 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1313 3.1f 3:fa9cde43:::rbd_data.11397bff333d.000000000000003a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3178173~33091] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:34.969 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.283783+0000", 2024-09-15T00:11:34.969 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.6405123400000008, 2024-09-15T00:11:34.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049414089999999999, 2024-09-15T00:11:34.969 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.970 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.970 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.970 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.970 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.970 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.970 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1313 2024-09-15T00:11:34.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.970 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283783+0000", 2024-09-15T00:11:34.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283783+0000", 2024-09-15T00:11:34.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283784+0000", 2024-09-15T00:11:34.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8100000000000002e-07 2024-09-15T00:11:34.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283806+0000", 2024-09-15T00:11:34.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1970000000000001e-05 2024-09-15T00:11:34.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283816+0000", 2024-09-15T00:11:34.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0641e-05 2024-09-15T00:11:34.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283821+0000", 2024-09-15T00:11:34.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0039999999999999e-06 2024-09-15T00:11:34.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283860+0000", 2024-09-15T00:11:34.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9054000000000003e-05 2024-09-15T00:11:34.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.283950+0000", 2024-09-15T00:11:34.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9907000000000005e-05 2024-09-15T00:11:34.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:34.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.284036+0000", 2024-09-15T00:11:34.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5774999999999998e-05 2024-09-15T00:11:34.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.284885+0000", 2024-09-15T00:11:34.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00084878899999999997 2024-09-15T00:11:34.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.288668+0000", 2024-09-15T00:11:34.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0037833369999999999 2024-09-15T00:11:34.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.288700+0000", 2024-09-15T00:11:34.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1962000000000003e-05 2024-09-15T00:11:34.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.288724+0000", 2024-09-15T00:11:34.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4189000000000002e-05 2024-09-15T00:11:34.975 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.975 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.975 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3160 3.1d 3:bf735e48:::rbd_data.1139b531b717.0000000000000064:head [set-alloc-hint object_size 2097152 write_size 2097152,write 925696~3756] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.624419+0000", 2024-09-15T00:11:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.299876598, 2024-09-15T00:11:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050107880000000004, 2024-09-15T00:11:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3160 2024-09-15T00:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624419+0000", 2024-09-15T00:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624419+0000", 2024-09-15T00:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624420+0000", 2024-09-15T00:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3790000000000001e-06 2024-09-15T00:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624428+0000", 2024-09-15T00:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1289999999999996e-06 2024-09-15T00:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624440+0000", 2024-09-15T00:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1252e-05 2024-09-15T00:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624445+0000", 2024-09-15T00:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5249999999999996e-06 2024-09-15T00:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624492+0000", 2024-09-15T00:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6517999999999997e-05 2024-09-15T00:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624605+0000", 2024-09-15T00:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113588 2024-09-15T00:11:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624748+0000", 2024-09-15T00:11:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014249 2024-09-15T00:11:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.626822+0000", 2024-09-15T00:11:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020740310000000001 2024-09-15T00:11:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.629341+0000", 2024-09-15T00:11:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0025189880000000002 2024-09-15T00:11:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.629384+0000", 2024-09-15T00:11:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3788000000000002e-05 2024-09-15T00:11:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.629430+0000", 2024-09-15T00:11:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5099999999999998e-05 2024-09-15T00:11:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3183 3.d e33/23 3:b2b2ff6e:::rbd_data.1139b531b717.000000000000006b:head v 33'19, mlcod=33'17)", 2024-09-15T00:11:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.658700+0000", 2024-09-15T00:11:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.2655957469999999, 2024-09-15T00:11:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051160099999999998, 2024-09-15T00:11:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658700+0000", 2024-09-15T00:11:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658700+0000", 2024-09-15T00:11:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658700+0000", 2024-09-15T00:11:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7400000000000003e-07 2024-09-15T00:11:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658707+0000", 2024-09-15T00:11:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4760000000000003e-06 2024-09-15T00:11:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658725+0000", 2024-09-15T00:11:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8539999999999999e-05 2024-09-15T00:11:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658732+0000", 2024-09-15T00:11:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1590000000000004e-06 2024-09-15T00:11:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658837+0000", 2024-09-15T00:11:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104941 2024-09-15T00:11:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.658857+0000", 2024-09-15T00:11:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9179999999999999e-05 2024-09-15T00:11:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663771+0000", 2024-09-15T00:11:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049141660000000002 2024-09-15T00:11:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663816+0000", 2024-09-15T00:11:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4873999999999998e-05 2024-09-15T00:11:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3184 3.d e33/23 3:b57e2e8d:::rbd_data.1139b531b717.000000000000006c:head v 33'20, mlcod=33'17)", 2024-09-15T00:11:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.659024+0000", 2024-09-15T00:11:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.265271598, 2024-09-15T00:11:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048185770000000001, 2024-09-15T00:11:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659024+0000", 2024-09-15T00:11:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659024+0000", 2024-09-15T00:11:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659024+0000", 2024-09-15T00:11:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8599999999999998e-07 2024-09-15T00:11:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659030+0000", 2024-09-15T00:11:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3430000000000003e-06 2024-09-15T00:11:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659036+0000", 2024-09-15T00:11:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4849999999999997e-06 2024-09-15T00:11:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659040+0000", 2024-09-15T00:11:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.206e-06 2024-09-15T00:11:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659080+0000", 2024-09-15T00:11:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9217000000000001e-05 2024-09-15T00:11:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659097+0000", 2024-09-15T00:11:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6969999999999998e-05 2024-09-15T00:11:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663819+0000", 2024-09-15T00:11:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047219289999999997 2024-09-15T00:11:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663842+0000", 2024-09-15T00:11:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3740999999999999e-05 2024-09-15T00:11:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3186 3.5 3:a3ee9f36:::rbd_data.1139b531b717.000000000000006e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.659078+0000", 2024-09-15T00:11:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.265217399, 2024-09-15T00:11:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004869852, 2024-09-15T00:11:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3186 2024-09-15T00:11:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659078+0000", 2024-09-15T00:11:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659078+0000", 2024-09-15T00:11:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659079+0000", 2024-09-15T00:11:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4499999999999996e-07 2024-09-15T00:11:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659083+0000", 2024-09-15T00:11:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8689999999999997e-06 2024-09-15T00:11:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659091+0000", 2024-09-15T00:11:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0639999999999994e-06 2024-09-15T00:11:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659104+0000", 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3049e-05 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659149+0000", 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5108000000000002e-05 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659228+0000", 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.941e-05 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:34.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.659312+0000", 2024-09-15T00:11:34.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3829e-05 2024-09-15T00:11:34.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:34.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.661416+0000", 2024-09-15T00:11:34.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021034080000000002 2024-09-15T00:11:34.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:34.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663849+0000", 2024-09-15T00:11:34.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0024330340000000002 2024-09-15T00:11:34.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:34.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663904+0000", 2024-09-15T00:11:34.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5535999999999998e-05 2024-09-15T00:11:34.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:34.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.663948+0000", 2024-09-15T00:11:34.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3699999999999998e-05 2024-09-15T00:11:34.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.996 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:34.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:34.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.996 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3221 3.17 3:eb942860:::rbd_data.1139b531b717.0000000000000050:head [set-alloc-hint object_size 2097152 write_size 2097152,write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:34.997 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.689861+0000", 2024-09-15T00:11:34.997 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.2344345170000004, 2024-09-15T00:11:34.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005121919, 2024-09-15T00:11:34.997 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:34.997 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:34.997 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:34.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:34.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:34.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:34.997 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3221 2024-09-15T00:11:34.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.997 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:34.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:34.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.689861+0000", 2024-09-15T00:11:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.689861+0000", 2024-09-15T00:11:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.689862+0000", 2024-09-15T00:11:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9999999999999997e-07 2024-09-15T00:11:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.689872+0000", 2024-09-15T00:11:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0739e-05 2024-09-15T00:11:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.689889+0000", 2024-09-15T00:11:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6948000000000001e-05 2024-09-15T00:11:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.689894+0000", 2024-09-15T00:11:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7709999999999997e-06 2024-09-15T00:11:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.689943+0000", 2024-09-15T00:11:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8916999999999999e-05 2024-09-15T00:11:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.690038+0000", 2024-09-15T00:11:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5320000000000002e-05 2024-09-15T00:11:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.690150+0000", 2024-09-15T00:11:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000111716 2024-09-15T00:11:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.690683+0000", 2024-09-15T00:11:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053344999999999998 2024-09-15T00:11:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.694907+0000", 2024-09-15T00:11:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042237389999999998 2024-09-15T00:11:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.694940+0000", 2024-09-15T00:11:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3046999999999997e-05 2024-09-15T00:11:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.694983+0000", 2024-09-15T00:11:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2571999999999998e-05 2024-09-15T00:11:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3422 3.16 3:683c2b90:::rbd_data.1139b531b717.0000000000000007:head [stat,truncate 548864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.868521+0000", 2024-09-15T00:11:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0557748509999998, 2024-09-15T00:11:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051108050000000004, 2024-09-15T00:11:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3422 2024-09-15T00:11:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868521+0000", 2024-09-15T00:11:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868521+0000", 2024-09-15T00:11:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868521+0000", 2024-09-15T00:11:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9400000000000004e-07 2024-09-15T00:11:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868532+0000", 2024-09-15T00:11:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1e-05 2024-09-15T00:11:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868544+0000", 2024-09-15T00:11:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1673e-05 2024-09-15T00:11:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868551+0000", 2024-09-15T00:11:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9469999999999997e-06 2024-09-15T00:11:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868598+0000", 2024-09-15T00:11:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7228999999999998e-05 2024-09-15T00:11:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868665+0000", 2024-09-15T00:11:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6334999999999994e-05 2024-09-15T00:11:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.868775+0000", 2024-09-15T00:11:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110649 2024-09-15T00:11:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.869761+0000", 2024-09-15T00:11:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00098550000000000005 2024-09-15T00:11:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.873536+0000", 2024-09-15T00:11:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0037752739999999999 2024-09-15T00:11:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.873589+0000", 2024-09-15T00:11:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3019000000000002e-05 2024-09-15T00:11:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.873631+0000", 2024-09-15T00:11:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2284999999999999e-05 2024-09-15T00:11:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3509 3.17 3:e9bf9e8f:::rbd_data.1139b531b717.000000000000002c:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1048576~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.951418+0000", 2024-09-15T00:11:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.9728771900000002, 2024-09-15T00:11:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053473030000000003, 2024-09-15T00:11:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3509 2024-09-15T00:11:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951418+0000", 2024-09-15T00:11:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951418+0000", 2024-09-15T00:11:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951420+0000", 2024-09-15T00:11:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4929999999999999e-06 2024-09-15T00:11:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951429+0000", 2024-09-15T00:11:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2879999999999998e-06 2024-09-15T00:11:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951439+0000", 2024-09-15T00:11:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0205e-05 2024-09-15T00:11:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951444+0000", 2024-09-15T00:11:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0139999999999998e-06 2024-09-15T00:11:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951487+0000", 2024-09-15T00:11:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2364999999999998e-05 2024-09-15T00:11:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951567+0000", 2024-09-15T00:11:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0248999999999997e-05 2024-09-15T00:11:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.951657+0000", 2024-09-15T00:11:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9717999999999997e-05 2024-09-15T00:11:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955025+0000", 2024-09-15T00:11:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003368149 2024-09-15T00:11:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.956667+0000", 2024-09-15T00:11:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0016422640000000001 2024-09-15T00:11:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.956722+0000", 2024-09-15T00:11:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4926999999999998e-05 2024-09-15T00:11:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.956765+0000", 2024-09-15T00:11:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3631000000000003e-05 2024-09-15T00:11:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3584 3.17 3:ef86f319:::rbd_data.1139b531b717.000000000000006d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 498027~1685] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T00:11:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:30.032469+0000", 2024-09-15T00:11:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8918262610000003, 2024-09-15T00:11:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0056325170000000001, 2024-09-15T00:11:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3584 2024-09-15T00:11:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032469+0000", 2024-09-15T00:11:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032469+0000", 2024-09-15T00:11:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032470+0000", 2024-09-15T00:11:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3710000000000001e-06 2024-09-15T00:11:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032479+0000", 2024-09-15T00:11:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5839999999999995e-06 2024-09-15T00:11:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032492+0000", 2024-09-15T00:11:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3125000000000001e-05 2024-09-15T00:11:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032499+0000", 2024-09-15T00:11:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:11:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032553+0000", 2024-09-15T00:11:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3761000000000002e-05 2024-09-15T00:11:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-15T00:11:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.032557+0000", 2024-09-15T00:11:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6659999999999999e-06 2024-09-15T00:11:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.035250+0000", 2024-09-15T00:11:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026922890000000001 2024-09-15T00:11:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.035319+0000", 2024-09-15T00:11:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8919999999999997e-05 2024-09-15T00:11:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.035441+0000", 2024-09-15T00:11:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012218799999999999 2024-09-15T00:11:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.036328+0000", 2024-09-15T00:11:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00088673199999999999 2024-09-15T00:11:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.038036+0000", 2024-09-15T00:11:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00170856 2024-09-15T00:11:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.038073+0000", 2024-09-15T00:11:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7324999999999999e-05 2024-09-15T00:11:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:30.038102+0000", 2024-09-15T00:11:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8194000000000001e-05 2024-09-15T00:11:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3696 3.17 3:e9bf9e8f:::rbd_data.1139b531b717.000000000000002c:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.077406+0000", 2024-09-15T00:11:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.846888968, 2024-09-15T00:11:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051398219999999996, 2024-09-15T00:11:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:35.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:35.023 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3696 2024-09-15T00:11:35.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.023 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.077406+0000", 2024-09-15T00:11:35.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.077406+0000", 2024-09-15T00:11:35.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.077408+0000", 2024-09-15T00:11:35.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.102e-06 2024-09-15T00:11:35.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.077412+0000", 2024-09-15T00:11:35.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7210000000000002e-06 2024-09-15T00:11:35.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.077424+0000", 2024-09-15T00:11:35.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.184e-05 2024-09-15T00:11:35.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.077431+0000", 2024-09-15T00:11:35.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.528e-06 2024-09-15T00:11:35.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.077501+0000", 2024-09-15T00:11:35.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0020999999999999e-05 2024-09-15T00:11:35.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.077581+0000", 2024-09-15T00:11:35.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9958999999999998e-05 2024-09-15T00:11:35.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:35.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.077734+0000", 2024-09-15T00:11:35.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015331700000000001 2024-09-15T00:11:35.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:35.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.080903+0000", 2024-09-15T00:11:35.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031688139999999998 2024-09-15T00:11:35.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:35.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.082492+0000", 2024-09-15T00:11:35.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001589563 2024-09-15T00:11:35.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.082522+0000", 2024-09-15T00:11:35.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9546000000000001e-05 2024-09-15T00:11:35.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.082546+0000", 2024-09-15T00:11:35.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4411000000000001e-05 2024-09-15T00:11:35.028 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.028 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.028 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.028 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3710 3.5 3:a3ee9f36:::rbd_data.1139b531b717.000000000000006e:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:35.028 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.081532+0000", 2024-09-15T00:11:35.028 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.8427629159999999, 2024-09-15T00:11:35.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0054723050000000002, 2024-09-15T00:11:35.028 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.029 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.029 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:35.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:35.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:35.029 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3710 2024-09-15T00:11:35.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.029 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081532+0000", 2024-09-15T00:11:35.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081532+0000", 2024-09-15T00:11:35.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081533+0000", 2024-09-15T00:11:35.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9899999999999999e-07 2024-09-15T00:11:35.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081538+0000", 2024-09-15T00:11:35.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.724e-06 2024-09-15T00:11:35.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081549+0000", 2024-09-15T00:11:35.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.101e-05 2024-09-15T00:11:35.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081556+0000", 2024-09-15T00:11:35.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9879999999999998e-06 2024-09-15T00:11:35.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081616+0000", 2024-09-15T00:11:35.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0239999999999999e-05 2024-09-15T00:11:35.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081691+0000", 2024-09-15T00:11:35.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4678000000000006e-05 2024-09-15T00:11:35.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:35.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081833+0000", 2024-09-15T00:11:35.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014229800000000001 2024-09-15T00:11:35.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:35.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.082760+0000", 2024-09-15T00:11:35.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00092685 2024-09-15T00:11:35.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:35.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.086953+0000", 2024-09-15T00:11:35.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041931720000000002 2024-09-15T00:11:35.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.086978+0000", 2024-09-15T00:11:35.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5046e-05 2024-09-15T00:11:35.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.087005+0000", 2024-09-15T00:11:35.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.65e-05 2024-09-15T00:11:35.034 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.034 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.034 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.034 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3711 3.17 3:ebd45212:::rbd_data.1139b531b717.000000000000006f:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:35.034 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.081832+0000", 2024-09-15T00:11:35.034 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.8424629769999998, 2024-09-15T00:11:35.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051790359999999997, 2024-09-15T00:11:35.034 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.034 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.035 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.035 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:35.035 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:35.035 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:35.035 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3711 2024-09-15T00:11:35.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.035 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081832+0000", 2024-09-15T00:11:35.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081832+0000", 2024-09-15T00:11:35.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081833+0000", 2024-09-15T00:11:35.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2600000000000002e-07 2024-09-15T00:11:35.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081837+0000", 2024-09-15T00:11:35.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8600000000000003e-06 2024-09-15T00:11:35.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081846+0000", 2024-09-15T00:11:35.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6090000000000007e-06 2024-09-15T00:11:35.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081853+0000", 2024-09-15T00:11:35.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6699999999999997e-06 2024-09-15T00:11:35.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081909+0000", 2024-09-15T00:11:35.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5996999999999999e-05 2024-09-15T00:11:35.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.081978+0000", 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9331000000000004e-05 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.082117+0000", 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013808599999999999 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.083583+0000", 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0014669259999999999 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:35.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.086940+0000", 2024-09-15T00:11:35.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033568000000000001 2024-09-15T00:11:35.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.086970+0000", 2024-09-15T00:11:35.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9343e-05 2024-09-15T00:11:35.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.087011+0000", 2024-09-15T00:11:35.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1788e-05 2024-09-15T00:11:35.039 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.040 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.040 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.040 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3736 3.d e35/23 3:b1632ea2:::rbd_data.1139b531b717.0000000000000009:head v 35'32, mlcod=33'29)", 2024-09-15T00:11:35.040 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.099759+0000", 2024-09-15T00:11:35.040 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.8245366700000001, 2024-09-15T00:11:35.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053403909999999999, 2024-09-15T00:11:35.040 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.040 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.040 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.040 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099759+0000", 2024-09-15T00:11:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099759+0000", 2024-09-15T00:11:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099760+0000", 2024-09-15T00:11:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6800000000000009e-07 2024-09-15T00:11:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099769+0000", 2024-09-15T00:11:35.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7839999999999998e-06 2024-09-15T00:11:35.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099782+0000", 2024-09-15T00:11:35.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2787e-05 2024-09-15T00:11:35.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099788+0000", 2024-09-15T00:11:35.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4009999999999996e-06 2024-09-15T00:11:35.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099838+0000", 2024-09-15T00:11:35.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0861e-05 2024-09-15T00:11:35.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099857+0000", 2024-09-15T00:11:35.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8790000000000001e-05 2024-09-15T00:11:35.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.105059+0000", 2024-09-15T00:11:35.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052020950000000003 2024-09-15T00:11:35.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.105099+0000", 2024-09-15T00:11:35.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9705e-05 2024-09-15T00:11:35.044 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.044 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.044 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.044 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.044 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.044 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:35.046 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:11:35.046 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:35.044+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:35.046 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:11:35.056 INFO:tasks.ceph.mon.a.smithi171.stderr:2024-09-15T00:11:35.054+0000 7f0425d35640 -1 mon.a@0(leader).osd e38 definitely_dead 0 2024-09-15T00:11:35.115 INFO:teuthology.orchestra.run.smithi171.stdout:leaving image image_client.0-clone7 intact 2024-09-15T00:11:35.146 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:35.144+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:35.195 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:35.195 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:35.195 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:35.195 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:35.204 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:11:35.213 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-15T00:11:35.239 INFO:teuthology.orchestra.run.smithi171.stdout:921 punch from 0x15d49ee to 0x15d50bd, (0x6cf bytes) 2024-09-15T00:11:35.248 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:35.247+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:35.254 INFO:teuthology.orchestra.run.smithi171.stdout:924 writesame 0x89a827e thru 0x89aac4d data_size 0x18(0x29d0 bytes) 2024-09-15T00:11:35.266 INFO:teuthology.orchestra.run.smithi171.stdout:925 writesame 0xebdd384 thru 0xebe3bd0 data_size 0x1(0x684d bytes) 2024-09-15T00:11:35.267 INFO:teuthology.orchestra.run.smithi171.stdout:928 read 0xda2f8ba thru 0xda35aaa (0x61f1 bytes) 2024-09-15T00:11:35.283 INFO:teuthology.orchestra.run.smithi171.stdout:929 trunc from 0xebe3bd1 to 0x106ca77 2024-09-15T00:11:35.316 INFO:teuthology.orchestra.run.smithi171.stdout:930 read 0xc911e2 thru 0xc9a204 (0x9023 bytes) 2024-09-15T00:11:35.316 INFO:teuthology.orchestra.run.smithi171.stdout:931 punch from 0x8a400e to 0x8a8b47, (0x4b39 bytes) 2024-09-15T00:11:35.322 INFO:teuthology.orchestra.run.smithi171.stdout:932 trunc from 0x106ca77 to 0x6665af8 2024-09-15T00:11:35.323 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:35.323 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:35.323 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:35.323 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:35.324 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:35.328 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.328 INFO:teuthology.orchestra.run.smithi171.stdout:937 write 0x6505bdb thru 0x6510a25 (0xae4b bytes) 2024-09-15T00:11:35.333 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:11:35.334 INFO:teuthology.orchestra.run.smithi171.stdout:939 write 0x1fb1fdc thru 0x1fb60a8 (0x40cd bytes) 2024-09-15T00:11:35.337 INFO:teuthology.orchestra.run.smithi171.stdout:940 write 0x5f75aeb thru 0x5f7d3b2 (0x78c8 bytes) 2024-09-15T00:11:35.340 INFO:teuthology.orchestra.run.smithi171.stdout:941 read 0x26bb6aa thru 0x26bd53b (0x1e92 bytes) 2024-09-15T00:11:35.340 INFO:teuthology.orchestra.run.smithi171.stdout:942 writesame 0x57d18d2 thru 0x57d1bab data_size 0x5(0x2da bytes) 2024-09-15T00:11:35.342 INFO:teuthology.orchestra.run.smithi171.stdout:943 read 0x28fc686 thru 0x290402c (0x79a7 bytes) 2024-09-15T00:11:35.348 INFO:teuthology.orchestra.run.smithi171.stdout:946 trunc from 0x6665af8 to 0x1de7abc 2024-09-15T00:11:35.358 INFO:teuthology.orchestra.run.smithi171.stdout:947 read 0x3a9292 thru 0x3aef0f (0x5c7e bytes) 2024-09-15T00:11:35.375 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:35.373+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:35.375 INFO:teuthology.orchestra.run.smithi171.stdout:948 write 0xb6168e3 thru 0xb6201b7 (0x98d5 bytes) 2024-09-15T00:11:35.378 INFO:teuthology.orchestra.run.smithi171.stdout:949 writesame 0x21d8980 thru 0x21d8ef3 data_size 0x2(0x574 bytes) 2024-09-15T00:11:35.388 INFO:teuthology.orchestra.run.smithi171.stdout:951 trunc from 0xb6201b8 to 0x5cf6d04 2024-09-15T00:11:35.404 INFO:teuthology.orchestra.run.smithi171.stdout:952 trunc from 0x5cf6d04 to 0xad29248 2024-09-15T00:11:35.409 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.409 INFO:teuthology.orchestra.run.smithi171.stdout:954 read 0x1b3800f thru 0x1b4196a (0x995c bytes) 2024-09-15T00:11:35.410 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.410 INFO:teuthology.orchestra.run.smithi171.stdout:959 read 0x9ea0e12 thru 0x9ea964d (0x883c bytes) 2024-09-15T00:11:35.419 INFO:teuthology.orchestra.run.smithi171.stdout:960 writesame 0xddb5340 thru 0xddb6ecf data_size 0x48(0x1b90 bytes) 2024-09-15T00:11:35.419 INFO:teuthology.orchestra.run.smithi171.stdout:961 writesame 0x14362fd thru 0x1438ae0 data_size 0x25(0x27e4 bytes) 2024-09-15T00:11:35.421 INFO:teuthology.orchestra.run.smithi171.stdout:962 writesame 0x9183029 thru 0x9192dbc data_size 0x2(0xfd94 bytes) 2024-09-15T00:11:35.449 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:35.447+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:35.484 INFO:teuthology.orchestra.run.smithi171.stdout:963 read 0x76dbb2 thru 0x775d57 (0x81a6 bytes) 2024-09-15T00:11:35.485 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:35.485 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:35.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:35.485 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:35.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.485 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3037 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'86, mlcod=33'82)", 2024-09-15T00:11:35.485 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.497583+0000", 2024-09-15T00:11:35.486 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.9851006509999998, 2024-09-15T00:11:35.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077497779999999997, 2024-09-15T00:11:35.486 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.486 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.486 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.486 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497583+0000", 2024-09-15T00:11:35.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497583+0000", 2024-09-15T00:11:35.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497584+0000", 2024-09-15T00:11:35.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6599999999999996e-07 2024-09-15T00:11:35.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497588+0000", 2024-09-15T00:11:35.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3529999999999997e-06 2024-09-15T00:11:35.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497598+0000", 2024-09-15T00:11:35.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4839999999999993e-06 2024-09-15T00:11:35.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497602+0000", 2024-09-15T00:11:35.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:11:35.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497643+0000", 2024-09-15T00:11:35.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0326e-05 2024-09-15T00:11:35.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497662+0000", 2024-09-15T00:11:35.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9590999999999999e-05 2024-09-15T00:11:35.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505303+0000", 2024-09-15T00:11:35.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076408919999999998 2024-09-15T00:11:35.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505333+0000", 2024-09-15T00:11:35.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0071999999999999e-05 2024-09-15T00:11:35.489 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.490 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.490 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.490 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3038 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'87, mlcod=33'82)", 2024-09-15T00:11:35.490 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:35.490 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.9842561390000002, 2024-09-15T00:11:35.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069253919999999998, 2024-09-15T00:11:35.490 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.490 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.490 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.490 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:35.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:35.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:35.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6899999999999997e-07 2024-09-15T00:11:35.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498435+0000", 2024-09-15T00:11:35.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1330000000000004e-06 2024-09-15T00:11:35.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498444+0000", 2024-09-15T00:11:35.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6749999999999997e-06 2024-09-15T00:11:35.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498449+0000", 2024-09-15T00:11:35.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5449999999999997e-06 2024-09-15T00:11:35.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498485+0000", 2024-09-15T00:11:35.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5737e-05 2024-09-15T00:11:35.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498501+0000", 2024-09-15T00:11:35.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6637e-05 2024-09-15T00:11:35.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505334+0000", 2024-09-15T00:11:35.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0068333100000000004 2024-09-15T00:11:35.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505353+0000", 2024-09-15T00:11:35.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8785999999999999e-05 2024-09-15T00:11:35.494 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.494 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.494 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.494 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3039 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'88, mlcod=33'83)", 2024-09-15T00:11:35.494 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:35.494 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.9834655049999999, 2024-09-15T00:11:35.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0061577309999999996, 2024-09-15T00:11:35.495 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.495 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.495 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.495 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:35.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:35.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:35.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3099999999999997e-07 2024-09-15T00:11:35.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499224+0000", 2024-09-15T00:11:35.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.916e-06 2024-09-15T00:11:35.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499233+0000", 2024-09-15T00:11:35.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7949999999999996e-06 2024-09-15T00:11:35.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499239+0000", 2024-09-15T00:11:35.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0090000000000004e-06 2024-09-15T00:11:35.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499292+0000", 2024-09-15T00:11:35.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3049e-05 2024-09-15T00:11:35.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499312+0000", 2024-09-15T00:11:35.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0574e-05 2024-09-15T00:11:35.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505355+0000", 2024-09-15T00:11:35.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060423930000000001 2024-09-15T00:11:35.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505376+0000", 2024-09-15T00:11:35.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1364000000000001e-05 2024-09-15T00:11:35.498 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.498 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.498 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.499 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3040 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'89, mlcod=33'83)", 2024-09-15T00:11:35.499 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.499752+0000", 2024-09-15T00:11:35.499 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.9829317140000002, 2024-09-15T00:11:35.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0059059300000000002, 2024-09-15T00:11:35.499 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.499 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.499 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.499 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499752+0000", 2024-09-15T00:11:35.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499752+0000", 2024-09-15T00:11:35.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499753+0000", 2024-09-15T00:11:35.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1399999999999997e-07 2024-09-15T00:11:35.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499763+0000", 2024-09-15T00:11:35.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0365e-05 2024-09-15T00:11:35.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499774+0000", 2024-09-15T00:11:35.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0920999999999999e-05 2024-09-15T00:11:35.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499780+0000", 2024-09-15T00:11:35.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.02e-06 2024-09-15T00:11:35.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499829+0000", 2024-09-15T00:11:35.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9066000000000001e-05 2024-09-15T00:11:35.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499849+0000", 2024-09-15T00:11:35.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9596999999999999e-05 2024-09-15T00:11:35.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505633+0000", 2024-09-15T00:11:35.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057841189999999999 2024-09-15T00:11:35.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505658+0000", 2024-09-15T00:11:35.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5228e-05 2024-09-15T00:11:35.503 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.503 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.503 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.503 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3041 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'90, mlcod=33'83)", 2024-09-15T00:11:35.503 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.500417+0000", 2024-09-15T00:11:35.503 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.982266901, 2024-09-15T00:11:35.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052558600000000002, 2024-09-15T00:11:35.503 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.504 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.504 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.504 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500417+0000", 2024-09-15T00:11:35.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500417+0000", 2024-09-15T00:11:35.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500418+0000", 2024-09-15T00:11:35.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6199999999999998e-07 2024-09-15T00:11:35.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500422+0000", 2024-09-15T00:11:35.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1169999999999997e-06 2024-09-15T00:11:35.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500429+0000", 2024-09-15T00:11:35.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6620000000000005e-06 2024-09-15T00:11:35.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500435+0000", 2024-09-15T00:11:35.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6130000000000003e-06 2024-09-15T00:11:35.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500471+0000", 2024-09-15T00:11:35.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5899999999999998e-05 2024-09-15T00:11:35.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500484+0000", 2024-09-15T00:11:35.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3502999999999999e-05 2024-09-15T00:11:35.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505660+0000", 2024-09-15T00:11:35.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051752930000000001 2024-09-15T00:11:35.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505673+0000", 2024-09-15T00:11:35.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3210000000000001e-05 2024-09-15T00:11:35.508 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.508 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.508 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.508 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3102 3.a 3:566166ea:::rbd_data.1139b531b717.000000000000000b:head [stat,truncate 385024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:35.508 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:35.508 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.9128243129999998, 2024-09-15T00:11:35.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060108410000000003, 2024-09-15T00:11:35.508 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.508 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.508 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.508 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:35.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:35.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:35.509 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3102 2024-09-15T00:11:35.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.509 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:35.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:35.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:35.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:11:35.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569863+0000", 2024-09-15T00:11:35.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.819e-06 2024-09-15T00:11:35.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569871+0000", 2024-09-15T00:11:35.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6690000000000006e-06 2024-09-15T00:11:35.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569882+0000", 2024-09-15T00:11:35.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1195999999999999e-05 2024-09-15T00:11:35.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.570647+0000", 2024-09-15T00:11:35.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00076547900000000001 2024-09-15T00:11:35.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.570734+0000", 2024-09-15T00:11:35.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7087000000000002e-05 2024-09-15T00:11:35.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:35.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.570863+0000", 2024-09-15T00:11:35.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012899799999999999 2024-09-15T00:11:35.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:35.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.573483+0000", 2024-09-15T00:11:35.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026192799999999999 2024-09-15T00:11:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.575796+0000", 2024-09-15T00:11:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0023136570000000002 2024-09-15T00:11:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.575827+0000", 2024-09-15T00:11:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0468e-05 2024-09-15T00:11:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.575871+0000", 2024-09-15T00:11:35.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3656999999999999e-05 2024-09-15T00:11:35.514 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.514 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.514 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.514 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3152 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'108, mlcod=33'104)", 2024-09-15T00:11:35.514 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.619166+0000", 2024-09-15T00:11:35.514 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.8635180980000001, 2024-09-15T00:11:35.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049522180000000004, 2024-09-15T00:11:35.514 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.514 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.514 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.514 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619166+0000", 2024-09-15T00:11:35.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619166+0000", 2024-09-15T00:11:35.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619166+0000", 2024-09-15T00:11:35.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:11:35.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619170+0000", 2024-09-15T00:11:35.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2990000000000001e-06 2024-09-15T00:11:35.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619177+0000", 2024-09-15T00:11:35.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7260000000000002e-06 2024-09-15T00:11:35.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619181+0000", 2024-09-15T00:11:35.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9759999999999997e-06 2024-09-15T00:11:35.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619215+0000", 2024-09-15T00:11:35.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4066000000000002e-05 2024-09-15T00:11:35.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619230+0000", 2024-09-15T00:11:35.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4392999999999999e-05 2024-09-15T00:11:35.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624081+0000", 2024-09-15T00:11:35.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048513560000000002 2024-09-15T00:11:35.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624118+0000", 2024-09-15T00:11:35.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6973e-05 2024-09-15T00:11:35.518 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.518 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.518 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.518 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3379 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'162, mlcod=33'158)", 2024-09-15T00:11:35.518 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.833370+0000", 2024-09-15T00:11:35.518 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.6493137510000002, 2024-09-15T00:11:35.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049772030000000004, 2024-09-15T00:11:35.519 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.519 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.519 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.519 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833370+0000", 2024-09-15T00:11:35.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833370+0000", 2024-09-15T00:11:35.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833371+0000", 2024-09-15T00:11:35.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.23e-07 2024-09-15T00:11:35.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833375+0000", 2024-09-15T00:11:35.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0389999999999998e-06 2024-09-15T00:11:35.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833384+0000", 2024-09-15T00:11:35.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0110000000000006e-06 2024-09-15T00:11:35.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833388+0000", 2024-09-15T00:11:35.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2490000000000003e-06 2024-09-15T00:11:35.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833450+0000", 2024-09-15T00:11:35.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1849999999999999e-05 2024-09-15T00:11:35.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833469+0000", 2024-09-15T00:11:35.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9222999999999999e-05 2024-09-15T00:11:35.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.838319+0000", 2024-09-15T00:11:35.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048498860000000003 2024-09-15T00:11:35.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.838347+0000", 2024-09-15T00:11:35.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8221999999999998e-05 2024-09-15T00:11:35.523 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.523 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.523 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.523 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3502 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'181, mlcod=33'177)", 2024-09-15T00:11:35.523 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:35.523 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5343691469999996, 2024-09-15T00:11:35.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0067634649999999998, 2024-09-15T00:11:35.523 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.523 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.523 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.523 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:35.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:35.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:35.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5199999999999997e-07 2024-09-15T00:11:35.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948319+0000", 2024-09-15T00:11:35.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0740000000000003e-06 2024-09-15T00:11:35.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948329+0000", 2024-09-15T00:11:35.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1409999999999994e-06 2024-09-15T00:11:35.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948335+0000", 2024-09-15T00:11:35.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6290000000000004e-06 2024-09-15T00:11:35.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948398+0000", 2024-09-15T00:11:35.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2422999999999995e-05 2024-09-15T00:11:35.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948412+0000", 2024-09-15T00:11:35.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4538999999999999e-05 2024-09-15T00:11:35.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.954988+0000", 2024-09-15T00:11:35.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.006575455 2024-09-15T00:11:35.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955078+0000", 2024-09-15T00:11:35.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0651999999999999e-05 2024-09-15T00:11:35.527 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.527 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.527 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.527 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3503 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'182, mlcod=33'177)", 2024-09-15T00:11:35.527 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.948857+0000", 2024-09-15T00:11:35.527 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5338265519999998, 2024-09-15T00:11:35.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0062792079999999997, 2024-09-15T00:11:35.527 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.528 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.528 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.528 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948857+0000", 2024-09-15T00:11:35.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948857+0000", 2024-09-15T00:11:35.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948858+0000", 2024-09-15T00:11:35.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6899999999999997e-07 2024-09-15T00:11:35.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948862+0000", 2024-09-15T00:11:35.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2429999999999999e-06 2024-09-15T00:11:35.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948872+0000", 2024-09-15T00:11:35.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5340000000000005e-06 2024-09-15T00:11:35.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948883+0000", 2024-09-15T00:11:35.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1304999999999999e-05 2024-09-15T00:11:35.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948917+0000", 2024-09-15T00:11:35.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3510999999999998e-05 2024-09-15T00:11:35.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948929+0000", 2024-09-15T00:11:35.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2473000000000001e-05 2024-09-15T00:11:35.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955082+0000", 2024-09-15T00:11:35.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0061525010000000003 2024-09-15T00:11:35.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955137+0000", 2024-09-15T00:11:35.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5071999999999997e-05 2024-09-15T00:11:35.531 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.531 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.531 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.531 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3504 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'183, mlcod=33'178)", 2024-09-15T00:11:35.532 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:35.532 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5330711749999999, 2024-09-15T00:11:35.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058979369999999998, 2024-09-15T00:11:35.532 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.532 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.532 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.532 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:35.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:35.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:35.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8400000000000004e-07 2024-09-15T00:11:35.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949620+0000", 2024-09-15T00:11:35.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9739999999999996e-06 2024-09-15T00:11:35.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949630+0000", 2024-09-15T00:11:35.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6399999999999992e-06 2024-09-15T00:11:35.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949635+0000", 2024-09-15T00:11:35.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:11:35.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949688+0000", 2024-09-15T00:11:35.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2667000000000002e-05 2024-09-15T00:11:35.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949701+0000", 2024-09-15T00:11:35.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.36e-05 2024-09-15T00:11:35.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955477+0000", 2024-09-15T00:11:35.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057763119999999996 2024-09-15T00:11:35.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955511+0000", 2024-09-15T00:11:35.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3337000000000003e-05 2024-09-15T00:11:35.536 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.536 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.536 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.536 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3505 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'184, mlcod=33'178)", 2024-09-15T00:11:35.536 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.950258+0000", 2024-09-15T00:11:35.536 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5324258339999997, 2024-09-15T00:11:35.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052830350000000002, 2024-09-15T00:11:35.536 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.536 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.536 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.536 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950258+0000", 2024-09-15T00:11:35.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950258+0000", 2024-09-15T00:11:35.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950259+0000", 2024-09-15T00:11:35.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7700000000000004e-07 2024-09-15T00:11:35.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950262+0000", 2024-09-15T00:11:35.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6839999999999998e-06 2024-09-15T00:11:35.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950271+0000", 2024-09-15T00:11:35.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8249999999999994e-06 2024-09-15T00:11:35.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950276+0000", 2024-09-15T00:11:35.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7600000000000002e-06 2024-09-15T00:11:35.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950311+0000", 2024-09-15T00:11:35.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4629999999999999e-05 2024-09-15T00:11:35.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950323+0000", 2024-09-15T00:11:35.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1853e-05 2024-09-15T00:11:35.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955513+0000", 2024-09-15T00:11:35.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051903599999999998 2024-09-15T00:11:35.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955541+0000", 2024-09-15T00:11:35.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8345999999999999e-05 2024-09-15T00:11:35.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.540 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.540 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3724 3.1e e35/23 3:792e48de:::rbd_object_map.1139b531b717:head v 35'223, mlcod=35'214)", 2024-09-15T00:11:35.540 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.097406+0000", 2024-09-15T00:11:35.540 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.3852782479999997, 2024-09-15T00:11:35.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050324560000000003, 2024-09-15T00:11:35.541 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.541 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.541 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.541 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097406+0000", 2024-09-15T00:11:35.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097406+0000", 2024-09-15T00:11:35.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097406+0000", 2024-09-15T00:11:35.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5299999999999999e-07 2024-09-15T00:11:35.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097409+0000", 2024-09-15T00:11:35.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2399999999999999e-06 2024-09-15T00:11:35.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097419+0000", 2024-09-15T00:11:35.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7920000000000004e-06 2024-09-15T00:11:35.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097425+0000", 2024-09-15T00:11:35.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7610000000000004e-06 2024-09-15T00:11:35.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097460+0000", 2024-09-15T00:11:35.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4997999999999999e-05 2024-09-15T00:11:35.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097472+0000", 2024-09-15T00:11:35.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1627000000000001e-05 2024-09-15T00:11:35.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102401+0000", 2024-09-15T00:11:35.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049290480000000001 2024-09-15T00:11:35.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102438+0000", 2024-09-15T00:11:35.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7537000000000003e-05 2024-09-15T00:11:35.544 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.544 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.545 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.545 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3726 3.1e e35/23 3:792e48de:::rbd_object_map.1139b531b717:head v 35'225, mlcod=35'220)", 2024-09-15T00:11:35.545 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.099757+0000", 2024-09-15T00:11:35.545 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.3829268429999999, 2024-09-15T00:11:35.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00535652, 2024-09-15T00:11:35.545 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.545 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.545 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.545 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099757+0000", 2024-09-15T00:11:35.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099757+0000", 2024-09-15T00:11:35.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099758+0000", 2024-09-15T00:11:35.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7400000000000002e-07 2024-09-15T00:11:35.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099764+0000", 2024-09-15T00:11:35.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0469999999999999e-06 2024-09-15T00:11:35.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099774+0000", 2024-09-15T00:11:35.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0118999999999999e-05 2024-09-15T00:11:35.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099781+0000", 2024-09-15T00:11:35.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7730000000000002e-06 2024-09-15T00:11:35.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099826+0000", 2024-09-15T00:11:35.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5229999999999999e-05 2024-09-15T00:11:35.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099844+0000", 2024-09-15T00:11:35.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8345000000000001e-05 2024-09-15T00:11:35.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.105063+0000", 2024-09-15T00:11:35.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052188920000000001 2024-09-15T00:11:35.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.105114+0000", 2024-09-15T00:11:35.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0340000000000003e-05 2024-09-15T00:11:35.549 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.549 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.549 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.549 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3737 3.a 3:53de29a7:::rbd_data.1139b531b717.000000000000000a:head [stat,truncate 270336] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:35.549 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.100151+0000", 2024-09-15T00:11:35.549 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.3825330300000003, 2024-09-15T00:11:35.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076797300000000001, 2024-09-15T00:11:35.550 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.550 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.550 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:35.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:35.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:35.550 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3737 2024-09-15T00:11:35.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.550 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100151+0000", 2024-09-15T00:11:35.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100151+0000", 2024-09-15T00:11:35.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100153+0000", 2024-09-15T00:11:35.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6530000000000001e-06 2024-09-15T00:11:35.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100158+0000", 2024-09-15T00:11:35.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7100000000000004e-06 2024-09-15T00:11:35.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100173+0000", 2024-09-15T00:11:35.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4242e-05 2024-09-15T00:11:35.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100180+0000", 2024-09-15T00:11:35.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7680000000000008e-06 2024-09-15T00:11:35.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100248+0000", 2024-09-15T00:11:35.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7459000000000006e-05 2024-09-15T00:11:35.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100321+0000", 2024-09-15T00:11:35.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2854000000000004e-05 2024-09-15T00:11:35.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:35.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100418+0000", 2024-09-15T00:11:35.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7827999999999999e-05 2024-09-15T00:11:35.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:35.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.105090+0000", 2024-09-15T00:11:35.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0046714349999999998 2024-09-15T00:11:35.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:35.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.107770+0000", 2024-09-15T00:11:35.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026799660000000002 2024-09-15T00:11:35.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.107801+0000", 2024-09-15T00:11:35.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1158000000000002e-05 2024-09-15T00:11:35.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.107831+0000", 2024-09-15T00:11:35.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9657000000000001e-05 2024-09-15T00:11:35.555 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.555 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.555 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.555 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3738 3.a 3:566166ea:::rbd_data.1139b531b717.000000000000000b:head [stat,truncate 270336] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:35.555 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.102790+0000", 2024-09-15T00:11:35.556 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.3798936059999996, 2024-09-15T00:11:35.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053085249999999997, 2024-09-15T00:11:35.556 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.556 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.556 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.556 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:35.556 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:35.556 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:35.556 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3738 2024-09-15T00:11:35.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.556 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102790+0000", 2024-09-15T00:11:35.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102790+0000", 2024-09-15T00:11:35.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102791+0000", 2024-09-15T00:11:35.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1900000000000001e-07 2024-09-15T00:11:35.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102796+0000", 2024-09-15T00:11:35.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5560000000000001e-06 2024-09-15T00:11:35.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102809+0000", 2024-09-15T00:11:35.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3213e-05 2024-09-15T00:11:35.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102817+0000", 2024-09-15T00:11:35.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6090000000000003e-06 2024-09-15T00:11:35.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102859+0000", 2024-09-15T00:11:35.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2536000000000001e-05 2024-09-15T00:11:35.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102933+0000", 2024-09-15T00:11:35.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3960000000000003e-05 2024-09-15T00:11:35.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:35.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.103108+0000", 2024-09-15T00:11:35.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00017522800000000001 2024-09-15T00:11:35.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:35.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.105349+0000", 2024-09-15T00:11:35.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002240377 2024-09-15T00:11:35.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:35.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.108036+0000", 2024-09-15T00:11:35.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002686964 2024-09-15T00:11:35.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.108066+0000", 2024-09-15T00:11:35.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9779999999999999e-05 2024-09-15T00:11:35.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.108099+0000", 2024-09-15T00:11:35.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3383000000000002e-05 2024-09-15T00:11:35.561 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.561 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.561 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.561 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18241 3.10 3:0c7717c4:::rbd_data.11393feb9e21.0000000000000005:a [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:35.561 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.284607+0000", 2024-09-15T00:11:35.562 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.19807675999999999, 2024-09-15T00:11:35.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048756839999999999, 2024-09-15T00:11:35.562 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.562 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.562 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:35.562 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:35.562 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:35.562 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:35.562 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18241 2024-09-15T00:11:35.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.562 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284607+0000", 2024-09-15T00:11:35.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284607+0000", 2024-09-15T00:11:35.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284608+0000", 2024-09-15T00:11:35.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:11:35.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284610+0000", 2024-09-15T00:11:35.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0320000000000002e-06 2024-09-15T00:11:35.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284616+0000", 2024-09-15T00:11:35.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3419999999999996e-06 2024-09-15T00:11:35.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284621+0000", 2024-09-15T00:11:35.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.006e-06 2024-09-15T00:11:35.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.289327+0000", 2024-09-15T00:11:35.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004706333 2024-09-15T00:11:35.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.289396+0000", 2024-09-15T00:11:35.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8207000000000006e-05 2024-09-15T00:11:35.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.289483+0000", 2024-09-15T00:11:35.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7325999999999997e-05 2024-09-15T00:11:35.565 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18242 3.10 3:0c7717c4:::rbd_data.11393feb9e21.0000000000000005:a [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.284628+0000", 2024-09-15T00:11:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.19805636800000001, 2024-09-15T00:11:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050370220000000004, 2024-09-15T00:11:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18242 2024-09-15T00:11:35.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.567 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284628+0000", 2024-09-15T00:11:35.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284628+0000", 2024-09-15T00:11:35.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284628+0000", 2024-09-15T00:11:35.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4099999999999999e-07 2024-09-15T00:11:35.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284630+0000", 2024-09-15T00:11:35.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0930000000000001e-06 2024-09-15T00:11:35.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284637+0000", 2024-09-15T00:11:35.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5640000000000002e-06 2024-09-15T00:11:35.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284650+0000", 2024-09-15T00:11:35.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.348e-05 2024-09-15T00:11:35.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.289510+0000", 2024-09-15T00:11:35.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048601979999999996 2024-09-15T00:11:35.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.289577+0000", 2024-09-15T00:11:35.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6563000000000006e-05 2024-09-15T00:11:35.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.289665+0000", 2024-09-15T00:11:35.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7682999999999994e-05 2024-09-15T00:11:35.570 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.570 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.570 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.570 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18243 3.10 3:0c7717c4:::rbd_data.11393feb9e21.0000000000000005:a [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:35.570 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.284658+0000", 2024-09-15T00:11:35.570 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.19802610400000001, 2024-09-15T00:11:35.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051959160000000001, 2024-09-15T00:11:35.571 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.571 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.571 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:35.571 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:35.571 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:35.571 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:35.571 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18243 2024-09-15T00:11:35.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.571 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284658+0000", 2024-09-15T00:11:35.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284658+0000", 2024-09-15T00:11:35.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284658+0000", 2024-09-15T00:11:35.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0400000000000002e-07 2024-09-15T00:11:35.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284660+0000", 2024-09-15T00:11:35.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.176e-06 2024-09-15T00:11:35.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284667+0000", 2024-09-15T00:11:35.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9530000000000002e-06 2024-09-15T00:11:35.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284672+0000", 2024-09-15T00:11:35.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9880000000000004e-06 2024-09-15T00:11:35.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.289693+0000", 2024-09-15T00:11:35.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050203840000000001 2024-09-15T00:11:35.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.289769+0000", 2024-09-15T00:11:35.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5785999999999996e-05 2024-09-15T00:11:35.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.289854+0000", 2024-09-15T00:11:35.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5224999999999998e-05 2024-09-15T00:11:35.574 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.574 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.574 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.575 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:18294 3.3 e38/28 3:c1f28201:::rbd_data.11398e1a091a.0000000000000000:head v 38'44, mlcod=37'42)", 2024-09-15T00:11:35.575 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.296284+0000", 2024-09-15T00:11:35.575 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.18639976799999999, 2024-09-15T00:11:35.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013754903000000001, 2024-09-15T00:11:35.575 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.575 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.575 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:35.575 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296284+0000", 2024-09-15T00:11:35.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296284+0000", 2024-09-15T00:11:35.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296285+0000", 2024-09-15T00:11:35.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7300000000000005e-07 2024-09-15T00:11:35.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296638+0000", 2024-09-15T00:11:35.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035318399999999998 2024-09-15T00:11:35.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296653+0000", 2024-09-15T00:11:35.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.468e-05 2024-09-15T00:11:35.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296661+0000", 2024-09-15T00:11:35.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8460000000000008e-06 2024-09-15T00:11:35.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296724+0000", 2024-09-15T00:11:35.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2947999999999994e-05 2024-09-15T00:11:35.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296744+0000", 2024-09-15T00:11:35.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0525000000000001e-05 2024-09-15T00:11:35.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:35.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.309995+0000", 2024-09-15T00:11:35.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013251284 2024-09-15T00:11:35.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:35.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.310039+0000", 2024-09-15T00:11:35.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3662999999999999e-05 2024-09-15T00:11:35.579 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.579 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.579 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.579 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.579 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.579 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:35.581 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:11:35.582 INFO:teuthology.orchestra.run.smithi171.stdout:964 write 0x7478664 thru 0x74826d1 (0xa06e bytes) 2024-09-15T00:11:35.582 INFO:teuthology.orchestra.run.smithi171.stdout:966 trunc from 0xddb6ed0 to 0x7d93e95 2024-09-15T00:11:35.582 INFO:teuthology.orchestra.run.smithi171.stdout:967 writesame 0x92748fc thru 0x927536c data_size 0xb(0xa71 bytes) 2024-09-15T00:11:35.582 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.582 INFO:teuthology.orchestra.run.smithi171.stdout:969 write 0x67de806 thru 0x67ed0e2 (0xe8dd bytes) 2024-09-15T00:11:35.582 INFO:teuthology.orchestra.run.smithi171.stdout:970 write 0x8406cc4 thru 0x840c50b (0x5848 bytes) 2024-09-15T00:11:35.582 INFO:teuthology.orchestra.run.smithi171.stdout:971 writesame 0xe98a0ed thru 0xe9983cf data_size 0x13(0xe2e3 bytes) 2024-09-15T00:11:35.583 INFO:teuthology.orchestra.run.smithi171.stdout:972 read 0x8262663 thru 0x826c896 (0xa234 bytes) 2024-09-15T00:11:35.583 INFO:teuthology.orchestra.run.smithi171.stdout:974 write 0x31a2ce5 thru 0x31a692e (0x3c4a bytes) 2024-09-15T00:11:35.583 INFO:teuthology.orchestra.run.smithi171.stdout:975 read 0x76c74dc thru 0x76c9b84 (0x26a9 bytes) 2024-09-15T00:11:35.583 INFO:teuthology.orchestra.run.smithi171.stdout:976 writesame 0xdaae849 thru 0xdab5124 data_size 0x6(0x68dc bytes) 2024-09-15T00:11:35.583 INFO:teuthology.orchestra.run.smithi171.stdout:977 write 0x4bc5cea thru 0x4bc64bc (0x7d3 bytes) 2024-09-15T00:11:35.583 INFO:teuthology.orchestra.run.smithi171.stdout:978 trunc from 0xe9983d0 to 0x8fdc468 2024-09-15T00:11:35.583 INFO:teuthology.orchestra.run.smithi171.stdout:979 read 0x53719dd thru 0x537a2a1 (0x88c5 bytes) 2024-09-15T00:11:35.583 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:35.578+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:35.584 INFO:teuthology.orchestra.run.smithi171.stdout:980 writesame 0xd898dff thru 0xd8a2c07 data_size 0x1(0x9e09 bytes) 2024-09-15T00:11:35.584 INFO:teuthology.orchestra.run.smithi171.stdout:981 punch from 0xb4fc935 to 0xb4ff6f5, (0x2dc0 bytes) 2024-09-15T00:11:35.587 INFO:teuthology.orchestra.run.smithi171.stdout:982 trunc from 0xd8a2c08 to 0x90dbd2c 2024-09-15T00:11:35.593 INFO:teuthology.orchestra.run.smithi171.stdout:983 read 0xee276 thru 0xfa7e2 (0xc56d bytes) 2024-09-15T00:11:35.594 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.594 INFO:teuthology.orchestra.run.smithi171.stdout:985 write 0x49e6c00 thru 0x49f25ec (0xb9ed bytes) 2024-09-15T00:11:35.599 INFO:teuthology.orchestra.run.smithi171.stdout:986 writesame 0x85bd185 thru 0x85ca624 data_size 0xbd(0xd4a0 bytes) 2024-09-15T00:11:35.602 INFO:teuthology.orchestra.run.smithi171.stdout:987 trunc from 0x90dbd2c to 0x924f891 2024-09-15T00:11:35.607 INFO:teuthology.orchestra.run.smithi171.stdout:988 punch from 0x4983bb4 to 0x499205e, (0xe4aa bytes) 2024-09-15T00:11:35.607 INFO:teuthology.orchestra.run.smithi171.stdout:989 read 0x187b9ba thru 0x18877d7 (0xbe1e bytes) 2024-09-15T00:11:35.608 INFO:teuthology.orchestra.run.smithi171.stdout:990 read 0x392e285 thru 0x3934adc (0x6858 bytes) 2024-09-15T00:11:35.614 INFO:teuthology.orchestra.run.smithi171.stdout:991 write 0x9d5ab47 thru 0x9d66acd (0xbf87 bytes) 2024-09-15T00:11:35.617 INFO:teuthology.orchestra.run.smithi171.stdout:993 write 0x4905e95 thru 0x4906114 (0x280 bytes) 2024-09-15T00:11:35.620 INFO:teuthology.orchestra.run.smithi171.stdout:994 punch from 0x899a545 to 0x89a56f5, (0xb1b0 bytes) 2024-09-15T00:11:35.620 INFO:teuthology.orchestra.run.smithi171.stdout:995 read 0x8dd3062 thru 0x8de04e3 (0xd482 bytes) 2024-09-15T00:11:35.632 INFO:teuthology.orchestra.run.smithi171.stdout:996 write 0xea73ee5 thru 0xea7b4bc (0x75d8 bytes) 2024-09-15T00:11:35.635 INFO:teuthology.orchestra.run.smithi171.stdout:1000 write 0x99f363b thru 0x9a020c6 (0xea8c bytes) 2024-09-15T00:11:35.640 INFO:teuthology.orchestra.run.smithi171.stdout:1001 punch from 0x7b370d7 to 0x7b377e5, (0x70e bytes) 2024-09-15T00:11:35.640 INFO:teuthology.orchestra.run.smithi171.stdout:1003 write 0xa032ba8 thru 0xa03ea08 (0xbe61 bytes) 2024-09-15T00:11:35.650 INFO:teuthology.orchestra.run.smithi171.stdout:1004 trunc from 0xea7b4bd to 0x9807a90 2024-09-15T00:11:35.670 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.670 INFO:teuthology.orchestra.run.smithi171.stdout:1007 punch from 0x143c5cd to 0x14495e6, (0xd019 bytes) 2024-09-15T00:11:35.671 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.671 INFO:teuthology.orchestra.run.smithi171.stdout:1009 write 0x42e58df thru 0x42ed473 (0x7b95 bytes) 2024-09-15T00:11:35.681 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:35.679+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:35.681 INFO:teuthology.orchestra.run.smithi171.stdout:1011 write 0xb63a879 thru 0xb6418fe (0x7086 bytes) 2024-09-15T00:11:35.688 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.688 INFO:teuthology.orchestra.run.smithi171.stdout:1014 punch from 0x7d20838 to 0x7d24601, (0x3dc9 bytes) 2024-09-15T00:11:35.692 INFO:teuthology.orchestra.run.smithi171.stdout:1015 trunc from 0xb6418ff to 0xe5cc240 2024-09-15T00:11:35.697 INFO:teuthology.orchestra.run.smithi171.stdout:1016 write 0x69fe0a1 thru 0x6a06e82 (0x8de2 bytes) 2024-09-15T00:11:35.712 INFO:teuthology.orchestra.run.smithi171.stdout:1018 writesame 0xeaec181 thru 0xeaf312f data_size 0x1(0x6faf bytes) 2024-09-15T00:11:35.712 INFO:teuthology.orchestra.run.smithi171.stdout:1019 write 0xbef11e3 thru 0xbefabca (0x99e8 bytes) 2024-09-15T00:11:35.716 INFO:teuthology.orchestra.run.smithi171.stdout:1020 punch from 0x21341b1 to 0x213b55f, (0x73ae bytes) 2024-09-15T00:11:35.716 INFO:teuthology.orchestra.run.smithi171.stdout:1021 read 0xe4dcc21 thru 0xe4eb351 (0xe731 bytes) 2024-09-15T00:11:35.716 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.731 INFO:teuthology.orchestra.run.smithi171.stdout:1027 trunc from 0xeaf3130 to 0x2241368 2024-09-15T00:11:35.738 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:35.738 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:35.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:18607 3.16 e38/23 3:6eea60b6:::rbd_object_map.11398e1a091a:head v 38'102, mlcod=38'100)", 2024-09-15T00:11:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.737407+0000", 2024-09-15T00:11:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00010962499999999999, 2024-09-15T00:11:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012490400000000001, 2024-09-15T00:11:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.737407+0000", 2024-09-15T00:11:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.737407+0000", 2024-09-15T00:11:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.737408+0000", 2024-09-15T00:11:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0800000000000004e-07 2024-09-15T00:11:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.737414+0000", 2024-09-15T00:11:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7690000000000002e-06 2024-09-15T00:11:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.737422+0000", 2024-09-15T00:11:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1899999999999995e-06 2024-09-15T00:11:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.737426+0000", 2024-09-15T00:11:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3290000000000004e-06 2024-09-15T00:11:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.737467+0000", 2024-09-15T00:11:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0451000000000003e-05 2024-09-15T00:11:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.737485+0000", 2024-09-15T00:11:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8277000000000001e-05 2024-09-15T00:11:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: ], 2024-09-15T00:11:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 1 2024-09-15T00:11:35.742 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:35.748 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:11:35.750 INFO:teuthology.orchestra.run.smithi171.stdout:1028 write 0x2bd319d thru 0x2be1f29 (0xed8d bytes) 2024-09-15T00:11:35.765 INFO:teuthology.orchestra.run.smithi171.stdout:1029 trunc from 0x2be1f2a to 0xc2f56ed 2024-09-15T00:11:35.775 INFO:teuthology.orchestra.run.smithi171.stdout:1030 trunc from 0xc2f56ed to 0x8683a6c 2024-09-15T00:11:35.781 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.781 INFO:teuthology.orchestra.run.smithi171.stdout:1034 read 0x42c9854 thru 0x42d8631 (0xedde bytes) 2024-09-15T00:11:35.782 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:35.780+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:35.788 INFO:teuthology.orchestra.run.smithi171.stdout:1035 trunc from 0x8683a6c to 0x294b9e9 2024-09-15T00:11:35.813 INFO:teuthology.orchestra.run.smithi171.stdout:1036 write 0xc6d48c1 thru 0xc6d7d7a (0x34ba bytes) 2024-09-15T00:11:35.816 INFO:teuthology.orchestra.run.smithi171.stdout:1037 writesame 0x1d80705 thru 0x1d81ad3 data_size 0x1(0x13cf bytes) 2024-09-15T00:11:35.818 INFO:teuthology.orchestra.run.smithi171.stdout:1039 punch from 0x95fe81f to 0x960956b, (0xad4c bytes) 2024-09-15T00:11:35.819 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.819 INFO:teuthology.orchestra.run.smithi171.stdout:1041 writesame 0x56224ba thru 0x562be8b data_size 0x3(0x99d2 bytes) 2024-09-15T00:11:35.842 INFO:teuthology.orchestra.run.smithi171.stdout:1043 read 0x5cd36fa thru 0x5cdeb2c (0xb433 bytes) 2024-09-15T00:11:35.842 INFO:teuthology.orchestra.run.smithi171.stdout:1044 write 0x2aeb533 thru 0x2af244e (0x6f1c bytes) 2024-09-15T00:11:35.846 INFO:teuthology.orchestra.run.smithi171.stdout:1047 read 0x7478cdd thru 0x7480b81 (0x7ea5 bytes) 2024-09-15T00:11:35.846 INFO:teuthology.orchestra.run.smithi171.stdout:1048 punch from 0x24cbb5f to 0x24d0cb6, (0x5157 bytes) 2024-09-15T00:11:35.846 INFO:teuthology.orchestra.run.smithi171.stdout:1049 punch from 0xeab996 to 0xeb7328, (0xb992 bytes) 2024-09-15T00:11:35.846 INFO:teuthology.orchestra.run.smithi171.stdout:1050 read 0xa98c41e thru 0xa995d24 (0x9907 bytes) 2024-09-15T00:11:35.858 INFO:teuthology.orchestra.run.smithi171.stdout:1051 trunc from 0xc6d7d7b to 0x26e9ea5 2024-09-15T00:11:35.882 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:35.881+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:35.883 INFO:teuthology.orchestra.run.smithi171.stdout:1052 trunc from 0x26e9ea5 to 0xb6a2853 2024-09-15T00:11:35.889 INFO:teuthology.orchestra.run.smithi171.stdout:1053 read 0x9192f4c thru 0x919ffba (0xd06f bytes) 2024-09-15T00:11:35.889 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.890 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.890 INFO:teuthology.orchestra.run.smithi171.stdout:1056 trunc from 0xb6a2853 to 0xa84352b 2024-09-15T00:11:35.895 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.899 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:35.899 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:35.899 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:35.899 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:35.899 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:35.902 INFO:teuthology.orchestra.run.smithi171.stdout:1062 trunc from 0xa84352b to 0x53377e0 2024-09-15T00:11:35.907 INFO:teuthology.orchestra.run.smithi171.stdout:1064 write 0x1dbdc1d thru 0x1dc6894 (0x8c78 bytes) 2024-09-15T00:11:35.909 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:11:35.910 INFO:teuthology.orchestra.run.smithi171.stdout:1066 read 0xdd7f30 thru 0xdd93bf (0x1490 bytes) 2024-09-15T00:11:35.910 INFO:teuthology.orchestra.run.smithi171.stdout:1067 writesame 0x2a67c4 thru 0x2ad932 data_size 0x1(0x716f bytes) 2024-09-15T00:11:35.957 INFO:teuthology.orchestra.run.smithi171.stdout:1068 read 0x4bb6caf thru 0x4bc68f4 (0xfc46 bytes) 2024-09-15T00:11:35.957 INFO:teuthology.orchestra.run.smithi171.stdout:1069 read 0x459e5f9 thru 0x45a277b (0x4183 bytes) 2024-09-15T00:11:35.957 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.964 INFO:teuthology.orchestra.run.smithi171.stdout:1071 write 0x622b7df thru 0x623a670 (0xee92 bytes) 2024-09-15T00:11:35.967 INFO:teuthology.orchestra.run.smithi171.stdout:1072 read 0x92d527 thru 0x9380f6 (0xabd0 bytes) 2024-09-15T00:11:35.968 INFO:teuthology.orchestra.run.smithi171.stdout:1073 read 0x4336e1b thru 0x43440ee (0xd2d4 bytes) 2024-09-15T00:11:35.968 INFO:teuthology.orchestra.run.smithi171.stdout:1075 write 0x58486ab thru 0x5850730 (0x8086 bytes) 2024-09-15T00:11:35.972 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.972 INFO:teuthology.orchestra.run.smithi171.stdout:1078 write 0x261950b thru 0x2624423 (0xaf19 bytes) 2024-09-15T00:11:35.976 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.976 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:35.977 INFO:teuthology.orchestra.run.smithi171.stdout:1081 trunc from 0x623a671 to 0x6eced8e 2024-09-15T00:11:35.983 INFO:teuthology.orchestra.run.smithi171.stdout:1083 write 0x693bb95 thru 0x6943001 (0x746d bytes) 2024-09-15T00:11:35.983 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:35.981+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:35.986 INFO:teuthology.orchestra.run.smithi171.stdout:1084 writesame 0x666aeab thru 0x6678fd0 data_size 0xa1(0xe126 bytes) 2024-09-15T00:11:35.989 INFO:teuthology.orchestra.run.smithi171.stdout:1085 read 0x14d6c97 thru 0x14e68d7 (0xfc41 bytes) 2024-09-15T00:11:35.990 INFO:teuthology.orchestra.run.smithi171.stdout:1087 trunc from 0x6eced8e to 0x65c9b25 2024-09-15T00:11:35.997 INFO:teuthology.orchestra.run.smithi171.stdout:1089 write 0xa6e17a thru 0xa7840a (0xa291 bytes) 2024-09-15T00:11:36.000 INFO:teuthology.orchestra.run.smithi171.stdout:1091 write 0x4aadb12 thru 0x4ab62f2 (0x87e1 bytes) 2024-09-15T00:11:36.015 INFO:teuthology.orchestra.run.smithi171.stdout:1092 write 0xa6afd52 thru 0xa6b69c2 (0x6c71 bytes) 2024-09-15T00:11:36.018 INFO:teuthology.orchestra.run.smithi171.stdout:1093 write 0x3904ae thru 0x39eadc (0xe62f bytes) 2024-09-15T00:11:36.020 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:36.020 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:36.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:36.020 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:36.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.020 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:36.020 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056512+0000", 2024-09-15T00:11:36.020 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 29.960796037000001, 2024-09-15T00:11:36.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.009307727, 2024-09-15T00:11:36.021 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.021 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.021 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:36.021 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:36.021 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:36.021 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:36.021 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 139 2024-09-15T00:11:36.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.021 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056512+0000", 2024-09-15T00:11:36.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056512+0000", 2024-09-15T00:11:36.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056513+0000", 2024-09-15T00:11:36.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5100000000000001e-07 2024-09-15T00:11:36.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056515+0000", 2024-09-15T00:11:36.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.26e-06 2024-09-15T00:11:36.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056519+0000", 2024-09-15T00:11:36.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0300000000000004e-06 2024-09-15T00:11:36.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056523+0000", 2024-09-15T00:11:36.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2379999999999999e-06 2024-09-15T00:11:36.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061418+0000", 2024-09-15T00:11:36.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048945630000000002 2024-09-15T00:11:36.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061474+0000", 2024-09-15T00:11:36.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6688999999999999e-05 2024-09-15T00:11:36.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:36.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061554+0000", 2024-09-15T00:11:36.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9810999999999999e-05 2024-09-15T00:11:36.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:36.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065502+0000", 2024-09-15T00:11:36.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039479720000000001 2024-09-15T00:11:36.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:36.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065776+0000", 2024-09-15T00:11:36.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00027390599999999998 2024-09-15T00:11:36.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:36.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065801+0000", 2024-09-15T00:11:36.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5219999999999999e-05 2024-09-15T00:11:36.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065820+0000", 2024-09-15T00:11:36.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8787000000000002e-05 2024-09-15T00:11:36.026 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.026 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.026 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.026 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:36.026 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:36.026 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 29.960780832000001, 2024-09-15T00:11:36.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0096064029999999995, 2024-09-15T00:11:36.026 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.026 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.026 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:36.027 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:36.027 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:36.027 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:36.027 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 140 2024-09-15T00:11:36.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.027 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:36.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:36.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:36.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4499999999999998e-07 2024-09-15T00:11:36.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056583+0000", 2024-09-15T00:11:36.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5377000000000002e-05 2024-09-15T00:11:36.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056588+0000", 2024-09-15T00:11:36.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0209999999999999e-06 2024-09-15T00:11:36.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056591+0000", 2024-09-15T00:11:36.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1089999999999998e-06 2024-09-15T00:11:36.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061790+0000", 2024-09-15T00:11:36.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051988759999999998 2024-09-15T00:11:36.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061837+0000", 2024-09-15T00:11:36.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6233000000000003e-05 2024-09-15T00:11:36.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:36.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061927+0000", 2024-09-15T00:11:36.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0525999999999994e-05 2024-09-15T00:11:36.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:36.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065584+0000", 2024-09-15T00:11:36.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036566630000000001 2024-09-15T00:11:36.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:36.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066084+0000", 2024-09-15T00:11:36.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000500557 2024-09-15T00:11:36.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:36.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066116+0000", 2024-09-15T00:11:36.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1272000000000001e-05 2024-09-15T00:11:36.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066134+0000", 2024-09-15T00:11:36.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8524000000000001e-05 2024-09-15T00:11:36.032 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.032 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.032 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.032 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:36.032 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:36.032 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 29.960711450000002, 2024-09-15T00:11:36.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0096621280000000007, 2024-09-15T00:11:36.032 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.032 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.032 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:36.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:36.033 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:36.033 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:36.033 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 141 2024-09-15T00:11:36.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.033 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:36.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:36.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:36.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9299999999999999e-07 2024-09-15T00:11:36.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056600+0000", 2024-09-15T00:11:36.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5330000000000001e-06 2024-09-15T00:11:36.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056606+0000", 2024-09-15T00:11:36.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5030000000000002e-06 2024-09-15T00:11:36.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056610+0000", 2024-09-15T00:11:36.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2279999999999998e-06 2024-09-15T00:11:36.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.064980+0000", 2024-09-15T00:11:36.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008370097 2024-09-15T00:11:36.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065030+0000", 2024-09-15T00:11:36.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0268000000000001e-05 2024-09-15T00:11:36.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:36.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065102+0000", 2024-09-15T00:11:36.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2120999999999996e-05 2024-09-15T00:11:36.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:36.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066159+0000", 2024-09-15T00:11:36.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001056881 2024-09-15T00:11:36.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:36.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066217+0000", 2024-09-15T00:11:36.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8344999999999998e-05 2024-09-15T00:11:36.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:36.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066241+0000", 2024-09-15T00:11:36.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3773000000000001e-05 2024-09-15T00:11:36.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066259+0000", 2024-09-15T00:11:36.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8085999999999999e-05 2024-09-15T00:11:36.037 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.038 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.038 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.038 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:155 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:36.038 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523404+0000", 2024-09-15T00:11:36.038 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 29.493904270000002, 2024-09-15T00:11:36.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092202580000000003, 2024-09-15T00:11:36.038 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.038 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.038 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:36.038 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:36.038 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:36.038 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:36.039 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 155 2024-09-15T00:11:36.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.039 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523404+0000", 2024-09-15T00:11:36.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523404+0000", 2024-09-15T00:11:36.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523405+0000", 2024-09-15T00:11:36.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:11:36.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523523+0000", 2024-09-15T00:11:36.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00011842500000000001 2024-09-15T00:11:36.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523531+0000", 2024-09-15T00:11:36.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7649999999999993e-06 2024-09-15T00:11:36.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523544+0000", 2024-09-15T00:11:36.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3243e-05 2024-09-15T00:11:36.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.524482+0000", 2024-09-15T00:11:36.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00093750599999999995 2024-09-15T00:11:36.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.524527+0000", 2024-09-15T00:11:36.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5296000000000001e-05 2024-09-15T00:11:36.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:36.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.524600+0000", 2024-09-15T00:11:36.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3271999999999999e-05 2024-09-15T00:11:36.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:36.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527438+0000", 2024-09-15T00:11:36.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002837783 2024-09-15T00:11:36.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:36.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532545+0000", 2024-09-15T00:11:36.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051066690000000003 2024-09-15T00:11:36.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:36.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532590+0000", 2024-09-15T00:11:36.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4929999999999998e-05 2024-09-15T00:11:36.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532624+0000", 2024-09-15T00:11:36.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4838000000000001e-05 2024-09-15T00:11:36.043 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.043 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.043 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.044 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:36.044 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:36.044 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 29.493710591999999, 2024-09-15T00:11:36.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092491780000000003, 2024-09-15T00:11:36.044 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.044 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.044 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:36.044 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:36.044 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:36.044 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:36.044 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 157 2024-09-15T00:11:36.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.044 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:36.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:36.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:36.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0500000000000004e-07 2024-09-15T00:11:36.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523602+0000", 2024-09-15T00:11:36.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5930000000000002e-06 2024-09-15T00:11:36.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523609+0000", 2024-09-15T00:11:36.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9970000000000001e-06 2024-09-15T00:11:36.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523614+0000", 2024-09-15T00:11:36.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8640000000000004e-06 2024-09-15T00:11:36.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525797+0000", 2024-09-15T00:11:36.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021830500000000002 2024-09-15T00:11:36.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525880+0000", 2024-09-15T00:11:36.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3394999999999997e-05 2024-09-15T00:11:36.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:36.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525997+0000", 2024-09-15T00:11:36.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000116971 2024-09-15T00:11:36.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:36.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527448+0000", 2024-09-15T00:11:36.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0014509690000000001 2024-09-15T00:11:36.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:36.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532758+0000", 2024-09-15T00:11:36.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053093319999999999 2024-09-15T00:11:36.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:36.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532804+0000", 2024-09-15T00:11:36.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5995000000000003e-05 2024-09-15T00:11:36.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532847+0000", 2024-09-15T00:11:36.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3507000000000002e-05 2024-09-15T00:11:36.049 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.049 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.049 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.049 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:36.049 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523622+0000", 2024-09-15T00:11:36.050 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 29.493686116999999, 2024-09-15T00:11:36.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0093097089999999993, 2024-09-15T00:11:36.050 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.050 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.050 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:36.050 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:36.050 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:36.050 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:36.050 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 158 2024-09-15T00:11:36.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.050 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523622+0000", 2024-09-15T00:11:36.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523622+0000", 2024-09-15T00:11:36.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523623+0000", 2024-09-15T00:11:36.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5400000000000002e-07 2024-09-15T00:11:36.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523689+0000", 2024-09-15T00:11:36.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6434999999999997e-05 2024-09-15T00:11:36.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523698+0000", 2024-09-15T00:11:36.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2229999999999996e-06 2024-09-15T00:11:36.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523705+0000", 2024-09-15T00:11:36.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6969999999999996e-06 2024-09-15T00:11:36.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526370+0000", 2024-09-15T00:11:36.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026647989999999998 2024-09-15T00:11:36.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526415+0000", 2024-09-15T00:11:36.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5278999999999998e-05 2024-09-15T00:11:36.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:36.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526484+0000", 2024-09-15T00:11:36.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8487999999999999e-05 2024-09-15T00:11:36.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:36.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.528098+0000", 2024-09-15T00:11:36.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0016147480000000001 2024-09-15T00:11:36.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:36.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532887+0000", 2024-09-15T00:11:36.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047889860000000003 2024-09-15T00:11:36.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:36.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532912+0000", 2024-09-15T00:11:36.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4694000000000001e-05 2024-09-15T00:11:36.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532932+0000", 2024-09-15T00:11:36.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9905999999999999e-05 2024-09-15T00:11:36.055 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.055 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.055 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.055 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:36.055 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:36.055 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 29.493577804000001, 2024-09-15T00:11:36.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0093142290000000003, 2024-09-15T00:11:36.055 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.055 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.056 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:36.056 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:36.056 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:36.056 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:36.056 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 159 2024-09-15T00:11:36.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.056 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:36.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6899999999999997e-07 2024-09-15T00:11:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523735+0000", 2024-09-15T00:11:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7359999999999999e-06 2024-09-15T00:11:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523745+0000", 2024-09-15T00:11:36.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0149000000000001e-05 2024-09-15T00:11:36.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523752+0000", 2024-09-15T00:11:36.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7220000000000002e-06 2024-09-15T00:11:36.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526967+0000", 2024-09-15T00:11:36.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032149280000000001 2024-09-15T00:11:36.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527039+0000", 2024-09-15T00:11:36.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2261000000000005e-05 2024-09-15T00:11:36.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:36.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527148+0000", 2024-09-15T00:11:36.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000108743 2024-09-15T00:11:36.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:36.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.528988+0000", 2024-09-15T00:11:36.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001840008 2024-09-15T00:11:36.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:36.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532965+0000", 2024-09-15T00:11:36.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039768980000000004 2024-09-15T00:11:36.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:36.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.533008+0000", 2024-09-15T00:11:36.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3673e-05 2024-09-15T00:11:36.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.533045+0000", 2024-09-15T00:11:36.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6541999999999997e-05 2024-09-15T00:11:36.061 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.061 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.061 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.061 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:36.061 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:11:36.061 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.9387889450000007, 2024-09-15T00:11:36.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:11:36.061 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.061 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.061 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:36.062 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:36.062 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:36.062 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:36.062 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:11:36.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.062 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:11:36.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:11:36.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:11:36.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:11:36.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:11:36.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:11:36.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:11:36.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:11:36.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:11:36.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:11:36.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:11:36.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:11:36.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:11:36.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:11:36.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:36.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:11:36.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:11:36.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:36.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:11:36.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:11:36.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:36.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:11:36.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:11:36.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:36.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:11:36.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:11:36.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:11:36.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:11:36.067 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.067 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.067 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.067 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:11:36.067 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:11:36.067 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.9260484029999994, 2024-09-15T00:11:36.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:11:36.067 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.067 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.067 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:36.067 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:11:36.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:11:36.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:11:36.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:11:36.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:11:36.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:11:36.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:11:36.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:11:36.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:11:36.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:11:36.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:11:36.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:11:36.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:11:36.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:11:36.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:11:36.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:11:36.071 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.071 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.071 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.071 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:36.072 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:11:36.072 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.7936926440000001, 2024-09-15T00:11:36.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:11:36.072 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.072 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.072 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:36.072 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:36.072 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:36.072 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:36.072 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:11:36.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.072 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:11:36.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:11:36.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:11:36.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:11:36.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:11:36.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:11:36.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:11:36.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:11:36.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:11:36.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:11:36.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:11:36.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:11:36.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:11:36.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:11:36.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:36.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:11:36.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:11:36.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:36.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:11:36.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:11:36.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:36.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:11:36.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:11:36.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:36.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:11:36.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:11:36.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:11:36.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:11:36.080 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.080 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.080 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.080 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:11:36.080 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:11:36.080 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.7802309990000005, 2024-09-15T00:11:36.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:11:36.080 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.080 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.080 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:36.081 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:11:36.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:11:36.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:11:36.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:11:36.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:11:36.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:11:36.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:11:36.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:11:36.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:11:36.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:11:36.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:11:36.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:11:36.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:11:36.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:11:36.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:11:36.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:11:36.085 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.085 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.085 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.085 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1612 3.e 3:77b26f8c:::rbd_data.11397bff333d.0000000000000023:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:36.085 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.663923+0000", 2024-09-15T00:11:36.085 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.3533854789999999, 2024-09-15T00:11:36.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017776304999999999, 2024-09-15T00:11:36.085 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.085 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.086 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:36.086 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:36.086 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:36.086 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:36.086 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1612 2024-09-15T00:11:36.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.086 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663923+0000", 2024-09-15T00:11:36.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663923+0000", 2024-09-15T00:11:36.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663925+0000", 2024-09-15T00:11:36.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5290000000000001e-06 2024-09-15T00:11:36.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663934+0000", 2024-09-15T00:11:36.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2040000000000003e-06 2024-09-15T00:11:36.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663948+0000", 2024-09-15T00:11:36.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3978999999999999e-05 2024-09-15T00:11:36.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663954+0000", 2024-09-15T00:11:36.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6059999999999999e-06 2024-09-15T00:11:36.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.664002+0000", 2024-09-15T00:11:36.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8127999999999997e-05 2024-09-15T00:11:36.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.664055+0000", 2024-09-15T00:11:36.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2237999999999997e-05 2024-09-15T00:11:36.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:36.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.665257+0000", 2024-09-15T00:11:36.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0012021989999999999 2024-09-15T00:11:36.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:36.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681584+0000", 2024-09-15T00:11:36.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016327264000000001 2024-09-15T00:11:36.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:36.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681620+0000", 2024-09-15T00:11:36.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6340000000000001e-05 2024-09-15T00:11:36.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:36.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681661+0000", 2024-09-15T00:11:36.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0259000000000002e-05 2024-09-15T00:11:36.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681699+0000", 2024-09-15T00:11:36.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8559000000000001e-05 2024-09-15T00:11:36.092 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.092 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.092 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.093 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1612 3.e e30/23 ondisk, result = 0)", 2024-09-15T00:11:36.093 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.668460+0000", 2024-09-15T00:11:36.093 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.3488482499999996, 2024-09-15T00:11:36.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013245329, 2024-09-15T00:11:36.093 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.093 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.093 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:36.093 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668460+0000", 2024-09-15T00:11:36.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668460+0000", 2024-09-15T00:11:36.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668461+0000", 2024-09-15T00:11:36.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3600000000000002e-07 2024-09-15T00:11:36.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668467+0000", 2024-09-15T00:11:36.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6559999999999997e-06 2024-09-15T00:11:36.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668477+0000", 2024-09-15T00:11:36.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0539e-05 2024-09-15T00:11:36.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668483+0000", 2024-09-15T00:11:36.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1580000000000002e-06 2024-09-15T00:11:36.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681608+0000", 2024-09-15T00:11:36.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013125009 2024-09-15T00:11:36.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681616+0000", 2024-09-15T00:11:36.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2880000000000004e-06 2024-09-15T00:11:36.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681706+0000", 2024-09-15T00:11:36.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9843e-05 2024-09-15T00:11:36.097 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.097 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.097 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.098 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3226 3.d 3:b1632ea2:::rbd_data.1139b531b717.0000000000000009:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:36.098 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.701042+0000", 2024-09-15T00:11:36.098 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.3162669210000004, 2024-09-15T00:11:36.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016783762000000001, 2024-09-15T00:11:36.098 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.098 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.098 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:36.098 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:36.098 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:36.098 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:36.098 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3226 2024-09-15T00:11:36.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.099 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701042+0000", 2024-09-15T00:11:36.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701042+0000", 2024-09-15T00:11:36.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701042+0000", 2024-09-15T00:11:36.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:11:36.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701045+0000", 2024-09-15T00:11:36.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1379999999999999e-06 2024-09-15T00:11:36.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701053+0000", 2024-09-15T00:11:36.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6350000000000006e-06 2024-09-15T00:11:36.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701058+0000", 2024-09-15T00:11:36.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3650000000000003e-06 2024-09-15T00:11:36.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701097+0000", 2024-09-15T00:11:36.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8769000000000001e-05 2024-09-15T00:11:36.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701164+0000", 2024-09-15T00:11:36.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7143000000000004e-05 2024-09-15T00:11:36.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:36.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.702192+0000", 2024-09-15T00:11:36.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00102808 2024-09-15T00:11:36.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:36.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717734+0000", 2024-09-15T00:11:36.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015541806 2024-09-15T00:11:36.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:36.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717748+0000", 2024-09-15T00:11:36.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3893999999999999e-05 2024-09-15T00:11:36.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:36.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717786+0000", 2024-09-15T00:11:36.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7996e-05 2024-09-15T00:11:36.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717825+0000", 2024-09-15T00:11:36.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9397000000000002e-05 2024-09-15T00:11:36.104 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.104 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.104 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.105 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:3226 3.d e33/23 ondisk, result = 0)", 2024-09-15T00:11:36.105 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.705838+0000", 2024-09-15T00:11:36.105 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.3114701660000003, 2024-09-15T00:11:36.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011902983000000001, 2024-09-15T00:11:36.105 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.105 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.105 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:36.105 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705838+0000", 2024-09-15T00:11:36.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705838+0000", 2024-09-15T00:11:36.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705839+0000", 2024-09-15T00:11:36.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9099999999999991e-07 2024-09-15T00:11:36.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705846+0000", 2024-09-15T00:11:36.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.251e-06 2024-09-15T00:11:36.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705860+0000", 2024-09-15T00:11:36.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4849e-05 2024-09-15T00:11:36.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705868+0000", 2024-09-15T00:11:36.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6629999999999993e-06 2024-09-15T00:11:36.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717669+0000", 2024-09-15T00:11:36.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011800966 2024-09-15T00:11:36.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717681+0000", 2024-09-15T00:11:36.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2201000000000001e-05 2024-09-15T00:11:36.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717741+0000", 2024-09-15T00:11:36.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0062000000000002e-05 2024-09-15T00:11:36.109 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.109 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.109 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.109 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18293 3.13 3:cd02983d:::rbd_data.11398e1a091a.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1310720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:36.110 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.294718+0000", 2024-09-15T00:11:36.110 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.72259062200000002, 2024-09-15T00:11:36.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012851031000000001, 2024-09-15T00:11:36.110 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.110 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.110 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:36.110 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:36.110 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:36.110 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:36.110 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18293 2024-09-15T00:11:36.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.110 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294718+0000", 2024-09-15T00:11:36.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294718+0000", 2024-09-15T00:11:36.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294719+0000", 2024-09-15T00:11:36.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.271e-06 2024-09-15T00:11:36.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294724+0000", 2024-09-15T00:11:36.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:11:36.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294742+0000", 2024-09-15T00:11:36.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8012999999999998e-05 2024-09-15T00:11:36.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294750+0000", 2024-09-15T00:11:36.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5120000000000002e-06 2024-09-15T00:11:36.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294801+0000", 2024-09-15T00:11:36.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1020000000000003e-05 2024-09-15T00:11:36.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294916+0000", 2024-09-15T00:11:36.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00011497600000000001 2024-09-15T00:11:36.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:36.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.295074+0000", 2024-09-15T00:11:36.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000158381 2024-09-15T00:11:36.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:36.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.298948+0000", 2024-09-15T00:11:36.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038734329999999999 2024-09-15T00:11:36.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:36.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.307455+0000", 2024-09-15T00:11:36.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0085077360000000001 2024-09-15T00:11:36.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:36.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.307525+0000", 2024-09-15T00:11:36.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9958999999999999e-05 2024-09-15T00:11:36.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.307569+0000", 2024-09-15T00:11:36.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3616999999999999e-05 2024-09-15T00:11:36.116 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.116 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.116 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.116 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:36.116 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:11:36.117 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.59522951599999996, 2024-09-15T00:11:36.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:11:36.117 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.117 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.117 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:36.117 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:36.117 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:36.117 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:36.117 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:11:36.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.117 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:11:36.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:11:36.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:11:36.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:11:36.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:11:36.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:11:36.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:11:36.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:11:36.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:11:36.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:11:36.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:11:36.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:11:36.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:11:36.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:11:36.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:36.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:11:36.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:11:36.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:36.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:11:36.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:11:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:11:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:11:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:11:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:11:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:11:36.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:11:36.123 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.123 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.123 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.123 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:11:36.123 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:11:36.123 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.58235324899999996, 2024-09-15T00:11:36.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:11:36.123 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.124 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.124 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:36.124 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:11:36.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:11:36.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:11:36.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:11:36.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:11:36.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:11:36.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:11:36.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:11:36.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:11:36.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:11:36.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:11:36.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:11:36.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:11:36.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:11:36.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:11:36.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:11:36.128 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.128 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.128 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.128 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18673 3.13 3:c8c60c5e:::rbd_data.11398e1a091a.000000000000002c:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:36.128 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.819584+0000", 2024-09-15T00:11:36.128 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.197724175, 2024-09-15T00:11:36.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.021809331000000001, 2024-09-15T00:11:36.128 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.128 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.129 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:36.129 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:36.129 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:36.129 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:36.129 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18673 2024-09-15T00:11:36.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.129 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819584+0000", 2024-09-15T00:11:36.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819584+0000", 2024-09-15T00:11:36.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819585+0000", 2024-09-15T00:11:36.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4600000000000003e-07 2024-09-15T00:11:36.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819590+0000", 2024-09-15T00:11:36.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2760000000000002e-06 2024-09-15T00:11:36.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819599+0000", 2024-09-15T00:11:36.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2809999999999997e-06 2024-09-15T00:11:36.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819604+0000", 2024-09-15T00:11:36.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8380000000000001e-06 2024-09-15T00:11:36.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819651+0000", 2024-09-15T00:11:36.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6978999999999999e-05 2024-09-15T00:11:36.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819740+0000", 2024-09-15T00:11:36.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9562999999999996e-05 2024-09-15T00:11:36.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:36.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.820752+0000", 2024-09-15T00:11:36.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0010121170000000001 2024-09-15T00:11:36.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:36.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.841221+0000", 2024-09-15T00:11:36.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.020468857999999999 2024-09-15T00:11:36.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:36.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.841276+0000", 2024-09-15T00:11:36.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5322999999999999e-05 2024-09-15T00:11:36.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:36.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.841341+0000", 2024-09-15T00:11:36.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4332999999999995e-05 2024-09-15T00:11:36.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.841394+0000", 2024-09-15T00:11:36.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2816999999999999e-05 2024-09-15T00:11:36.134 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.134 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.135 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.135 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18673 3.13 e38/23 ondisk, result = 0)", 2024-09-15T00:11:36.135 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.823700+0000", 2024-09-15T00:11:36.135 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.19360844199999999, 2024-09-15T00:11:36.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017702414, 2024-09-15T00:11:36.135 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:36.135 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:36.135 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:36.135 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:36.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:36.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.823700+0000", 2024-09-15T00:11:36.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:36.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.823700+0000", 2024-09-15T00:11:36.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:36.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:36.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.823701+0000", 2024-09-15T00:11:36.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9299999999999997e-07 2024-09-15T00:11:36.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:36.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.823705+0000", 2024-09-15T00:11:36.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8079999999999998e-06 2024-09-15T00:11:36.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:36.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.823714+0000", 2024-09-15T00:11:36.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:11:36.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:36.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.823719+0000", 2024-09-15T00:11:36.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.916e-06 2024-09-15T00:11:36.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:36.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.841258+0000", 2024-09-15T00:11:36.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017538221 2024-09-15T00:11:36.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:36.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.841269+0000", 2024-09-15T00:11:36.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1583000000000001e-05 2024-09-15T00:11:36.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:36.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:36.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:36.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.841402+0000", 2024-09-15T00:11:36.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000133351 2024-09-15T00:11:36.139 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.139 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.139 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.139 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:36.139 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:36.139 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:36.141 INFO:teuthology.orchestra.run.smithi171.stdout:1094 write 0x88adbcd thru 0x88b908c (0xb4c0 bytes) 2024-09-15T00:11:36.141 INFO:teuthology.orchestra.run.smithi171.stdout:1095 write 0x7b0ff45 thru 0x7b1f79d (0xf859 bytes) 2024-09-15T00:11:36.142 INFO:teuthology.orchestra.run.smithi171.stdout:1096 punch from 0x8190177 to 0x8197bf0, (0x7a79 bytes) 2024-09-15T00:11:36.142 INFO:teuthology.orchestra.run.smithi171.stdout:1097 write 0x5ea325c thru 0x5eb0f5a (0xdcff bytes) 2024-09-15T00:11:36.142 INFO:teuthology.orchestra.run.smithi171.stdout:1098 read 0x3c8999c thru 0x3c91b3c (0x81a1 bytes) 2024-09-15T00:11:36.142 INFO:teuthology.orchestra.run.smithi171.stderr:marked out osd.2. 2024-09-15T00:11:36.142 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:36.139+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:36.143 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is unset 2024-09-15T00:11:36.143 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:36.144 INFO:teuthology.orchestra.run.smithi171.stdout:1099 writesame 0xd131d3e thru 0xd13c494 data_size 0x1(0xa757 bytes) 2024-09-15T00:11:36.145 INFO:teuthology.orchestra.run.smithi171.stdout:1100 write 0x1da0d57 thru 0x1daa63a (0x98e4 bytes) 2024-09-15T00:11:36.242 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:36.240+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:36.288 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:36.288 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:36.288 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:36.288 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:36.296 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:11:36.381 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:36.379+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:36.436 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:11:36.441 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:36.441 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:36.441 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:36.441 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:36.441 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:36.443 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:36.442+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:36.450 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:11:36.544 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:36.542+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:36.560 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:36.560 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:36.577 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:36.578 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:36.594 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:36.594 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:36.597 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:36.598 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:36.607 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:36.610 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:36.611 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:36.628 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:36.628 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:36.645 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:36.645 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:36.645 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:36.643+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:36.661 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:36.661 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:36.678 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:36.678 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:36.716 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:36.716 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:36.725 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:36.766 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:36.765+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:36.847 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:36.846+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:36.869 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:36.869 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:36.878 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:36.948 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:36.946+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:37.022 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:37.022 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:37.030 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:37.074 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:37.073+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:37.138 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:37.139 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:37.147 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:37.189 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:37.187+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:37.244 INFO:teuthology.orchestra.run.smithi171.stdout:1101 writesame 0x57c8523 thru 0x57d4505 data_size 0x1(0xbfe3 bytes) 2024-09-15T00:11:37.251 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:37.250+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:37.292 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:37.293 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:37.305 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:37.348 INFO:teuthology.orchestra.run.smithi171.stdout:1102 trunc from 0xd13c495 to 0xb22516b 2024-09-15T00:11:37.352 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:37.350+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:37.353 INFO:teuthology.orchestra.run.smithi171.stdout:1103 write 0x8097209 thru 0x809dc54 (0x6a4c bytes) 2024-09-15T00:11:37.357 INFO:teuthology.orchestra.run.smithi171.stdout:1104 write 0x519009e thru 0x519cef5 (0xce58 bytes) 2024-09-15T00:11:37.449 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:37.449 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:37.453 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:37.452+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:37.457 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:37.555 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:37.553+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:37.602 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:37.602 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:37.610 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:37.655 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:37.654+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:37.713 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:37.713 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:37.721 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:37.756 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:37.755+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:37.825 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:37.825 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:37.834 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:37.878 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:37.876+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:37.938 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:37.938 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:37.947 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:37.988 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:37.986+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:38.060 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:38.058+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:38.091 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:38.091 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:38.100 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:38.146 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:11:38.191 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:38.189+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:38.202 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:38.202 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:38.211 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:38.232 INFO:teuthology.orchestra.run.smithi171.stdout:1106 write 0x314dd37 thru 0x315cec1 (0xf18b bytes) 2024-09-15T00:11:38.239 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.239 INFO:teuthology.orchestra.run.smithi171.stdout:1108 read 0xa5ee0b8 thru 0xa5f51d8 (0x7121 bytes) 2024-09-15T00:11:38.239 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.239 INFO:teuthology.orchestra.run.smithi171.stdout:1111 writesame 0xa187729 thru 0xa1942d0 data_size 0x98(0xcba8 bytes) 2024-09-15T00:11:38.245 INFO:teuthology.orchestra.run.smithi171.stdout:1112 writesame 0x6d7aa64 thru 0x6d873eb data_size 0x4(0xc988 bytes) 2024-09-15T00:11:38.260 INFO:teuthology.orchestra.run.smithi171.stdout:1113 write 0x9dbbd29 thru 0x9dbda62 (0x1d3a bytes) 2024-09-15T00:11:38.261 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:38.259+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:38.267 INFO:teuthology.orchestra.run.smithi171.stdout:1114 write 0x2216586 thru 0x221a253 (0x3cce bytes) 2024-09-15T00:11:38.273 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.273 INFO:teuthology.orchestra.run.smithi171.stdout:1116 read 0xe899d8 thru 0xe98f19 (0xf542 bytes) 2024-09-15T00:11:38.276 INFO:teuthology.orchestra.run.smithi171.stdout:1117 trunc from 0xb22516b to 0xd6e74b6 2024-09-15T00:11:38.282 INFO:teuthology.orchestra.run.smithi171.stdout:1118 punch from 0x15e5916 to 0x15e984b, (0x3f35 bytes) 2024-09-15T00:11:38.282 INFO:teuthology.orchestra.run.smithi171.stdout:1119 read 0x607a70 thru 0x60888c (0xe1d bytes) 2024-09-15T00:11:38.282 INFO:teuthology.orchestra.run.smithi171.stdout:1120 write 0x483fdb4 thru 0x4848d2c (0x8f79 bytes) 2024-09-15T00:11:38.290 INFO:teuthology.orchestra.run.smithi171.stdout:1121 read 0x4e0cc3f thru 0x4e16ab2 (0x9e74 bytes) 2024-09-15T00:11:38.290 INFO:teuthology.orchestra.run.smithi171.stdout:1122 writesame 0x3c3c839 thru 0x3c3f626 data_size 0x1(0x2dee bytes) 2024-09-15T00:11:38.308 INFO:teuthology.orchestra.run.smithi171.stdout:1123 read 0x3c1a313 thru 0x3c1bd81 (0x1a6f bytes) 2024-09-15T00:11:38.309 INFO:teuthology.orchestra.run.smithi171.stdout:1124 read 0xa6ffbb7 thru 0xa7037e9 (0x3c33 bytes) 2024-09-15T00:11:38.309 INFO:teuthology.orchestra.run.smithi171.stdout:1125 punch from 0x5617e7c to 0x561d673, (0x57f7 bytes) 2024-09-15T00:11:38.315 INFO:teuthology.orchestra.run.smithi171.stdout:1126 write 0xea6d6d4 thru 0xea7aadb (0xd408 bytes) 2024-09-15T00:11:38.319 INFO:teuthology.orchestra.run.smithi171.stdout:1127 read 0x5fadcfc thru 0x5fb4fbd (0x72c2 bytes) 2024-09-15T00:11:38.319 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.319 INFO:teuthology.orchestra.run.smithi171.stdout:1130 write 0x4c3af9b thru 0x4c3c449 (0x14af bytes) 2024-09-15T00:11:38.322 INFO:teuthology.orchestra.run.smithi171.stdout:1131 trunc from 0xea7aadc to 0xc8cb3dc 2024-09-15T00:11:38.331 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:38.331 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:38.334 INFO:teuthology.orchestra.run.smithi171.stdout:1132 read 0x3bff7d1 thru 0x3c0dbae (0xe3de bytes) 2024-09-15T00:11:38.335 INFO:teuthology.orchestra.run.smithi171.stdout:1133 read 0x9a47463 thru 0x9a544d2 (0xd070 bytes) 2024-09-15T00:11:38.341 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:38.342 INFO:teuthology.orchestra.run.smithi171.stdout:1135 writesame 0xcba3e2f thru 0xcba8324 data_size 0xe(0x44f6 bytes) 2024-09-15T00:11:38.342 INFO:teuthology.orchestra.run.smithi171.stdout:1136 read 0x4a7928b thru 0x4a7bfa0 (0x2d16 bytes) 2024-09-15T00:11:38.350 INFO:teuthology.orchestra.run.smithi171.stdout:1137 trunc from 0xcba8325 to 0x5b272dd 2024-09-15T00:11:38.363 INFO:teuthology.orchestra.run.smithi171.stdout:1138 write 0x2ab9916 thru 0x2abb456 (0x1b41 bytes) 2024-09-15T00:11:38.366 INFO:teuthology.orchestra.run.smithi171.stdout:1139 punch from 0x3c8b628 to 0x3c9613b, (0xab13 bytes) 2024-09-15T00:11:38.366 INFO:teuthology.orchestra.run.smithi171.stdout:1143 read 0x592ac43 thru 0x593aaca (0xfe88 bytes) 2024-09-15T00:11:38.366 INFO:teuthology.orchestra.run.smithi171.stdout:1144 read 0x2708315 thru 0x2710888 (0x8574 bytes) 2024-09-15T00:11:38.373 INFO:teuthology.orchestra.run.smithi171.stdout:1145 write 0x5e85c4e thru 0x5e8fe1d (0xa1d0 bytes) 2024-09-15T00:11:38.375 INFO:teuthology.orchestra.run.smithi171.stdout:1146 punch from 0xe819b4 to 0xe83f53, (0x259f bytes) 2024-09-15T00:11:38.375 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.375 INFO:teuthology.orchestra.run.smithi171.stdout:1148 read 0x183dfc2 thru 0x1847e52 (0x9e91 bytes) 2024-09-15T00:11:38.375 INFO:teuthology.orchestra.run.smithi171.stdout:1149 read 0x4e35420 thru 0x4e3c52a (0x710b bytes) 2024-09-15T00:11:38.384 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:38.383+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:38.387 INFO:teuthology.orchestra.run.smithi171.stdout:1151 write 0xb5a9669 thru 0xb5aeac1 (0x5459 bytes) 2024-09-15T00:11:38.399 INFO:teuthology.orchestra.run.smithi171.stdout:1152 writesame 0xe139e8b thru 0xe13ab4d data_size 0x1b(0xcc3 bytes) 2024-09-15T00:11:38.414 INFO:teuthology.orchestra.run.smithi171.stdout:1153 trunc from 0xe13ab4e to 0x20e3e93 2024-09-15T00:11:38.452 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:38.452 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:38.452 INFO:teuthology.orchestra.run.smithi171.stdout:1156 write 0xb6dedc9 thru 0xb6ec023 (0xd25b bytes) 2024-09-15T00:11:38.456 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.456 INFO:teuthology.orchestra.run.smithi171.stdout:1160 writesame 0x9f985a3 thru 0x9f9a7ee data_size 0xa(0x224c bytes) 2024-09-15T00:11:38.459 INFO:teuthology.orchestra.run.smithi171.stdout:1161 read 0x4db3a62 thru 0x4db6c25 (0x31c4 bytes) 2024-09-15T00:11:38.459 INFO:teuthology.orchestra.run.smithi171.stdout:1162 read 0x2308778 thru 0x2311b91 (0x941a bytes) 2024-09-15T00:11:38.460 INFO:teuthology.orchestra.run.smithi171.stdout:1163 punch from 0xa9ffb4c to 0xaa0c1ed, (0xc6a1 bytes) 2024-09-15T00:11:38.460 INFO:teuthology.orchestra.run.smithi171.stdout:1164 write 0x28e7ae2 thru 0x28e8d07 (0x1226 bytes) 2024-09-15T00:11:38.462 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:38.463 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:38.461+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:38.464 INFO:teuthology.orchestra.run.smithi171.stdout:1167 read 0x1bc60af thru 0x1bd02c6 (0xa218 bytes) 2024-09-15T00:11:38.464 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.464 INFO:teuthology.orchestra.run.smithi171.stdout:1169 write 0x76155bb thru 0x76188ff (0x3345 bytes) 2024-09-15T00:11:38.469 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.469 INFO:teuthology.orchestra.run.smithi171.stdout:1172 write 0x8371492 thru 0x8371e4e (0x9bd bytes) 2024-09-15T00:11:38.480 INFO:teuthology.orchestra.run.smithi171.stdout:1173 write 0xd77ca9c thru 0xd77cac0 (0x25 bytes) 2024-09-15T00:11:38.484 INFO:teuthology.orchestra.run.smithi171.stdout:1175 writesame 0xa95be54 thru 0xa967e73 data_size 0x74(0xc020 bytes) 2024-09-15T00:11:38.488 INFO:teuthology.orchestra.run.smithi171.stdout:1176 write 0x7c1c001 thru 0x7c28094 (0xc094 bytes) 2024-09-15T00:11:38.492 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.493 INFO:teuthology.orchestra.run.smithi171.stdout:1179 write 0xd69a1a8 thru 0xd6a2fa9 (0x8e02 bytes) 2024-09-15T00:11:38.497 INFO:teuthology.orchestra.run.smithi171.stdout:1180 read 0x370d7a thru 0x376e0a (0x6091 bytes) 2024-09-15T00:11:38.497 INFO:teuthology.orchestra.run.smithi171.stdout:1183 punch from 0xc52e57d to 0xc53926d, (0xacf0 bytes) 2024-09-15T00:11:38.497 INFO:teuthology.orchestra.run.smithi171.stdout:1186 punch from 0xbd58e9d to 0xbd67d31, (0xee94 bytes) 2024-09-15T00:11:38.497 INFO:teuthology.orchestra.run.smithi171.stdout:1187 write 0x74fb511 thru 0x75061e6 (0xacd6 bytes) 2024-09-15T00:11:38.501 INFO:teuthology.orchestra.run.smithi171.stdout:1188 write 0xbf7acde thru 0xbf7ce95 (0x21b8 bytes) 2024-09-15T00:11:38.504 INFO:teuthology.orchestra.run.smithi171.stdout:1189 read 0x64e847 thru 0x654ca9 (0x6463 bytes) 2024-09-15T00:11:38.505 INFO:teuthology.orchestra.run.smithi171.stdout:1190 write 0x32f37c3 thru 0x32f3e0f (0x64d bytes) 2024-09-15T00:11:38.507 INFO:teuthology.orchestra.run.smithi171.stdout:1193 read 0x768e3c6 thru 0x76977ac (0x93e7 bytes) 2024-09-15T00:11:38.507 INFO:teuthology.orchestra.run.smithi171.stdout:1194 punch from 0x7645c6b to 0x7649197, (0x352c bytes) 2024-09-15T00:11:38.508 INFO:teuthology.orchestra.run.smithi171.stdout:1198 write 0xd08a128 thru 0xd08b023 (0xefc bytes) 2024-09-15T00:11:38.510 INFO:teuthology.orchestra.run.smithi171.stdout:1199 read 0xa0fadc thru 0xa18918 (0x8e3d bytes) 2024-09-15T00:11:38.511 INFO:teuthology.orchestra.run.smithi171.stdout:1201 writesame 0x20d2139 thru 0x20db8a9 data_size 0x1(0x9771 bytes) 2024-09-15T00:11:38.564 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:38.562+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:38.572 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.572 INFO:teuthology.orchestra.run.smithi171.stdout:1204 write 0x38e3012 thru 0x38eb7f0 (0x87df bytes) 2024-09-15T00:11:38.575 INFO:teuthology.orchestra.run.smithi171.stdout:1205 read 0xacad6ff thru 0xacb07b0 (0x30b2 bytes) 2024-09-15T00:11:38.576 INFO:teuthology.orchestra.run.smithi171.stdout:1206 read 0x4aa519d thru 0x4aa7ec7 (0x2d2b bytes) 2024-09-15T00:11:38.576 INFO:teuthology.orchestra.run.smithi171.stdout:1207 read 0x9351b59 thru 0x93530a2 (0x154a bytes) 2024-09-15T00:11:38.576 INFO:teuthology.orchestra.run.smithi171.stdout:1211 punch from 0xd1ddf54 to 0xd1e63e7, (0x8493 bytes) 2024-09-15T00:11:38.576 INFO:teuthology.orchestra.run.smithi171.stdout:1215 punch from 0xa80d8f7 to 0xa818772, (0xae7b bytes) 2024-09-15T00:11:38.576 INFO:teuthology.orchestra.run.smithi171.stdout:1216 read 0x7983af1 thru 0x798fe9a (0xc3aa bytes) 2024-09-15T00:11:38.576 INFO:teuthology.orchestra.run.smithi171.stdout:1217 read 0x76cf0e5 thru 0x76cf6ba (0x5d6 bytes) 2024-09-15T00:11:38.576 INFO:teuthology.orchestra.run.smithi171.stdout:1218 writesame 0xa88f5f2 thru 0xa898002 data_size 0x1(0x8a11 bytes) 2024-09-15T00:11:38.612 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:38.612 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:38.619 INFO:teuthology.orchestra.run.smithi171.stdout:1219 trunc from 0xd77cac1 to 0x2e0256 2024-09-15T00:11:38.621 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:11:38.664 INFO:teuthology.orchestra.run.smithi171.stdout:1221 write 0xd4aa7d4 thru 0xd4b4431 (0x9c5e bytes) 2024-09-15T00:11:38.664 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:38.663+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:38.666 INFO:teuthology.orchestra.run.smithi171.stdout:1222 write 0x126e018 thru 0x126ef5d (0xf46 bytes) 2024-09-15T00:11:38.669 INFO:teuthology.orchestra.run.smithi171.stdout:1223 write 0xcc4c832 thru 0xcc57173 (0xa942 bytes) 2024-09-15T00:11:38.679 INFO:teuthology.orchestra.run.smithi171.stdout:1225 trunc from 0xd4b4432 to 0x867087d 2024-09-15T00:11:38.690 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:11:38.701 INFO:teuthology.orchestra.run.smithi171.stdout:1227 write 0xdbcb967 thru 0xdbcd407 (0x1aa1 bytes) 2024-09-15T00:11:38.704 INFO:teuthology.orchestra.run.smithi171.stdout:1228 read 0x2e0a105 thru 0x2e0b4bd (0x13b9 bytes) 2024-09-15T00:11:38.704 INFO:teuthology.orchestra.run.smithi171.stdout:1236 write 0xc09a4d1 thru 0xc09cc1f (0x274f bytes) 2024-09-15T00:11:38.707 INFO:teuthology.orchestra.run.smithi171.stdout:1237 read 0x98a750b thru 0x98ae493 (0x6f89 bytes) 2024-09-15T00:11:38.715 INFO:teuthology.orchestra.run.smithi171.stdout:1239 trunc from 0xdbcd408 to 0x73f43f7 2024-09-15T00:11:38.727 INFO:teuthology.orchestra.run.smithi171.stdout:1240 punch from 0x720832b to 0x72173bb, (0xf090 bytes) 2024-09-15T00:11:38.732 INFO:teuthology.orchestra.run.smithi171.stdout:1241 trunc from 0x73f43f7 to 0x381590c 2024-09-15T00:11:38.736 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:38.736 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:38.745 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:11:38.748 INFO:teuthology.orchestra.run.smithi171.stdout:1242 writesame 0x8d1b2ed thru 0x8d2057b data_size 0x5(0x528f bytes) 2024-09-15T00:11:38.760 INFO:teuthology.orchestra.run.smithi171.stdout:1244 writesame 0xd00d3f8 thru 0xd01b9be data_size 0x1(0xe5c7 bytes) 2024-09-15T00:11:38.760 INFO:teuthology.orchestra.run.smithi171.stdout:1247 read 0x96fc667 thru 0x9708979 (0xc313 bytes) 2024-09-15T00:11:38.760 INFO:teuthology.orchestra.run.smithi171.stdout:1248 punch from 0xc4ad979 to 0xc4b8977, (0xaffe bytes) 2024-09-15T00:11:38.760 INFO:teuthology.orchestra.run.smithi171.stdout:1249 read 0x3f170c8 thru 0x3f26866 (0xf79f bytes) 2024-09-15T00:11:38.765 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:38.764+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:38.771 INFO:teuthology.orchestra.run.smithi171.stdout:1250 trunc from 0xd01b9bf to 0x3c9d03a 2024-09-15T00:11:38.776 INFO:teuthology.orchestra.run.smithi171.stdout:1251 read 0x22c682d thru 0x22d4399 (0xdb6d bytes) 2024-09-15T00:11:38.794 INFO:teuthology.orchestra.run.smithi171.stdout:1253 write 0xdfc7465 thru 0xdfd41db (0xcd77 bytes) 2024-09-15T00:11:38.797 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.814 INFO:teuthology.orchestra.run.smithi171.stdout:1258 trunc from 0xdfd41dc to 0x4ef2b3 2024-09-15T00:11:38.827 INFO:teuthology.orchestra.run.smithi171.stdout:1259 read 0x3de488 thru 0x3eccda (0xe853 bytes) 2024-09-15T00:11:38.828 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.828 INFO:teuthology.orchestra.run.smithi171.stdout:1261 punch from 0x2148aa to 0x21f370, (0xaac6 bytes) 2024-09-15T00:11:38.832 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:38.832 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:38.837 INFO:teuthology.orchestra.run.smithi171.stdout:1262 trunc from 0x4ef2b3 to 0x833b869 2024-09-15T00:11:38.845 INFO:teuthology.orchestra.run.smithi171.stdout:1264 read 0x5a66954 thru 0x5a6cdcb (0x6478 bytes) 2024-09-15T00:11:38.845 INFO:teuthology.orchestra.run.smithi171.stdout:1266 read 0x2a6f28b thru 0x2a7bdff (0xcb75 bytes) 2024-09-15T00:11:38.849 INFO:teuthology.orchestra.run.smithi171.stdout:1268 writesame 0x850a0f9 thru 0x850d938 data_size 0x64(0x3840 bytes) 2024-09-15T00:11:38.851 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:38.851 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:38.856 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:38.856 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:38.858 INFO:teuthology.orchestra.run.smithi171.stdout:1269 trunc from 0x850d939 to 0x125fa47 2024-09-15T00:11:38.866 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:38.864+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:38.867 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:11:38.870 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:38.870 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:38.883 INFO:teuthology.orchestra.run.smithi171.stdout:1270 writesame 0xa8f5f2d thru 0xa8fd8f6 data_size 0x22(0x79ca bytes) 2024-09-15T00:11:38.888 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:38.888 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:38.890 INFO:teuthology.orchestra.run.smithi171.stdout:1271 trunc from 0xa8fd8f7 to 0x4e5760d 2024-09-15T00:11:38.896 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.896 INFO:teuthology.orchestra.run.smithi171.stdout:1273 read 0x41ce09b thru 0x41d097d (0x28e3 bytes) 2024-09-15T00:11:38.896 INFO:teuthology.orchestra.run.smithi171.stdout:1274 read 0x12cb523 thru 0x12d3e42 (0x8920 bytes) 2024-09-15T00:11:38.896 INFO:teuthology.orchestra.run.smithi171.stdout:1278 read 0x3327b89 thru 0x3334d51 (0xd1c9 bytes) 2024-09-15T00:11:38.896 INFO:teuthology.orchestra.run.smithi171.stdout:1280 read 0x348713f thru 0x3488751 (0x1613 bytes) 2024-09-15T00:11:38.904 INFO:teuthology.orchestra.run.smithi171.stdout:1281 trunc from 0x4e5760d to 0xbaa87f0 2024-09-15T00:11:38.906 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:38.906 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:38.909 INFO:teuthology.orchestra.run.smithi171.stdout:1282 write 0x6b6d0ae thru 0x6b788e2 (0xb835 bytes) 2024-09-15T00:11:38.914 INFO:teuthology.orchestra.run.smithi171.stdout:1283 read 0x5b4cbff thru 0x5b4dd46 (0x1148 bytes) 2024-09-15T00:11:38.914 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.914 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.914 INFO:teuthology.orchestra.run.smithi171.stdout:1286 punch from 0x943c0f9 to 0x944904c, (0xcf53 bytes) 2024-09-15T00:11:38.914 INFO:teuthology.orchestra.run.smithi171.stdout:1287 read 0x352ff4a thru 0x353969f (0x9756 bytes) 2024-09-15T00:11:38.914 INFO:teuthology.orchestra.run.smithi171.stdout:1289 writesame 0x484b92 thru 0x48aea9 data_size 0x54(0x6318 bytes) 2024-09-15T00:11:38.916 INFO:teuthology.orchestra.run.smithi171.stdout:1291 read 0xaf8f8a3 thru 0xaf97574 (0x7cd2 bytes) 2024-09-15T00:11:38.916 INFO:teuthology.orchestra.run.smithi171.stdout:1292 read 0x5f3ee2b thru 0x5f488db (0x9ab1 bytes) 2024-09-15T00:11:38.916 INFO:teuthology.orchestra.run.smithi171.stdout:1293 read 0x660d7a7 thru 0x6614b0c (0x7366 bytes) 2024-09-15T00:11:38.916 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.916 INFO:teuthology.orchestra.run.smithi171.stdout:1295 writesame 0x64d42c6 thru 0x64d9aec data_size 0x1(0x5827 bytes) 2024-09-15T00:11:38.923 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:38.923 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:38.941 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:38.941 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:38.957 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:38.957 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:38.959 INFO:teuthology.orchestra.run.smithi171.stdout:1296 writesame 0xced1124 thru 0xcedf37d data_size 0x7(0xe25a bytes) 2024-09-15T00:11:38.959 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:38.959 INFO:teuthology.orchestra.run.smithi171.stdout:1299 writesame 0x7379a06 thru 0x73815fe data_size 0x47(0x7bf9 bytes) 2024-09-15T00:11:38.962 INFO:teuthology.orchestra.run.smithi171.stdout:1300 read 0x6e10675 thru 0x6e1c5fb (0xbf87 bytes) 2024-09-15T00:11:38.962 INFO:teuthology.orchestra.run.smithi171.stdout:1303 read 0x3ee403b thru 0x3ee51d8 (0x119e bytes) 2024-09-15T00:11:38.967 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:38.965+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:38.973 INFO:teuthology.orchestra.run.smithi171.stdout:1304 trunc from 0xcedf37e to 0x39b6105 2024-09-15T00:11:38.984 INFO:teuthology.orchestra.run.smithi171.stdout:1305 writesame 0x20147ff thru 0x201bbe2 data_size 0x2(0x73e4 bytes) 2024-09-15T00:11:39.021 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:39.021 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:39.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:39.021 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:39.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.021 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3037 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'86, mlcod=33'82)", 2024-09-15T00:11:39.022 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.497583+0000", 2024-09-15T00:11:39.022 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.5204389379999999, 2024-09-15T00:11:39.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077497779999999997, 2024-09-15T00:11:39.022 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.022 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.022 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.022 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497583+0000", 2024-09-15T00:11:39.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497583+0000", 2024-09-15T00:11:39.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497584+0000", 2024-09-15T00:11:39.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6599999999999996e-07 2024-09-15T00:11:39.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497588+0000", 2024-09-15T00:11:39.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3529999999999997e-06 2024-09-15T00:11:39.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497598+0000", 2024-09-15T00:11:39.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4839999999999993e-06 2024-09-15T00:11:39.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497602+0000", 2024-09-15T00:11:39.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:11:39.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497643+0000", 2024-09-15T00:11:39.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0326e-05 2024-09-15T00:11:39.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497662+0000", 2024-09-15T00:11:39.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9590999999999999e-05 2024-09-15T00:11:39.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505303+0000", 2024-09-15T00:11:39.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076408919999999998 2024-09-15T00:11:39.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505333+0000", 2024-09-15T00:11:39.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0071999999999999e-05 2024-09-15T00:11:39.028 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.028 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.028 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.028 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3038 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'87, mlcod=33'82)", 2024-09-15T00:11:39.028 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:39.028 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.5195944259999994, 2024-09-15T00:11:39.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069253919999999998, 2024-09-15T00:11:39.028 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.028 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.028 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.028 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:39.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:39.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:39.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6899999999999997e-07 2024-09-15T00:11:39.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498435+0000", 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1330000000000004e-06 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498444+0000", 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6749999999999997e-06 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498449+0000", 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5449999999999997e-06 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498485+0000", 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5737e-05 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498501+0000", 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6637e-05 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505334+0000", 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0068333100000000004 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505353+0000", 2024-09-15T00:11:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8785999999999999e-05 2024-09-15T00:11:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3039 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'88, mlcod=33'83)", 2024-09-15T00:11:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.5188037919999999, 2024-09-15T00:11:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0061577309999999996, 2024-09-15T00:11:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:39.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:39.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:39.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3099999999999997e-07 2024-09-15T00:11:39.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499224+0000", 2024-09-15T00:11:39.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.916e-06 2024-09-15T00:11:39.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499233+0000", 2024-09-15T00:11:39.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7949999999999996e-06 2024-09-15T00:11:39.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499239+0000", 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0090000000000004e-06 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499292+0000", 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3049e-05 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499312+0000", 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0574e-05 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505355+0000", 2024-09-15T00:11:39.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060423930000000001 2024-09-15T00:11:39.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505376+0000", 2024-09-15T00:11:39.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1364000000000001e-05 2024-09-15T00:11:39.036 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.036 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.036 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.036 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3040 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'89, mlcod=33'83)", 2024-09-15T00:11:39.036 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.499752+0000", 2024-09-15T00:11:39.036 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.5182700009999994, 2024-09-15T00:11:39.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0059059300000000002, 2024-09-15T00:11:39.037 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.037 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.037 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.037 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499752+0000", 2024-09-15T00:11:39.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499752+0000", 2024-09-15T00:11:39.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499753+0000", 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1399999999999997e-07 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499763+0000", 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0365e-05 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499774+0000", 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0920999999999999e-05 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499780+0000", 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.02e-06 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499829+0000", 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9066000000000001e-05 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499849+0000", 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9596999999999999e-05 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505633+0000", 2024-09-15T00:11:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057841189999999999 2024-09-15T00:11:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505658+0000", 2024-09-15T00:11:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5228e-05 2024-09-15T00:11:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3041 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'90, mlcod=33'83)", 2024-09-15T00:11:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.500417+0000", 2024-09-15T00:11:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.5176051879999992, 2024-09-15T00:11:39.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052558600000000002, 2024-09-15T00:11:39.041 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.041 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.041 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.041 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500417+0000", 2024-09-15T00:11:39.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500417+0000", 2024-09-15T00:11:39.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500418+0000", 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6199999999999998e-07 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500422+0000", 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1169999999999997e-06 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500429+0000", 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6620000000000005e-06 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500435+0000", 2024-09-15T00:11:39.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6130000000000003e-06 2024-09-15T00:11:39.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500471+0000", 2024-09-15T00:11:39.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5899999999999998e-05 2024-09-15T00:11:39.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.500484+0000", 2024-09-15T00:11:39.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3502999999999999e-05 2024-09-15T00:11:39.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505660+0000", 2024-09-15T00:11:39.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051752930000000001 2024-09-15T00:11:39.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505673+0000", 2024-09-15T00:11:39.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3210000000000001e-05 2024-09-15T00:11:39.044 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.045 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.045 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.045 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3102 3.a 3:566166ea:::rbd_data.1139b531b717.000000000000000b:head [stat,truncate 385024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:39.045 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:39.045 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.4481625999999999, 2024-09-15T00:11:39.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060108410000000003, 2024-09-15T00:11:39.045 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.045 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.045 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:39.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:39.045 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3102 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:39.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569863+0000", 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.819e-06 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569871+0000", 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6690000000000006e-06 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569882+0000", 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1195999999999999e-05 2024-09-15T00:11:39.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.570647+0000", 2024-09-15T00:11:39.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00076547900000000001 2024-09-15T00:11:39.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.570734+0000", 2024-09-15T00:11:39.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7087000000000002e-05 2024-09-15T00:11:39.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:39.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.570863+0000", 2024-09-15T00:11:39.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012899799999999999 2024-09-15T00:11:39.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.573483+0000", 2024-09-15T00:11:39.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026192799999999999 2024-09-15T00:11:39.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.575796+0000", 2024-09-15T00:11:39.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0023136570000000002 2024-09-15T00:11:39.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.575827+0000", 2024-09-15T00:11:39.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0468e-05 2024-09-15T00:11:39.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.575871+0000", 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3656999999999999e-05 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3152 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'108, mlcod=33'104)", 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.619166+0000", 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.3988563850000002, 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049522180000000004, 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.050 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619166+0000", 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619166+0000", 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619166+0000", 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:11:39.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619170+0000", 2024-09-15T00:11:39.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2990000000000001e-06 2024-09-15T00:11:39.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619177+0000", 2024-09-15T00:11:39.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7260000000000002e-06 2024-09-15T00:11:39.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619181+0000", 2024-09-15T00:11:39.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9759999999999997e-06 2024-09-15T00:11:39.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619215+0000", 2024-09-15T00:11:39.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4066000000000002e-05 2024-09-15T00:11:39.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.619230+0000", 2024-09-15T00:11:39.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4392999999999999e-05 2024-09-15T00:11:39.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624081+0000", 2024-09-15T00:11:39.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048513560000000002 2024-09-15T00:11:39.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.624118+0000", 2024-09-15T00:11:39.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6973e-05 2024-09-15T00:11:39.054 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.054 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.054 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.054 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3379 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'162, mlcod=33'158)", 2024-09-15T00:11:39.054 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.833370+0000", 2024-09-15T00:11:39.055 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.1846520379999994, 2024-09-15T00:11:39.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049772030000000004, 2024-09-15T00:11:39.055 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.055 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.055 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.055 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833370+0000", 2024-09-15T00:11:39.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833370+0000", 2024-09-15T00:11:39.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833371+0000", 2024-09-15T00:11:39.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.23e-07 2024-09-15T00:11:39.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833375+0000", 2024-09-15T00:11:39.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0389999999999998e-06 2024-09-15T00:11:39.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833384+0000", 2024-09-15T00:11:39.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0110000000000006e-06 2024-09-15T00:11:39.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833388+0000", 2024-09-15T00:11:39.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2490000000000003e-06 2024-09-15T00:11:39.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833450+0000", 2024-09-15T00:11:39.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1849999999999999e-05 2024-09-15T00:11:39.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.833469+0000", 2024-09-15T00:11:39.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9222999999999999e-05 2024-09-15T00:11:39.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.838319+0000", 2024-09-15T00:11:39.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048498860000000003 2024-09-15T00:11:39.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.838347+0000", 2024-09-15T00:11:39.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8221999999999998e-05 2024-09-15T00:11:39.059 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.059 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.059 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.059 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3502 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'181, mlcod=33'177)", 2024-09-15T00:11:39.059 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:39.059 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.0697074339999997, 2024-09-15T00:11:39.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0067634649999999998, 2024-09-15T00:11:39.059 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.060 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.060 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.060 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:39.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:39.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:39.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5199999999999997e-07 2024-09-15T00:11:39.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948319+0000", 2024-09-15T00:11:39.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0740000000000003e-06 2024-09-15T00:11:39.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948329+0000", 2024-09-15T00:11:39.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1409999999999994e-06 2024-09-15T00:11:39.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948335+0000", 2024-09-15T00:11:39.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6290000000000004e-06 2024-09-15T00:11:39.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948398+0000", 2024-09-15T00:11:39.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2422999999999995e-05 2024-09-15T00:11:39.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948412+0000", 2024-09-15T00:11:39.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4538999999999999e-05 2024-09-15T00:11:39.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.954988+0000", 2024-09-15T00:11:39.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.006575455 2024-09-15T00:11:39.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955078+0000", 2024-09-15T00:11:39.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0651999999999999e-05 2024-09-15T00:11:39.064 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.064 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.064 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.064 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3503 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'182, mlcod=33'177)", 2024-09-15T00:11:39.064 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.948857+0000", 2024-09-15T00:11:39.064 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.0691648390000008, 2024-09-15T00:11:39.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0062792079999999997, 2024-09-15T00:11:39.064 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.064 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.065 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.065 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948857+0000", 2024-09-15T00:11:39.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948857+0000", 2024-09-15T00:11:39.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948858+0000", 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6899999999999997e-07 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948862+0000", 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2429999999999999e-06 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948872+0000", 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5340000000000005e-06 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948883+0000", 2024-09-15T00:11:39.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1304999999999999e-05 2024-09-15T00:11:39.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948917+0000", 2024-09-15T00:11:39.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3510999999999998e-05 2024-09-15T00:11:39.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948929+0000", 2024-09-15T00:11:39.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2473000000000001e-05 2024-09-15T00:11:39.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955082+0000", 2024-09-15T00:11:39.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0061525010000000003 2024-09-15T00:11:39.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955137+0000", 2024-09-15T00:11:39.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5071999999999997e-05 2024-09-15T00:11:39.068 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.068 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.068 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.068 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3504 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'183, mlcod=33'178)", 2024-09-15T00:11:39.068 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:39.069 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.068409462, 2024-09-15T00:11:39.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058979369999999998, 2024-09-15T00:11:39.069 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.069 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.069 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.069 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:39.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:39.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:39.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8400000000000004e-07 2024-09-15T00:11:39.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949620+0000", 2024-09-15T00:11:39.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9739999999999996e-06 2024-09-15T00:11:39.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949630+0000", 2024-09-15T00:11:39.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6399999999999992e-06 2024-09-15T00:11:39.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949635+0000", 2024-09-15T00:11:39.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:11:39.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949688+0000", 2024-09-15T00:11:39.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2667000000000002e-05 2024-09-15T00:11:39.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949701+0000", 2024-09-15T00:11:39.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.36e-05 2024-09-15T00:11:39.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955477+0000", 2024-09-15T00:11:39.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057763119999999996 2024-09-15T00:11:39.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955511+0000", 2024-09-15T00:11:39.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3337000000000003e-05 2024-09-15T00:11:39.073 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.073 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.073 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.073 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3505 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'184, mlcod=33'178)", 2024-09-15T00:11:39.073 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.950258+0000", 2024-09-15T00:11:39.074 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.0677641209999997, 2024-09-15T00:11:39.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052830350000000002, 2024-09-15T00:11:39.074 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.074 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.074 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.074 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950258+0000", 2024-09-15T00:11:39.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950258+0000", 2024-09-15T00:11:39.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950259+0000", 2024-09-15T00:11:39.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7700000000000004e-07 2024-09-15T00:11:39.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950262+0000", 2024-09-15T00:11:39.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6839999999999998e-06 2024-09-15T00:11:39.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950271+0000", 2024-09-15T00:11:39.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8249999999999994e-06 2024-09-15T00:11:39.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950276+0000", 2024-09-15T00:11:39.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7600000000000002e-06 2024-09-15T00:11:39.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950311+0000", 2024-09-15T00:11:39.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4629999999999999e-05 2024-09-15T00:11:39.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.950323+0000", 2024-09-15T00:11:39.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1853e-05 2024-09-15T00:11:39.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955513+0000", 2024-09-15T00:11:39.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051903599999999998 2024-09-15T00:11:39.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955541+0000", 2024-09-15T00:11:39.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8345999999999999e-05 2024-09-15T00:11:39.078 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.078 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.078 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.078 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3724 3.1e e35/23 3:792e48de:::rbd_object_map.1139b531b717:head v 35'223, mlcod=35'214)", 2024-09-15T00:11:39.078 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.097406+0000", 2024-09-15T00:11:39.078 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 7.9206165349999997, 2024-09-15T00:11:39.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050324560000000003, 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097406+0000", 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097406+0000", 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097406+0000", 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5299999999999999e-07 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097409+0000", 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2399999999999999e-06 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097419+0000", 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7920000000000004e-06 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097425+0000", 2024-09-15T00:11:39.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7610000000000004e-06 2024-09-15T00:11:39.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097460+0000", 2024-09-15T00:11:39.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4997999999999999e-05 2024-09-15T00:11:39.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.097472+0000", 2024-09-15T00:11:39.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1627000000000001e-05 2024-09-15T00:11:39.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102401+0000", 2024-09-15T00:11:39.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049290480000000001 2024-09-15T00:11:39.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102438+0000", 2024-09-15T00:11:39.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7537000000000003e-05 2024-09-15T00:11:39.082 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.082 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.082 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.082 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3726 3.1e e35/23 3:792e48de:::rbd_object_map.1139b531b717:head v 35'225, mlcod=35'220)", 2024-09-15T00:11:39.082 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.099757+0000", 2024-09-15T00:11:39.082 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 7.91826513, 2024-09-15T00:11:39.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00535652, 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099757+0000", 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099757+0000", 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099758+0000", 2024-09-15T00:11:39.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7400000000000002e-07 2024-09-15T00:11:39.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099764+0000", 2024-09-15T00:11:39.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0469999999999999e-06 2024-09-15T00:11:39.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099774+0000", 2024-09-15T00:11:39.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0118999999999999e-05 2024-09-15T00:11:39.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099781+0000", 2024-09-15T00:11:39.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7730000000000002e-06 2024-09-15T00:11:39.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099826+0000", 2024-09-15T00:11:39.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5229999999999999e-05 2024-09-15T00:11:39.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.099844+0000", 2024-09-15T00:11:39.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8345000000000001e-05 2024-09-15T00:11:39.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.105063+0000", 2024-09-15T00:11:39.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052188920000000001 2024-09-15T00:11:39.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.105114+0000", 2024-09-15T00:11:39.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0340000000000003e-05 2024-09-15T00:11:39.086 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.086 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.086 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.086 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3737 3.a 3:53de29a7:::rbd_data.1139b531b717.000000000000000a:head [stat,truncate 270336] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:39.086 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.100151+0000", 2024-09-15T00:11:39.087 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 7.9178713170000004, 2024-09-15T00:11:39.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076797300000000001, 2024-09-15T00:11:39.087 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.087 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.087 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:39.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:39.087 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3737 2024-09-15T00:11:39.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.087 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100151+0000", 2024-09-15T00:11:39.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100151+0000", 2024-09-15T00:11:39.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100153+0000", 2024-09-15T00:11:39.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6530000000000001e-06 2024-09-15T00:11:39.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100158+0000", 2024-09-15T00:11:39.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7100000000000004e-06 2024-09-15T00:11:39.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100173+0000", 2024-09-15T00:11:39.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4242e-05 2024-09-15T00:11:39.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100180+0000", 2024-09-15T00:11:39.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7680000000000008e-06 2024-09-15T00:11:39.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100248+0000", 2024-09-15T00:11:39.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7459000000000006e-05 2024-09-15T00:11:39.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100321+0000", 2024-09-15T00:11:39.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2854000000000004e-05 2024-09-15T00:11:39.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:39.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100418+0000", 2024-09-15T00:11:39.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7827999999999999e-05 2024-09-15T00:11:39.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.105090+0000", 2024-09-15T00:11:39.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0046714349999999998 2024-09-15T00:11:39.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.107770+0000", 2024-09-15T00:11:39.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026799660000000002 2024-09-15T00:11:39.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.107801+0000", 2024-09-15T00:11:39.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1158000000000002e-05 2024-09-15T00:11:39.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.107831+0000", 2024-09-15T00:11:39.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9657000000000001e-05 2024-09-15T00:11:39.095 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.095 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.095 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.096 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3738 3.a 3:566166ea:::rbd_data.1139b531b717.000000000000000b:head [stat,truncate 270336] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:39.096 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.102790+0000", 2024-09-15T00:11:39.096 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 7.9152318929999996, 2024-09-15T00:11:39.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053085249999999997, 2024-09-15T00:11:39.096 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.097 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.097 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:39.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:39.097 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3738 2024-09-15T00:11:39.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.097 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102790+0000", 2024-09-15T00:11:39.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102790+0000", 2024-09-15T00:11:39.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102791+0000", 2024-09-15T00:11:39.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1900000000000001e-07 2024-09-15T00:11:39.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102796+0000", 2024-09-15T00:11:39.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5560000000000001e-06 2024-09-15T00:11:39.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102809+0000", 2024-09-15T00:11:39.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3213e-05 2024-09-15T00:11:39.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102817+0000", 2024-09-15T00:11:39.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6090000000000003e-06 2024-09-15T00:11:39.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102859+0000", 2024-09-15T00:11:39.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2536000000000001e-05 2024-09-15T00:11:39.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.102933+0000", 2024-09-15T00:11:39.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3960000000000003e-05 2024-09-15T00:11:39.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:39.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.103108+0000", 2024-09-15T00:11:39.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00017522800000000001 2024-09-15T00:11:39.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.105349+0000", 2024-09-15T00:11:39.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002240377 2024-09-15T00:11:39.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.108036+0000", 2024-09-15T00:11:39.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002686964 2024-09-15T00:11:39.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.108066+0000", 2024-09-15T00:11:39.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9779999999999999e-05 2024-09-15T00:11:39.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.108099+0000", 2024-09-15T00:11:39.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3383000000000002e-05 2024-09-15T00:11:39.102 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.103 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.103 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.103 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18242 3.10 3:0c7717c4:::rbd_data.11393feb9e21.0000000000000005:a [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:39.103 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.284628+0000", 2024-09-15T00:11:39.103 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.7333946550000001, 2024-09-15T00:11:39.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050370220000000004, 2024-09-15T00:11:39.103 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.103 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.103 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:39.103 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.103 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:39.103 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:39.104 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18242 2024-09-15T00:11:39.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.104 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284628+0000", 2024-09-15T00:11:39.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284628+0000", 2024-09-15T00:11:39.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284628+0000", 2024-09-15T00:11:39.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4099999999999999e-07 2024-09-15T00:11:39.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284630+0000", 2024-09-15T00:11:39.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0930000000000001e-06 2024-09-15T00:11:39.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284637+0000", 2024-09-15T00:11:39.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5640000000000002e-06 2024-09-15T00:11:39.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284650+0000", 2024-09-15T00:11:39.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.348e-05 2024-09-15T00:11:39.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.289510+0000", 2024-09-15T00:11:39.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048601979999999996 2024-09-15T00:11:39.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.289577+0000", 2024-09-15T00:11:39.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6563000000000006e-05 2024-09-15T00:11:39.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.289665+0000", 2024-09-15T00:11:39.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7682999999999994e-05 2024-09-15T00:11:39.108 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.108 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.108 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.109 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18243 3.10 3:0c7717c4:::rbd_data.11393feb9e21.0000000000000005:a [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:39.109 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.284658+0000", 2024-09-15T00:11:39.109 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.7333643909999998, 2024-09-15T00:11:39.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051959160000000001, 2024-09-15T00:11:39.109 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.109 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.109 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:39.109 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.109 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:39.109 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:39.109 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18243 2024-09-15T00:11:39.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.109 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284658+0000", 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284658+0000", 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284658+0000", 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0400000000000002e-07 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284660+0000", 2024-09-15T00:11:39.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.176e-06 2024-09-15T00:11:39.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284667+0000", 2024-09-15T00:11:39.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9530000000000002e-06 2024-09-15T00:11:39.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.284672+0000", 2024-09-15T00:11:39.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9880000000000004e-06 2024-09-15T00:11:39.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.289693+0000", 2024-09-15T00:11:39.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050203840000000001 2024-09-15T00:11:39.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.289769+0000", 2024-09-15T00:11:39.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5785999999999996e-05 2024-09-15T00:11:39.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.289854+0000", 2024-09-15T00:11:39.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5224999999999998e-05 2024-09-15T00:11:39.112 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.112 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.113 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.113 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:18294 3.3 e38/28 3:c1f28201:::rbd_data.11398e1a091a.0000000000000000:head v 38'44, mlcod=37'42)", 2024-09-15T00:11:39.113 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.296284+0000", 2024-09-15T00:11:39.113 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.7217380549999999, 2024-09-15T00:11:39.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013754903000000001, 2024-09-15T00:11:39.113 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.113 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.113 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.113 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296284+0000", 2024-09-15T00:11:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296284+0000", 2024-09-15T00:11:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296285+0000", 2024-09-15T00:11:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7300000000000005e-07 2024-09-15T00:11:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296638+0000", 2024-09-15T00:11:39.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035318399999999998 2024-09-15T00:11:39.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296653+0000", 2024-09-15T00:11:39.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.468e-05 2024-09-15T00:11:39.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296661+0000", 2024-09-15T00:11:39.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8460000000000008e-06 2024-09-15T00:11:39.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296724+0000", 2024-09-15T00:11:39.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2947999999999994e-05 2024-09-15T00:11:39.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296744+0000", 2024-09-15T00:11:39.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0525000000000001e-05 2024-09-15T00:11:39.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.309995+0000", 2024-09-15T00:11:39.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013251284 2024-09-15T00:11:39.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.310039+0000", 2024-09-15T00:11:39.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3662999999999999e-05 2024-09-15T00:11:39.117 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.117 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.117 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.117 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:18785 3.3 e38/28 3:c4795c41:::rbd_data.11398e1a091a.000000000000000e:head v 38'52, mlcod=38'50)", 2024-09-15T00:11:39.117 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:36.018259+0000", 2024-09-15T00:11:39.117 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.9997629639999999, 2024-09-15T00:11:39.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048925419999999997, 2024-09-15T00:11:39.117 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.117 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.118 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.118 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:36.018259+0000", 2024-09-15T00:11:39.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:36.018259+0000", 2024-09-15T00:11:39.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:36.018260+0000", 2024-09-15T00:11:39.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2200000000000002e-07 2024-09-15T00:11:39.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:36.018301+0000", 2024-09-15T00:11:39.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1044999999999998e-05 2024-09-15T00:11:39.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:36.018312+0000", 2024-09-15T00:11:39.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0699999999999999e-05 2024-09-15T00:11:39.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:36.018318+0000", 2024-09-15T00:11:39.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8619999999999999e-06 2024-09-15T00:11:39.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:36.018379+0000", 2024-09-15T00:11:39.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0665999999999998e-05 2024-09-15T00:11:39.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:36.018402+0000", 2024-09-15T00:11:39.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3192000000000001e-05 2024-09-15T00:11:39.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:36.023108+0000", 2024-09-15T00:11:39.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047062609999999998 2024-09-15T00:11:39.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:36.023152+0000", 2024-09-15T00:11:39.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3893999999999997e-05 2024-09-15T00:11:39.121 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.121 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.121 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.121 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.122 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.122 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:39.124 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:11:39.125 INFO:teuthology.orchestra.run.smithi171.stdout:1309 write 0xca01b2a thru 0xca0231d (0x7f4 bytes) 2024-09-15T00:11:39.125 INFO:teuthology.orchestra.run.smithi171.stdout:1312 read 0xa9197a2 thru 0xa92491d (0xb17c bytes) 2024-09-15T00:11:39.125 INFO:teuthology.orchestra.run.smithi171.stdout:1313 writesame 0x8fe027 thru 0x9047d0 data_size 0x3(0x67aa bytes) 2024-09-15T00:11:39.125 INFO:teuthology.orchestra.run.smithi171.stdout:1314 read 0x8a48bbd thru 0x8a567d5 (0xdc19 bytes) 2024-09-15T00:11:39.125 INFO:teuthology.orchestra.run.smithi171.stdout:1315 punch from 0x39657bc to 0x3971959, (0xc19d bytes) 2024-09-15T00:11:39.125 INFO:teuthology.orchestra.run.smithi171.stdout:1316 read 0xbc68173 thru 0xbc6ce9e (0x4d2c bytes) 2024-09-15T00:11:39.125 INFO:teuthology.orchestra.run.smithi171.stdout:1317 write 0xc58a597 thru 0xc58ade6 (0x850 bytes) 2024-09-15T00:11:39.125 INFO:teuthology.orchestra.run.smithi171.stdout:1318 write 0x1355fa9 thru 0x1362b03 (0xcb5b bytes) 2024-09-15T00:11:39.125 INFO:teuthology.orchestra.run.smithi171.stdout:1319 trunc from 0xca0231e to 0xc7f791f 2024-09-15T00:11:39.126 INFO:teuthology.orchestra.run.smithi171.stdout:1322 write 0x9abfacd thru 0x9ac7123 (0x7657 bytes) 2024-09-15T00:11:39.126 INFO:teuthology.orchestra.run.smithi171.stdout:1323 write 0x7cdb6dc thru 0x7cdf8bc (0x41e1 bytes) 2024-09-15T00:11:39.126 INFO:teuthology.orchestra.run.smithi171.stdout:1324 write 0x215e31b thru 0x215ef81 (0xc67 bytes) 2024-09-15T00:11:39.126 INFO:teuthology.orchestra.run.smithi171.stdout:1326 punch from 0x6cdb4f4 to 0x6ce589f, (0xa3ab bytes) 2024-09-15T00:11:39.126 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:39.126 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:39.126 INFO:teuthology.orchestra.run.smithi171.stdout:1330 write 0x68dcd13 thru 0x68e2b63 (0x5e51 bytes) 2024-09-15T00:11:39.126 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is unset 2024-09-15T00:11:39.127 INFO:teuthology.orchestra.run.smithi171.stdout:1332 write 0xd5aba84 thru 0xd5abb8d (0x10a bytes) 2024-09-15T00:11:39.127 INFO:teuthology.orchestra.run.smithi171.stdout:1333 write 0x6a9a6a9 thru 0x6a9ea0a (0x4362 bytes) 2024-09-15T00:11:39.127 INFO:teuthology.orchestra.run.smithi171.stdout:1335 write 0xbdb5d77 thru 0xbdc1367 (0xb5f1 bytes) 2024-09-15T00:11:39.128 INFO:teuthology.orchestra.run.smithi171.stdout:1337 punch from 0x3901706 to 0x3906e95, (0x578f bytes) 2024-09-15T00:11:39.129 INFO:teuthology.orchestra.run.smithi171.stdout:1338 writesame 0x73101aa thru 0x7312547 data_size 0x2(0x239e bytes) 2024-09-15T00:11:39.129 INFO:teuthology.orchestra.run.smithi171.stdout:1340 write 0x7f453bf thru 0x7f51808 (0xc44a bytes) 2024-09-15T00:11:39.129 INFO:teuthology.orchestra.run.smithi171.stdout:1342 trunc from 0xd5abb8e to 0xbd3fff6 2024-09-15T00:11:39.129 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:39.129 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:39.129 INFO:teuthology.orchestra.run.smithi171.stdout:1345 trunc from 0xbd3fff6 to 0xdaecd53 2024-09-15T00:11:39.129 INFO:teuthology.orchestra.run.smithi171.stdout:1347 trunc from 0xdaecd53 to 0x6db9d5d 2024-09-15T00:11:39.129 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:39.121+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:39.142 INFO:teuthology.orchestra.run.smithi171.stdout:1348 write 0xb23a696 thru 0xb2484b8 (0xde23 bytes) 2024-09-15T00:11:39.146 INFO:teuthology.orchestra.run.smithi171.stdout:1349 clone 9 order 21 su 65536 sc 15 2024-09-15T00:11:39.224 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:39.222+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:39.234 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:39.234 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:39.234 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:39.234 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:39.243 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:11:39.324 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:39.323+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:39.345 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:39.345 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:39.346 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:39.346 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:39.346 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:39.354 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:11:39.426 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:39.424+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:39.458 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:39.459 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:39.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:39.459 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:39.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.459 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:39.459 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056512+0000", 2024-09-15T00:11:39.459 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.399696847999998, 2024-09-15T00:11:39.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.009307727, 2024-09-15T00:11:39.459 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.459 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.459 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.459 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.459 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:39.460 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:39.460 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 139 2024-09-15T00:11:39.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.460 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056512+0000", 2024-09-15T00:11:39.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056512+0000", 2024-09-15T00:11:39.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056513+0000", 2024-09-15T00:11:39.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5100000000000001e-07 2024-09-15T00:11:39.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056515+0000", 2024-09-15T00:11:39.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.26e-06 2024-09-15T00:11:39.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056519+0000", 2024-09-15T00:11:39.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0300000000000004e-06 2024-09-15T00:11:39.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056523+0000", 2024-09-15T00:11:39.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2379999999999999e-06 2024-09-15T00:11:39.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061418+0000", 2024-09-15T00:11:39.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048945630000000002 2024-09-15T00:11:39.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061474+0000", 2024-09-15T00:11:39.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6688999999999999e-05 2024-09-15T00:11:39.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:39.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061554+0000", 2024-09-15T00:11:39.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9810999999999999e-05 2024-09-15T00:11:39.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065502+0000", 2024-09-15T00:11:39.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039479720000000001 2024-09-15T00:11:39.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065776+0000", 2024-09-15T00:11:39.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00027390599999999998 2024-09-15T00:11:39.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065801+0000", 2024-09-15T00:11:39.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5219999999999999e-05 2024-09-15T00:11:39.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065820+0000", 2024-09-15T00:11:39.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8787000000000002e-05 2024-09-15T00:11:39.465 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.465 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.465 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.465 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:39.465 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:39.465 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.399681643000001, 2024-09-15T00:11:39.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0096064029999999995, 2024-09-15T00:11:39.465 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.465 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.465 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:39.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:39.466 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 140 2024-09-15T00:11:39.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.466 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:39.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:39.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056528+0000", 2024-09-15T00:11:39.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4499999999999998e-07 2024-09-15T00:11:39.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056583+0000", 2024-09-15T00:11:39.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5377000000000002e-05 2024-09-15T00:11:39.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056588+0000", 2024-09-15T00:11:39.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0209999999999999e-06 2024-09-15T00:11:39.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056591+0000", 2024-09-15T00:11:39.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1089999999999998e-06 2024-09-15T00:11:39.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061790+0000", 2024-09-15T00:11:39.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051988759999999998 2024-09-15T00:11:39.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061837+0000", 2024-09-15T00:11:39.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6233000000000003e-05 2024-09-15T00:11:39.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:39.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.061927+0000", 2024-09-15T00:11:39.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0525999999999994e-05 2024-09-15T00:11:39.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065584+0000", 2024-09-15T00:11:39.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036566630000000001 2024-09-15T00:11:39.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066084+0000", 2024-09-15T00:11:39.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000500557 2024-09-15T00:11:39.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066116+0000", 2024-09-15T00:11:39.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1272000000000001e-05 2024-09-15T00:11:39.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066134+0000", 2024-09-15T00:11:39.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8524000000000001e-05 2024-09-15T00:11:39.471 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.471 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.471 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.471 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:39.471 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:39.471 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.399612261000001, 2024-09-15T00:11:39.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0096621280000000007, 2024-09-15T00:11:39.471 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.471 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.471 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.471 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.472 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:39.472 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:39.472 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 141 2024-09-15T00:11:39.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.472 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:39.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:39.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056597+0000", 2024-09-15T00:11:39.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9299999999999999e-07 2024-09-15T00:11:39.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056600+0000", 2024-09-15T00:11:39.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5330000000000001e-06 2024-09-15T00:11:39.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056606+0000", 2024-09-15T00:11:39.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5030000000000002e-06 2024-09-15T00:11:39.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.056610+0000", 2024-09-15T00:11:39.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2279999999999998e-06 2024-09-15T00:11:39.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.064980+0000", 2024-09-15T00:11:39.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008370097 2024-09-15T00:11:39.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065030+0000", 2024-09-15T00:11:39.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0268000000000001e-05 2024-09-15T00:11:39.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:39.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.065102+0000", 2024-09-15T00:11:39.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2120999999999996e-05 2024-09-15T00:11:39.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066159+0000", 2024-09-15T00:11:39.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001056881 2024-09-15T00:11:39.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066217+0000", 2024-09-15T00:11:39.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8344999999999998e-05 2024-09-15T00:11:39.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066241+0000", 2024-09-15T00:11:39.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3773000000000001e-05 2024-09-15T00:11:39.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.066259+0000", 2024-09-15T00:11:39.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8085999999999999e-05 2024-09-15T00:11:39.476 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.476 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.476 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.477 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:155 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:39.477 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523404+0000", 2024-09-15T00:11:39.477 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.932805080999998, 2024-09-15T00:11:39.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092202580000000003, 2024-09-15T00:11:39.477 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.477 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.477 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.477 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.478 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:39.478 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:39.478 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 155 2024-09-15T00:11:39.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.478 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523404+0000", 2024-09-15T00:11:39.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523404+0000", 2024-09-15T00:11:39.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523405+0000", 2024-09-15T00:11:39.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:11:39.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523523+0000", 2024-09-15T00:11:39.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00011842500000000001 2024-09-15T00:11:39.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523531+0000", 2024-09-15T00:11:39.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7649999999999993e-06 2024-09-15T00:11:39.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523544+0000", 2024-09-15T00:11:39.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3243e-05 2024-09-15T00:11:39.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.524482+0000", 2024-09-15T00:11:39.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00093750599999999995 2024-09-15T00:11:39.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.524527+0000", 2024-09-15T00:11:39.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5296000000000001e-05 2024-09-15T00:11:39.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:39.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.524600+0000", 2024-09-15T00:11:39.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3271999999999999e-05 2024-09-15T00:11:39.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527438+0000", 2024-09-15T00:11:39.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002837783 2024-09-15T00:11:39.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532545+0000", 2024-09-15T00:11:39.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051066690000000003 2024-09-15T00:11:39.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532590+0000", 2024-09-15T00:11:39.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4929999999999998e-05 2024-09-15T00:11:39.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532624+0000", 2024-09-15T00:11:39.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4838000000000001e-05 2024-09-15T00:11:39.483 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.483 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.483 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.483 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:39.483 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:39.483 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.932611403000003, 2024-09-15T00:11:39.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092491780000000003, 2024-09-15T00:11:39.483 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.483 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.483 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.483 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.483 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:39.484 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:39.484 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 157 2024-09-15T00:11:39.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.484 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:39.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:39.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523598+0000", 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0500000000000004e-07 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523602+0000", 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5930000000000002e-06 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523609+0000", 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9970000000000001e-06 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523614+0000", 2024-09-15T00:11:39.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8640000000000004e-06 2024-09-15T00:11:39.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525797+0000", 2024-09-15T00:11:39.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021830500000000002 2024-09-15T00:11:39.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525880+0000", 2024-09-15T00:11:39.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3394999999999997e-05 2024-09-15T00:11:39.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:39.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.525997+0000", 2024-09-15T00:11:39.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000116971 2024-09-15T00:11:39.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527448+0000", 2024-09-15T00:11:39.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0014509690000000001 2024-09-15T00:11:39.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532758+0000", 2024-09-15T00:11:39.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053093319999999999 2024-09-15T00:11:39.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532804+0000", 2024-09-15T00:11:39.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5995000000000003e-05 2024-09-15T00:11:39.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532847+0000", 2024-09-15T00:11:39.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3507000000000002e-05 2024-09-15T00:11:39.488 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.488 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.488 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.488 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:39.489 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523622+0000", 2024-09-15T00:11:39.489 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.932586927999999, 2024-09-15T00:11:39.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0093097089999999993, 2024-09-15T00:11:39.489 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.489 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.489 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.489 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.489 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:39.489 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:39.489 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 158 2024-09-15T00:11:39.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.489 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523622+0000", 2024-09-15T00:11:39.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523622+0000", 2024-09-15T00:11:39.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523623+0000", 2024-09-15T00:11:39.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5400000000000002e-07 2024-09-15T00:11:39.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523689+0000", 2024-09-15T00:11:39.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6434999999999997e-05 2024-09-15T00:11:39.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523698+0000", 2024-09-15T00:11:39.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2229999999999996e-06 2024-09-15T00:11:39.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523705+0000", 2024-09-15T00:11:39.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6969999999999996e-06 2024-09-15T00:11:39.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526370+0000", 2024-09-15T00:11:39.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026647989999999998 2024-09-15T00:11:39.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526415+0000", 2024-09-15T00:11:39.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5278999999999998e-05 2024-09-15T00:11:39.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:39.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526484+0000", 2024-09-15T00:11:39.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8487999999999999e-05 2024-09-15T00:11:39.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.528098+0000", 2024-09-15T00:11:39.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0016147480000000001 2024-09-15T00:11:39.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532887+0000", 2024-09-15T00:11:39.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047889860000000003 2024-09-15T00:11:39.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532912+0000", 2024-09-15T00:11:39.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4694000000000001e-05 2024-09-15T00:11:39.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532932+0000", 2024-09-15T00:11:39.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9905999999999999e-05 2024-09-15T00:11:39.494 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.494 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.494 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.494 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T00:11:39.494 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:39.494 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.932478615000001, 2024-09-15T00:11:39.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0093142290000000003, 2024-09-15T00:11:39.494 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.494 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4215", 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/1198395731", 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 159 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:39.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523731+0000", 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6899999999999997e-07 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523735+0000", 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7359999999999999e-06 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523745+0000", 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0149000000000001e-05 2024-09-15T00:11:39.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.523752+0000", 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7220000000000002e-06 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.526967+0000", 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032149280000000001 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527039+0000", 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2261000000000005e-05 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:11:39.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.527148+0000", 2024-09-15T00:11:39.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000108743 2024-09-15T00:11:39.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.528988+0000", 2024-09-15T00:11:39.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001840008 2024-09-15T00:11:39.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.532965+0000", 2024-09-15T00:11:39.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039768980000000004 2024-09-15T00:11:39.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.533008+0000", 2024-09-15T00:11:39.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3673e-05 2024-09-15T00:11:39.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:06.533045+0000", 2024-09-15T00:11:39.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6541999999999997e-05 2024-09-15T00:11:39.499 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.499 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.499 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.377689756000001, 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:11:39.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:11:39.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:11:39.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:11:39.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:11:39.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:11:39.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:11:39.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:11:39.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:11:39.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:11:39.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:11:39.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:11:39.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:11:39.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:11:39.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:39.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:11:39.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:11:39.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:11:39.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:11:39.505 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.505 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.505 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.505 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:11:39.505 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:11:39.505 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.364949213999999, 2024-09-15T00:11:39.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:11:39.505 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.505 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.505 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:39.505 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:11:39.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:11:39.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:11:39.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:11:39.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:11:39.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:11:39.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:11:39.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:11:39.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:11:39.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:11:39.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:11:39.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:11:39.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:11:39.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:11:39.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:11:39.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:11:39.509 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.509 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.509 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.509 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:39.510 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:11:39.510 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.232593455, 2024-09-15T00:11:39.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:11:39.510 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.510 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.510 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.510 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.510 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:39.510 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:39.510 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:11:39.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.510 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:11:39.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:11:39.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:11:39.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:11:39.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:11:39.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:11:39.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:11:39.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:11:39.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:11:39.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:11:39.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:11:39.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:11:39.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:11:39.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:11:39.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:39.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:11:39.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:11:39.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:11:39.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:11:39.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:11:39.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:11:39.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:11:39.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:11:39.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:11:39.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:11:39.515 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.515 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.515 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.515 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:11:39.515 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:11:39.515 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.21913181, 2024-09-15T00:11:39.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:11:39.516 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.516 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.516 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:39.516 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:11:39.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:11:39.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:11:39.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:11:39.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:11:39.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:11:39.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:11:39.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:11:39.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:11:39.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:11:39.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:11:39.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:11:39.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:11:39.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:11:39.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:11:39.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:11:39.519 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.519 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.519 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.519 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1612 3.e 3:77b26f8c:::rbd_data.11397bff333d.0000000000000023:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:39.519 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.663923+0000", 2024-09-15T00:11:39.519 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.79228629, 2024-09-15T00:11:39.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017776304999999999, 2024-09-15T00:11:39.520 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.520 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.520 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:39.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:39.520 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1612 2024-09-15T00:11:39.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.520 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663923+0000", 2024-09-15T00:11:39.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663923+0000", 2024-09-15T00:11:39.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663925+0000", 2024-09-15T00:11:39.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5290000000000001e-06 2024-09-15T00:11:39.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663934+0000", 2024-09-15T00:11:39.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2040000000000003e-06 2024-09-15T00:11:39.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663948+0000", 2024-09-15T00:11:39.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3978999999999999e-05 2024-09-15T00:11:39.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.663954+0000", 2024-09-15T00:11:39.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6059999999999999e-06 2024-09-15T00:11:39.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.664002+0000", 2024-09-15T00:11:39.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8127999999999997e-05 2024-09-15T00:11:39.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.664055+0000", 2024-09-15T00:11:39.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2237999999999997e-05 2024-09-15T00:11:39.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:39.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.665257+0000", 2024-09-15T00:11:39.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0012021989999999999 2024-09-15T00:11:39.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681584+0000", 2024-09-15T00:11:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016327264000000001 2024-09-15T00:11:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681620+0000", 2024-09-15T00:11:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6340000000000001e-05 2024-09-15T00:11:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681661+0000", 2024-09-15T00:11:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0259000000000002e-05 2024-09-15T00:11:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681699+0000", 2024-09-15T00:11:39.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8559000000000001e-05 2024-09-15T00:11:39.525 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.525 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.525 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.525 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1612 3.e e30/23 ondisk, result = 0)", 2024-09-15T00:11:39.525 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.668460+0000", 2024-09-15T00:11:39.525 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.787749061, 2024-09-15T00:11:39.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013245329, 2024-09-15T00:11:39.525 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.525 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.525 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:39.526 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668460+0000", 2024-09-15T00:11:39.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668460+0000", 2024-09-15T00:11:39.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668461+0000", 2024-09-15T00:11:39.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3600000000000002e-07 2024-09-15T00:11:39.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668467+0000", 2024-09-15T00:11:39.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6559999999999997e-06 2024-09-15T00:11:39.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668477+0000", 2024-09-15T00:11:39.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0539e-05 2024-09-15T00:11:39.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.668483+0000", 2024-09-15T00:11:39.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1580000000000002e-06 2024-09-15T00:11:39.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681608+0000", 2024-09-15T00:11:39.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013125009 2024-09-15T00:11:39.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681616+0000", 2024-09-15T00:11:39.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2880000000000004e-06 2024-09-15T00:11:39.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.681706+0000", 2024-09-15T00:11:39.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9843e-05 2024-09-15T00:11:39.529 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.529 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.529 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.529 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3226 3.d 3:b1632ea2:::rbd_data.1139b531b717.0000000000000009:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:39.529 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.701042+0000", 2024-09-15T00:11:39.529 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.7551677320000003, 2024-09-15T00:11:39.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016783762000000001, 2024-09-15T00:11:39.529 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.529 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.529 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.530 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.530 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:39.530 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:39.530 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3226 2024-09-15T00:11:39.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.530 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701042+0000", 2024-09-15T00:11:39.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701042+0000", 2024-09-15T00:11:39.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701042+0000", 2024-09-15T00:11:39.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:11:39.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701045+0000", 2024-09-15T00:11:39.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1379999999999999e-06 2024-09-15T00:11:39.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701053+0000", 2024-09-15T00:11:39.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6350000000000006e-06 2024-09-15T00:11:39.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701058+0000", 2024-09-15T00:11:39.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3650000000000003e-06 2024-09-15T00:11:39.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701097+0000", 2024-09-15T00:11:39.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8769000000000001e-05 2024-09-15T00:11:39.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.701164+0000", 2024-09-15T00:11:39.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7143000000000004e-05 2024-09-15T00:11:39.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:39.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.702192+0000", 2024-09-15T00:11:39.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00102808 2024-09-15T00:11:39.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717734+0000", 2024-09-15T00:11:39.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015541806 2024-09-15T00:11:39.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717748+0000", 2024-09-15T00:11:39.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3893999999999999e-05 2024-09-15T00:11:39.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717786+0000", 2024-09-15T00:11:39.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7996e-05 2024-09-15T00:11:39.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717825+0000", 2024-09-15T00:11:39.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9397000000000002e-05 2024-09-15T00:11:39.535 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.535 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.535 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.535 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:3226 3.d e33/23 ondisk, result = 0)", 2024-09-15T00:11:39.535 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.705838+0000", 2024-09-15T00:11:39.535 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.7503709769999993, 2024-09-15T00:11:39.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011902983000000001, 2024-09-15T00:11:39.535 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.535 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.535 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:39.536 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705838+0000", 2024-09-15T00:11:39.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705838+0000", 2024-09-15T00:11:39.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705839+0000", 2024-09-15T00:11:39.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9099999999999991e-07 2024-09-15T00:11:39.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705846+0000", 2024-09-15T00:11:39.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.251e-06 2024-09-15T00:11:39.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705860+0000", 2024-09-15T00:11:39.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4849e-05 2024-09-15T00:11:39.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.705868+0000", 2024-09-15T00:11:39.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6629999999999993e-06 2024-09-15T00:11:39.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717669+0000", 2024-09-15T00:11:39.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011800966 2024-09-15T00:11:39.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717681+0000", 2024-09-15T00:11:39.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2201000000000001e-05 2024-09-15T00:11:39.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.717741+0000", 2024-09-15T00:11:39.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0062000000000002e-05 2024-09-15T00:11:39.539 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.539 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.539 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.539 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18293 3.13 3:cd02983d:::rbd_data.11398e1a091a.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1310720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:39.539 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.294718+0000", 2024-09-15T00:11:39.539 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1614914330000001, 2024-09-15T00:11:39.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012851031000000001, 2024-09-15T00:11:39.540 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.540 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.540 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.540 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.540 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:39.540 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:39.540 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18293 2024-09-15T00:11:39.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.540 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294718+0000", 2024-09-15T00:11:39.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294718+0000", 2024-09-15T00:11:39.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294719+0000", 2024-09-15T00:11:39.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.271e-06 2024-09-15T00:11:39.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294724+0000", 2024-09-15T00:11:39.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:11:39.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294742+0000", 2024-09-15T00:11:39.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8012999999999998e-05 2024-09-15T00:11:39.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294750+0000", 2024-09-15T00:11:39.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5120000000000002e-06 2024-09-15T00:11:39.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294801+0000", 2024-09-15T00:11:39.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1020000000000003e-05 2024-09-15T00:11:39.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.294916+0000", 2024-09-15T00:11:39.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00011497600000000001 2024-09-15T00:11:39.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:39.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.295074+0000", 2024-09-15T00:11:39.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000158381 2024-09-15T00:11:39.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.298948+0000", 2024-09-15T00:11:39.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038734329999999999 2024-09-15T00:11:39.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.307455+0000", 2024-09-15T00:11:39.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0085077360000000001 2024-09-15T00:11:39.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.307525+0000", 2024-09-15T00:11:39.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9958999999999999e-05 2024-09-15T00:11:39.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.307569+0000", 2024-09-15T00:11:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3616999999999999e-05 2024-09-15T00:11:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:11:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.0341303269999997, 2024-09-15T00:11:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:11:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:11:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:11:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:11:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:11:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:11:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:11:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:11:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:11:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:11:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:11:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:11:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:11:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:11:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:11:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:11:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:11:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:11:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:11:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:11:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:11:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:11:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:11:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:11:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:11:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:11:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:11:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:11:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.0212540600000004, 2024-09-15T00:11:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:11:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:11:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:11:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:11:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:11:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:11:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:11:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:11:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:11:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:11:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:11:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:11:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:11:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:11:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:11:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:11:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:11:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18673 3.13 3:c8c60c5e:::rbd_data.11398e1a091a.000000000000002c:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.819584+0000", 2024-09-15T00:11:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.6366249860000002, 2024-09-15T00:11:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.021809331000000001, 2024-09-15T00:11:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18673 2024-09-15T00:11:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819584+0000", 2024-09-15T00:11:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819584+0000", 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819585+0000", 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4600000000000003e-07 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819590+0000", 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2760000000000002e-06 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819599+0000", 2024-09-15T00:11:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2809999999999997e-06 2024-09-15T00:11:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819604+0000", 2024-09-15T00:11:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8380000000000001e-06 2024-09-15T00:11:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819651+0000", 2024-09-15T00:11:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6978999999999999e-05 2024-09-15T00:11:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.819740+0000", 2024-09-15T00:11:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9562999999999996e-05 2024-09-15T00:11:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.820752+0000", 2024-09-15T00:11:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0010121170000000001 2024-09-15T00:11:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.841221+0000", 2024-09-15T00:11:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.020468857999999999 2024-09-15T00:11:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.841276+0000", 2024-09-15T00:11:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5322999999999999e-05 2024-09-15T00:11:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.841341+0000", 2024-09-15T00:11:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4332999999999995e-05 2024-09-15T00:11:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.841394+0000", 2024-09-15T00:11:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2816999999999999e-05 2024-09-15T00:11:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18673 3.13 e38/23 ondisk, result = 0)", 2024-09-15T00:11:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.823700+0000", 2024-09-15T00:11:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.6325092530000003, 2024-09-15T00:11:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017702414, 2024-09-15T00:11:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.823700+0000", 2024-09-15T00:11:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.823700+0000", 2024-09-15T00:11:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.823701+0000", 2024-09-15T00:11:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9299999999999997e-07 2024-09-15T00:11:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.823705+0000", 2024-09-15T00:11:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8079999999999998e-06 2024-09-15T00:11:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.823714+0000", 2024-09-15T00:11:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:11:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.823719+0000", 2024-09-15T00:11:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.916e-06 2024-09-15T00:11:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.841258+0000", 2024-09-15T00:11:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017538221 2024-09-15T00:11:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.841269+0000", 2024-09-15T00:11:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1583000000000001e-05 2024-09-15T00:11:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.841402+0000", 2024-09-15T00:11:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000133351 2024-09-15T00:11:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:39.565 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:39.567 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:39.567 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:39.565+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:39.668 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:39.666+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:39.672 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:39.672 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:39.672 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:39.672 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:39.681 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:11:39.769 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:39.767+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:39.784 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:39.785 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:39.785 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:39.785 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:39.785 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:39.793 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:11:39.869 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:39.868+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:39.899 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:39.899 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:39.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:39.899 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:39.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.899 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-15T00:11:39.900 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:11:39.900 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 34.361574554000001, 2024-09-15T00:11:39.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0035574980000000001, 2024-09-15T00:11:39.900 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.900 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.900 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.900 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:11:39.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:11:39.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535370+0000", 2024-09-15T00:11:39.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-15T00:11:39.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535438+0000", 2024-09-15T00:11:39.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199000000000002e-05 2024-09-15T00:11:39.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535446+0000", 2024-09-15T00:11:39.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1640000000000001e-06 2024-09-15T00:11:39.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535450+0000", 2024-09-15T00:11:39.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1520000000000002e-06 2024-09-15T00:11:39.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535491+0000", 2024-09-15T00:11:39.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0969e-05 2024-09-15T00:11:39.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535510+0000", 2024-09-15T00:11:39.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9031000000000001e-05 2024-09-15T00:11:39.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.538893+0000", 2024-09-15T00:11:39.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033823870000000002 2024-09-15T00:11:39.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.538927+0000", 2024-09-15T00:11:39.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4085999999999999e-05 2024-09-15T00:11:39.904 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.904 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.904 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.904 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:70 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-09-15T00:11:39.904 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:11:39.904 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 34.360834334000003, 2024-09-15T00:11:39.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057871490000000001, 2024-09-15T00:11:39.905 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.905 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.905 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.905 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:11:39.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:11:39.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536110+0000", 2024-09-15T00:11:39.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-15T00:11:39.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536115+0000", 2024-09-15T00:11:39.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8960000000000002e-06 2024-09-15T00:11:39.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536123+0000", 2024-09-15T00:11:39.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3979999999999999e-06 2024-09-15T00:11:39.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536128+0000", 2024-09-15T00:11:39.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5569999999999997e-06 2024-09-15T00:11:39.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537011+0000", 2024-09-15T00:11:39.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088276299999999995 2024-09-15T00:11:39.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537034+0000", 2024-09-15T00:11:39.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3448999999999999e-05 2024-09-15T00:11:39.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541867+0000", 2024-09-15T00:11:39.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048332949999999996 2024-09-15T00:11:39.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541896+0000", 2024-09-15T00:11:39.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9179999999999998e-05 2024-09-15T00:11:39.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.909 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.909 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-15T00:11:39.909 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:39.909 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 34.333215881000001, 2024-09-15T00:11:39.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027567730000000001, 2024-09-15T00:11:39.909 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.909 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.909 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.909 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:39.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:39.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:39.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5599999999999995e-07 2024-09-15T00:11:39.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563787+0000", 2024-09-15T00:11:39.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9092000000000003e-05 2024-09-15T00:11:39.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563796+0000", 2024-09-15T00:11:39.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0059999999999998e-06 2024-09-15T00:11:39.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563801+0000", 2024-09-15T00:11:39.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3290000000000004e-06 2024-09-15T00:11:39.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563830+0000", 2024-09-15T00:11:39.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9369999999999998e-05 2024-09-15T00:11:39.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563842+0000", 2024-09-15T00:11:39.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1646e-05 2024-09-15T00:11:39.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.566459+0000", 2024-09-15T00:11:39.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026174549999999999 2024-09-15T00:11:39.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.566485+0000", 2024-09-15T00:11:39.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5318999999999999e-05 2024-09-15T00:11:39.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.913 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.913 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:106 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-09-15T00:11:39.913 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:11:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 34.200313418, 2024-09-15T00:11:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029384250000000002, 2024-09-15T00:11:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:11:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:11:39.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696631+0000", 2024-09-15T00:11:39.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4299999999999998e-07 2024-09-15T00:11:39.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696692+0000", 2024-09-15T00:11:39.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1229999999999995e-05 2024-09-15T00:11:39.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696699+0000", 2024-09-15T00:11:39.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5399999999999998e-06 2024-09-15T00:11:39.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696705+0000", 2024-09-15T00:11:39.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5770000000000001e-06 2024-09-15T00:11:39.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696732+0000", 2024-09-15T00:11:39.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7022e-05 2024-09-15T00:11:39.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696743+0000", 2024-09-15T00:11:39.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1426999999999999e-05 2024-09-15T00:11:39.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699542+0000", 2024-09-15T00:11:39.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002798521 2024-09-15T00:11:39.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699569+0000", 2024-09-15T00:11:39.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6665e-05 2024-09-15T00:11:39.918 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.918 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.918 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.918 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:107 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-09-15T00:11:39.918 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:39.918 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 34.199417988, 2024-09-15T00:11:39.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048949229999999998, 2024-09-15T00:11:39.918 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.918 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.918 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.918 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:39.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:39.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:39.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5799999999999999e-07 2024-09-15T00:11:39.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697531+0000", 2024-09-15T00:11:39.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3100000000000002e-06 2024-09-15T00:11:39.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697539+0000", 2024-09-15T00:11:39.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6410000000000008e-06 2024-09-15T00:11:39.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697543+0000", 2024-09-15T00:11:39.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1060000000000002e-06 2024-09-15T00:11:39.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697573+0000", 2024-09-15T00:11:39.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9978e-05 2024-09-15T00:11:39.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697585+0000", 2024-09-15T00:11:39.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1279e-05 2024-09-15T00:11:39.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702398+0000", 2024-09-15T00:11:39.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004812981 2024-09-15T00:11:39.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702421+0000", 2024-09-15T00:11:39.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3070000000000001e-05 2024-09-15T00:11:39.922 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.922 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.922 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.922 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:108 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'66)", 2024-09-15T00:11:39.923 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:11:39.923 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 34.198524497999998, 2024-09-15T00:11:39.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040179439999999999, 2024-09-15T00:11:39.923 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.923 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.923 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.923 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:11:39.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:11:39.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698420+0000", 2024-09-15T00:11:39.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0900000000000002e-07 2024-09-15T00:11:39.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698423+0000", 2024-09-15T00:11:39.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8319999999999999e-06 2024-09-15T00:11:39.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698431+0000", 2024-09-15T00:11:39.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8949999999999998e-06 2024-09-15T00:11:39.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698435+0000", 2024-09-15T00:11:39.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5939999999999998e-06 2024-09-15T00:11:39.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698469+0000", 2024-09-15T00:11:39.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4258000000000003e-05 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698481+0000", 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1763000000000001e-05 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702422+0000", 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003940603 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702437+0000", 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5489999999999999e-05 2024-09-15T00:11:39.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.927 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.927 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.927 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:109 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'66)", 2024-09-15T00:11:39.927 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:39.927 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 34.197994061999999, 2024-09-15T00:11:39.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038555479999999999, 2024-09-15T00:11:39.927 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.927 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.927 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.927 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:39.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:39.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:39.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.13e-07 2024-09-15T00:11:39.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699018+0000", 2024-09-15T00:11:39.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7540999999999996e-05 2024-09-15T00:11:39.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699024+0000", 2024-09-15T00:11:39.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4869999999999999e-06 2024-09-15T00:11:39.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699044+0000", 2024-09-15T00:11:39.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9400999999999999e-05 2024-09-15T00:11:39.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699104+0000", 2024-09-15T00:11:39.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0331999999999998e-05 2024-09-15T00:11:39.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699117+0000", 2024-09-15T00:11:39.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2789000000000001e-05 2024-09-15T00:11:39.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702777+0000", 2024-09-15T00:11:39.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0036604900000000002 2024-09-15T00:11:39.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702805+0000", 2024-09-15T00:11:39.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7994999999999999e-05 2024-09-15T00:11:39.931 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.931 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.931 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.931 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:110 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'72, mlcod=16'66)", 2024-09-15T00:11:39.931 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:11:39.931 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 34.197234528999999, 2024-09-15T00:11:39.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031108189999999999, 2024-09-15T00:11:39.932 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.932 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.932 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.932 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:11:39.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:11:39.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699710+0000", 2024-09-15T00:11:39.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:11:39.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699714+0000", 2024-09-15T00:11:39.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0010000000000003e-06 2024-09-15T00:11:39.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699722+0000", 2024-09-15T00:11:39.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4840000000000004e-06 2024-09-15T00:11:39.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699726+0000", 2024-09-15T00:11:39.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:11:39.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699755+0000", 2024-09-15T00:11:39.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8433999999999999e-05 2024-09-15T00:11:39.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699775+0000", 2024-09-15T00:11:39.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0017000000000002e-05 2024-09-15T00:11:39.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702807+0000", 2024-09-15T00:11:39.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030320239999999999 2024-09-15T00:11:39.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702820+0000", 2024-09-15T00:11:39.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3169999999999999e-05 2024-09-15T00:11:39.935 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.935 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.935 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.936 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:120 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'82, mlcod=16'78)", 2024-09-15T00:11:39.936 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:11:39.936 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 34.191331183000003, 2024-09-15T00:11:39.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028965829999999999, 2024-09-15T00:11:39.936 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.936 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.936 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.936 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:11:39.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:11:39.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705613+0000", 2024-09-15T00:11:39.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1800000000000001e-07 2024-09-15T00:11:39.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705684+0000", 2024-09-15T00:11:39.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0903999999999997e-05 2024-09-15T00:11:39.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705691+0000", 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4290000000000004e-06 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705695+0000", 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4790000000000001e-06 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705729+0000", 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4415999999999997e-05 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705741+0000", 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1443999999999999e-05 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.708476+0000", 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027356630000000002 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.708509+0000", 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2830000000000002e-05 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 34.190544717999998, 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047824809999999999, 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:39.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:39.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:11:39.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706471+0000", 2024-09-15T00:11:39.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1496999999999997e-05 2024-09-15T00:11:39.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706478+0000", 2024-09-15T00:11:39.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7460000000000003e-06 2024-09-15T00:11:39.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706481+0000", 2024-09-15T00:11:39.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.258e-06 2024-09-15T00:11:39.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706513+0000", 2024-09-15T00:11:39.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2567000000000002e-05 2024-09-15T00:11:39.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706524+0000", 2024-09-15T00:11:39.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1028e-05 2024-09-15T00:11:39.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.711150+0000", 2024-09-15T00:11:39.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046259450000000002 2024-09-15T00:11:39.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.711181+0000", 2024-09-15T00:11:39.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1041000000000003e-05 2024-09-15T00:11:39.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.944 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.944 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:136 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-15T00:11:39.944 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:39.944 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.837107783999997, 2024-09-15T00:11:39.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031270970000000001, 2024-09-15T00:11:39.944 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.944 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.944 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.944 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:39.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:39.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3000000000000002e-07 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059839+0000", 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3050000000000001e-06 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059847+0000", 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.622e-06 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059851+0000", 2024-09-15T00:11:39.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8120000000000001e-06 2024-09-15T00:11:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059884+0000", 2024-09-15T00:11:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2620999999999998e-05 2024-09-15T00:11:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059894+0000", 2024-09-15T00:11:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0924000000000001e-05 2024-09-15T00:11:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062935+0000", 2024-09-15T00:11:39.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030406830000000002 2024-09-15T00:11:39.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062963+0000", 2024-09-15T00:11:39.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7800000000000001e-05 2024-09-15T00:11:39.948 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.948 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.948 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.948 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:139 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'92)", 2024-09-15T00:11:39.948 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:39.948 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.835320561000003, 2024-09-15T00:11:39.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040690240000000001, 2024-09-15T00:11:39.948 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.949 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.949 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.949 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:39.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:39.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061631+0000", 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.17e-06 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061636+0000", 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7679999999999997e-06 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061639+0000", 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9069999999999999e-06 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062733+0000", 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010937119999999999 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062744+0000", 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1439e-05 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065658+0000", 2024-09-15T00:11:39.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029134970000000001 2024-09-15T00:11:39.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065692+0000", 2024-09-15T00:11:39.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4147999999999999e-05 2024-09-15T00:11:39.952 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.952 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.952 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.952 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'92)", 2024-09-15T00:11:39.952 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:39.952 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.834931499, 2024-09-15T00:11:39.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003986608, 2024-09-15T00:11:39.953 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.953 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.953 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.953 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:39.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:39.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:39.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7000000000000001e-07 2024-09-15T00:11:39.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062075+0000", 2024-09-15T00:11:39.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2638000000000006e-05 2024-09-15T00:11:39.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062083+0000", 2024-09-15T00:11:39.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4330000000000004e-06 2024-09-15T00:11:39.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062087+0000", 2024-09-15T00:11:39.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3799999999999998e-06 2024-09-15T00:11:39.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.063001+0000", 2024-09-15T00:11:39.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091419500000000002 2024-09-15T00:11:39.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.063012+0000", 2024-09-15T00:11:39.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1117999999999999e-05 2024-09-15T00:11:39.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065977+0000", 2024-09-15T00:11:39.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029650840000000002 2024-09-15T00:11:39.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065999+0000", 2024-09-15T00:11:39.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1489999999999999e-05 2024-09-15T00:11:39.957 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.957 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.957 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.957 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:154 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'108, mlcod=16'105)", 2024-09-15T00:11:39.957 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:39.957 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.372683874000003, 2024-09-15T00:11:39.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053187770000000002, 2024-09-15T00:11:39.957 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.957 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.957 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.957 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:39.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:39.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:39.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:11:39.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524264+0000", 2024-09-15T00:11:39.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.738e-06 2024-09-15T00:11:39.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524270+0000", 2024-09-15T00:11:39.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9379999999999997e-06 2024-09-15T00:11:39.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524273+0000", 2024-09-15T00:11:39.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6370000000000001e-06 2024-09-15T00:11:39.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524310+0000", 2024-09-15T00:11:39.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6266999999999997e-05 2024-09-15T00:11:39.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524322+0000", 2024-09-15T00:11:39.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2741999999999999e-05 2024-09-15T00:11:39.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.529547+0000", 2024-09-15T00:11:39.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005225001 2024-09-15T00:11:39.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.529579+0000", 2024-09-15T00:11:39.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1065999999999997e-05 2024-09-15T00:11:39.961 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.961 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.961 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.961 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-15T00:11:39.962 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:11:39.962 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.372246236000002, 2024-09-15T00:11:39.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077021529999999998, 2024-09-15T00:11:39.962 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.962 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.962 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.962 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:11:39.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:11:39.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524698+0000", 2024-09-15T00:11:39.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6300000000000001e-07 2024-09-15T00:11:39.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524778+0000", 2024-09-15T00:11:39.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0598000000000003e-05 2024-09-15T00:11:39.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524785+0000", 2024-09-15T00:11:39.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6359999999999997e-06 2024-09-15T00:11:39.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524788+0000", 2024-09-15T00:11:39.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2540000000000001e-06 2024-09-15T00:11:39.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524817+0000", 2024-09-15T00:11:39.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8784000000000001e-05 2024-09-15T00:11:39.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524828+0000", 2024-09-15T00:11:39.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-15T00:11:39.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532360+0000", 2024-09-15T00:11:39.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007531757 2024-09-15T00:11:39.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532400+0000", 2024-09-15T00:11:39.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9835000000000001e-05 2024-09-15T00:11:39.966 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.966 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.966 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.968 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=16'106)", 2024-09-15T00:11:39.968 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:39.968 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.371304082999998, 2024-09-15T00:11:39.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067923050000000002, 2024-09-15T00:11:39.969 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.969 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.969 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.969 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:39.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:39.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:39.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-15T00:11:39.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525644+0000", 2024-09-15T00:11:39.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.709e-06 2024-09-15T00:11:39.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525651+0000", 2024-09-15T00:11:39.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7970000000000003e-06 2024-09-15T00:11:39.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525654+0000", 2024-09-15T00:11:39.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.433e-06 2024-09-15T00:11:39.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525685+0000", 2024-09-15T00:11:39.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0970000000000003e-05 2024-09-15T00:11:39.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525697+0000", 2024-09-15T00:11:39.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1759e-05 2024-09-15T00:11:39.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532401+0000", 2024-09-15T00:11:39.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006704425 2024-09-15T00:11:39.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532432+0000", 2024-09-15T00:11:39.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0709e-05 2024-09-15T00:11:39.973 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.973 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.973 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.973 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'106)", 2024-09-15T00:11:39.973 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:39.973 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.370841829, 2024-09-15T00:11:39.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063539190000000004, 2024-09-15T00:11:39.974 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.974 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.974 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.974 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:39.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:39.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:39.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:11:39.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526107+0000", 2024-09-15T00:11:39.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:11:39.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526117+0000", 2024-09-15T00:11:39.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7750000000000004e-06 2024-09-15T00:11:39.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526121+0000", 2024-09-15T00:11:39.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:11:39.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526163+0000", 2024-09-15T00:11:39.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1360999999999999e-05 2024-09-15T00:11:39.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526175+0000", 2024-09-15T00:11:39.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2729e-05 2024-09-15T00:11:39.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532433+0000", 2024-09-15T00:11:39.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062576999999999997 2024-09-15T00:11:39.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532456+0000", 2024-09-15T00:11:39.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2773e-05 2024-09-15T00:11:39.977 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.977 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.977 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.978 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'106)", 2024-09-15T00:11:39.978 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:11:39.978 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.370372427, 2024-09-15T00:11:39.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059082689999999998, 2024-09-15T00:11:39.978 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.978 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.978 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.978 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:11:39.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:11:39.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526572+0000", 2024-09-15T00:11:39.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9500000000000003e-07 2024-09-15T00:11:39.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526641+0000", 2024-09-15T00:11:39.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8858999999999999e-05 2024-09-15T00:11:39.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526648+0000", 2024-09-15T00:11:39.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:11:39.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526654+0000", 2024-09-15T00:11:39.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7509999999999997e-06 2024-09-15T00:11:39.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526689+0000", 2024-09-15T00:11:39.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5800000000000003e-05 2024-09-15T00:11:39.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526700+0000", 2024-09-15T00:11:39.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0467e-05 2024-09-15T00:11:39.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532457+0000", 2024-09-15T00:11:39.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057568180000000004 2024-09-15T00:11:39.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532479+0000", 2024-09-15T00:11:39.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2739e-05 2024-09-15T00:11:39.982 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.982 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.982 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.982 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'106)", 2024-09-15T00:11:39.982 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:11:39.982 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.369716162000003, 2024-09-15T00:11:39.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005274801, 2024-09-15T00:11:39.982 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.982 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.983 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.983 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:11:39.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:11:39.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527228+0000", 2024-09-15T00:11:39.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1600000000000001e-07 2024-09-15T00:11:39.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527232+0000", 2024-09-15T00:11:39.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8920000000000002e-06 2024-09-15T00:11:39.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527239+0000", 2024-09-15T00:11:39.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2880000000000004e-06 2024-09-15T00:11:39.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527243+0000", 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.143e-06 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527275+0000", 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1866000000000003e-05 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527288+0000", 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469999999999999e-05 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532480+0000", 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051927129999999998 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532502+0000", 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1912999999999998e-05 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:25 3.10 e26/23 3:0d9e4e9e:::rbd_header.11395e6383ec:head v 26'2, mlcod=0'0)", 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:21.619783+0000", 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.277160496, 2024-09-15T00:11:39.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031992600000000002, 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619783+0000", 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619783+0000", 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619784+0000", 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9900000000000009e-07 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619795+0000", 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0689e-05 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619809+0000", 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3757000000000001e-05 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:39.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619818+0000", 2024-09-15T00:11:39.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4909999999999994e-06 2024-09-15T00:11:39.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:39.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619868+0000", 2024-09-15T00:11:39.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0046999999999997e-05 2024-09-15T00:11:39.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:39.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619887+0000", 2024-09-15T00:11:39.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780999999999999e-05 2024-09-15T00:11:39.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:39.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.622947+0000", 2024-09-15T00:11:39.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030601069999999998 2024-09-15T00:11:39.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:39.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:39.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:39.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.622982+0000", 2024-09-15T00:11:39.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5389000000000003e-05 2024-09-15T00:11:39.990 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.990 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.990 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.990 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:39.990 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:39.990 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:39.992 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:39.992 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:39.990+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:40.092 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:40.090+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:40.146 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:40.146 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:40.146 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:40.155 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:40.162 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone8 from 0xb2484b9 (overlap 0x2e0256) to 0xdcd80 2024-09-15T00:11:40.193 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:40.192+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:40.266 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:40.266 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:40.275 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:40.288 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-15T00:11:40.332 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:40.330+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:40.395 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1183 3.4 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.947741+0000", 2024-09-15T00:11:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.445028134999999, 2024-09-15T00:11:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024684798000000001, 2024-09-15T00:11:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1183 2024-09-15T00:11:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947741+0000", 2024-09-15T00:11:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947741+0000", 2024-09-15T00:11:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947743+0000", 2024-09-15T00:11:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.446e-06 2024-09-15T00:11:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947750+0000", 2024-09-15T00:11:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7640000000000005e-06 2024-09-15T00:11:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947774+0000", 2024-09-15T00:11:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3958e-05 2024-09-15T00:11:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947783+0000", 2024-09-15T00:11:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8410000000000006e-06 2024-09-15T00:11:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947840+0000", 2024-09-15T00:11:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6784999999999999e-05 2024-09-15T00:11:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947947+0000", 2024-09-15T00:11:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010688000000000001 2024-09-15T00:11:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.948963+0000", 2024-09-15T00:11:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001016482 2024-09-15T00:11:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972328+0000", 2024-09-15T00:11:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023364843999999999 2024-09-15T00:11:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972341+0000", 2024-09-15T00:11:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2777e-05 2024-09-15T00:11:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972378+0000", 2024-09-15T00:11:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6749999999999999e-05 2024-09-15T00:11:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972426+0000", 2024-09-15T00:11:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8270999999999999e-05 2024-09-15T00:11:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:1183 3.4 e29/23 ondisk, result = 0)", 2024-09-15T00:11:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.951923+0000", 2024-09-15T00:11:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.440845952, 2024-09-15T00:11:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020411286000000001, 2024-09-15T00:11:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951923+0000", 2024-09-15T00:11:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951923+0000", 2024-09-15T00:11:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951924+0000", 2024-09-15T00:11:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8700000000000004e-07 2024-09-15T00:11:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951928+0000", 2024-09-15T00:11:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1110000000000001e-06 2024-09-15T00:11:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951939+0000", 2024-09-15T00:11:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0752e-05 2024-09-15T00:11:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951945+0000", 2024-09-15T00:11:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1009999999999997e-06 2024-09-15T00:11:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972316+0000", 2024-09-15T00:11:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020370335999999999 2024-09-15T00:11:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972323+0000", 2024-09-15T00:11:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9189999999999999e-06 2024-09-15T00:11:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972335+0000", 2024-09-15T00:11:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.118e-05 2024-09-15T00:11:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:11:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.212944601, 2024-09-15T00:11:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:11:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:11:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:11:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:11:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:11:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:11:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:11:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:11:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:11:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:11:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:11:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:11:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:11:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:11:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:11:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:11:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:11:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:11:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:11:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:11:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:11:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.209516691999999, 2024-09-15T00:11:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032661645000000003, 2024-09-15T00:11:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for ondisk", 2024-09-15T00:11:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:11:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:11:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:11:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:11:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5300000000000004e-07 2024-09-15T00:11:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183257+0000", 2024-09-15T00:11:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9380000000000002e-06 2024-09-15T00:11:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183265+0000", 2024-09-15T00:11:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6599999999999995e-06 2024-09-15T00:11:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183270+0000", 2024-09-15T00:11:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6489999999999996e-06 2024-09-15T00:11:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215767+0000", 2024-09-15T00:11:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032496490000000003 2024-09-15T00:11:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for ondisk", 2024-09-15T00:11:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215819+0000", 2024-09-15T00:11:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1647999999999997e-05 2024-09-15T00:11:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215914+0000", 2024-09-15T00:11:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5706999999999997e-05 2024-09-15T00:11:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:1287 3.6 e30/28 ondisk, result = 0)", 2024-09-15T00:11:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.186816+0000", 2024-09-15T00:11:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.205953106000001, 2024-09-15T00:11:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.029151897999999999, 2024-09-15T00:11:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186816+0000", 2024-09-15T00:11:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186816+0000", 2024-09-15T00:11:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186817+0000", 2024-09-15T00:11:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0729999999999999e-06 2024-09-15T00:11:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186822+0000", 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.006e-06 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186837+0000", 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4819e-05 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186846+0000", 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1519999999999998e-06 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215852+0000", 2024-09-15T00:11:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.029005882 2024-09-15T00:11:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215859+0000", 2024-09-15T00:11:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7499999999999997e-06 2024-09-15T00:11:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215968+0000", 2024-09-15T00:11:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010921600000000001 2024-09-15T00:11:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1329 3.19 3:9ac4ae77:::rbd_data.11397bff333d.0000000000000032:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.307972+0000", 2024-09-15T00:11:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.084796913, 2024-09-15T00:11:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0096326950000000001, 2024-09-15T00:11:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1329 2024-09-15T00:11:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.307972+0000", 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.307972+0000", 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.307973+0000", 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3300000000000002e-07 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.307975+0000", 2024-09-15T00:11:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1569999999999998e-06 2024-09-15T00:11:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.307983+0000", 2024-09-15T00:11:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5730000000000002e-06 2024-09-15T00:11:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.307987+0000", 2024-09-15T00:11:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9110000000000003e-06 2024-09-15T00:11:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308446+0000", 2024-09-15T00:11:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045920200000000003 2024-09-15T00:11:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308536+0000", 2024-09-15T00:11:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0469000000000004e-05 2024-09-15T00:11:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308635+0000", 2024-09-15T00:11:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8732000000000003e-05 2024-09-15T00:11:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311792+0000", 2024-09-15T00:11:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003156603 2024-09-15T00:11:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.317550+0000", 2024-09-15T00:11:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057583169999999998 2024-09-15T00:11:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.317580+0000", 2024-09-15T00:11:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9821000000000001e-05 2024-09-15T00:11:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.317605+0000", 2024-09-15T00:11:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5377000000000001e-05 2024-09-15T00:11:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1330 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.309238+0000", 2024-09-15T00:11:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.083531002000001, 2024-09-15T00:11:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0082918479999999992, 2024-09-15T00:11:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1330 2024-09-15T00:11:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309238+0000", 2024-09-15T00:11:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309238+0000", 2024-09-15T00:11:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309239+0000", 2024-09-15T00:11:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9700000000000005e-07 2024-09-15T00:11:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309243+0000", 2024-09-15T00:11:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.912e-06 2024-09-15T00:11:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309253+0000", 2024-09-15T00:11:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0059e-05 2024-09-15T00:11:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309260+0000", 2024-09-15T00:11:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2990000000000002e-06 2024-09-15T00:11:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309312+0000", 2024-09-15T00:11:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2120000000000002e-05 2024-09-15T00:11:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309388+0000", 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6794999999999999e-05 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309449+0000", 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0572000000000002e-05 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.312204+0000", 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027552599999999998 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.317441+0000", 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005236653 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.317491+0000", 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9944000000000002e-05 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.317530+0000", 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9337e-05 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.431 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.432 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1728 3.19 3:9bb394f7:::rbd_data.11397bff333d.0000000000000022:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:40.432 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.880130+0000", 2024-09-15T00:11:40.432 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.512639588000001, 2024-09-15T00:11:40.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0066262090000000001, 2024-09-15T00:11:40.432 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.432 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.432 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.432 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.432 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.432 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.432 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1728 2024-09-15T00:11:40.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.432 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880130+0000", 2024-09-15T00:11:40.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880130+0000", 2024-09-15T00:11:40.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880130+0000", 2024-09-15T00:11:40.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-15T00:11:40.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880136+0000", 2024-09-15T00:11:40.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4090000000000002e-06 2024-09-15T00:11:40.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880143+0000", 2024-09-15T00:11:40.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7579999999999992e-06 2024-09-15T00:11:40.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880149+0000", 2024-09-15T00:11:40.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8619999999999999e-06 2024-09-15T00:11:40.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880221+0000", 2024-09-15T00:11:40.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2213999999999997e-05 2024-09-15T00:11:40.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880310+0000", 2024-09-15T00:11:40.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8931999999999995e-05 2024-09-15T00:11:40.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:40.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880418+0000", 2024-09-15T00:11:40.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010771 2024-09-15T00:11:40.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:40.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.881398+0000", 2024-09-15T00:11:40.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097939699999999991 2024-09-15T00:11:40.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:40.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.886659+0000", 2024-09-15T00:11:40.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052613460000000001 2024-09-15T00:11:40.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.886714+0000", 2024-09-15T00:11:40.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4666000000000001e-05 2024-09-15T00:11:40.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.886756+0000", 2024-09-15T00:11:40.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2412e-05 2024-09-15T00:11:40.438 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.438 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.438 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.438 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18227 3.4 3:23fbab8a:::rbd_data.11393feb9e21.0000000000000003:a [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:40.438 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285135+0000", 2024-09-15T00:11:40.438 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.1076339270000002, 2024-09-15T00:11:40.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067484800000000003, 2024-09-15T00:11:40.438 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.438 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.439 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:40.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.439 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18227 2024-09-15T00:11:40.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.439 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285135+0000", 2024-09-15T00:11:40.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285135+0000", 2024-09-15T00:11:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285136+0000", 2024-09-15T00:11:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1299999999999999e-07 2024-09-15T00:11:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285137+0000", 2024-09-15T00:11:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0559999999999999e-06 2024-09-15T00:11:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285141+0000", 2024-09-15T00:11:40.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0430000000000001e-06 2024-09-15T00:11:40.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285143+0000", 2024-09-15T00:11:40.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4949999999999998e-06 2024-09-15T00:11:40.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.291717+0000", 2024-09-15T00:11:40.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0065740140000000004 2024-09-15T00:11:40.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.291791+0000", 2024-09-15T00:11:40.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3466000000000004e-05 2024-09-15T00:11:40.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.291884+0000", 2024-09-15T00:11:40.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3193e-05 2024-09-15T00:11:40.442 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.442 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.442 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.443 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18228 3.4 3:23fbab8a:::rbd_data.11393feb9e21.0000000000000003:a [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:40.443 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285146+0000", 2024-09-15T00:11:40.443 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.107623137, 2024-09-15T00:11:40.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0068998740000000003, 2024-09-15T00:11:40.443 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.443 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.443 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:40.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.443 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18228 2024-09-15T00:11:40.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.443 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285146+0000", 2024-09-15T00:11:40.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285146+0000", 2024-09-15T00:11:40.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285146+0000", 2024-09-15T00:11:40.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0599999999999999e-07 2024-09-15T00:11:40.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285154+0000", 2024-09-15T00:11:40.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3209999999999999e-06 2024-09-15T00:11:40.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285158+0000", 2024-09-15T00:11:40.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4059999999999998e-06 2024-09-15T00:11:40.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285161+0000", 2024-09-15T00:11:40.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.83e-06 2024-09-15T00:11:40.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.291927+0000", 2024-09-15T00:11:40.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067663749999999998 2024-09-15T00:11:40.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.291986+0000", 2024-09-15T00:11:40.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8906000000000001e-05 2024-09-15T00:11:40.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.292046+0000", 2024-09-15T00:11:40.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9830000000000001e-05 2024-09-15T00:11:40.447 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.447 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.447 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.447 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18245 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:40.447 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285164+0000", 2024-09-15T00:11:40.447 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.1076049149999996, 2024-09-15T00:11:40.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0076021179999999997, 2024-09-15T00:11:40.447 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.448 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.448 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:40.448 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.448 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.448 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.448 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18245 2024-09-15T00:11:40.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.448 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285164+0000", 2024-09-15T00:11:40.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285164+0000", 2024-09-15T00:11:40.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285165+0000", 2024-09-15T00:11:40.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:11:40.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285166+0000", 2024-09-15T00:11:40.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.17e-06 2024-09-15T00:11:40.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285170+0000", 2024-09-15T00:11:40.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:11:40.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285173+0000", 2024-09-15T00:11:40.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5040000000000001e-06 2024-09-15T00:11:40.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.292129+0000", 2024-09-15T00:11:40.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0069561900000000001 2024-09-15T00:11:40.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.292220+0000", 2024-09-15T00:11:40.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1267999999999994e-05 2024-09-15T00:11:40.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.292767+0000", 2024-09-15T00:11:40.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00054657900000000005 2024-09-15T00:11:40.451 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.451 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.451 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.452 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18246 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:40.452 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285175+0000", 2024-09-15T00:11:40.452 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.1075938609999998, 2024-09-15T00:11:40.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077482820000000004, 2024-09-15T00:11:40.452 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.452 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.452 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:40.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.452 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18246 2024-09-15T00:11:40.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.452 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285175+0000", 2024-09-15T00:11:40.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285175+0000", 2024-09-15T00:11:40.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285176+0000", 2024-09-15T00:11:40.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:11:40.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285177+0000", 2024-09-15T00:11:40.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.111e-06 2024-09-15T00:11:40.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285181+0000", 2024-09-15T00:11:40.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0489999999999997e-06 2024-09-15T00:11:40.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285184+0000", 2024-09-15T00:11:40.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.954e-06 2024-09-15T00:11:40.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.292790+0000", 2024-09-15T00:11:40.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0076064410000000002 2024-09-15T00:11:40.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.292862+0000", 2024-09-15T00:11:40.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1994000000000002e-05 2024-09-15T00:11:40.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.292924+0000", 2024-09-15T00:11:40.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1531000000000004e-05 2024-09-15T00:11:40.456 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.456 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.456 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.457 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18247 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:40.457 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285187+0000", 2024-09-15T00:11:40.457 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.1075825290000001, 2024-09-15T00:11:40.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079134459999999993, 2024-09-15T00:11:40.457 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.457 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.457 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:40.457 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.457 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.457 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.457 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18247 2024-09-15T00:11:40.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.457 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285187+0000", 2024-09-15T00:11:40.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285187+0000", 2024-09-15T00:11:40.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285187+0000", 2024-09-15T00:11:40.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-15T00:11:40.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285188+0000", 2024-09-15T00:11:40.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-15T00:11:40.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285193+0000", 2024-09-15T00:11:40.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.814e-06 2024-09-15T00:11:40.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285197+0000", 2024-09-15T00:11:40.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1019999999999999e-06 2024-09-15T00:11:40.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.292960+0000", 2024-09-15T00:11:40.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077625539999999996 2024-09-15T00:11:40.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293018+0000", 2024-09-15T00:11:40.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8288000000000002e-05 2024-09-15T00:11:40.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293100+0000", 2024-09-15T00:11:40.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2330000000000006e-05 2024-09-15T00:11:40.460 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.461 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.461 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.461 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18248 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:40.461 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285203+0000", 2024-09-15T00:11:40.461 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.1075668060000003, 2024-09-15T00:11:40.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0081211630000000007, 2024-09-15T00:11:40.461 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.461 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.461 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:40.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.461 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18248 2024-09-15T00:11:40.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.462 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285203+0000", 2024-09-15T00:11:40.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285203+0000", 2024-09-15T00:11:40.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285203+0000", 2024-09-15T00:11:40.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.34e-07 2024-09-15T00:11:40.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285205+0000", 2024-09-15T00:11:40.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.903e-06 2024-09-15T00:11:40.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285212+0000", 2024-09-15T00:11:40.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9539999999999998e-06 2024-09-15T00:11:40.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285216+0000", 2024-09-15T00:11:40.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6349999999999997e-06 2024-09-15T00:11:40.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293148+0000", 2024-09-15T00:11:40.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007931713 2024-09-15T00:11:40.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293233+0000", 2024-09-15T00:11:40.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4943000000000003e-05 2024-09-15T00:11:40.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293324+0000", 2024-09-15T00:11:40.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0680999999999995e-05 2024-09-15T00:11:40.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.465 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.465 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18249 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:40.465 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285223+0000", 2024-09-15T00:11:40.466 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.1075461219999996, 2024-09-15T00:11:40.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0082666660000000006, 2024-09-15T00:11:40.466 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.466 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.466 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:40.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.466 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18249 2024-09-15T00:11:40.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.466 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285223+0000", 2024-09-15T00:11:40.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285223+0000", 2024-09-15T00:11:40.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285224+0000", 2024-09-15T00:11:40.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:11:40.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285226+0000", 2024-09-15T00:11:40.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4310000000000001e-06 2024-09-15T00:11:40.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285235+0000", 2024-09-15T00:11:40.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9020000000000005e-06 2024-09-15T00:11:40.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285240+0000", 2024-09-15T00:11:40.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4410000000000001e-06 2024-09-15T00:11:40.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293372+0000", 2024-09-15T00:11:40.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0081318740000000007 2024-09-15T00:11:40.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293435+0000", 2024-09-15T00:11:40.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3215999999999999e-05 2024-09-15T00:11:40.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293490+0000", 2024-09-15T00:11:40.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4401999999999999e-05 2024-09-15T00:11:40.470 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.470 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.470 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.470 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18250 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:40.470 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285248+0000", 2024-09-15T00:11:40.470 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.1075210860000002, 2024-09-15T00:11:40.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0083998030000000008, 2024-09-15T00:11:40.470 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.470 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.470 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:40.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.471 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.471 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.471 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18250 2024-09-15T00:11:40.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.471 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285248+0000", 2024-09-15T00:11:40.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285248+0000", 2024-09-15T00:11:40.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285249+0000", 2024-09-15T00:11:40.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:11:40.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285251+0000", 2024-09-15T00:11:40.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5730000000000002e-06 2024-09-15T00:11:40.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285261+0000", 2024-09-15T00:11:40.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8970000000000003e-06 2024-09-15T00:11:40.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285266+0000", 2024-09-15T00:11:40.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3530000000000002e-06 2024-09-15T00:11:40.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293525+0000", 2024-09-15T00:11:40.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0082582539999999996 2024-09-15T00:11:40.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293588+0000", 2024-09-15T00:11:40.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3633000000000006e-05 2024-09-15T00:11:40.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293648+0000", 2024-09-15T00:11:40.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9667000000000003e-05 2024-09-15T00:11:40.474 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.474 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.474 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.475 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18251 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:40.475 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285273+0000", 2024-09-15T00:11:40.475 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.1074963599999998, 2024-09-15T00:11:40.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0085317829999999994, 2024-09-15T00:11:40.475 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.475 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.475 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:40.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.475 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18251 2024-09-15T00:11:40.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.475 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285273+0000", 2024-09-15T00:11:40.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285273+0000", 2024-09-15T00:11:40.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285273+0000", 2024-09-15T00:11:40.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4999999999999998e-07 2024-09-15T00:11:40.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285276+0000", 2024-09-15T00:11:40.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3590000000000002e-06 2024-09-15T00:11:40.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285282+0000", 2024-09-15T00:11:40.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-15T00:11:40.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285286+0000", 2024-09-15T00:11:40.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9570000000000004e-06 2024-09-15T00:11:40.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293687+0000", 2024-09-15T00:11:40.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0084014599999999995 2024-09-15T00:11:40.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293733+0000", 2024-09-15T00:11:40.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5572999999999999e-05 2024-09-15T00:11:40.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293805+0000", 2024-09-15T00:11:40.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1901000000000002e-05 2024-09-15T00:11:40.479 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.479 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.479 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.479 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18252 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:40.479 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285290+0000", 2024-09-15T00:11:40.479 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.1074794570000002, 2024-09-15T00:11:40.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0086438569999999996, 2024-09-15T00:11:40.479 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.479 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.480 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:40.480 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.480 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.480 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.480 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18252 2024-09-15T00:11:40.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.480 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285290+0000", 2024-09-15T00:11:40.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285290+0000", 2024-09-15T00:11:40.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285290+0000", 2024-09-15T00:11:40.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-15T00:11:40.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285291+0000", 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2640000000000001e-06 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285296+0000", 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4440000000000001e-06 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285298+0000", 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5160000000000001e-06 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293834+0000", 2024-09-15T00:11:40.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0085355929999999993 2024-09-15T00:11:40.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293879+0000", 2024-09-15T00:11:40.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5464999999999999e-05 2024-09-15T00:11:40.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293934+0000", 2024-09-15T00:11:40.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4354000000000002e-05 2024-09-15T00:11:40.483 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.483 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.483 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.484 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:40.484 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:11:40.484 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.4831034220000001, 2024-09-15T00:11:40.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:11:40.484 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.484 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.484 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.484 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:40.484 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:40.484 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:40.484 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:11:40.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.484 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:11:40.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:40.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:11:40.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:11:40.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:40.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:11:40.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:11:40.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:11:40.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:11:40.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:11:40.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:11:40.489 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.489 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.489 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.489 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:11:40.489 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:11:40.489 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.4761450590000003, 2024-09-15T00:11:40.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:11:40.490 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.490 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.490 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:40.490 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:11:40.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:11:40.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:11:40.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:11:40.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:11:40.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:11:40.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:11:40.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:11:40.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:11:40.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:11:40.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:11:40.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:11:40.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:11:40.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:11:40.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:11:40.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:11:40.494 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.494 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.494 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.494 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.494 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.494 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:40.495 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:40.393+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:40.496 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:40.497 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:40.495+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:40.553 INFO:teuthology.orchestra.run.smithi171.stdout:1350 read 0x247d097 thru 0x248277a (0x56e4 bytes) 2024-09-15T00:11:40.555 INFO:teuthology.orchestra.run.smithi171.stdout:1352 trunc from 0xb2484b9 to 0xa51dbbc 2024-09-15T00:11:40.588 INFO:teuthology.orchestra.run.smithi171.stdout:1354 trunc from 0xa51dbbc to 0x3acf8bb 2024-09-15T00:11:40.598 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:40.596+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:40.631 INFO:teuthology.orchestra.run.smithi171.stdout:1355 write 0x3327f30 thru 0x333182b (0x98fc bytes) 2024-09-15T00:11:40.633 INFO:teuthology.orchestra.run.smithi171.stdout:1356 write 0x1f1a3f4 thru 0x1f1e30f (0x3f1c bytes) 2024-09-15T00:11:40.642 INFO:teuthology.orchestra.run.smithi171.stdout:1357 write 0x5cdb80c thru 0x5cdccf1 (0x14e6 bytes) 2024-09-15T00:11:40.645 INFO:teuthology.orchestra.run.smithi171.stdout:1358 write 0x2442e10 thru 0x2452632 (0xf823 bytes) 2024-09-15T00:11:40.649 INFO:teuthology.orchestra.run.smithi171.stdout:1359 read 0x49992f1 thru 0x499f4aa (0x61ba bytes) 2024-09-15T00:11:40.651 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:40.651 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:40.651 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:40.651 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:40.659 INFO:teuthology.orchestra.run.smithi171.stdout:1360 trunc from 0x5cdccf2 to 0xe06a51e 2024-09-15T00:11:40.664 INFO:teuthology.orchestra.run.smithi171.stdout:1363 read 0xc1b5f92 thru 0xc1c32be (0xd32d bytes) 2024-09-15T00:11:40.664 INFO:teuthology.orchestra.run.smithi171.stdout:1364 read 0x8e672eb thru 0x8e67806 (0x51c bytes) 2024-09-15T00:11:40.665 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:40.669 INFO:teuthology.orchestra.run.smithi171.stdout:1365 trunc from 0xe06a51e to 0xa2851ec 2024-09-15T00:11:40.673 INFO:teuthology.orchestra.run.smithi171.stdout:1368 read 0x393c6b3 thru 0x393f850 (0x319e bytes) 2024-09-15T00:11:40.679 INFO:teuthology.orchestra.run.smithi171.stdout:1370 trunc from 0xa2851ec to 0xe6b6998 2024-09-15T00:11:40.684 INFO:teuthology.orchestra.run.smithi171.stdout:1371 write 0x4e43e76 thru 0x4e45479 (0x1604 bytes) 2024-09-15T00:11:40.687 INFO:teuthology.orchestra.run.smithi171.stdout:1372 read 0x766686b thru 0x7676583 (0xfd19 bytes) 2024-09-15T00:11:40.687 INFO:teuthology.orchestra.run.smithi171.stdout:1373 read 0x7bfb0b1 thru 0x7c02365 (0x72b5 bytes) 2024-09-15T00:11:40.687 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:40.687 INFO:teuthology.orchestra.run.smithi171.stdout:1376 punch from 0xc9da6a2 to 0xc9e107d, (0x69db bytes) 2024-09-15T00:11:40.687 INFO:teuthology.orchestra.run.smithi171.stdout:1377 writesame 0xe0fa5ae thru 0xe0fc371 data_size 0x14(0x1dc4 bytes) 2024-09-15T00:11:40.689 INFO:teuthology.orchestra.run.smithi171.stdout:1378 writesame 0x710cebb thru 0x7118e5e data_size 0x2(0xbfa4 bytes) 2024-09-15T00:11:40.698 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:40.697+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:40.701 INFO:teuthology.orchestra.run.smithi171.stdout:1379 read 0x7a7ff88 thru 0x7a80b71 (0xbea bytes) 2024-09-15T00:11:40.701 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:40.701 INFO:teuthology.orchestra.run.smithi171.stdout:1382 write 0xaa7d67c thru 0xaa83371 (0x5cf6 bytes) 2024-09-15T00:11:40.706 INFO:teuthology.orchestra.run.smithi171.stdout:1383 read 0xc13bc4b thru 0xc14a0e4 (0xe49a bytes) 2024-09-15T00:11:40.706 INFO:teuthology.orchestra.run.smithi171.stdout:1385 writesame 0x4812bdc thru 0x4816931 data_size 0x3(0x3d56 bytes) 2024-09-15T00:11:40.729 INFO:teuthology.orchestra.run.smithi171.stdout:1386 trunc from 0xe6b6998 to 0x3cbec4f 2024-09-15T00:11:40.744 INFO:teuthology.orchestra.run.smithi171.stdout:1387 read 0x286edd2 thru 0x287722d (0x845c bytes) 2024-09-15T00:11:40.745 INFO:teuthology.orchestra.run.smithi171.stdout:1389 punch from 0xf8445b to 0xf8b76a, (0x730f bytes) 2024-09-15T00:11:40.745 INFO:teuthology.orchestra.run.smithi171.stdout:1391 read 0x1fb4151 thru 0x1fc3bfd (0xfaad bytes) 2024-09-15T00:11:40.761 INFO:teuthology.orchestra.run.smithi171.stdout:1392 write 0xb8be7fa thru 0xb8c404f (0x5856 bytes) 2024-09-15T00:11:40.765 INFO:teuthology.orchestra.run.smithi171.stdout:1394 read 0x49d5414 thru 0x49d8882 (0x346f bytes) 2024-09-15T00:11:40.765 INFO:teuthology.orchestra.run.smithi171.stdout:1396 write 0x86a610a thru 0x86a98fb (0x37f2 bytes) 2024-09-15T00:11:40.768 INFO:teuthology.orchestra.run.smithi171.stdout:1397 write 0x6543b9c thru 0x6546d13 (0x3178 bytes) 2024-09-15T00:11:40.772 INFO:teuthology.orchestra.run.smithi171.stdout:1398 read 0x1fecedd thru 0x1ff5a5e (0x8b82 bytes) 2024-09-15T00:11:40.772 INFO:teuthology.orchestra.run.smithi171.stdout:1399 punch from 0x46b922 to 0x474ae3, (0x91c1 bytes) 2024-09-15T00:11:40.772 INFO:teuthology.orchestra.run.smithi171.stdout:1401 punch from 0x9de66f3 to 0x9dedcdb, (0x75e8 bytes) 2024-09-15T00:11:40.776 INFO:teuthology.orchestra.run.smithi171.stdout:1402 trunc from 0xb8c4050 to 0x8d27407 2024-09-15T00:11:40.784 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:40.784 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:40.784 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:40.785 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:40.788 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:40.789 INFO:teuthology.orchestra.run.smithi171.stdout:1403 read 0x2e06e56 thru 0x2e0d3ff (0x65aa bytes) 2024-09-15T00:11:40.789 INFO:teuthology.orchestra.run.smithi171.stdout:1405 write 0x78159ea thru 0x7818479 (0x2a90 bytes) 2024-09-15T00:11:40.790 INFO:teuthology.orchestra.run.smithi171.stdout:1406 read 0x5c3b1d4 thru 0x5c3fe8b (0x4cb8 bytes) 2024-09-15T00:11:40.790 INFO:teuthology.orchestra.run.smithi171.stdout:1407 read 0x766b986 thru 0x766d11e (0x1799 bytes) 2024-09-15T00:11:40.790 INFO:teuthology.orchestra.run.smithi171.stdout:1410 read 0x40d4db6 thru 0x40df53e (0xa789 bytes) 2024-09-15T00:11:40.790 INFO:teuthology.orchestra.run.smithi171.stdout:1411 read 0x331c319 thru 0x33237e1 (0x74c9 bytes) 2024-09-15T00:11:40.791 INFO:teuthology.orchestra.run.smithi171.stdout:1412 writesame 0x46e1921 thru 0x46e526a data_size 0x1(0x394a bytes) 2024-09-15T00:11:40.794 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:40.817 INFO:teuthology.orchestra.run.smithi171.stdout:1413 punch from 0x40d1d11 to 0x40d600e, (0x42fd bytes) 2024-09-15T00:11:40.817 INFO:teuthology.orchestra.run.smithi171.stdout:1414 read 0x8463bf1 thru 0x846b1d9 (0x75e9 bytes) 2024-09-15T00:11:40.817 INFO:teuthology.orchestra.run.smithi171.stdout:1415 read 0x517cbb5 thru 0x5182092 (0x54de bytes) 2024-09-15T00:11:40.818 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:40.816+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:40.823 INFO:teuthology.orchestra.run.smithi171.stdout:1416 write 0xa1b249d thru 0xa1ba55b (0x80bf bytes) 2024-09-15T00:11:40.836 INFO:teuthology.orchestra.run.smithi171.stdout:1417 write 0xe96b644 thru 0xe97716e (0xbb2b bytes) 2024-09-15T00:11:40.840 INFO:teuthology.orchestra.run.smithi171.stdout:1418 read 0x9ed7f20 thru 0x9ee48bc (0xc99d bytes) 2024-09-15T00:11:40.840 INFO:teuthology.orchestra.run.smithi171.stdout:1419 write 0x3989ff9 thru 0x398cd4a (0x2d52 bytes) 2024-09-15T00:11:40.842 INFO:teuthology.orchestra.run.smithi171.stdout:1420 write 0x33474f7 thru 0x33529fa (0xb504 bytes) 2024-09-15T00:11:40.847 INFO:teuthology.orchestra.run.smithi171.stdout:1421 trunc from 0xe97716f to 0xc89e4fe 2024-09-15T00:11:40.855 INFO:teuthology.orchestra.run.smithi171.stdout:1422 writesame 0x983523f thru 0x9836d61 data_size 0x1(0x1b23 bytes) 2024-09-15T00:11:40.863 INFO:teuthology.orchestra.run.smithi171.stdout:1423 trunc from 0xc89e4fe to 0xa082c57 2024-09-15T00:11:40.869 INFO:teuthology.orchestra.run.smithi171.stdout:1425 punch from 0x760703 to 0x769373, (0x8c70 bytes) 2024-09-15T00:11:40.869 INFO:teuthology.orchestra.run.smithi171.stdout:1426 writesame 0x244ce5f thru 0x244f2bb data_size 0x1d(0x245d bytes) 2024-09-15T00:11:40.879 INFO:teuthology.orchestra.run.smithi171.stdout:1430 writesame 0xb1c9494 thru 0xb1d802c data_size 0x1(0xeb99 bytes) 2024-09-15T00:11:40.879 INFO:teuthology.orchestra.run.smithi171.stdout:1433 read 0x1d560 thru 0x2ca6f (0xf510 bytes) 2024-09-15T00:11:40.880 INFO:teuthology.orchestra.run.smithi171.stdout:1435 punch from 0x24e8a3c to 0x24f3763, (0xad27 bytes) 2024-09-15T00:11:40.889 INFO:teuthology.orchestra.run.smithi171.stdout:1437 write 0xe2c82b3 thru 0xe2cd69a (0x53e8 bytes) 2024-09-15T00:11:40.891 INFO:teuthology.orchestra.run.smithi171.stdout:1438 writesame 0x757fa65 thru 0x7584b11 data_size 0x1(0x50ad bytes) 2024-09-15T00:11:40.899 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:40.898+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:40.925 INFO:teuthology.orchestra.run.smithi171.stdout:1439 writesame 0xbd48b25 thru 0xbd582b9 data_size 0x3(0xf795 bytes) 2024-09-15T00:11:40.930 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:40.930 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:40.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:462 3.0 e29/23 3:04479396:::rbd_id.image_client.0-clone2:head v 29'14, mlcod=27'12)", 2024-09-15T00:11:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.911852541, 2024-09-15T00:11:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030963029999999999, 2024-09-15T00:11:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:11:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016208+0000", 2024-09-15T00:11:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3669999999999997e-06 2024-09-15T00:11:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016250+0000", 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1845999999999999e-05 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016260+0000", 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9750000000000002e-06 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016342+0000", 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1595999999999996e-05 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016363+0000", 2024-09-15T00:11:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1073e-05 2024-09-15T00:11:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019246+0000", 2024-09-15T00:11:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028835240000000002 2024-09-15T00:11:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019296+0000", 2024-09-15T00:11:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9533999999999997e-05 2024-09-15T00:11:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:463 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'9, mlcod=27'7)", 2024-09-15T00:11:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.907239537000001, 2024-09-15T00:11:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040015709999999998, 2024-09-15T00:11:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7599999999999999e-07 2024-09-15T00:11:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020818+0000", 2024-09-15T00:11:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6609999999999999e-06 2024-09-15T00:11:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020826+0000", 2024-09-15T00:11:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4770000000000003e-06 2024-09-15T00:11:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020833+0000", 2024-09-15T00:11:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0809999999999996e-06 2024-09-15T00:11:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020878+0000", 2024-09-15T00:11:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4752000000000001e-05 2024-09-15T00:11:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020892+0000", 2024-09-15T00:11:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3899e-05 2024-09-15T00:11:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024767+0000", 2024-09-15T00:11:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038752259999999998 2024-09-15T00:11:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024814+0000", 2024-09-15T00:11:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7299000000000002e-05 2024-09-15T00:11:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:464 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'10, mlcod=27'8)", 2024-09-15T00:11:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.902319694999999, 2024-09-15T00:11:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048109290000000002, 2024-09-15T00:11:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-15T00:11:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025743+0000", 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5789999999999993e-06 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025752+0000", 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8440000000000004e-06 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025757+0000", 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2610000000000001e-06 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025796+0000", 2024-09-15T00:11:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9428999999999998e-05 2024-09-15T00:11:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025810+0000", 2024-09-15T00:11:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3129000000000001e-05 2024-09-15T00:11:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030514+0000", 2024-09-15T00:11:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047040099999999998 2024-09-15T00:11:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030544+0000", 2024-09-15T00:11:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9966000000000001e-05 2024-09-15T00:11:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:610 3.4 e29/23 3:255b0913:::rbd_data.11397bff333d.0000000000000039:head v 29'2, mlcod=0'0)", 2024-09-15T00:11:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:11:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.6995761, 2024-09-15T00:11:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003163014, 2024-09-15T00:11:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:11:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:11:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228477+0000", 2024-09-15T00:11:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1100000000000004e-07 2024-09-15T00:11:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228483+0000", 2024-09-15T00:11:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3720000000000001e-06 2024-09-15T00:11:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228497+0000", 2024-09-15T00:11:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3944000000000001e-05 2024-09-15T00:11:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228505+0000", 2024-09-15T00:11:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8140000000000001e-06 2024-09-15T00:11:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228549+0000", 2024-09-15T00:11:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4116000000000002e-05 2024-09-15T00:11:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228568+0000", 2024-09-15T00:11:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8473000000000001e-05 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231599+0000", 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030315590000000001 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231639+0000", 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9824999999999999e-05 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:766 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'81, mlcod=29'78)", 2024-09-15T00:11:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:11:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.495027023, 2024-09-15T00:11:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029025880000000002, 2024-09-15T00:11:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:11:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:11:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433026+0000", 2024-09-15T00:11:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:11:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433030+0000", 2024-09-15T00:11:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6280000000000004e-06 2024-09-15T00:11:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433038+0000", 2024-09-15T00:11:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.072e-06 2024-09-15T00:11:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433045+0000", 2024-09-15T00:11:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1510000000000001e-06 2024-09-15T00:11:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433101+0000", 2024-09-15T00:11:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6336999999999999e-05 2024-09-15T00:11:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433113+0000", 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1639e-05 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435897+0000", 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027845779999999998 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435928+0000", 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0689999999999999e-05 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:832 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'100, mlcod=29'97)", 2024-09-15T00:11:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.402781523, 2024-09-15T00:11:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005498925, 2024-09-15T00:11:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7999999999999995e-07 2024-09-15T00:11:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525276+0000", 2024-09-15T00:11:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6739999999999996e-06 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525287+0000", 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0906e-05 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525293+0000", 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4409999999999998e-06 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525329+0000", 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5237000000000002e-05 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525340+0000", 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1782e-05 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530710+0000", 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053695460000000002 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530770+0000", 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9658999999999999e-05 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:876 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'114, mlcod=29'111)", 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.231742522999999, 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028465830000000002, 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2399999999999998e-07 2024-09-15T00:11:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696316+0000", 2024-09-15T00:11:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2800000000000003e-06 2024-09-15T00:11:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696326+0000", 2024-09-15T00:11:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0546e-05 2024-09-15T00:11:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696333+0000", 2024-09-15T00:11:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7660000000000001e-06 2024-09-15T00:11:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696369+0000", 2024-09-15T00:11:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5933000000000003e-05 2024-09-15T00:11:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696381+0000", 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2417e-05 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699124+0000", 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027428000000000001 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699156+0000", 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2217e-05 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1313 3.1f e30/28 3:fa9cde43:::rbd_data.11397bff333d.000000000000003a:head v 30'3, mlcod=26'1)", 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.643902885999999, 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044064359999999997, 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:11:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284150+0000", 2024-09-15T00:11:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3799999999999999e-07 2024-09-15T00:11:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284228+0000", 2024-09-15T00:11:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:11:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284235+0000", 2024-09-15T00:11:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:11:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284239+0000", 2024-09-15T00:11:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3449999999999999e-06 2024-09-15T00:11:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284282+0000", 2024-09-15T00:11:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2506000000000003e-05 2024-09-15T00:11:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284297+0000", 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5523999999999999e-05 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288517+0000", 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042194019999999997 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288556+0000", 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8952000000000002e-05 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1321 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'7, mlcod=29'5)", 2024-09-15T00:11:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.619809627, 2024-09-15T00:11:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003131254, 2024-09-15T00:11:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308250+0000", 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6109999999999999e-06 2024-09-15T00:11:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308264+0000", 2024-09-15T00:11:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4473e-05 2024-09-15T00:11:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308272+0000", 2024-09-15T00:11:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3540000000000002e-06 2024-09-15T00:11:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308340+0000", 2024-09-15T00:11:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8305000000000004e-05 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308361+0000", 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0409999999999999e-05 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311332+0000", 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029715950000000001 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311374+0000", 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1786000000000003e-05 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.9 epoch 30/23 rep_tid 277 entries 30'217 (0'0) error 3:942069cf:::rbd_header.11397bff333d:head by client.4409.0:1373 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'214)", 2024-09-15T00:11:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:11:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.57425703, 2024-09-15T00:11:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034617960000000001, 2024-09-15T00:11:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:11:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:11:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:11:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353796+0000", 2024-09-15T00:11:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6699999999999999e-07 2024-09-15T00:11:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353803+0000", 2024-09-15T00:11:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.832e-06 2024-09-15T00:11:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353814+0000", 2024-09-15T00:11:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1815e-05 2024-09-15T00:11:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353822+0000", 2024-09-15T00:11:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7440000000000007e-06 2024-09-15T00:11:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353910+0000", 2024-09-15T00:11:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7491000000000006e-05 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.357257+0000", 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033474469999999999 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1374 3.9 e30/23 3:942069cf:::rbd_header.11397bff333d:head v 30'218, mlcod=30'215)", 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.572857387999999, 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047299830000000001, 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:11:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:11:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355196+0000", 2024-09-15T00:11:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3900000000000004e-07 2024-09-15T00:11:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355200+0000", 2024-09-15T00:11:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5700000000000003e-06 2024-09-15T00:11:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355210+0000", 2024-09-15T00:11:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6050000000000004e-06 2024-09-15T00:11:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355215+0000", 2024-09-15T00:11:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.933e-06 2024-09-15T00:11:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355249+0000", 2024-09-15T00:11:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4110999999999999e-05 2024-09-15T00:11:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355260+0000", 2024-09-15T00:11:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1235e-05 2024-09-15T00:11:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359895+0000", 2024-09-15T00:11:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046345550000000003 2024-09-15T00:11:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359925+0000", 2024-09-15T00:11:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0335e-05 2024-09-15T00:11:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1385 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'221, mlcod=30'218)", 2024-09-15T00:11:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:11:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.560400853000001, 2024-09-15T00:11:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030443950000000001, 2024-09-15T00:11:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:11:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:11:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367652+0000", 2024-09-15T00:11:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:11:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367657+0000", 2024-09-15T00:11:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5410000000000002e-06 2024-09-15T00:11:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367666+0000", 2024-09-15T00:11:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9280000000000005e-06 2024-09-15T00:11:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367671+0000", 2024-09-15T00:11:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7870000000000001e-06 2024-09-15T00:11:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367707+0000", 2024-09-15T00:11:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6102000000000001e-05 2024-09-15T00:11:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367719+0000", 2024-09-15T00:11:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1571e-05 2024-09-15T00:11:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370666+0000", 2024-09-15T00:11:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029474750000000002 2024-09-15T00:11:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370696+0000", 2024-09-15T00:11:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9402000000000001e-05 2024-09-15T00:11:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1662 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'300, mlcod=30'297)", 2024-09-15T00:11:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:11:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.140036216, 2024-09-15T00:11:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031296090000000002, 2024-09-15T00:11:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:11:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788017+0000", 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8999999999999996e-07 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788021+0000", 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4970000000000003e-06 2024-09-15T00:11:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788027+0000", 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1840000000000004e-06 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788032+0000", 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.135e-06 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788070+0000", 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7073999999999998e-05 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788081+0000", 2024-09-15T00:11:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0987e-05 2024-09-15T00:11:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791116+0000", 2024-09-15T00:11:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030355209999999998 2024-09-15T00:11:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791146+0000", 2024-09-15T00:11:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9621e-05 2024-09-15T00:11:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1677 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'305, mlcod=30'302)", 2024-09-15T00:11:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:11:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.116307854, 2024-09-15T00:11:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031364320000000002, 2024-09-15T00:11:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:11:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:11:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811745+0000", 2024-09-15T00:11:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:11:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811749+0000", 2024-09-15T00:11:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9369999999999997e-06 2024-09-15T00:11:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811756+0000", 2024-09-15T00:11:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:11:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811765+0000", 2024-09-15T00:11:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7930000000000003e-06 2024-09-15T00:11:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811797+0000", 2024-09-15T00:11:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2203999999999999e-05 2024-09-15T00:11:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811810+0000", 2024-09-15T00:11:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2359e-05 2024-09-15T00:11:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814852+0000", 2024-09-15T00:11:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003042276 2024-09-15T00:11:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814881+0000", 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9037999999999999e-05 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1724 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'9, mlcod=30'7)", 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.047382170000001, 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003115769, 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:11:40.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:11:40.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880671+0000", 2024-09-15T00:11:40.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4900000000000005e-07 2024-09-15T00:11:40.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880678+0000", 2024-09-15T00:11:40.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0979999999999996e-06 2024-09-15T00:11:40.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880691+0000", 2024-09-15T00:11:40.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3288999999999999e-05 2024-09-15T00:11:40.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880698+0000", 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9240000000000001e-06 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880757+0000", 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8866000000000002e-05 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880790+0000", 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2926999999999998e-05 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883738+0000", 2024-09-15T00:11:40.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002948418 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883786+0000", 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7398000000000003e-05 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1790 3.9 e30/23 3:9676e01b:::rbd_data.11397bff333d.0000000000000030:head v 30'338, mlcod=30'335)", 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.9684998, 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044496980000000002, 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.992 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:40.993 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:40.993 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:40.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:40.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:11:40.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:40.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:11:40.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:40.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:40.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959553+0000", 2024-09-15T00:11:40.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3600000000000004e-07 2024-09-15T00:11:40.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:40.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959569+0000", 2024-09-15T00:11:40.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6152999999999999e-05 2024-09-15T00:11:40.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:40.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959577+0000", 2024-09-15T00:11:40.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5179999999999998e-06 2024-09-15T00:11:40.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:40.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959581+0000", 2024-09-15T00:11:40.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3669999999999999e-06 2024-09-15T00:11:40.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:40.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959619+0000", 2024-09-15T00:11:40.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8209e-05 2024-09-15T00:11:40.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:40.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959632+0000", 2024-09-15T00:11:40.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2405000000000001e-05 2024-09-15T00:11:40.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:40.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.963954+0000", 2024-09-15T00:11:40.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043226660000000002 2024-09-15T00:11:40.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:40.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.964002+0000", 2024-09-15T00:11:40.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7843999999999998e-05 2024-09-15T00:11:40.999 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.999 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:40.999 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:40.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:40.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:40.999 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:3177 3.4 e33/23 3:23818c00:::rbd_data.1139b531b717.0000000000000022:head v 33'58, mlcod=33'56)", 2024-09-15T00:11:40.999 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:11:40.999 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 11.270211234, 2024-09-15T00:11:40.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030564120000000001, 2024-09-15T00:11:40.999 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:40.999 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:41.000 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:41.000 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:41.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:41.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:11:41.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:41.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:41.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:11:41.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:41.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:41.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657842+0000", 2024-09-15T00:11:41.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.19e-06 2024-09-15T00:11:41.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:41.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657853+0000", 2024-09-15T00:11:41.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:11:41.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:41.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657871+0000", 2024-09-15T00:11:41.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8233e-05 2024-09-15T00:11:41.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:41.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657880+0000", 2024-09-15T00:11:41.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8319999999999995e-06 2024-09-15T00:11:41.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:41.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657949+0000", 2024-09-15T00:11:41.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8847e-05 2024-09-15T00:11:41.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:41.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657977+0000", 2024-09-15T00:11:41.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8308000000000001e-05 2024-09-15T00:11:41.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:41.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.660851+0000", 2024-09-15T00:11:41.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028743190000000002 2024-09-15T00:11:41.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:41.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.660897+0000", 2024-09-15T00:11:41.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6142e-05 2024-09-15T00:11:41.003 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:41.004 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:41.004 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:41.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.004 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:3602 3.1f e34/28 3:fd28af01:::rbd_data.1139b531b717.0000000000000032:head v 34'24, mlcod=33'22)", 2024-09-15T00:11:41.004 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:11:41.004 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.882813839000001, 2024-09-15T00:11:41.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034159709999999999, 2024-09-15T00:11:41.004 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:41.004 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:41.004 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:41.004 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:41.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:41.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:11:41.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:41.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:41.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:11:41.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:41.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:41.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045239+0000", 2024-09-15T00:11:41.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:11:41.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:41.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045252+0000", 2024-09-15T00:11:41.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2891000000000001e-05 2024-09-15T00:11:41.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:41.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045263+0000", 2024-09-15T00:11:41.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0875e-05 2024-09-15T00:11:41.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:41.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045271+0000", 2024-09-15T00:11:41.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3529999999999995e-06 2024-09-15T00:11:41.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:41.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045338+0000", 2024-09-15T00:11:41.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7377000000000002e-05 2024-09-15T00:11:41.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:41.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045359+0000", 2024-09-15T00:11:41.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1078e-05 2024-09-15T00:11:41.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:41.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.048621+0000", 2024-09-15T00:11:41.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003261543 2024-09-15T00:11:41.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:41.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.048654+0000", 2024-09-15T00:11:41.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3550999999999998e-05 2024-09-15T00:11:41.008 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:41.008 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:41.008 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:41.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.008 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:4253 3.4 e35/23 3:242f5431:::rbd_data.11396389f8f2.0000000000000000:head v 35'87, mlcod=35'85)", 2024-09-15T00:11:41.009 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:11:41.009 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.5353249810000005, 2024-09-15T00:11:41.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002944997, 2024-09-15T00:11:41.009 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:41.009 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:41.009 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:41.009 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:41.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:41.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:11:41.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:41.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:41.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:11:41.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:41.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:41.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392728+0000", 2024-09-15T00:11:41.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.06e-07 2024-09-15T00:11:41.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:41.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392782+0000", 2024-09-15T00:11:41.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.448e-05 2024-09-15T00:11:41.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:41.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392794+0000", 2024-09-15T00:11:41.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1768e-05 2024-09-15T00:11:41.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:41.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392801+0000", 2024-09-15T00:11:41.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9589999999999998e-06 2024-09-15T00:11:41.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:41.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392845+0000", 2024-09-15T00:11:41.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3884999999999997e-05 2024-09-15T00:11:41.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:41.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392860+0000", 2024-09-15T00:11:41.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5364999999999999e-05 2024-09-15T00:11:41.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:41.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.395641+0000", 2024-09-15T00:11:41.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027807629999999999 2024-09-15T00:11:41.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:41.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.395672+0000", 2024-09-15T00:11:41.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1170999999999997e-05 2024-09-15T00:11:41.013 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:41.013 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:41.013 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:41.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.013 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:18791 3.0 e38/23 3:05ecfc97:::rbd_data.11398e1a091a.0000000000000030:head v 38'66, mlcod=38'64)", 2024-09-15T00:11:41.013 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:11:41.013 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8924094450000002, 2024-09-15T00:11:41.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029447940000000001, 2024-09-15T00:11:41.013 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:41.013 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:41.013 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:41.013 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035644+0000", 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4e-07 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:41.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035684+0000", 2024-09-15T00:11:41.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9857999999999997e-05 2024-09-15T00:11:41.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:41.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035694+0000", 2024-09-15T00:11:41.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0259e-05 2024-09-15T00:11:41.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:41.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035699+0000", 2024-09-15T00:11:41.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6389999999999997e-06 2024-09-15T00:11:41.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:41.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035745+0000", 2024-09-15T00:11:41.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5640999999999999e-05 2024-09-15T00:11:41.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:41.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035764+0000", 2024-09-15T00:11:41.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9368000000000002e-05 2024-09-15T00:11:41.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:41.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.038551+0000", 2024-09-15T00:11:41.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027860879999999999 2024-09-15T00:11:41.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:41.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:41.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:41.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.038588+0000", 2024-09-15T00:11:41.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7101000000000003e-05 2024-09-15T00:11:41.017 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:41.017 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:41.017 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:41.017 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:41.017 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:41.017 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:41.018 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:11:41.019 INFO:teuthology.orchestra.run.smithi171.stdout:1440 read 0xb3a309d thru 0xb3a73a1 (0x4305 bytes) 2024-09-15T00:11:41.019 INFO:teuthology.orchestra.run.smithi171.stdout:1441 punch from 0x1b38037 to 0x1b3a634, (0x25fd bytes) 2024-09-15T00:11:41.019 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:41.019 INFO:teuthology.orchestra.run.smithi171.stdout:1445 write 0x3784dcf thru 0x3793b9e (0xedd0 bytes) 2024-09-15T00:11:41.021 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:41.021 INFO:teuthology.orchestra.run.smithi171.stdout:1446 clone 10 order 18 su 4096 sc 5 2024-09-15T00:11:41.021 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:41.018+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:41.119 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:41.118+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:41.129 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:11:41.145 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7] out_osds: [0, 2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T00:11:41.146 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T00:11:41.176 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:41.176 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:41.176 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:41.176 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:41.183 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone9 from 0xe2cd69b (overlap 0x3acf8bb) to 0xbc31bf 2024-09-15T00:11:41.186 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:41.195 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:41.196 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:41.221 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:41.219+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:41.226 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:41.226 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:41.255 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:41.255 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:41.279 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:41.279 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:41.299 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:41.299 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:41.318 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:41.318 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:41.321 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:41.319+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:41.345 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:41.345 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:41.352 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:41.352 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:41.357 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:41.377 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-15T00:11:41.382 INFO:teuthology.orchestra.run.smithi171.stdout:1447 writesame 0x565fdc8 thru 0x56679bd data_size 0x7b(0x7bf6 bytes) 2024-09-15T00:11:41.387 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:41.387 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:41.394 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:41.395 INFO:teuthology.orchestra.run.smithi171.stdout:1453 punch from 0xfa4958 to 0xfa72d9, (0x2981 bytes) 2024-09-15T00:11:41.395 INFO:teuthology.orchestra.run.smithi171.stdout:1454 trunc from 0xe2cd69b to 0xe889a1e 2024-09-15T00:11:41.401 INFO:teuthology.orchestra.run.smithi171.stdout:1456 read 0xe1e061e thru 0xe1e8502 (0x7ee5 bytes) 2024-09-15T00:11:41.401 INFO:teuthology.orchestra.run.smithi171.stdout:1457 punch from 0x5b1bec3 to 0x5b2a00c, (0xe149 bytes) 2024-09-15T00:11:41.401 INFO:teuthology.orchestra.run.smithi171.stdout:1458 punch from 0xb9acaac to 0xb9b5390, (0x88e4 bytes) 2024-09-15T00:11:41.401 INFO:teuthology.orchestra.run.smithi171.stdout:1459 writesame 0x405e524 thru 0x406a2f7 data_size 0x2(0xbdd4 bytes) 2024-09-15T00:11:41.422 INFO:teuthology.orchestra.run.smithi171.stdout:1462 punch from 0x5b033f to 0x5b45bb, (0x427c bytes) 2024-09-15T00:11:41.423 INFO:teuthology.orchestra.run.smithi171.stdout:1463 read 0x55c4c5f thru 0x55d21d6 (0xd578 bytes) 2024-09-15T00:11:41.423 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:41.421+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:41.425 INFO:teuthology.orchestra.run.smithi171.stdout:1464 write 0x956df1e thru 0x9578117 (0xa1fa bytes) 2024-09-15T00:11:41.433 INFO:teuthology.orchestra.run.smithi171.stdout:1466 write 0x6221399 thru 0x6222429 (0x1091 bytes) 2024-09-15T00:11:41.438 INFO:teuthology.orchestra.run.smithi171.stdout:1467 writesame 0xa293139 thru 0xa294091 data_size 0x1(0xf59 bytes) 2024-09-15T00:11:41.444 INFO:teuthology.orchestra.run.smithi171.stdout:1468 punch from 0x673de29 to 0x674aa4d, (0xcc24 bytes) 2024-09-15T00:11:41.445 INFO:teuthology.orchestra.run.smithi171.stdout:1469 read 0xdf055f4 thru 0xdf151ae (0xfbbb bytes) 2024-09-15T00:11:41.447 INFO:teuthology.orchestra.run.smithi171.stdout:1470 writesame 0x5aa2fa1 thru 0x5aa7b0d data_size 0x1(0x4b6d bytes) 2024-09-15T00:11:41.461 INFO:teuthology.orchestra.run.smithi171.stdout:1471 write 0x6f465dc thru 0x6f537ab (0xd1d0 bytes) 2024-09-15T00:11:41.473 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:41.473 INFO:teuthology.orchestra.run.smithi171.stdout:1473 read 0x99c79e3 thru 0x99d5609 (0xdc27 bytes) 2024-09-15T00:11:41.474 INFO:teuthology.orchestra.run.smithi171.stdout:1474 writesame 0x25d2113 thru 0x25e08b9 data_size 0x1(0xe7a7 bytes) 2024-09-15T00:11:41.508 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:41.508 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:41.519 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:41.525 INFO:teuthology.orchestra.run.smithi171.stdout:1475 writesame 0x2a96b9b thru 0x2aa45f6 data_size 0xac(0xda5c bytes) 2024-09-15T00:11:41.526 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:41.524+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:41.549 INFO:teuthology.orchestra.run.smithi171.stdout:1479 writesame 0x1d14a98 thru 0x1d150ee data_size 0x1(0x657 bytes) 2024-09-15T00:11:41.559 INFO:teuthology.orchestra.run.smithi171.stdout:1481 read 0x7a5bd13 thru 0x7a684ea (0xc7d8 bytes) 2024-09-15T00:11:41.559 INFO:teuthology.orchestra.run.smithi171.stdout:1482 write 0x682f5f7 thru 0x683ac73 (0xb67d bytes) 2024-09-15T00:11:41.574 INFO:teuthology.orchestra.run.smithi171.stdout:1483 write 0xb4bba66 thru 0xb4bf2d0 (0x386b bytes) 2024-09-15T00:11:41.588 INFO:teuthology.orchestra.run.smithi171.stdout:1484 trunc from 0xe889a1e to 0xd281fc6 2024-09-15T00:11:41.600 INFO:teuthology.orchestra.run.smithi171.stdout:1485 write 0x465768e thru 0x4666bc1 (0xf534 bytes) 2024-09-15T00:11:41.613 INFO:teuthology.orchestra.run.smithi171.stdout:1486 write 0xbd4364c thru 0xbd46472 (0x2e27 bytes) 2024-09-15T00:11:41.621 INFO:teuthology.orchestra.run.smithi171.stdout:1487 write 0x173c0f5 thru 0x173f35b (0x3267 bytes) 2024-09-15T00:11:41.624 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:41.622+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:41.637 INFO:teuthology.orchestra.run.smithi171.stdout:1488 read 0x52e0078 thru 0x52e050f (0x498 bytes) 2024-09-15T00:11:41.638 INFO:teuthology.orchestra.run.smithi171.stdout:1489 write 0x4464386 thru 0x446d2fe (0x8f79 bytes) 2024-09-15T00:11:41.646 INFO:teuthology.orchestra.run.smithi171.stdout:1490 writesame 0x31e4eca thru 0x31f0201 data_size 0x9b(0xb338 bytes) 2024-09-15T00:11:41.654 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:41.654 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:41.666 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:41.673 INFO:teuthology.orchestra.run.smithi171.stdout:1491 trunc from 0xd281fc6 to 0x74a34ca 2024-09-15T00:11:41.696 INFO:teuthology.orchestra.run.smithi171.stdout:1492 read 0x2aecbec thru 0x2afca2e (0xfe43 bytes) 2024-09-15T00:11:41.696 INFO:teuthology.orchestra.run.smithi171.stdout:1493 read 0x4c48f63 thru 0x4c4cb4d (0x3beb bytes) 2024-09-15T00:11:41.696 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:41.697 INFO:teuthology.orchestra.run.smithi171.stdout:1495 read 0x345b3ad thru 0x34650e3 (0x9d37 bytes) 2024-09-15T00:11:41.698 INFO:teuthology.orchestra.run.smithi171.stdout:1496 write 0x4c64361 thru 0x4c73c4e (0xf8ee bytes) 2024-09-15T00:11:41.725 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:41.724+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:41.726 INFO:teuthology.orchestra.run.smithi171.stdout:1497 writesame 0xd9c95a2 thru 0xd9d836f data_size 0x2(0xedce bytes) 2024-09-15T00:11:41.727 INFO:teuthology.orchestra.run.smithi171.stdout:1498 read 0x5eef6e5 thru 0x5efa032 (0xa94e bytes) 2024-09-15T00:11:41.729 INFO:teuthology.orchestra.run.smithi171.stdout:1500 write 0x68124ba thru 0x68193a7 (0x6eee bytes) 2024-09-15T00:11:41.731 INFO:teuthology.orchestra.run.smithi171.stdout:1501 write 0x4f32cf4 thru 0x4f3ee1b (0xc128 bytes) 2024-09-15T00:11:41.745 INFO:teuthology.orchestra.run.smithi171.stdout:1502 write 0x9c31ab5 thru 0x9c343a8 (0x28f4 bytes) 2024-09-15T00:11:41.763 INFO:teuthology.orchestra.run.smithi171.stdout:1504 writesame 0xdca3f89 thru 0xdca7328 data_size 0x20(0x33a0 bytes) 2024-09-15T00:11:41.769 INFO:teuthology.orchestra.run.smithi171.stdout:1505 trunc from 0xdca7329 to 0x853bce3 2024-09-15T00:11:41.780 INFO:teuthology.orchestra.run.smithi171.stdout:1506 write 0x7e884c1 thru 0x7e9570a (0xd24a bytes) 2024-09-15T00:11:41.790 INFO:teuthology.orchestra.run.smithi171.stdout:1507 write 0x2f14bad thru 0x2f17ae6 (0x2f3a bytes) 2024-09-15T00:11:41.818 INFO:teuthology.orchestra.run.smithi171.stdout:1510 write 0xabc3605 thru 0xabcd772 (0xa16e bytes) 2024-09-15T00:11:41.826 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:41.824+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:41.826 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:41.827 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:41.827 INFO:teuthology.orchestra.run.smithi171.stdout:1515 punch from 0xd88fca to 0xd93fd9, (0xb00f bytes) 2024-09-15T00:11:41.827 INFO:teuthology.orchestra.run.smithi171.stdout:1516 read 0x3f2c75d thru 0x3f38bfc (0xc4a0 bytes) 2024-09-15T00:11:41.827 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:41.827 INFO:teuthology.orchestra.run.smithi171.stdout:1519 read 0x906c816 thru 0x90755a7 (0x8d92 bytes) 2024-09-15T00:11:41.827 INFO:teuthology.orchestra.run.smithi171.stdout:1521 write 0x1bc70d2 thru 0x1bca24c (0x317b bytes) 2024-09-15T00:11:41.828 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:41.828 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:41.840 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:41.842 INFO:teuthology.orchestra.run.smithi171.stdout:1522 punch from 0x598f4e1 to 0x599d8e3, (0xe402 bytes) 2024-09-15T00:11:41.842 INFO:teuthology.orchestra.run.smithi171.stdout:1523 read 0x4df17f7 thru 0x4df3fe7 (0x27f1 bytes) 2024-09-15T00:11:41.842 INFO:teuthology.orchestra.run.smithi171.stdout:1525 writesame 0x6e3f636 thru 0x6e40d87 data_size 0xf(0x1752 bytes) 2024-09-15T00:11:41.853 INFO:teuthology.orchestra.run.smithi171.stdout:1526 write 0xc1193e2 thru 0xc11bd0f (0x292e bytes) 2024-09-15T00:11:41.858 INFO:teuthology.orchestra.run.smithi171.stdout:1528 write 0x9bd8310 thru 0x9be29ab (0xa69c bytes) 2024-09-15T00:11:41.864 INFO:teuthology.orchestra.run.smithi171.stdout:1530 writesame 0x93277f3 thru 0x9333e25 data_size 0xd(0xc633 bytes) 2024-09-15T00:11:41.872 INFO:teuthology.orchestra.run.smithi171.stdout:1531 write 0x44a2a2a thru 0x44a42e0 (0x18b7 bytes) 2024-09-15T00:11:41.873 INFO:teuthology.orchestra.run.smithi171.stdout:1533 read 0x42f9d59 thru 0x4305749 (0xb9f1 bytes) 2024-09-15T00:11:41.875 INFO:teuthology.orchestra.run.smithi171.stdout:1534 read 0x551bdc2 thru 0x5520599 (0x47d8 bytes) 2024-09-15T00:11:41.875 INFO:teuthology.orchestra.run.smithi171.stdout:1535 punch from 0x8cf6c57 to 0x8d061b0, (0xf559 bytes) 2024-09-15T00:11:41.875 INFO:teuthology.orchestra.run.smithi171.stdout:1536 writesame 0x6b7a5b2 thru 0x6b86e8a data_size 0x57(0xc8d9 bytes) 2024-09-15T00:11:41.900 INFO:teuthology.orchestra.run.smithi171.stdout:1537 trunc from 0xc11bd10 to 0x565b131 2024-09-15T00:11:41.927 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:41.926+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:41.935 INFO:teuthology.orchestra.run.smithi171.stdout:1539 trunc from 0x565b131 to 0xd2033e4 2024-09-15T00:11:41.941 INFO:teuthology.orchestra.run.smithi171.stdout:1541 write 0xff4080 thru 0xffc0a7 (0x8028 bytes) 2024-09-15T00:11:41.975 INFO:teuthology.orchestra.run.smithi171.stdout:1542 trunc from 0xd2033e4 to 0x3cd8559 2024-09-15T00:11:41.999 INFO:teuthology.orchestra.run.smithi171.stdout:1543 read 0x2fec263 thru 0x2ffb5aa (0xf348 bytes) 2024-09-15T00:11:42.005 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:42.005 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:42.017 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:42.018 INFO:teuthology.orchestra.run.smithi171.stdout:1545 writesame 0xc508f92 thru 0xc511a01 data_size 0x28(0x8a70 bytes) 2024-09-15T00:11:42.018 INFO:teuthology.orchestra.run.smithi171.stdout:1546 read 0x40a7b04 thru 0x40acdda (0x52d7 bytes) 2024-09-15T00:11:42.018 INFO:teuthology.orchestra.run.smithi171.stdout:1548 punch from 0x53deb9 to 0x54291d, (0x4a64 bytes) 2024-09-15T00:11:42.018 INFO:teuthology.orchestra.run.smithi171.stdout:1550 write 0x1a9b24b thru 0x1aa6acc (0xb882 bytes) 2024-09-15T00:11:42.036 INFO:teuthology.orchestra.run.smithi171.stdout:1551 clone 11 order 24 su 131072 sc 11 2024-09-15T00:11:42.062 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:42.061+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:42.081 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is set 2024-09-15T00:11:42.082 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T00:11:42.098 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T00:11:42.128 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:42.127+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:42.131 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:42.131 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:42.141 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:42.181 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone10 from 0xc511a02 (overlap 0x3cd8559) to 0x0 2024-09-15T00:11:42.230 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:42.228+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:42.331 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:42.329+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:42.341 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:42.341 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:42.354 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:42.432 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:42.431+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:42.492 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:42.492 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:42.507 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:11:42.533 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:42.531+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:42.635 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:42.633+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:42.736 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:42.737 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:42.738 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:42.734+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:42.750 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:11:42.837 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:42.836+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:42.898 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:42.898 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:42.907 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:11:42.910 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-15T00:11:42.928 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:42.940 INFO:teuthology.orchestra.run.smithi171.stdout:1553 write 0xe3dafa7 thru 0xe3e7de1 (0xce3b bytes) 2024-09-15T00:11:42.946 INFO:teuthology.orchestra.run.smithi171.stdout:1554 write 0xd3d0704 thru 0xd3de100 (0xd9fd bytes) 2024-09-15T00:11:42.948 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:42.946+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:42.952 INFO:teuthology.orchestra.run.smithi171.stdout:1555 read 0xd84d548 thru 0xd84df6d (0xa26 bytes) 2024-09-15T00:11:42.952 INFO:teuthology.orchestra.run.smithi171.stdout:1556 writesame 0x6f6fdb thru 0x6fc020 data_size 0x19(0x5046 bytes) 2024-09-15T00:11:43.025 INFO:teuthology.orchestra.run.smithi171.stdout:1558 writesame 0x6426ed9 thru 0x6434ccf data_size 0x5d(0xddf7 bytes) 2024-09-15T00:11:43.041 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:43.039+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:43.075 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:43.075 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:43.084 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:11:43.088 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T00:11:43.099 INFO:teuthology.orchestra.run.smithi171.stdout:1559 punch from 0x9eb51d9 to 0x9ec0e4c, (0xbc73 bytes) 2024-09-15T00:11:43.099 INFO:teuthology.orchestra.run.smithi171.stdout:1560 write 0x1b75149 thru 0x1b7dd7e (0x8c36 bytes) 2024-09-15T00:11:43.105 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-15T00:11:43.147 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:43.145+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:43.179 INFO:teuthology.orchestra.run.smithi171.stdout:1561 read 0x9bdf657 thru 0x9be541d (0x5dc7 bytes) 2024-09-15T00:11:43.179 INFO:teuthology.orchestra.run.smithi171.stdout:1562 read 0xd989ff9 thru 0xd98fa93 (0x5a9b bytes) 2024-09-15T00:11:43.179 INFO:teuthology.orchestra.run.smithi171.stdout:1564 write 0x6681b32 thru 0x66918d7 (0xfda6 bytes) 2024-09-15T00:11:43.214 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:43.214 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:43.223 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:11:43.245 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:43.243+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:43.253 INFO:teuthology.orchestra.run.smithi171.stdout:1565 writesame 0x640914 thru 0x6423a6 data_size 0x1(0x1a93 bytes) 2024-09-15T00:11:43.257 INFO:teuthology.orchestra.run.smithi171.stdout:1566 write 0xd9a0fce thru 0xd9a58fe (0x4931 bytes) 2024-09-15T00:11:43.272 INFO:teuthology.orchestra.run.smithi171.stdout:1567 trunc from 0xe3e7de2 to 0x50594dd 2024-09-15T00:11:43.347 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:43.344+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:43.373 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:43.373 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:43.387 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:11:43.402 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:11:43.473 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:43.471+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:43.547 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:43.545+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:43.569 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:43.570 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:43.581 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:43.613 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:43.613 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:43.623 INFO:teuthology.orchestra.run.smithi171.stdout:1568 read 0x4213c4 thru 0x424955 (0x3592 bytes) 2024-09-15T00:11:43.629 INFO:teuthology.orchestra.run.smithi171.stdout:1570 trunc from 0x50594dd to 0x86b5f5b 2024-09-15T00:11:43.634 INFO:teuthology.orchestra.run.smithi171.stdout:1571 read 0x178afec thru 0x1796801 (0xb816 bytes) 2024-09-15T00:11:43.634 INFO:teuthology.orchestra.run.smithi171.stdout:1573 write 0x83f144d thru 0x83f7d22 (0x68d6 bytes) 2024-09-15T00:11:43.637 INFO:teuthology.orchestra.run.smithi171.stdout:1574 write 0x7b65fd2 thru 0x7b68ac8 (0x2af7 bytes) 2024-09-15T00:11:43.642 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:43.642 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:43.643 INFO:teuthology.orchestra.run.smithi171.stdout:1575 trunc from 0x86b5f5b to 0xbd36d35 2024-09-15T00:11:43.648 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:43.646+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:43.648 INFO:teuthology.orchestra.run.smithi171.stdout:1576 read 0xa298088 thru 0xa2a2e9d (0xae16 bytes) 2024-09-15T00:11:43.657 INFO:teuthology.orchestra.run.smithi171.stdout:1578 write 0xd5a3874 thru 0xd5adcc9 (0xa456 bytes) 2024-09-15T00:11:43.659 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:43.660 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:43.661 INFO:teuthology.orchestra.run.smithi171.stdout:1580 trunc from 0xd5adcca to 0xc4e33f3 2024-09-15T00:11:43.678 INFO:teuthology.orchestra.run.smithi171.stdout:1584 trunc from 0xc4e33f3 to 0x5957080 2024-09-15T00:11:43.683 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:43.683 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:43.704 INFO:teuthology.orchestra.run.smithi171.stdout:1586 writesame 0xb658ebc thru 0xb6597ee data_size 0x5(0x933 bytes) 2024-09-15T00:11:43.704 INFO:teuthology.orchestra.run.smithi171.stdout:1587 writesame 0x15f3bf3 thru 0x15fe614 data_size 0x6(0xaa22 bytes) 2024-09-15T00:11:43.715 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:43.716 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:43.718 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:43.718 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:43.718 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:43.718 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:43.718 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:43.726 INFO:teuthology.orchestra.run.smithi171.stdout:1590 trunc from 0xb6597ef to 0x4e9731b 2024-09-15T00:11:43.733 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:11:43.735 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:43.735 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:43.738 INFO:teuthology.orchestra.run.smithi171.stdout:1591 trunc from 0x4e9731b to 0x9615dfa 2024-09-15T00:11:43.743 INFO:teuthology.orchestra.run.smithi171.stdout:1592 punch from 0x16236ba to 0x16254d1, (0x1e17 bytes) 2024-09-15T00:11:43.743 INFO:teuthology.orchestra.run.smithi171.stdout:1597 read 0x3a0f573 thru 0x3a13ca1 (0x472f bytes) 2024-09-15T00:11:43.749 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:43.747+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:43.753 INFO:teuthology.orchestra.run.smithi171.stdout:1598 write 0xc798bf7 thru 0xc7a5aae (0xceb8 bytes) 2024-09-15T00:11:43.756 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:43.756 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:43.760 INFO:teuthology.orchestra.run.smithi171.stdout:1599 writesame 0xac48c8a thru 0xac4e228 data_size 0x1(0x559f bytes) 2024-09-15T00:11:43.773 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:43.773 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:43.804 INFO:teuthology.orchestra.run.smithi171.stdout:1600 trunc from 0xc7a5aaf to 0x9fcd46f 2024-09-15T00:11:43.819 INFO:teuthology.orchestra.run.smithi171.stdout:1601 clone 12 order 25 su 131072 sc 14 2024-09-15T00:11:43.845 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:43.845 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:43.845 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:43.849 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:43.848+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:43.854 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:11:43.950 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:43.949+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:43.999 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:43.999 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:43.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:44.000 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:44.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.000 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-15T00:11:44.000 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:11:44.000 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.461416988000003, 2024-09-15T00:11:44.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0035574980000000001, 2024-09-15T00:11:44.000 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.000 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.000 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.000 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:11:44.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:11:44.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535370+0000", 2024-09-15T00:11:44.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-15T00:11:44.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535438+0000", 2024-09-15T00:11:44.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199000000000002e-05 2024-09-15T00:11:44.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535446+0000", 2024-09-15T00:11:44.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1640000000000001e-06 2024-09-15T00:11:44.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535450+0000", 2024-09-15T00:11:44.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1520000000000002e-06 2024-09-15T00:11:44.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535491+0000", 2024-09-15T00:11:44.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0969e-05 2024-09-15T00:11:44.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535510+0000", 2024-09-15T00:11:44.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9031000000000001e-05 2024-09-15T00:11:44.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.538893+0000", 2024-09-15T00:11:44.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033823870000000002 2024-09-15T00:11:44.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.538927+0000", 2024-09-15T00:11:44.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4085999999999999e-05 2024-09-15T00:11:44.004 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.004 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.004 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.004 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:70 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-09-15T00:11:44.004 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:11:44.004 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.460676767999999, 2024-09-15T00:11:44.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057871490000000001, 2024-09-15T00:11:44.004 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.005 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.005 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.005 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:11:44.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:11:44.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536110+0000", 2024-09-15T00:11:44.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-15T00:11:44.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536115+0000", 2024-09-15T00:11:44.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8960000000000002e-06 2024-09-15T00:11:44.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536123+0000", 2024-09-15T00:11:44.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3979999999999999e-06 2024-09-15T00:11:44.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536128+0000", 2024-09-15T00:11:44.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5569999999999997e-06 2024-09-15T00:11:44.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537011+0000", 2024-09-15T00:11:44.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088276299999999995 2024-09-15T00:11:44.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537034+0000", 2024-09-15T00:11:44.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3448999999999999e-05 2024-09-15T00:11:44.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541867+0000", 2024-09-15T00:11:44.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048332949999999996 2024-09-15T00:11:44.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541896+0000", 2024-09-15T00:11:44.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9179999999999998e-05 2024-09-15T00:11:44.009 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.009 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.009 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.009 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-15T00:11:44.009 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:44.009 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.433058314999997, 2024-09-15T00:11:44.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027567730000000001, 2024-09-15T00:11:44.009 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.009 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.009 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.009 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:44.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:44.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:44.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5599999999999995e-07 2024-09-15T00:11:44.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563787+0000", 2024-09-15T00:11:44.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9092000000000003e-05 2024-09-15T00:11:44.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563796+0000", 2024-09-15T00:11:44.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0059999999999998e-06 2024-09-15T00:11:44.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563801+0000", 2024-09-15T00:11:44.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3290000000000004e-06 2024-09-15T00:11:44.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563830+0000", 2024-09-15T00:11:44.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9369999999999998e-05 2024-09-15T00:11:44.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563842+0000", 2024-09-15T00:11:44.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1646e-05 2024-09-15T00:11:44.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.566459+0000", 2024-09-15T00:11:44.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026174549999999999 2024-09-15T00:11:44.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.566485+0000", 2024-09-15T00:11:44.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5318999999999999e-05 2024-09-15T00:11:44.013 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.013 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.013 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.013 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:106 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-09-15T00:11:44.014 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:11:44.014 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.300155852000003, 2024-09-15T00:11:44.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029384250000000002, 2024-09-15T00:11:44.014 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.014 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.014 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.014 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:11:44.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:11:44.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696631+0000", 2024-09-15T00:11:44.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4299999999999998e-07 2024-09-15T00:11:44.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696692+0000", 2024-09-15T00:11:44.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1229999999999995e-05 2024-09-15T00:11:44.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696699+0000", 2024-09-15T00:11:44.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5399999999999998e-06 2024-09-15T00:11:44.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696705+0000", 2024-09-15T00:11:44.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5770000000000001e-06 2024-09-15T00:11:44.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696732+0000", 2024-09-15T00:11:44.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7022e-05 2024-09-15T00:11:44.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696743+0000", 2024-09-15T00:11:44.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1426999999999999e-05 2024-09-15T00:11:44.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699542+0000", 2024-09-15T00:11:44.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002798521 2024-09-15T00:11:44.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699569+0000", 2024-09-15T00:11:44.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6665e-05 2024-09-15T00:11:44.018 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.018 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.018 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.018 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:107 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-09-15T00:11:44.018 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:44.018 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.299260422000003, 2024-09-15T00:11:44.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048949229999999998, 2024-09-15T00:11:44.019 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.019 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.019 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.019 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:44.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:44.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:44.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5799999999999999e-07 2024-09-15T00:11:44.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697531+0000", 2024-09-15T00:11:44.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3100000000000002e-06 2024-09-15T00:11:44.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697539+0000", 2024-09-15T00:11:44.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6410000000000008e-06 2024-09-15T00:11:44.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697543+0000", 2024-09-15T00:11:44.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1060000000000002e-06 2024-09-15T00:11:44.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697573+0000", 2024-09-15T00:11:44.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9978e-05 2024-09-15T00:11:44.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697585+0000", 2024-09-15T00:11:44.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1279e-05 2024-09-15T00:11:44.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702398+0000", 2024-09-15T00:11:44.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004812981 2024-09-15T00:11:44.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702421+0000", 2024-09-15T00:11:44.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3070000000000001e-05 2024-09-15T00:11:44.022 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.022 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.022 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.023 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:108 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'66)", 2024-09-15T00:11:44.023 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:11:44.023 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.298366932, 2024-09-15T00:11:44.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040179439999999999, 2024-09-15T00:11:44.023 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.023 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.023 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.023 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:11:44.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:11:44.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698420+0000", 2024-09-15T00:11:44.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0900000000000002e-07 2024-09-15T00:11:44.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698423+0000", 2024-09-15T00:11:44.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8319999999999999e-06 2024-09-15T00:11:44.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698431+0000", 2024-09-15T00:11:44.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8949999999999998e-06 2024-09-15T00:11:44.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698435+0000", 2024-09-15T00:11:44.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5939999999999998e-06 2024-09-15T00:11:44.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698469+0000", 2024-09-15T00:11:44.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4258000000000003e-05 2024-09-15T00:11:44.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698481+0000", 2024-09-15T00:11:44.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1763000000000001e-05 2024-09-15T00:11:44.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702422+0000", 2024-09-15T00:11:44.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003940603 2024-09-15T00:11:44.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702437+0000", 2024-09-15T00:11:44.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5489999999999999e-05 2024-09-15T00:11:44.027 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.028 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.028 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.028 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:109 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'66)", 2024-09-15T00:11:44.028 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:44.028 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.297836496000002, 2024-09-15T00:11:44.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038555479999999999, 2024-09-15T00:11:44.028 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.028 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.028 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.028 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:44.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:44.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:44.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.13e-07 2024-09-15T00:11:44.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699018+0000", 2024-09-15T00:11:44.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7540999999999996e-05 2024-09-15T00:11:44.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699024+0000", 2024-09-15T00:11:44.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4869999999999999e-06 2024-09-15T00:11:44.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699044+0000", 2024-09-15T00:11:44.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9400999999999999e-05 2024-09-15T00:11:44.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699104+0000", 2024-09-15T00:11:44.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0331999999999998e-05 2024-09-15T00:11:44.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699117+0000", 2024-09-15T00:11:44.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2789000000000001e-05 2024-09-15T00:11:44.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702777+0000", 2024-09-15T00:11:44.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0036604900000000002 2024-09-15T00:11:44.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702805+0000", 2024-09-15T00:11:44.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7994999999999999e-05 2024-09-15T00:11:44.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.032 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.033 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:110 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'72, mlcod=16'66)", 2024-09-15T00:11:44.033 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:11:44.033 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.297076963000002, 2024-09-15T00:11:44.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031108189999999999, 2024-09-15T00:11:44.033 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.033 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.033 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.033 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:11:44.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:11:44.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699710+0000", 2024-09-15T00:11:44.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:11:44.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699714+0000", 2024-09-15T00:11:44.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0010000000000003e-06 2024-09-15T00:11:44.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699722+0000", 2024-09-15T00:11:44.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4840000000000004e-06 2024-09-15T00:11:44.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699726+0000", 2024-09-15T00:11:44.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:11:44.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699755+0000", 2024-09-15T00:11:44.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8433999999999999e-05 2024-09-15T00:11:44.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699775+0000", 2024-09-15T00:11:44.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0017000000000002e-05 2024-09-15T00:11:44.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702807+0000", 2024-09-15T00:11:44.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030320239999999999 2024-09-15T00:11:44.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702820+0000", 2024-09-15T00:11:44.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3169999999999999e-05 2024-09-15T00:11:44.037 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.037 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.037 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.037 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:120 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'82, mlcod=16'78)", 2024-09-15T00:11:44.037 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:11:44.037 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.291173616999998, 2024-09-15T00:11:44.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028965829999999999, 2024-09-15T00:11:44.038 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.038 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.038 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.038 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:11:44.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:11:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705613+0000", 2024-09-15T00:11:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1800000000000001e-07 2024-09-15T00:11:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705684+0000", 2024-09-15T00:11:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0903999999999997e-05 2024-09-15T00:11:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705691+0000", 2024-09-15T00:11:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4290000000000004e-06 2024-09-15T00:11:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705695+0000", 2024-09-15T00:11:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4790000000000001e-06 2024-09-15T00:11:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705729+0000", 2024-09-15T00:11:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4415999999999997e-05 2024-09-15T00:11:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705741+0000", 2024-09-15T00:11:44.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1443999999999999e-05 2024-09-15T00:11:44.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.708476+0000", 2024-09-15T00:11:44.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027356630000000002 2024-09-15T00:11:44.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.708509+0000", 2024-09-15T00:11:44.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2830000000000002e-05 2024-09-15T00:11:44.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.041 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.042 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-15T00:11:44.042 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:44.042 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.290387152000001, 2024-09-15T00:11:44.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047824809999999999, 2024-09-15T00:11:44.042 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.042 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.042 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.042 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:44.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:44.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:44.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:11:44.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706471+0000", 2024-09-15T00:11:44.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1496999999999997e-05 2024-09-15T00:11:44.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706478+0000", 2024-09-15T00:11:44.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7460000000000003e-06 2024-09-15T00:11:44.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706481+0000", 2024-09-15T00:11:44.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.258e-06 2024-09-15T00:11:44.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706513+0000", 2024-09-15T00:11:44.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2567000000000002e-05 2024-09-15T00:11:44.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706524+0000", 2024-09-15T00:11:44.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1028e-05 2024-09-15T00:11:44.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.711150+0000", 2024-09-15T00:11:44.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046259450000000002 2024-09-15T00:11:44.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.711181+0000", 2024-09-15T00:11:44.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1041000000000003e-05 2024-09-15T00:11:44.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.046 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.046 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.046 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:136 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-15T00:11:44.046 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:44.046 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.936950218, 2024-09-15T00:11:44.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031270970000000001, 2024-09-15T00:11:44.046 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.046 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.046 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.046 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:44.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:44.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:44.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3000000000000002e-07 2024-09-15T00:11:44.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059839+0000", 2024-09-15T00:11:44.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3050000000000001e-06 2024-09-15T00:11:44.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059847+0000", 2024-09-15T00:11:44.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.622e-06 2024-09-15T00:11:44.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059851+0000", 2024-09-15T00:11:44.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8120000000000001e-06 2024-09-15T00:11:44.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059884+0000", 2024-09-15T00:11:44.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2620999999999998e-05 2024-09-15T00:11:44.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059894+0000", 2024-09-15T00:11:44.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0924000000000001e-05 2024-09-15T00:11:44.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062935+0000", 2024-09-15T00:11:44.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030406830000000002 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062963+0000", 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7800000000000001e-05 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:139 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'92)", 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.935162994999999, 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040690240000000001, 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.050 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.051 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.051 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:44.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:44.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:44.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:11:44.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061631+0000", 2024-09-15T00:11:44.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.17e-06 2024-09-15T00:11:44.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061636+0000", 2024-09-15T00:11:44.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7679999999999997e-06 2024-09-15T00:11:44.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061639+0000", 2024-09-15T00:11:44.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9069999999999999e-06 2024-09-15T00:11:44.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062733+0000", 2024-09-15T00:11:44.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010937119999999999 2024-09-15T00:11:44.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062744+0000", 2024-09-15T00:11:44.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1439e-05 2024-09-15T00:11:44.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065658+0000", 2024-09-15T00:11:44.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029134970000000001 2024-09-15T00:11:44.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065692+0000", 2024-09-15T00:11:44.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4147999999999999e-05 2024-09-15T00:11:44.054 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.054 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.054 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'92)", 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.934773933000002, 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003986608, 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:44.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:44.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7000000000000001e-07 2024-09-15T00:11:44.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062075+0000", 2024-09-15T00:11:44.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2638000000000006e-05 2024-09-15T00:11:44.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062083+0000", 2024-09-15T00:11:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4330000000000004e-06 2024-09-15T00:11:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062087+0000", 2024-09-15T00:11:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3799999999999998e-06 2024-09-15T00:11:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.063001+0000", 2024-09-15T00:11:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091419500000000002 2024-09-15T00:11:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.063012+0000", 2024-09-15T00:11:44.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1117999999999999e-05 2024-09-15T00:11:44.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065977+0000", 2024-09-15T00:11:44.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029650840000000002 2024-09-15T00:11:44.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065999+0000", 2024-09-15T00:11:44.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1489999999999999e-05 2024-09-15T00:11:44.058 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.058 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.058 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:154 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'108, mlcod=16'105)", 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.472526307999999, 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053187770000000002, 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:44.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:44.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:11:44.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524264+0000", 2024-09-15T00:11:44.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.738e-06 2024-09-15T00:11:44.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524270+0000", 2024-09-15T00:11:44.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9379999999999997e-06 2024-09-15T00:11:44.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524273+0000", 2024-09-15T00:11:44.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6370000000000001e-06 2024-09-15T00:11:44.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524310+0000", 2024-09-15T00:11:44.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6266999999999997e-05 2024-09-15T00:11:44.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524322+0000", 2024-09-15T00:11:44.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2741999999999999e-05 2024-09-15T00:11:44.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.529547+0000", 2024-09-15T00:11:44.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005225001 2024-09-15T00:11:44.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.529579+0000", 2024-09-15T00:11:44.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1065999999999997e-05 2024-09-15T00:11:44.065 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.065 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.065 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.065 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-15T00:11:44.066 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:11:44.066 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.472088669999998, 2024-09-15T00:11:44.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077021529999999998, 2024-09-15T00:11:44.066 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.066 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.066 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.066 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:11:44.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524698+0000", 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6300000000000001e-07 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524778+0000", 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0598000000000003e-05 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524785+0000", 2024-09-15T00:11:44.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6359999999999997e-06 2024-09-15T00:11:44.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524788+0000", 2024-09-15T00:11:44.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2540000000000001e-06 2024-09-15T00:11:44.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524817+0000", 2024-09-15T00:11:44.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8784000000000001e-05 2024-09-15T00:11:44.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524828+0000", 2024-09-15T00:11:44.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-15T00:11:44.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532360+0000", 2024-09-15T00:11:44.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007531757 2024-09-15T00:11:44.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532400+0000", 2024-09-15T00:11:44.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9835000000000001e-05 2024-09-15T00:11:44.070 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.070 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.070 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.070 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=16'106)", 2024-09-15T00:11:44.070 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:44.070 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.471146517000001, 2024-09-15T00:11:44.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067923050000000002, 2024-09-15T00:11:44.070 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.070 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:44.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-15T00:11:44.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525644+0000", 2024-09-15T00:11:44.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.709e-06 2024-09-15T00:11:44.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525651+0000", 2024-09-15T00:11:44.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7970000000000003e-06 2024-09-15T00:11:44.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525654+0000", 2024-09-15T00:11:44.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.433e-06 2024-09-15T00:11:44.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525685+0000", 2024-09-15T00:11:44.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0970000000000003e-05 2024-09-15T00:11:44.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525697+0000", 2024-09-15T00:11:44.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1759e-05 2024-09-15T00:11:44.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532401+0000", 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006704425 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532432+0000", 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0709e-05 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'106)", 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.470684263000003, 2024-09-15T00:11:44.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063539190000000004, 2024-09-15T00:11:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:44.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:11:44.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526107+0000", 2024-09-15T00:11:44.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:11:44.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526117+0000", 2024-09-15T00:11:44.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7750000000000004e-06 2024-09-15T00:11:44.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526121+0000", 2024-09-15T00:11:44.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:11:44.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526163+0000", 2024-09-15T00:11:44.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1360999999999999e-05 2024-09-15T00:11:44.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526175+0000", 2024-09-15T00:11:44.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2729e-05 2024-09-15T00:11:44.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532433+0000", 2024-09-15T00:11:44.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062576999999999997 2024-09-15T00:11:44.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532456+0000", 2024-09-15T00:11:44.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2773e-05 2024-09-15T00:11:44.078 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.078 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.078 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.079 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'106)", 2024-09-15T00:11:44.079 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:11:44.079 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.470214861000002, 2024-09-15T00:11:44.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059082689999999998, 2024-09-15T00:11:44.079 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.079 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.079 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.079 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:11:44.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:11:44.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526572+0000", 2024-09-15T00:11:44.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9500000000000003e-07 2024-09-15T00:11:44.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526641+0000", 2024-09-15T00:11:44.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8858999999999999e-05 2024-09-15T00:11:44.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526648+0000", 2024-09-15T00:11:44.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:11:44.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526654+0000", 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7509999999999997e-06 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526689+0000", 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5800000000000003e-05 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526700+0000", 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0467e-05 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532457+0000", 2024-09-15T00:11:44.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057568180000000004 2024-09-15T00:11:44.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532479+0000", 2024-09-15T00:11:44.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2739e-05 2024-09-15T00:11:44.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.083 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.083 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'106)", 2024-09-15T00:11:44.083 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:11:44.083 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.469558595999999, 2024-09-15T00:11:44.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005274801, 2024-09-15T00:11:44.084 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.084 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.084 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.084 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:11:44.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:11:44.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527228+0000", 2024-09-15T00:11:44.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1600000000000001e-07 2024-09-15T00:11:44.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527232+0000", 2024-09-15T00:11:44.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8920000000000002e-06 2024-09-15T00:11:44.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527239+0000", 2024-09-15T00:11:44.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2880000000000004e-06 2024-09-15T00:11:44.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527243+0000", 2024-09-15T00:11:44.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.143e-06 2024-09-15T00:11:44.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527275+0000", 2024-09-15T00:11:44.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1866000000000003e-05 2024-09-15T00:11:44.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527288+0000", 2024-09-15T00:11:44.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469999999999999e-05 2024-09-15T00:11:44.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532480+0000", 2024-09-15T00:11:44.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051927129999999998 2024-09-15T00:11:44.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532502+0000", 2024-09-15T00:11:44.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1912999999999998e-05 2024-09-15T00:11:44.087 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.087 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.087 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.088 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:25 3.10 e26/23 3:0d9e4e9e:::rbd_header.11395e6383ec:head v 26'2, mlcod=0'0)", 2024-09-15T00:11:44.088 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:21.619783+0000", 2024-09-15T00:11:44.088 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.37700293, 2024-09-15T00:11:44.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031992600000000002, 2024-09-15T00:11:44.088 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.088 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.088 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.088 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619783+0000", 2024-09-15T00:11:44.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619783+0000", 2024-09-15T00:11:44.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619784+0000", 2024-09-15T00:11:44.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9900000000000009e-07 2024-09-15T00:11:44.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619795+0000", 2024-09-15T00:11:44.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0689e-05 2024-09-15T00:11:44.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619809+0000", 2024-09-15T00:11:44.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3757000000000001e-05 2024-09-15T00:11:44.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619818+0000", 2024-09-15T00:11:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4909999999999994e-06 2024-09-15T00:11:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619868+0000", 2024-09-15T00:11:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0046999999999997e-05 2024-09-15T00:11:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619887+0000", 2024-09-15T00:11:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780999999999999e-05 2024-09-15T00:11:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.622947+0000", 2024-09-15T00:11:44.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030601069999999998 2024-09-15T00:11:44.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.622982+0000", 2024-09-15T00:11:44.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5389000000000003e-05 2024-09-15T00:11:44.092 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.092 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.092 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.092 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.092 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.092 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:44.094 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:44.094 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:44.092+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:44.099 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:11:44.099 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.4 to 1 (10000) 2024-09-15T00:11:44.115 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-15T00:11:44.199 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:44.198+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:44.213 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone11 from 0x9fcd46f (overlap 0x4e9731b) to 0x37036f9 2024-09-15T00:11:44.216 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:44.217 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:44.217 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:44.217 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:44.232 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:44.316 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:44.315+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:44.366 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-15T00:11:44.367 INFO:teuthology.orchestra.run.smithi171.stdout:1602 read 0x3b37f28 thru 0x3b4328f (0xb368 bytes) 2024-09-15T00:11:44.369 INFO:teuthology.orchestra.run.smithi171.stdout:1603 trunc from 0x9fcd46f to 0xa9066a4 2024-09-15T00:11:44.377 INFO:teuthology.orchestra.run.smithi171.stdout:1604 write 0x1754d37 thru 0x175fa5e (0xad28 bytes) 2024-09-15T00:11:44.393 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:44.393 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:44.393 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:44.396 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:44.395+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:44.402 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:44.422 INFO:teuthology.orchestra.run.smithi171.stdout:1605 write 0x1f8dac4 thru 0x1f91da7 (0x42e4 bytes) 2024-09-15T00:11:44.460 INFO:teuthology.orchestra.run.smithi171.stdout:1606 writesame 0x278a67 thru 0x27fb24 data_size 0x1(0x70be bytes) 2024-09-15T00:11:44.497 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:44.496+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:44.524 INFO:teuthology.orchestra.run.smithi171.stdout:1608 write 0xb99a051 thru 0xb9a14e7 (0x7497 bytes) 2024-09-15T00:11:44.536 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:44.536 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:44.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:44.536 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:44.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.537 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1183 3.4 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:44.537 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.947741+0000", 2024-09-15T00:11:44.537 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.585375583000001, 2024-09-15T00:11:44.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024684798000000001, 2024-09-15T00:11:44.537 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.537 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.537 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.538 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1183 2024-09-15T00:11:44.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.538 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947741+0000", 2024-09-15T00:11:44.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947741+0000", 2024-09-15T00:11:44.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947743+0000", 2024-09-15T00:11:44.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.446e-06 2024-09-15T00:11:44.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947750+0000", 2024-09-15T00:11:44.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7640000000000005e-06 2024-09-15T00:11:44.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947774+0000", 2024-09-15T00:11:44.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3958e-05 2024-09-15T00:11:44.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947783+0000", 2024-09-15T00:11:44.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8410000000000006e-06 2024-09-15T00:11:44.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947840+0000", 2024-09-15T00:11:44.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6784999999999999e-05 2024-09-15T00:11:44.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947947+0000", 2024-09-15T00:11:44.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010688000000000001 2024-09-15T00:11:44.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:44.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.948963+0000", 2024-09-15T00:11:44.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001016482 2024-09-15T00:11:44.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:44.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972328+0000", 2024-09-15T00:11:44.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023364843999999999 2024-09-15T00:11:44.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:44.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972341+0000", 2024-09-15T00:11:44.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2777e-05 2024-09-15T00:11:44.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972378+0000", 2024-09-15T00:11:44.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6749999999999999e-05 2024-09-15T00:11:44.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972426+0000", 2024-09-15T00:11:44.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8270999999999999e-05 2024-09-15T00:11:44.542 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.543 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.543 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.543 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:1183 3.4 e29/23 ondisk, result = 0)", 2024-09-15T00:11:44.543 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.951923+0000", 2024-09-15T00:11:44.543 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.5811934, 2024-09-15T00:11:44.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020411286000000001, 2024-09-15T00:11:44.543 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.543 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.543 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:44.543 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951923+0000", 2024-09-15T00:11:44.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951923+0000", 2024-09-15T00:11:44.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951924+0000", 2024-09-15T00:11:44.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8700000000000004e-07 2024-09-15T00:11:44.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951928+0000", 2024-09-15T00:11:44.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1110000000000001e-06 2024-09-15T00:11:44.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951939+0000", 2024-09-15T00:11:44.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0752e-05 2024-09-15T00:11:44.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951945+0000", 2024-09-15T00:11:44.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1009999999999997e-06 2024-09-15T00:11:44.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972316+0000", 2024-09-15T00:11:44.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020370335999999999 2024-09-15T00:11:44.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972323+0000", 2024-09-15T00:11:44.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9189999999999999e-06 2024-09-15T00:11:44.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972335+0000", 2024-09-15T00:11:44.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.118e-05 2024-09-15T00:11:44.546 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.546 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.547 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.547 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:44.547 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:11:44.547 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.353292049, 2024-09-15T00:11:44.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:11:44.547 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.547 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.547 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.547 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.547 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.547 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.548 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:11:44.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.548 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:11:44.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:11:44.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:11:44.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:11:44.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:11:44.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:11:44.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:11:44.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:11:44.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:11:44.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:11:44.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:11:44.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:11:44.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:11:44.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:11:44.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:44.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:11:44.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:11:44.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:44.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:11:44.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:11:44.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:44.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:11:44.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:11:44.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:11:44.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:11:44.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:11:44.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:11:44.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.553 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.554 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:44.554 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:11:44.554 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.349864140000001, 2024-09-15T00:11:44.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032661645000000003, 2024-09-15T00:11:44.554 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.554 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.554 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for ondisk", 2024-09-15T00:11:44.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.554 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:11:44.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.554 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:11:44.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:11:44.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:11:44.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5300000000000004e-07 2024-09-15T00:11:44.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183257+0000", 2024-09-15T00:11:44.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9380000000000002e-06 2024-09-15T00:11:44.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183265+0000", 2024-09-15T00:11:44.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6599999999999995e-06 2024-09-15T00:11:44.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183270+0000", 2024-09-15T00:11:44.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6489999999999996e-06 2024-09-15T00:11:44.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215767+0000", 2024-09-15T00:11:44.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032496490000000003 2024-09-15T00:11:44.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for ondisk", 2024-09-15T00:11:44.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215819+0000", 2024-09-15T00:11:44.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1647999999999997e-05 2024-09-15T00:11:44.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215914+0000", 2024-09-15T00:11:44.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5706999999999997e-05 2024-09-15T00:11:44.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.558 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.559 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:1287 3.6 e30/28 ondisk, result = 0)", 2024-09-15T00:11:44.559 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.186816+0000", 2024-09-15T00:11:44.559 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.346300553999999, 2024-09-15T00:11:44.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.029151897999999999, 2024-09-15T00:11:44.559 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.559 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.560 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:44.560 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186816+0000", 2024-09-15T00:11:44.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186816+0000", 2024-09-15T00:11:44.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186817+0000", 2024-09-15T00:11:44.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0729999999999999e-06 2024-09-15T00:11:44.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186822+0000", 2024-09-15T00:11:44.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.006e-06 2024-09-15T00:11:44.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186837+0000", 2024-09-15T00:11:44.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4819e-05 2024-09-15T00:11:44.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186846+0000", 2024-09-15T00:11:44.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1519999999999998e-06 2024-09-15T00:11:44.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215852+0000", 2024-09-15T00:11:44.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.029005882 2024-09-15T00:11:44.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215859+0000", 2024-09-15T00:11:44.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7499999999999997e-06 2024-09-15T00:11:44.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215968+0000", 2024-09-15T00:11:44.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010921600000000001 2024-09-15T00:11:44.564 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.565 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.565 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.565 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1329 3.19 3:9ac4ae77:::rbd_data.11397bff333d.0000000000000032:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:44.565 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.307972+0000", 2024-09-15T00:11:44.565 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.225144361000002, 2024-09-15T00:11:44.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0096326950000000001, 2024-09-15T00:11:44.565 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.565 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.565 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.566 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.566 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.566 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.566 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1329 2024-09-15T00:11:44.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.566 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.307972+0000", 2024-09-15T00:11:44.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.307972+0000", 2024-09-15T00:11:44.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.307973+0000", 2024-09-15T00:11:44.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3300000000000002e-07 2024-09-15T00:11:44.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.307975+0000", 2024-09-15T00:11:44.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1569999999999998e-06 2024-09-15T00:11:44.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.307983+0000", 2024-09-15T00:11:44.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5730000000000002e-06 2024-09-15T00:11:44.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.307987+0000", 2024-09-15T00:11:44.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9110000000000003e-06 2024-09-15T00:11:44.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308446+0000", 2024-09-15T00:11:44.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045920200000000003 2024-09-15T00:11:44.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308536+0000", 2024-09-15T00:11:44.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0469000000000004e-05 2024-09-15T00:11:44.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:44.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308635+0000", 2024-09-15T00:11:44.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8732000000000003e-05 2024-09-15T00:11:44.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:44.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311792+0000", 2024-09-15T00:11:44.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003156603 2024-09-15T00:11:44.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:44.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.317550+0000", 2024-09-15T00:11:44.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057583169999999998 2024-09-15T00:11:44.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.317580+0000", 2024-09-15T00:11:44.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9821000000000001e-05 2024-09-15T00:11:44.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.317605+0000", 2024-09-15T00:11:44.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5377000000000001e-05 2024-09-15T00:11:44.571 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.571 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.572 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.572 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1330 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:44.572 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.309238+0000", 2024-09-15T00:11:44.572 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.223878450000001, 2024-09-15T00:11:44.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0082918479999999992, 2024-09-15T00:11:44.572 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.572 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.572 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.572 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.572 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.573 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.573 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1330 2024-09-15T00:11:44.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.573 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309238+0000", 2024-09-15T00:11:44.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309238+0000", 2024-09-15T00:11:44.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309239+0000", 2024-09-15T00:11:44.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9700000000000005e-07 2024-09-15T00:11:44.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309243+0000", 2024-09-15T00:11:44.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.912e-06 2024-09-15T00:11:44.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309253+0000", 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0059e-05 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309260+0000", 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2990000000000002e-06 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309312+0000", 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2120000000000002e-05 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309388+0000", 2024-09-15T00:11:44.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6794999999999999e-05 2024-09-15T00:11:44.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:44.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.309449+0000", 2024-09-15T00:11:44.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0572000000000002e-05 2024-09-15T00:11:44.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:44.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.312204+0000", 2024-09-15T00:11:44.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027552599999999998 2024-09-15T00:11:44.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:44.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.317441+0000", 2024-09-15T00:11:44.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005236653 2024-09-15T00:11:44.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.317491+0000", 2024-09-15T00:11:44.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9944000000000002e-05 2024-09-15T00:11:44.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.317530+0000", 2024-09-15T00:11:44.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9337e-05 2024-09-15T00:11:44.577 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.577 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.578 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.578 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18245 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:44.578 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285164+0000", 2024-09-15T00:11:44.578 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.2479523629999996, 2024-09-15T00:11:44.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0076021179999999997, 2024-09-15T00:11:44.578 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.578 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.578 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:44.579 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.579 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.579 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.579 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18245 2024-09-15T00:11:44.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.579 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285164+0000", 2024-09-15T00:11:44.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285164+0000", 2024-09-15T00:11:44.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285165+0000", 2024-09-15T00:11:44.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:11:44.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285166+0000", 2024-09-15T00:11:44.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.17e-06 2024-09-15T00:11:44.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285170+0000", 2024-09-15T00:11:44.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:11:44.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285173+0000", 2024-09-15T00:11:44.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5040000000000001e-06 2024-09-15T00:11:44.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.292129+0000", 2024-09-15T00:11:44.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0069561900000000001 2024-09-15T00:11:44.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.292220+0000", 2024-09-15T00:11:44.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1267999999999994e-05 2024-09-15T00:11:44.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.292767+0000", 2024-09-15T00:11:44.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00054657900000000005 2024-09-15T00:11:44.582 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.582 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.582 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.582 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18246 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:44.582 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285175+0000", 2024-09-15T00:11:44.582 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.2479413089999998, 2024-09-15T00:11:44.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077482820000000004, 2024-09-15T00:11:44.583 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.583 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.583 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:44.583 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.583 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.583 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.583 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18246 2024-09-15T00:11:44.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.583 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285175+0000", 2024-09-15T00:11:44.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285175+0000", 2024-09-15T00:11:44.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285176+0000", 2024-09-15T00:11:44.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:11:44.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285177+0000", 2024-09-15T00:11:44.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.111e-06 2024-09-15T00:11:44.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285181+0000", 2024-09-15T00:11:44.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0489999999999997e-06 2024-09-15T00:11:44.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285184+0000", 2024-09-15T00:11:44.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.954e-06 2024-09-15T00:11:44.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.292790+0000", 2024-09-15T00:11:44.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0076064410000000002 2024-09-15T00:11:44.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.292862+0000", 2024-09-15T00:11:44.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1994000000000002e-05 2024-09-15T00:11:44.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.292924+0000", 2024-09-15T00:11:44.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1531000000000004e-05 2024-09-15T00:11:44.586 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.586 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.586 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.587 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18247 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:44.587 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285187+0000", 2024-09-15T00:11:44.587 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.2479299770000001, 2024-09-15T00:11:44.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079134459999999993, 2024-09-15T00:11:44.587 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.587 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.587 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:44.587 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.587 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.587 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.587 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18247 2024-09-15T00:11:44.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.587 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285187+0000", 2024-09-15T00:11:44.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285187+0000", 2024-09-15T00:11:44.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285187+0000", 2024-09-15T00:11:44.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-15T00:11:44.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285188+0000", 2024-09-15T00:11:44.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-15T00:11:44.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285193+0000", 2024-09-15T00:11:44.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.814e-06 2024-09-15T00:11:44.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285197+0000", 2024-09-15T00:11:44.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1019999999999999e-06 2024-09-15T00:11:44.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.292960+0000", 2024-09-15T00:11:44.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077625539999999996 2024-09-15T00:11:44.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293018+0000", 2024-09-15T00:11:44.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8288000000000002e-05 2024-09-15T00:11:44.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293100+0000", 2024-09-15T00:11:44.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2330000000000006e-05 2024-09-15T00:11:44.591 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.591 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.591 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.591 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18248 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:44.591 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285203+0000", 2024-09-15T00:11:44.591 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.2479142539999994, 2024-09-15T00:11:44.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0081211630000000007, 2024-09-15T00:11:44.591 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.591 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.592 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:44.592 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.592 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.592 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.592 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18248 2024-09-15T00:11:44.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.592 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285203+0000", 2024-09-15T00:11:44.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285203+0000", 2024-09-15T00:11:44.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285203+0000", 2024-09-15T00:11:44.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.34e-07 2024-09-15T00:11:44.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285205+0000", 2024-09-15T00:11:44.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.903e-06 2024-09-15T00:11:44.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285212+0000", 2024-09-15T00:11:44.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9539999999999998e-06 2024-09-15T00:11:44.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285216+0000", 2024-09-15T00:11:44.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6349999999999997e-06 2024-09-15T00:11:44.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293148+0000", 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007931713 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293233+0000", 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4943000000000003e-05 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293324+0000", 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0680999999999995e-05 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.596 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18249 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:44.596 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285223+0000", 2024-09-15T00:11:44.596 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.2478935700000005, 2024-09-15T00:11:44.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0082666660000000006, 2024-09-15T00:11:44.596 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.596 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.596 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:44.596 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.596 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.596 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.596 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18249 2024-09-15T00:11:44.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.596 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285223+0000", 2024-09-15T00:11:44.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285223+0000", 2024-09-15T00:11:44.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285224+0000", 2024-09-15T00:11:44.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:11:44.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285226+0000", 2024-09-15T00:11:44.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4310000000000001e-06 2024-09-15T00:11:44.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285235+0000", 2024-09-15T00:11:44.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9020000000000005e-06 2024-09-15T00:11:44.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285240+0000", 2024-09-15T00:11:44.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4410000000000001e-06 2024-09-15T00:11:44.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293372+0000", 2024-09-15T00:11:44.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0081318740000000007 2024-09-15T00:11:44.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293435+0000", 2024-09-15T00:11:44.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3215999999999999e-05 2024-09-15T00:11:44.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293490+0000", 2024-09-15T00:11:44.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4401999999999999e-05 2024-09-15T00:11:44.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.600 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.601 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18250 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:44.601 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285248+0000", 2024-09-15T00:11:44.601 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.2478685340000002, 2024-09-15T00:11:44.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0083998030000000008, 2024-09-15T00:11:44.601 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.601 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.601 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:44.601 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.601 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.601 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.601 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18250 2024-09-15T00:11:44.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.601 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285248+0000", 2024-09-15T00:11:44.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285248+0000", 2024-09-15T00:11:44.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285249+0000", 2024-09-15T00:11:44.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:11:44.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285251+0000", 2024-09-15T00:11:44.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5730000000000002e-06 2024-09-15T00:11:44.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285261+0000", 2024-09-15T00:11:44.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8970000000000003e-06 2024-09-15T00:11:44.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285266+0000", 2024-09-15T00:11:44.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3530000000000002e-06 2024-09-15T00:11:44.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293525+0000", 2024-09-15T00:11:44.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0082582539999999996 2024-09-15T00:11:44.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293588+0000", 2024-09-15T00:11:44.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3633000000000006e-05 2024-09-15T00:11:44.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293648+0000", 2024-09-15T00:11:44.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9667000000000003e-05 2024-09-15T00:11:44.605 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.605 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.605 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.605 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18251 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:44.605 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285273+0000", 2024-09-15T00:11:44.605 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.2478438080000007, 2024-09-15T00:11:44.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0085317829999999994, 2024-09-15T00:11:44.605 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.605 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.605 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:44.606 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.606 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.606 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.606 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18251 2024-09-15T00:11:44.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.606 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285273+0000", 2024-09-15T00:11:44.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285273+0000", 2024-09-15T00:11:44.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285273+0000", 2024-09-15T00:11:44.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4999999999999998e-07 2024-09-15T00:11:44.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285276+0000", 2024-09-15T00:11:44.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3590000000000002e-06 2024-09-15T00:11:44.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285282+0000", 2024-09-15T00:11:44.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.083e-06 2024-09-15T00:11:44.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285286+0000", 2024-09-15T00:11:44.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9570000000000004e-06 2024-09-15T00:11:44.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293687+0000", 2024-09-15T00:11:44.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0084014599999999995 2024-09-15T00:11:44.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293733+0000", 2024-09-15T00:11:44.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5572999999999999e-05 2024-09-15T00:11:44.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293805+0000", 2024-09-15T00:11:44.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1901000000000002e-05 2024-09-15T00:11:44.609 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.609 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.609 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.609 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18252 3.4 3:24dfc0e7:::rbd_data.11393feb9e21.0000000000000006:a [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:44.609 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.285290+0000", 2024-09-15T00:11:44.609 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.2478269050000002, 2024-09-15T00:11:44.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0086438569999999996, 2024-09-15T00:11:44.610 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.610 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.610 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:44.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.610 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18252 2024-09-15T00:11:44.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.610 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285290+0000", 2024-09-15T00:11:44.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285290+0000", 2024-09-15T00:11:44.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285290+0000", 2024-09-15T00:11:44.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2100000000000001e-07 2024-09-15T00:11:44.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285291+0000", 2024-09-15T00:11:44.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2640000000000001e-06 2024-09-15T00:11:44.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285296+0000", 2024-09-15T00:11:44.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4440000000000001e-06 2024-09-15T00:11:44.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.285298+0000", 2024-09-15T00:11:44.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5160000000000001e-06 2024-09-15T00:11:44.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293834+0000", 2024-09-15T00:11:44.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0085355929999999993 2024-09-15T00:11:44.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293879+0000", 2024-09-15T00:11:44.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5464999999999999e-05 2024-09-15T00:11:44.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.293934+0000", 2024-09-15T00:11:44.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4354000000000002e-05 2024-09-15T00:11:44.614 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.614 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.614 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.614 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:44.614 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:11:44.614 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.6234508699999992, 2024-09-15T00:11:44.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:11:44.614 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.614 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.614 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.614 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.615 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:11:44.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.615 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:11:44.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:11:44.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:11:44.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:11:44.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:11:44.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:11:44.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:11:44.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:11:44.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:11:44.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:11:44.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:11:44.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:11:44.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:11:44.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:11:44.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:44.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:11:44.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:11:44.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:44.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:11:44.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:11:44.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:44.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:11:44.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:11:44.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:11:44.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:11:44.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:11:44.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:11:44.619 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.620 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.620 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.620 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:11:44.620 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:11:44.620 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.6164925070000002, 2024-09-15T00:11:44.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:11:44.620 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.620 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.620 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:44.620 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:11:44.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:11:44.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:11:44.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:11:44.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:11:44.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:11:44.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:11:44.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:11:44.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:11:44.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:11:44.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:11:44.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:11:44.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:11:44.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:11:44.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:11:44.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:11:44.624 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.624 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.624 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.624 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:20389 3.4 3:2749bfcc:::rbd_data.113917258088.0000000000000015:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T00:11:44.624 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:41.185081+0000", 2024-09-15T00:11:44.624 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.3480357019999998, 2024-09-15T00:11:44.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0078787619999999992, 2024-09-15T00:11:44.624 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.624 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.625 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.625 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.625 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.625 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.625 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 20389 2024-09-15T00:11:44.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.625 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.185081+0000", 2024-09-15T00:11:44.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.185081+0000", 2024-09-15T00:11:44.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.185084+0000", 2024-09-15T00:11:44.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7300000000000001e-06 2024-09-15T00:11:44.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.185104+0000", 2024-09-15T00:11:44.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9925e-05 2024-09-15T00:11:44.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.185132+0000", 2024-09-15T00:11:44.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7804999999999999e-05 2024-09-15T00:11:44.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.185148+0000", 2024-09-15T00:11:44.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6174000000000001e-05 2024-09-15T00:11:44.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.185245+0000", 2024-09-15T00:11:44.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7077999999999994e-05 2024-09-15T00:11:44.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.185346+0000", 2024-09-15T00:11:44.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101442 2024-09-15T00:11:44.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:44.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.185554+0000", 2024-09-15T00:11:44.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020743000000000001 2024-09-15T00:11:44.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:44.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.187059+0000", 2024-09-15T00:11:44.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015057849999999999 2024-09-15T00:11:44.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:44.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.192850+0000", 2024-09-15T00:11:44.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057909700000000003 2024-09-15T00:11:44.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.192912+0000", 2024-09-15T00:11:44.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1797999999999993e-05 2024-09-15T00:11:44.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.192960+0000", 2024-09-15T00:11:44.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7624999999999999e-05 2024-09-15T00:11:44.630 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.630 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.630 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.630 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:20394 3.4 3:278af919:::rbd_data.113917258088.000000000000001a:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T00:11:44.630 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:41.187814+0000", 2024-09-15T00:11:44.630 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.3453024029999998, 2024-09-15T00:11:44.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077213259999999997, 2024-09-15T00:11:44.630 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.630 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.630 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.630 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.631 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.631 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.631 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 20394 2024-09-15T00:11:44.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.631 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.187814+0000", 2024-09-15T00:11:44.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.187814+0000", 2024-09-15T00:11:44.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.187816+0000", 2024-09-15T00:11:44.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.725e-06 2024-09-15T00:11:44.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.187822+0000", 2024-09-15T00:11:44.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0120000000000002e-06 2024-09-15T00:11:44.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.187838+0000", 2024-09-15T00:11:44.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5517999999999999e-05 2024-09-15T00:11:44.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.187846+0000", 2024-09-15T00:11:44.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1240000000000005e-06 2024-09-15T00:11:44.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.187891+0000", 2024-09-15T00:11:44.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5296000000000001e-05 2024-09-15T00:11:44.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.187949+0000", 2024-09-15T00:11:44.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7537999999999997e-05 2024-09-15T00:11:44.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:44.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.188067+0000", 2024-09-15T00:11:44.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011806700000000001 2024-09-15T00:11:44.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:44.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.189300+0000", 2024-09-15T00:11:44.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012329680000000001 2024-09-15T00:11:44.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:44.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.195444+0000", 2024-09-15T00:11:44.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061445629999999996 2024-09-15T00:11:44.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.195490+0000", 2024-09-15T00:11:44.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5859999999999998e-05 2024-09-15T00:11:44.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.195536+0000", 2024-09-15T00:11:44.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5655000000000002e-05 2024-09-15T00:11:44.636 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.636 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.636 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.637 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:20396 3.6 3:620d4347:::rbd_data.113917258088.000000000000001c:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T00:11:44.637 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:41.189690+0000", 2024-09-15T00:11:44.637 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.3434265129999998, 2024-09-15T00:11:44.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0084295040000000009, 2024-09-15T00:11:44.637 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:44.637 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:44.637 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:44.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:44.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:44.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:44.637 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 20396 2024-09-15T00:11:44.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.638 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:44.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:44.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.189690+0000", 2024-09-15T00:11:44.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:44.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.189690+0000", 2024-09-15T00:11:44.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:44.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:44.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.189692+0000", 2024-09-15T00:11:44.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2920000000000001e-06 2024-09-15T00:11:44.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:44.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.189698+0000", 2024-09-15T00:11:44.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6139999999999997e-06 2024-09-15T00:11:44.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:44.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.189712+0000", 2024-09-15T00:11:44.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4290999999999999e-05 2024-09-15T00:11:44.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:44.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.189720+0000", 2024-09-15T00:11:44.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.011e-06 2024-09-15T00:11:44.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:44.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.189781+0000", 2024-09-15T00:11:44.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0656999999999999e-05 2024-09-15T00:11:44.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.189832+0000", 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0723000000000002e-05 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.189940+0000", 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000108379 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.190616+0000", 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000676058 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:44.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.198006+0000", 2024-09-15T00:11:44.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073894829999999996 2024-09-15T00:11:44.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:44.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.198069+0000", 2024-09-15T00:11:44.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3256999999999994e-05 2024-09-15T00:11:44.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:44.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:44.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:44.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:41.198120+0000", 2024-09-15T00:11:44.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0739000000000003e-05 2024-09-15T00:11:44.643 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.643 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.643 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.643 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:44.643 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:44.643 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:44.644 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:44.645 INFO:teuthology.orchestra.run.smithi171.stdout:1610 punch from 0xa760aa to 0xa7d233, (0x7189 bytes) 2024-09-15T00:11:44.645 INFO:teuthology.orchestra.run.smithi171.stdout:1611 read 0x17f1df7 thru 0x18010c7 (0xf2d1 bytes) 2024-09-15T00:11:44.645 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:44.645 INFO:teuthology.orchestra.run.smithi171.stdout:1614 write 0xe66a69d thru 0xe67737e (0xcce2 bytes) 2024-09-15T00:11:44.646 INFO:teuthology.orchestra.run.smithi171.stdout:1616 punch from 0xaa4f97f to 0xaa547b4, (0x4e35 bytes) 2024-09-15T00:11:44.646 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:44.646 INFO:teuthology.orchestra.run.smithi171.stdout:1619 trunc from 0xe67737f to 0xc7b799a 2024-09-15T00:11:44.646 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:44.642+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:44.726 INFO:teuthology.orchestra.run.smithi171.stdout:1620 punch from 0x885435 to 0x8894c4, (0x408f bytes) 2024-09-15T00:11:44.726 INFO:teuthology.orchestra.run.smithi171.stdout:1622 write 0x9a8a273 thru 0x9a98814 (0xe5a2 bytes) 2024-09-15T00:11:44.730 INFO:teuthology.orchestra.run.smithi171.stdout:1623 read 0xb6a06f6 thru 0xb6a6e84 (0x678f bytes) 2024-09-15T00:11:44.730 INFO:teuthology.orchestra.run.smithi171.stdout:1624 punch from 0x822b4b1 to 0x822b9b3, (0x502 bytes) 2024-09-15T00:11:44.736 INFO:teuthology.orchestra.run.smithi171.stdout:1626 write 0xd71b038 thru 0xd7258c1 (0xa88a bytes) 2024-09-15T00:11:44.738 INFO:teuthology.orchestra.run.smithi171.stdout:1627 writesame 0x3bdf1e3 thru 0x3be3596 data_size 0xe(0x43b4 bytes) 2024-09-15T00:11:44.740 INFO:teuthology.orchestra.run.smithi171.stdout:1628 punch from 0xd180b51 to 0xd19078f, (0xfc3e bytes) 2024-09-15T00:11:44.740 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:44.740 INFO:teuthology.orchestra.run.smithi171.stdout:1630 write 0x1360ea0 thru 0x1368791 (0x78f2 bytes) 2024-09-15T00:11:44.742 INFO:teuthology.orchestra.run.smithi171.stdout:1631 write 0x973716d thru 0x97413a9 (0xa23d bytes) 2024-09-15T00:11:44.744 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:44.743+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:44.747 INFO:teuthology.orchestra.run.smithi171.stdout:1632 trunc from 0xd7258c2 to 0xec774d5 2024-09-15T00:11:44.752 INFO:teuthology.orchestra.run.smithi171.stdout:1635 writesame 0xc30e7a8 thru 0xc311b89 data_size 0x1d(0x33e2 bytes) 2024-09-15T00:11:44.753 INFO:teuthology.orchestra.run.smithi171.stdout:1636 read 0xb359622 thru 0xb35bffb (0x29da bytes) 2024-09-15T00:11:44.754 INFO:teuthology.orchestra.run.smithi171.stdout:1637 write 0x6f1546f thru 0x6f1a3df (0x4f71 bytes) 2024-09-15T00:11:44.756 INFO:teuthology.orchestra.run.smithi171.stdout:1639 read 0x7c67f25 thru 0x7c6f8bd (0x7999 bytes) 2024-09-15T00:11:44.756 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:44.756 INFO:teuthology.orchestra.run.smithi171.stdout:1642 write 0x82476e2 thru 0x82548bd (0xd1dc bytes) 2024-09-15T00:11:44.760 INFO:teuthology.orchestra.run.smithi171.stdout:1643 read 0x267416d thru 0x267d627 (0x94bb bytes) 2024-09-15T00:11:44.760 INFO:teuthology.orchestra.run.smithi171.stdout:1644 punch from 0x4c8d82a to 0x4c93bde, (0x63b4 bytes) 2024-09-15T00:11:44.761 INFO:teuthology.orchestra.run.smithi171.stdout:1646 punch from 0x8274185 to 0x8279ca0, (0x5b1b bytes) 2024-09-15T00:11:44.763 INFO:teuthology.orchestra.run.smithi171.stdout:1647 trunc from 0xec774d5 to 0xc4159c6 2024-09-15T00:11:44.777 INFO:teuthology.orchestra.run.smithi171.stdout:1649 write 0xdeb7f3f thru 0xdec274c (0xa80e bytes) 2024-09-15T00:11:44.779 INFO:teuthology.orchestra.run.smithi171.stdout:1652 write 0x26c8784 thru 0x26cd72d (0x4faa bytes) 2024-09-15T00:11:44.781 INFO:teuthology.orchestra.run.smithi171.stdout:1653 read 0x6756b49 thru 0x67586c2 (0x1b7a bytes) 2024-09-15T00:11:44.781 INFO:teuthology.orchestra.run.smithi171.stdout:1654 writesame 0x9d9e42 thru 0x9e9495 data_size 0x5(0xf654 bytes) 2024-09-15T00:11:44.788 INFO:teuthology.orchestra.run.smithi171.stdout:1656 writesame 0x78f90f3 thru 0x78f955c data_size 0x5(0x46a bytes) 2024-09-15T00:11:44.789 INFO:teuthology.orchestra.run.smithi171.stdout:1657 writesame 0x431bf0 thru 0x43e5e9 data_size 0x1(0xc9fa bytes) 2024-09-15T00:11:44.799 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:44.799 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:44.799 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:44.799 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:44.808 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:44.846 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:44.844+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:44.885 INFO:teuthology.orchestra.run.smithi171.stdout:1658 read 0xb83656f thru 0xb83d11d (0x6baf bytes) 2024-09-15T00:11:44.886 INFO:teuthology.orchestra.run.smithi171.stdout:1659 read 0x5466714 thru 0x54685cf (0x1ebc bytes) 2024-09-15T00:11:44.886 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:44.886 INFO:teuthology.orchestra.run.smithi171.stdout:1661 write 0x283c678 thru 0x2844a92 (0x841b bytes) 2024-09-15T00:11:44.888 INFO:teuthology.orchestra.run.smithi171.stdout:1662 read 0xa3746c4 thru 0xa37f74f (0xb08c bytes) 2024-09-15T00:11:44.889 INFO:teuthology.orchestra.run.smithi171.stdout:1663 punch from 0xd8ae131 to 0xd8bb562, (0xd431 bytes) 2024-09-15T00:11:44.889 INFO:teuthology.orchestra.run.smithi171.stdout:1665 read 0xb61a03f thru 0xb62753f (0xd501 bytes) 2024-09-15T00:11:44.889 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:44.889 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:44.889 INFO:teuthology.orchestra.run.smithi171.stdout:1669 read 0xb673c4b thru 0xb680b1c (0xced2 bytes) 2024-09-15T00:11:44.889 INFO:teuthology.orchestra.run.smithi171.stdout:1670 write 0xdb50baa thru 0xdb5348c (0x28e3 bytes) 2024-09-15T00:11:44.891 INFO:teuthology.orchestra.run.smithi171.stdout:1671 read 0x6c97038 thru 0x6c9fb32 (0x8afb bytes) 2024-09-15T00:11:44.891 INFO:teuthology.orchestra.run.smithi171.stdout:1672 read 0xaad287e thru 0xaadaaee (0x8271 bytes) 2024-09-15T00:11:44.891 INFO:teuthology.orchestra.run.smithi171.stdout:1673 read 0x9871640 thru 0x987d55c (0xbf1d bytes) 2024-09-15T00:11:44.892 INFO:teuthology.orchestra.run.smithi171.stdout:1674 punch from 0xb2349d5 to 0xb235ba3, (0x11ce bytes) 2024-09-15T00:11:44.906 INFO:teuthology.orchestra.run.smithi171.stdout:1675 trunc from 0xdec274d to 0x1e21c35 2024-09-15T00:11:44.917 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:44.925 INFO:teuthology.orchestra.run.smithi171.stdout:1678 trunc from 0x1e21c35 to 0x84861fd 2024-09-15T00:11:44.930 INFO:teuthology.orchestra.run.smithi171.stdout:1683 punch from 0x342bc2c to 0x3430436, (0x480a bytes) 2024-09-15T00:11:44.930 INFO:teuthology.orchestra.run.smithi171.stdout:1684 read 0xf5db95 thru 0xf610e2 (0x354e bytes) 2024-09-15T00:11:44.931 INFO:teuthology.orchestra.run.smithi171.stdout:1685 write 0x5104c72 thru 0x5104fd7 (0x366 bytes) 2024-09-15T00:11:44.932 INFO:teuthology.orchestra.run.smithi171.stdout:1686 read 0x6581263 thru 0x6588c10 (0x79ae bytes) 2024-09-15T00:11:44.932 INFO:teuthology.orchestra.run.smithi171.stdout:1687 punch from 0x4f3184 to 0x4f539b, (0x2217 bytes) 2024-09-15T00:11:44.933 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:44.933 INFO:teuthology.orchestra.run.smithi171.stdout:1689 write 0x4347c9e thru 0x4352bf3 (0xaf56 bytes) 2024-09-15T00:11:44.945 INFO:teuthology.orchestra.run.smithi171.stdout:1690 write 0xbd0dd3d thru 0xbd1256d (0x4831 bytes) 2024-09-15T00:11:44.947 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:44.947 INFO:teuthology.orchestra.run.smithi171.stdout:1694 read 0x7f2c25f thru 0x7f34500 (0x82a2 bytes) 2024-09-15T00:11:44.947 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:44.945+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:44.947 INFO:teuthology.orchestra.run.smithi171.stdout:1695 write 0x2b318fe thru 0x2b3bad3 (0xa1d6 bytes) 2024-09-15T00:11:44.950 INFO:teuthology.orchestra.run.smithi171.stdout:1696 write 0x9abdcca thru 0x9ac2233 (0x456a bytes) 2024-09-15T00:11:44.951 INFO:teuthology.orchestra.run.smithi171.stdout:1697 writesame 0x6830c4d thru 0x6832629 data_size 0x1(0x19dd bytes) 2024-09-15T00:11:44.955 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:44.955 INFO:teuthology.orchestra.run.smithi171.stdout:1700 write 0xaf542d5 thru 0xaf5aba3 (0x68cf bytes) 2024-09-15T00:11:44.957 INFO:teuthology.orchestra.run.smithi171.stdout:1701 writesame 0x4dd994c thru 0x4de5dce data_size 0x29(0xc483 bytes) 2024-09-15T00:11:44.959 INFO:teuthology.orchestra.run.smithi171.stdout:1702 punch from 0x34ffedc to 0x35018cb, (0x19ef bytes) 2024-09-15T00:11:44.963 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:44.963 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:44.971 INFO:teuthology.orchestra.run.smithi171.stdout:1703 trunc from 0xbd1256e to 0x14864bd 2024-09-15T00:11:44.978 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:44.981 INFO:teuthology.orchestra.run.smithi171.stdout:1704 read 0x1f2e33 thru 0x1fcff2 (0xa1c0 bytes) 2024-09-15T00:11:44.982 INFO:teuthology.orchestra.run.smithi171.stdout:1705 punch from 0x12b76f4 to 0x12bb586, (0x3e92 bytes) 2024-09-15T00:11:45.000 INFO:teuthology.orchestra.run.smithi171.stdout:1707 writesame 0xbfb5214 thru 0xbfb986a data_size 0x1(0x4657 bytes) 2024-09-15T00:11:45.006 INFO:teuthology.orchestra.run.smithi171.stdout:1708 trunc from 0xbfb986b to 0x7b169a9 2024-09-15T00:11:45.013 INFO:teuthology.orchestra.run.smithi171.stdout:1709 punch from 0x700f78 to 0x7030e0, (0x2168 bytes) 2024-09-15T00:11:45.022 INFO:teuthology.orchestra.run.smithi171.stdout:1710 write 0xa4e6184 thru 0xa4e9f96 (0x3e13 bytes) 2024-09-15T00:11:45.023 INFO:teuthology.orchestra.run.smithi171.stdout:1711 read 0x7e7aaff thru 0x7e89cdb (0xf1dd bytes) 2024-09-15T00:11:45.024 INFO:teuthology.orchestra.run.smithi171.stdout:1713 flatten 2024-09-15T00:11:45.034 INFO:teuthology.orchestra.run.smithi171.stdout:1715 trunc from 0xa4e9f97 to 0x8e3ebf5 2024-09-15T00:11:45.043 INFO:teuthology.orchestra.run.smithi171.stdout:1717 write 0x583aff6 thru 0x584a97c (0xf987 bytes) 2024-09-15T00:11:45.048 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:45.046+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:45.053 INFO:teuthology.orchestra.run.smithi171.stdout:1718 trunc from 0x8e3ebf5 to 0x41fdca9 2024-09-15T00:11:45.077 INFO:teuthology.orchestra.run.smithi171.stdout:1719 write 0xb7a8c1a thru 0xb7acc70 (0x4057 bytes) 2024-09-15T00:11:45.080 INFO:teuthology.orchestra.run.smithi171.stdout:1720 trunc from 0xb7acc71 to 0xa6aefc8 2024-09-15T00:11:45.087 INFO:teuthology.orchestra.run.smithi171.stdout:1722 read 0x44be4e8 thru 0x44c17ff (0x3318 bytes) 2024-09-15T00:11:45.087 INFO:teuthology.orchestra.run.smithi171.stdout:1723 punch from 0x13b0cdf to 0x13b46c9, (0x39ea bytes) 2024-09-15T00:11:45.088 INFO:teuthology.orchestra.run.smithi171.stdout:1724 write 0x3fafcef thru 0x3fb3997 (0x3ca9 bytes) 2024-09-15T00:11:45.089 INFO:teuthology.orchestra.run.smithi171.stdout:1725 read 0xea22b1 thru 0xeae9b1 (0xc701 bytes) 2024-09-15T00:11:45.089 INFO:teuthology.orchestra.run.smithi171.stdout:1726 punch from 0xb8f15c to 0xb9481a, (0x56be bytes) 2024-09-15T00:11:45.090 INFO:teuthology.orchestra.run.smithi171.stdout:1727 write 0x7dd18eb thru 0x7dd7cd6 (0x63ec bytes) 2024-09-15T00:11:45.091 INFO:teuthology.orchestra.run.smithi171.stdout:1728 read 0x657c365 thru 0x658ae14 (0xeab0 bytes) 2024-09-15T00:11:45.097 INFO:teuthology.orchestra.run.smithi171.stdout:1730 trunc from 0xa6aefc8 to 0xe32d403 2024-09-15T00:11:45.103 INFO:teuthology.orchestra.run.smithi171.stdout:1731 read 0xb6702b3 thru 0xb672544 (0x2292 bytes) 2024-09-15T00:11:45.103 INFO:teuthology.orchestra.run.smithi171.stdout:1732 read 0x558f257 thru 0x559ded7 (0xec81 bytes) 2024-09-15T00:11:45.103 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:45.103 INFO:teuthology.orchestra.run.smithi171.stdout:1735 write 0x493a048 thru 0x493bac1 (0x1a7a bytes) 2024-09-15T00:11:45.105 INFO:teuthology.orchestra.run.smithi171.stdout:1736 read 0x584d57b thru 0x58595ca (0xc050 bytes) 2024-09-15T00:11:45.106 INFO:teuthology.orchestra.run.smithi171.stdout:1737 writesame 0xd8b7bcc thru 0xd8c6af4 data_size 0x4f(0xef29 bytes) 2024-09-15T00:11:45.106 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.5 to 1 (10000) 2024-09-15T00:11:45.113 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is set 2024-09-15T00:11:45.115 INFO:teuthology.orchestra.run.smithi171.stdout:1738 write 0x7a956f4 thru 0x7a9ea09 (0x9316 bytes) 2024-09-15T00:11:45.120 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:45.120 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:45.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:45.120 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:45.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.120 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:462 3.0 e29/23 3:04479396:::rbd_id.image_client.0-clone2:head v 29'14, mlcod=27'12)", 2024-09-15T00:11:45.120 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:45.120 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.10112123, 2024-09-15T00:11:45.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030963029999999999, 2024-09-15T00:11:45.120 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.120 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.120 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.120 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:45.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:45.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:45.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:11:45.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016208+0000", 2024-09-15T00:11:45.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3669999999999997e-06 2024-09-15T00:11:45.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016250+0000", 2024-09-15T00:11:45.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1845999999999999e-05 2024-09-15T00:11:45.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016260+0000", 2024-09-15T00:11:45.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9750000000000002e-06 2024-09-15T00:11:45.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016342+0000", 2024-09-15T00:11:45.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1595999999999996e-05 2024-09-15T00:11:45.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016363+0000", 2024-09-15T00:11:45.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1073e-05 2024-09-15T00:11:45.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019246+0000", 2024-09-15T00:11:45.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028835240000000002 2024-09-15T00:11:45.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019296+0000", 2024-09-15T00:11:45.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9533999999999997e-05 2024-09-15T00:11:45.124 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.124 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.124 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.125 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:463 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'9, mlcod=27'7)", 2024-09-15T00:11:45.125 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:45.125 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.096508226000001, 2024-09-15T00:11:45.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040015709999999998, 2024-09-15T00:11:45.125 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.125 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.125 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.125 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:45.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:45.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:45.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7599999999999999e-07 2024-09-15T00:11:45.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020818+0000", 2024-09-15T00:11:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6609999999999999e-06 2024-09-15T00:11:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020826+0000", 2024-09-15T00:11:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4770000000000003e-06 2024-09-15T00:11:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020833+0000", 2024-09-15T00:11:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0809999999999996e-06 2024-09-15T00:11:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020878+0000", 2024-09-15T00:11:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4752000000000001e-05 2024-09-15T00:11:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020892+0000", 2024-09-15T00:11:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3899e-05 2024-09-15T00:11:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024767+0000", 2024-09-15T00:11:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038752259999999998 2024-09-15T00:11:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024814+0000", 2024-09-15T00:11:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7299000000000002e-05 2024-09-15T00:11:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:464 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'10, mlcod=27'8)", 2024-09-15T00:11:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.091588384000001, 2024-09-15T00:11:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048109290000000002, 2024-09-15T00:11:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-15T00:11:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025743+0000", 2024-09-15T00:11:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5789999999999993e-06 2024-09-15T00:11:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025752+0000", 2024-09-15T00:11:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8440000000000004e-06 2024-09-15T00:11:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025757+0000", 2024-09-15T00:11:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2610000000000001e-06 2024-09-15T00:11:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025796+0000", 2024-09-15T00:11:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9428999999999998e-05 2024-09-15T00:11:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025810+0000", 2024-09-15T00:11:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3129000000000001e-05 2024-09-15T00:11:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030514+0000", 2024-09-15T00:11:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047040099999999998 2024-09-15T00:11:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030544+0000", 2024-09-15T00:11:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9966000000000001e-05 2024-09-15T00:11:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:610 3.4 e29/23 3:255b0913:::rbd_data.11397bff333d.0000000000000039:head v 29'2, mlcod=0'0)", 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.888844789, 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003163014, 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:11:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228477+0000", 2024-09-15T00:11:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1100000000000004e-07 2024-09-15T00:11:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228483+0000", 2024-09-15T00:11:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3720000000000001e-06 2024-09-15T00:11:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228497+0000", 2024-09-15T00:11:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3944000000000001e-05 2024-09-15T00:11:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228505+0000", 2024-09-15T00:11:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8140000000000001e-06 2024-09-15T00:11:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228549+0000", 2024-09-15T00:11:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4116000000000002e-05 2024-09-15T00:11:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228568+0000", 2024-09-15T00:11:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8473000000000001e-05 2024-09-15T00:11:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231599+0000", 2024-09-15T00:11:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030315590000000001 2024-09-15T00:11:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231639+0000", 2024-09-15T00:11:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9824999999999999e-05 2024-09-15T00:11:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:766 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'81, mlcod=29'78)", 2024-09-15T00:11:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:11:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.684295712000001, 2024-09-15T00:11:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029025880000000002, 2024-09-15T00:11:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:11:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:11:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433026+0000", 2024-09-15T00:11:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:11:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433030+0000", 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6280000000000004e-06 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433038+0000", 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.072e-06 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433045+0000", 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1510000000000001e-06 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433101+0000", 2024-09-15T00:11:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6336999999999999e-05 2024-09-15T00:11:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433113+0000", 2024-09-15T00:11:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1639e-05 2024-09-15T00:11:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435897+0000", 2024-09-15T00:11:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027845779999999998 2024-09-15T00:11:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435928+0000", 2024-09-15T00:11:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0689999999999999e-05 2024-09-15T00:11:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:832 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'100, mlcod=29'97)", 2024-09-15T00:11:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.592050212, 2024-09-15T00:11:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005498925, 2024-09-15T00:11:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7999999999999995e-07 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525276+0000", 2024-09-15T00:11:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6739999999999996e-06 2024-09-15T00:11:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525287+0000", 2024-09-15T00:11:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0906e-05 2024-09-15T00:11:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525293+0000", 2024-09-15T00:11:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4409999999999998e-06 2024-09-15T00:11:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525329+0000", 2024-09-15T00:11:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5237000000000002e-05 2024-09-15T00:11:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525340+0000", 2024-09-15T00:11:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1782e-05 2024-09-15T00:11:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530710+0000", 2024-09-15T00:11:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053695460000000002 2024-09-15T00:11:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530770+0000", 2024-09-15T00:11:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9658999999999999e-05 2024-09-15T00:11:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:876 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'114, mlcod=29'111)", 2024-09-15T00:11:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.421011212, 2024-09-15T00:11:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028465830000000002, 2024-09-15T00:11:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2399999999999998e-07 2024-09-15T00:11:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696316+0000", 2024-09-15T00:11:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2800000000000003e-06 2024-09-15T00:11:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696326+0000", 2024-09-15T00:11:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0546e-05 2024-09-15T00:11:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696333+0000", 2024-09-15T00:11:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7660000000000001e-06 2024-09-15T00:11:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696369+0000", 2024-09-15T00:11:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5933000000000003e-05 2024-09-15T00:11:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696381+0000", 2024-09-15T00:11:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2417e-05 2024-09-15T00:11:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699124+0000", 2024-09-15T00:11:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027428000000000001 2024-09-15T00:11:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699156+0000", 2024-09-15T00:11:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2217e-05 2024-09-15T00:11:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1313 3.1f e30/28 3:fa9cde43:::rbd_data.11397bff333d.000000000000003a:head v 30'3, mlcod=26'1)", 2024-09-15T00:11:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:11:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.833171575000001, 2024-09-15T00:11:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044064359999999997, 2024-09-15T00:11:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:11:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:11:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284150+0000", 2024-09-15T00:11:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3799999999999999e-07 2024-09-15T00:11:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284228+0000", 2024-09-15T00:11:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:11:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284235+0000", 2024-09-15T00:11:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:11:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284239+0000", 2024-09-15T00:11:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3449999999999999e-06 2024-09-15T00:11:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284282+0000", 2024-09-15T00:11:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2506000000000003e-05 2024-09-15T00:11:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284297+0000", 2024-09-15T00:11:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5523999999999999e-05 2024-09-15T00:11:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288517+0000", 2024-09-15T00:11:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042194019999999997 2024-09-15T00:11:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288556+0000", 2024-09-15T00:11:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8952000000000002e-05 2024-09-15T00:11:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1321 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'7, mlcod=29'5)", 2024-09-15T00:11:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.809078316000001, 2024-09-15T00:11:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003131254, 2024-09-15T00:11:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:11:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308250+0000", 2024-09-15T00:11:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6109999999999999e-06 2024-09-15T00:11:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308264+0000", 2024-09-15T00:11:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4473e-05 2024-09-15T00:11:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308272+0000", 2024-09-15T00:11:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3540000000000002e-06 2024-09-15T00:11:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308340+0000", 2024-09-15T00:11:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8305000000000004e-05 2024-09-15T00:11:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308361+0000", 2024-09-15T00:11:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0409999999999999e-05 2024-09-15T00:11:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311332+0000", 2024-09-15T00:11:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029715950000000001 2024-09-15T00:11:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311374+0000", 2024-09-15T00:11:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1786000000000003e-05 2024-09-15T00:11:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.9 epoch 30/23 rep_tid 277 entries 30'217 (0'0) error 3:942069cf:::rbd_header.11397bff333d:head by client.4409.0:1373 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'214)", 2024-09-15T00:11:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:11:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.763525719, 2024-09-15T00:11:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034617960000000001, 2024-09-15T00:11:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:11:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:11:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:11:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353796+0000", 2024-09-15T00:11:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6699999999999999e-07 2024-09-15T00:11:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353803+0000", 2024-09-15T00:11:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.832e-06 2024-09-15T00:11:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353814+0000", 2024-09-15T00:11:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1815e-05 2024-09-15T00:11:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353822+0000", 2024-09-15T00:11:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7440000000000007e-06 2024-09-15T00:11:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353910+0000", 2024-09-15T00:11:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7491000000000006e-05 2024-09-15T00:11:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.357257+0000", 2024-09-15T00:11:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033474469999999999 2024-09-15T00:11:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1374 3.9 e30/23 3:942069cf:::rbd_header.11397bff333d:head v 30'218, mlcod=30'215)", 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.762126077000001, 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047299830000000001, 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:11:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355196+0000", 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3900000000000004e-07 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355200+0000", 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5700000000000003e-06 2024-09-15T00:11:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355210+0000", 2024-09-15T00:11:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6050000000000004e-06 2024-09-15T00:11:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355215+0000", 2024-09-15T00:11:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.933e-06 2024-09-15T00:11:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355249+0000", 2024-09-15T00:11:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4110999999999999e-05 2024-09-15T00:11:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355260+0000", 2024-09-15T00:11:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1235e-05 2024-09-15T00:11:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359895+0000", 2024-09-15T00:11:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046345550000000003 2024-09-15T00:11:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359925+0000", 2024-09-15T00:11:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0335e-05 2024-09-15T00:11:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1385 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'221, mlcod=30'218)", 2024-09-15T00:11:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:11:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.749669541999999, 2024-09-15T00:11:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030443950000000001, 2024-09-15T00:11:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:11:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:11:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367652+0000", 2024-09-15T00:11:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:11:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367657+0000", 2024-09-15T00:11:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5410000000000002e-06 2024-09-15T00:11:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367666+0000", 2024-09-15T00:11:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9280000000000005e-06 2024-09-15T00:11:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367671+0000", 2024-09-15T00:11:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7870000000000001e-06 2024-09-15T00:11:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367707+0000", 2024-09-15T00:11:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6102000000000001e-05 2024-09-15T00:11:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367719+0000", 2024-09-15T00:11:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1571e-05 2024-09-15T00:11:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370666+0000", 2024-09-15T00:11:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029474750000000002 2024-09-15T00:11:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370696+0000", 2024-09-15T00:11:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9402000000000001e-05 2024-09-15T00:11:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1662 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'300, mlcod=30'297)", 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.329304905000001, 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031296090000000002, 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:11:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788017+0000", 2024-09-15T00:11:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8999999999999996e-07 2024-09-15T00:11:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788021+0000", 2024-09-15T00:11:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4970000000000003e-06 2024-09-15T00:11:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788027+0000", 2024-09-15T00:11:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1840000000000004e-06 2024-09-15T00:11:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788032+0000", 2024-09-15T00:11:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.135e-06 2024-09-15T00:11:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788070+0000", 2024-09-15T00:11:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7073999999999998e-05 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788081+0000", 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0987e-05 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791116+0000", 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030355209999999998 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791146+0000", 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9621e-05 2024-09-15T00:11:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1677 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'305, mlcod=30'302)", 2024-09-15T00:11:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:11:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.305576543000001, 2024-09-15T00:11:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031364320000000002, 2024-09-15T00:11:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:11:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:11:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811745+0000", 2024-09-15T00:11:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:11:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811749+0000", 2024-09-15T00:11:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9369999999999997e-06 2024-09-15T00:11:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811756+0000", 2024-09-15T00:11:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:11:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811765+0000", 2024-09-15T00:11:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7930000000000003e-06 2024-09-15T00:11:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811797+0000", 2024-09-15T00:11:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2203999999999999e-05 2024-09-15T00:11:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811810+0000", 2024-09-15T00:11:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2359e-05 2024-09-15T00:11:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814852+0000", 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003042276 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814881+0000", 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9037999999999999e-05 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1724 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'9, mlcod=30'7)", 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.236650859000001, 2024-09-15T00:11:45.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003115769, 2024-09-15T00:11:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:11:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:11:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880671+0000", 2024-09-15T00:11:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4900000000000005e-07 2024-09-15T00:11:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880678+0000", 2024-09-15T00:11:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0979999999999996e-06 2024-09-15T00:11:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880691+0000", 2024-09-15T00:11:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3288999999999999e-05 2024-09-15T00:11:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880698+0000", 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9240000000000001e-06 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880757+0000", 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8866000000000002e-05 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880790+0000", 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2926999999999998e-05 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883738+0000", 2024-09-15T00:11:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002948418 2024-09-15T00:11:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883786+0000", 2024-09-15T00:11:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7398000000000003e-05 2024-09-15T00:11:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1790 3.9 e30/23 3:9676e01b:::rbd_data.11397bff333d.0000000000000030:head v 30'338, mlcod=30'335)", 2024-09-15T00:11:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:11:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.157768488999999, 2024-09-15T00:11:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044496980000000002, 2024-09-15T00:11:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:11:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:11:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959553+0000", 2024-09-15T00:11:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3600000000000004e-07 2024-09-15T00:11:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959569+0000", 2024-09-15T00:11:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6152999999999999e-05 2024-09-15T00:11:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959577+0000", 2024-09-15T00:11:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5179999999999998e-06 2024-09-15T00:11:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959581+0000", 2024-09-15T00:11:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3669999999999999e-06 2024-09-15T00:11:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959619+0000", 2024-09-15T00:11:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8209e-05 2024-09-15T00:11:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959632+0000", 2024-09-15T00:11:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2405000000000001e-05 2024-09-15T00:11:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.963954+0000", 2024-09-15T00:11:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043226660000000002 2024-09-15T00:11:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.964002+0000", 2024-09-15T00:11:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7843999999999998e-05 2024-09-15T00:11:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:3177 3.4 e33/23 3:23818c00:::rbd_data.1139b531b717.0000000000000022:head v 33'58, mlcod=33'56)", 2024-09-15T00:11:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:11:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.459479923, 2024-09-15T00:11:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030564120000000001, 2024-09-15T00:11:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:11:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:11:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657842+0000", 2024-09-15T00:11:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.19e-06 2024-09-15T00:11:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657853+0000", 2024-09-15T00:11:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:11:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657871+0000", 2024-09-15T00:11:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8233e-05 2024-09-15T00:11:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657880+0000", 2024-09-15T00:11:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8319999999999995e-06 2024-09-15T00:11:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657949+0000", 2024-09-15T00:11:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8847e-05 2024-09-15T00:11:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657977+0000", 2024-09-15T00:11:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8308000000000001e-05 2024-09-15T00:11:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.660851+0000", 2024-09-15T00:11:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028743190000000002 2024-09-15T00:11:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.660897+0000", 2024-09-15T00:11:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6142e-05 2024-09-15T00:11:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:3602 3.1f e34/28 3:fd28af01:::rbd_data.1139b531b717.0000000000000032:head v 34'24, mlcod=33'22)", 2024-09-15T00:11:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:11:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.072082527999999, 2024-09-15T00:11:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034159709999999999, 2024-09-15T00:11:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045239+0000", 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045252+0000", 2024-09-15T00:11:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2891000000000001e-05 2024-09-15T00:11:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045263+0000", 2024-09-15T00:11:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0875e-05 2024-09-15T00:11:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045271+0000", 2024-09-15T00:11:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3529999999999995e-06 2024-09-15T00:11:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045338+0000", 2024-09-15T00:11:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7377000000000002e-05 2024-09-15T00:11:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045359+0000", 2024-09-15T00:11:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1078e-05 2024-09-15T00:11:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.048621+0000", 2024-09-15T00:11:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003261543 2024-09-15T00:11:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.048654+0000", 2024-09-15T00:11:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3550999999999998e-05 2024-09-15T00:11:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:4253 3.4 e35/23 3:242f5431:::rbd_data.11396389f8f2.0000000000000000:head v 35'87, mlcod=35'85)", 2024-09-15T00:11:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:11:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.724593670000001, 2024-09-15T00:11:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002944997, 2024-09-15T00:11:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392728+0000", 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.06e-07 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392782+0000", 2024-09-15T00:11:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.448e-05 2024-09-15T00:11:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392794+0000", 2024-09-15T00:11:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1768e-05 2024-09-15T00:11:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392801+0000", 2024-09-15T00:11:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9589999999999998e-06 2024-09-15T00:11:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392845+0000", 2024-09-15T00:11:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3884999999999997e-05 2024-09-15T00:11:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392860+0000", 2024-09-15T00:11:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5364999999999999e-05 2024-09-15T00:11:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.395641+0000", 2024-09-15T00:11:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027807629999999999 2024-09-15T00:11:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.395672+0000", 2024-09-15T00:11:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1170999999999997e-05 2024-09-15T00:11:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:18791 3.0 e38/23 3:05ecfc97:::rbd_data.11398e1a091a.0000000000000030:head v 38'66, mlcod=38'64)", 2024-09-15T00:11:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:11:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.0816781340000006, 2024-09-15T00:11:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029447940000000001, 2024-09-15T00:11:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:11:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:11:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035644+0000", 2024-09-15T00:11:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4e-07 2024-09-15T00:11:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035684+0000", 2024-09-15T00:11:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9857999999999997e-05 2024-09-15T00:11:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035694+0000", 2024-09-15T00:11:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0259e-05 2024-09-15T00:11:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035699+0000", 2024-09-15T00:11:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6389999999999997e-06 2024-09-15T00:11:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035745+0000", 2024-09-15T00:11:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5640999999999999e-05 2024-09-15T00:11:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035764+0000", 2024-09-15T00:11:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9368000000000002e-05 2024-09-15T00:11:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.038551+0000", 2024-09-15T00:11:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027860879999999999 2024-09-15T00:11:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.038588+0000", 2024-09-15T00:11:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7101000000000003e-05 2024-09-15T00:11:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.210 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:45.211 INFO:teuthology.orchestra.run.smithi171.stdout:1739 read 0x13cbbfa thru 0x13d1947 (0x5d4e bytes) 2024-09-15T00:11:45.211 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-15T00:11:45.212 INFO:teuthology.orchestra.run.smithi171.stdout:1740 punch from 0x1ea71d3 to 0x1eae4a2, (0x72cf bytes) 2024-09-15T00:11:45.212 INFO:teuthology.orchestra.run.smithi171.stdout:1742 write 0xafbdd37 thru 0xafc2ec0 (0x518a bytes) 2024-09-15T00:11:45.212 INFO:teuthology.orchestra.run.smithi171.stdout:1743 write 0xca5dbd7 thru 0xca6c180 (0xe5aa bytes) 2024-09-15T00:11:45.213 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:45.215 INFO:teuthology.orchestra.run.smithi171.stdout:1744 punch from 0xc1d496 to 0xc24436, (0x6fa0 bytes) 2024-09-15T00:11:45.215 INFO:teuthology.orchestra.run.smithi171.stdout:1746 trunc from 0xe32d403 to 0x1cef2e1 2024-09-15T00:11:45.215 INFO:teuthology.orchestra.run.smithi171.stdout:1747 trunc from 0x1cef2e1 to 0x7bfec2d 2024-09-15T00:11:45.215 INFO:teuthology.orchestra.run.smithi171.stdout:1748 write 0x4286260 thru 0x4289184 (0x2f25 bytes) 2024-09-15T00:11:45.215 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:45.216 INFO:teuthology.orchestra.run.smithi171.stdout:1753 read 0x1629859 thru 0x162f42e (0x5bd6 bytes) 2024-09-15T00:11:45.216 INFO:teuthology.orchestra.run.smithi171.stdout:1755 trunc from 0x7bfec2d to 0xec5943b 2024-09-15T00:11:45.216 INFO:teuthology.orchestra.run.smithi171.stdout:1756 write 0x3592587 thru 0x3592ce4 (0x75e bytes) 2024-09-15T00:11:45.216 INFO:teuthology.orchestra.run.smithi171.stdout:1757 write 0x5e19c4b thru 0x5e1d6da (0x3a90 bytes) 2024-09-15T00:11:45.216 INFO:teuthology.orchestra.run.smithi171.stdout:1759 writesame 0x8e9b5ad thru 0x8ea4884 data_size 0x4a(0x92d8 bytes) 2024-09-15T00:11:45.216 INFO:teuthology.orchestra.run.smithi171.stdout:1761 writesame 0x1d03ad2 thru 0x1d03bb7 data_size 0x5(0xe6 bytes) 2024-09-15T00:11:45.216 INFO:teuthology.orchestra.run.smithi171.stdout:1763 write 0x964acf thru 0x96b1a8 (0x66da bytes) 2024-09-15T00:11:45.216 INFO:teuthology.orchestra.run.smithi171.stdout:1765 writesame 0xe7300fa thru 0xe730a30 data_size 0x1(0x937 bytes) 2024-09-15T00:11:45.216 INFO:teuthology.orchestra.run.smithi171.stdout:1767 trunc from 0xec5943b to 0x44e8069 2024-09-15T00:11:45.216 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:45.209+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:45.217 INFO:teuthology.orchestra.run.smithi171.stdout:1768 writesame 0x1da1f3 thru 0x1db87f data_size 0x1(0x168d bytes) 2024-09-15T00:11:45.217 INFO:teuthology.orchestra.run.smithi171.stdout:1769 trunc from 0x44e8069 to 0x62109bf 2024-09-15T00:11:45.222 INFO:teuthology.orchestra.run.smithi171.stdout:1770 read 0x391353d thru 0x39192c1 (0x5d85 bytes) 2024-09-15T00:11:45.226 INFO:teuthology.orchestra.run.smithi171.stdout:1771 trunc from 0x62109bf to 0x3d107ed 2024-09-15T00:11:45.253 INFO:teuthology.orchestra.run.smithi171.stdout:1772 writesame 0xe6fe2bf thru 0xe7029d3 data_size 0x1(0x4715 bytes) 2024-09-15T00:11:45.255 INFO:teuthology.orchestra.run.smithi171.stdout:1773 read 0xace50ea thru 0xacefba5 (0xaabc bytes) 2024-09-15T00:11:45.263 INFO:teuthology.orchestra.run.smithi171.stdout:1774 trunc from 0xe7029d4 to 0x7764012 2024-09-15T00:11:45.270 INFO:teuthology.orchestra.run.smithi171.stdout:1775 read 0x2e78e12 thru 0x2e7cb12 (0x3d01 bytes) 2024-09-15T00:11:45.271 INFO:teuthology.orchestra.run.smithi171.stdout:1777 read 0x5808735 thru 0x580d99e (0x526a bytes) 2024-09-15T00:11:45.271 INFO:teuthology.orchestra.run.smithi171.stdout:1778 read 0xfb42b7 thru 0xfc0b43 (0xc88d bytes) 2024-09-15T00:11:45.272 INFO:teuthology.orchestra.run.smithi171.stdout:1779 writesame 0x46b4a31 thru 0x46b740b data_size 0x1(0x29db bytes) 2024-09-15T00:11:45.280 INFO:teuthology.orchestra.run.smithi171.stdout:1780 read 0x58e1b82 thru 0x58e9116 (0x7595 bytes) 2024-09-15T00:11:45.281 INFO:teuthology.orchestra.run.smithi171.stdout:1781 write 0x53acaa4 thru 0x53b922d (0xc78a bytes) 2024-09-15T00:11:45.294 INFO:teuthology.orchestra.run.smithi171.stdout:1782 write 0xa060332 thru 0xa0697fe (0x94cd bytes) 2024-09-15T00:11:45.309 INFO:teuthology.orchestra.run.smithi171.stdout:1784 trunc from 0xa0697ff to 0x8ef79c 2024-09-15T00:11:45.312 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:45.310+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:45.333 INFO:teuthology.orchestra.run.smithi171.stdout:1785 write 0x810e2e3 thru 0x810e332 (0x50 bytes) 2024-09-15T00:11:45.334 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:45.339 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:45.339 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:45.343 INFO:teuthology.orchestra.run.smithi171.stdout:1787 trunc from 0x810e333 to 0xd6c68c 2024-09-15T00:11:45.352 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:45.359 INFO:teuthology.orchestra.run.smithi171.stdout:1788 writesame 0x3781558 thru 0x3786cfe data_size 0x1(0x57a7 bytes) 2024-09-15T00:11:45.360 INFO:teuthology.orchestra.run.smithi171.stdout:1789 punch from 0x1e76760 to 0x1e7a5da, (0x3e7a bytes) 2024-09-15T00:11:45.364 INFO:teuthology.orchestra.run.smithi171.stdout:1790 trunc from 0x3786cff to 0x64384d3 2024-09-15T00:11:45.369 INFO:teuthology.orchestra.run.smithi171.stdout:1794 read 0x4b9f83e thru 0x4bac31d (0xcae0 bytes) 2024-09-15T00:11:45.370 INFO:teuthology.orchestra.run.smithi171.stdout:1795 write 0x37cf581 thru 0x37de676 (0xf0f6 bytes) 2024-09-15T00:11:45.373 INFO:teuthology.orchestra.run.smithi171.stdout:1797 write 0x39f9b96 thru 0x39fe23f (0x46aa bytes) 2024-09-15T00:11:45.375 INFO:teuthology.orchestra.run.smithi171.stdout:1798 writesame 0x1b3c137 thru 0x1b49d7e data_size 0x98(0xdc48 bytes) 2024-09-15T00:11:45.377 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:45.391 INFO:teuthology.orchestra.run.smithi171.stdout:1800 write 0xde24952 thru 0xde3191f (0xcfce bytes) 2024-09-15T00:11:45.393 INFO:teuthology.orchestra.run.smithi171.stdout:1801 read 0x5f34cf thru 0x5f34e5 (0x17 bytes) 2024-09-15T00:11:45.393 INFO:teuthology.orchestra.run.smithi171.stdout:1802 writesame 0x9fadc40 thru 0x9fb3837 data_size 0x6c(0x5bf8 bytes) 2024-09-15T00:11:45.395 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:45.395 INFO:teuthology.orchestra.run.smithi171.stdout:1804 write 0x7fe30a2 thru 0x7fe96c5 (0x6624 bytes) 2024-09-15T00:11:45.397 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:45.397 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:45.397 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:45.397 INFO:teuthology.orchestra.run.smithi171.stdout:1810 write 0x7098d5f thru 0x709e397 (0x5639 bytes) 2024-09-15T00:11:45.399 INFO:teuthology.orchestra.run.smithi171.stdout:1813 read 0xc1bdd6 thru 0xc1f683 (0x38ae bytes) 2024-09-15T00:11:45.400 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:45.400 INFO:teuthology.orchestra.run.smithi171.stdout:1817 read 0x5d6e818 thru 0x5d7dfed (0xf7d6 bytes) 2024-09-15T00:11:45.400 INFO:teuthology.orchestra.run.smithi171.stdout:1818 punch from 0x96f4010 to 0x96f8e4a, (0x4e3a bytes) 2024-09-15T00:11:45.400 INFO:teuthology.orchestra.run.smithi171.stdout:1819 read 0xdd3acad thru 0xdd40949 (0x5c9d bytes) 2024-09-15T00:11:45.401 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:45.401 INFO:teuthology.orchestra.run.smithi171.stdout:1822 read 0xb8d43ce thru 0xb8e2258 (0xde8b bytes) 2024-09-15T00:11:45.406 INFO:teuthology.orchestra.run.smithi171.stdout:1823 write 0xde2465f thru 0xde321a4 (0xdb46 bytes) 2024-09-15T00:11:45.410 INFO:teuthology.orchestra.run.smithi171.stdout:1824 trunc from 0xde321a5 to 0xdcc2253 2024-09-15T00:11:45.413 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:45.411+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:45.417 INFO:teuthology.orchestra.run.smithi171.stdout:1825 write 0xc839174 thru 0xc8398a1 (0x72e bytes) 2024-09-15T00:11:45.418 INFO:teuthology.orchestra.run.smithi171.stdout:1828 write 0x7986e4a thru 0x7995245 (0xe3fc bytes) 2024-09-15T00:11:45.437 INFO:teuthology.orchestra.run.smithi171.stdout:1829 trunc from 0xdcc2253 to 0xf38db9 2024-09-15T00:11:45.456 INFO:teuthology.orchestra.run.smithi171.stdout:1830 writesame 0x2815196 thru 0x2817468 data_size 0x1(0x22d3 bytes) 2024-09-15T00:11:45.457 INFO:teuthology.orchestra.run.smithi171.stdout:1831 writesame 0xbfb7b3 thru 0xc02712 data_size 0xa2(0x6f60 bytes) 2024-09-15T00:11:45.458 INFO:teuthology.orchestra.run.smithi171.stdout:1832 read 0x1d54cb thru 0x1dd748 (0x827e bytes) 2024-09-15T00:11:45.459 INFO:teuthology.orchestra.run.smithi171.stdout:1833 write 0x1eae38f thru 0x1eaeb53 (0x7c5 bytes) 2024-09-15T00:11:45.461 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:45.461 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:45.461 INFO:teuthology.orchestra.run.smithi171.stdout:1836 write 0x37e083 thru 0x38a86c (0xc7ea bytes) 2024-09-15T00:11:45.465 INFO:teuthology.orchestra.run.smithi171.stdout:1837 read 0x230d919 thru 0x23125e8 (0x4cd0 bytes) 2024-09-15T00:11:45.465 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:45.465 INFO:teuthology.orchestra.run.smithi171.stdout:1839 read 0x20bac99 thru 0x20be9b2 (0x3d1a bytes) 2024-09-15T00:11:45.478 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:45.479 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:45.479 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:45.479 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:45.479 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:45.483 INFO:teuthology.orchestra.run.smithi171.stdout:1840 write 0xd23d910 thru 0xd242d31 (0x5422 bytes) 2024-09-15T00:11:45.487 INFO:teuthology.orchestra.run.smithi171.stdout:1841 clone 13 order 21 su 16384 sc 4 2024-09-15T00:11:45.488 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:45.529 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:45.528+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:45.615 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:45.613+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:45.633 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:45.633 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:45.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:45.634 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:45.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.634 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1240 3.2 3:4001636a:::rbd_data.11397bff333d.0000000000000007:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:45.634 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.045955+0000", 2024-09-15T00:11:45.634 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.584870809000002, 2024-09-15T00:11:45.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017913958000000001, 2024-09-15T00:11:45.634 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.634 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.634 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.634 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.634 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.634 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.634 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1240 2024-09-15T00:11:45.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.635 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045955+0000", 2024-09-15T00:11:45.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045955+0000", 2024-09-15T00:11:45.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045957+0000", 2024-09-15T00:11:45.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6640000000000001e-06 2024-09-15T00:11:45.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045964+0000", 2024-09-15T00:11:45.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3629999999999996e-06 2024-09-15T00:11:45.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045979+0000", 2024-09-15T00:11:45.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4436e-05 2024-09-15T00:11:45.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.045989+0000", 2024-09-15T00:11:45.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0307e-05 2024-09-15T00:11:45.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.046042+0000", 2024-09-15T00:11:45.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3134999999999999e-05 2024-09-15T00:11:45.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.046106+0000", 2024-09-15T00:11:45.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3807999999999996e-05 2024-09-15T00:11:45.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:45.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.047043+0000", 2024-09-15T00:11:45.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000937165 2024-09-15T00:11:45.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:45.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063721+0000", 2024-09-15T00:11:45.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016678193000000001 2024-09-15T00:11:45.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:45.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063743+0000", 2024-09-15T00:11:45.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1163999999999999e-05 2024-09-15T00:11:45.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063810+0000", 2024-09-15T00:11:45.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7317000000000006e-05 2024-09-15T00:11:45.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.063869+0000", 2024-09-15T00:11:45.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9406e-05 2024-09-15T00:11:45.639 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.639 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.639 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.640 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:47798 3.7 3:e6004524:::rbd_data.113916cf513.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1789239~44328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.640 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.387407+0000", 2024-09-15T00:11:45.640 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.2434190089999999, 2024-09-15T00:11:45.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033697973999999999, 2024-09-15T00:11:45.640 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.640 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.640 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.640 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.640 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.640 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.640 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 47798 2024-09-15T00:11:45.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.641 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.387407+0000", 2024-09-15T00:11:45.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.387407+0000", 2024-09-15T00:11:45.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.387408+0000", 2024-09-15T00:11:45.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2580000000000001e-06 2024-09-15T00:11:45.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389094+0000", 2024-09-15T00:11:45.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001685572 2024-09-15T00:11:45.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389116+0000", 2024-09-15T00:11:45.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2379999999999999e-05 2024-09-15T00:11:45.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389126+0000", 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0044000000000001e-05 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389203+0000", 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6945000000000003e-05 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389381+0000", 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000177908 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:45.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389890+0000", 2024-09-15T00:11:45.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00050844599999999996 2024-09-15T00:11:45.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:45.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.420647+0000", 2024-09-15T00:11:45.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030757201000000001 2024-09-15T00:11:45.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:45.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.420970+0000", 2024-09-15T00:11:45.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032290200000000002 2024-09-15T00:11:45.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.421037+0000", 2024-09-15T00:11:45.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6883000000000003e-05 2024-09-15T00:11:45.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.421105+0000", 2024-09-15T00:11:45.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8435000000000005e-05 2024-09-15T00:11:45.645 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.645 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.645 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.645 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:47892 3.12 e47/28 3:4ef0cbcd:::rbd_data.113916cf513.0000000000000000:head v 47'139, mlcod=45'137)", 2024-09-15T00:11:45.645 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.437387+0000", 2024-09-15T00:11:45.645 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.1934392199999999, 2024-09-15T00:11:45.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021482557999999999, 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.437387+0000", 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.437387+0000", 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.437389+0000", 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6390000000000001e-06 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.440729+0000", 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033401490000000002 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.440750+0000", 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1376e-05 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.440767+0000", 2024-09-15T00:11:45.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7127000000000001e-05 2024-09-15T00:11:45.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.440847+0000", 2024-09-15T00:11:45.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689000000000002e-05 2024-09-15T00:11:45.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.440876+0000", 2024-09-15T00:11:45.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8663e-05 2024-09-15T00:11:45.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.458828+0000", 2024-09-15T00:11:45.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017952633999999999 2024-09-15T00:11:45.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.458869+0000", 2024-09-15T00:11:45.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1281e-05 2024-09-15T00:11:45.649 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.649 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.649 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.649 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48090 3.b 3:d42b6c47:::rbd_data.113916cf513.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 13893632~5352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.534140+0000", 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.096686236, 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022685002999999999, 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48090 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.534140+0000", 2024-09-15T00:11:45.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.534140+0000", 2024-09-15T00:11:45.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.534141+0000", 2024-09-15T00:11:45.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2699999999999999e-06 2024-09-15T00:11:45.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.534970+0000", 2024-09-15T00:11:45.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00082854499999999998 2024-09-15T00:11:45.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.534991+0000", 2024-09-15T00:11:45.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1058999999999999e-05 2024-09-15T00:11:45.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.535004+0000", 2024-09-15T00:11:45.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2830000000000001e-05 2024-09-15T00:11:45.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.535078+0000", 2024-09-15T00:11:45.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4695000000000002e-05 2024-09-15T00:11:45.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.535209+0000", 2024-09-15T00:11:45.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013081900000000001 2024-09-15T00:11:45.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:45.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.535595+0000", 2024-09-15T00:11:45.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000385993 2024-09-15T00:11:45.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:45.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.553922+0000", 2024-09-15T00:11:45.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018327011000000001 2024-09-15T00:11:45.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:45.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.556695+0000", 2024-09-15T00:11:45.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00277261 2024-09-15T00:11:45.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.556785+0000", 2024-09-15T00:11:45.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0649000000000006e-05 2024-09-15T00:11:45.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.556825+0000", 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9521999999999999e-05 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48891 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 5242880~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.608312+0000", 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.0225140829999999, 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018339534000000001, 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.655 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.656 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48891 2024-09-15T00:11:45.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.656 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608312+0000", 2024-09-15T00:11:45.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608312+0000", 2024-09-15T00:11:45.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608313+0000", 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1200000000000003e-07 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608315+0000", 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9079999999999999e-06 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608325+0000", 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9199999999999999e-06 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608331+0000", 2024-09-15T00:11:45.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.587e-06 2024-09-15T00:11:45.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626093+0000", 2024-09-15T00:11:45.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017762527 2024-09-15T00:11:45.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626140+0000", 2024-09-15T00:11:45.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6649e-05 2024-09-15T00:11:45.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626652+0000", 2024-09-15T00:11:45.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051143100000000002 2024-09-15T00:11:45.659 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.659 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.659 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.659 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48892 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.659 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.608348+0000", 2024-09-15T00:11:45.659 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.022478545, 2024-09-15T00:11:45.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018518995, 2024-09-15T00:11:45.659 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.659 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.660 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:45.660 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.660 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.660 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.660 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48892 2024-09-15T00:11:45.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.660 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608348+0000", 2024-09-15T00:11:45.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608348+0000", 2024-09-15T00:11:45.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608348+0000", 2024-09-15T00:11:45.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-15T00:11:45.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608351+0000", 2024-09-15T00:11:45.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.88e-06 2024-09-15T00:11:45.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608358+0000", 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1840000000000002e-06 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608364+0000", 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3499999999999996e-06 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626677+0000", 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018313781000000001 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626770+0000", 2024-09-15T00:11:45.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3209999999999997e-05 2024-09-15T00:11:45.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626867+0000", 2024-09-15T00:11:45.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6043000000000002e-05 2024-09-15T00:11:45.663 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.663 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.663 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.663 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48893 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.663 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.608371+0000", 2024-09-15T00:11:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.0224547639999999, 2024-09-15T00:11:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018717826, 2024-09-15T00:11:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48893 2024-09-15T00:11:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608371+0000", 2024-09-15T00:11:45.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608371+0000", 2024-09-15T00:11:45.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608372+0000", 2024-09-15T00:11:45.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:11:45.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608374+0000", 2024-09-15T00:11:45.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.357e-06 2024-09-15T00:11:45.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608380+0000", 2024-09-15T00:11:45.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9410000000000004e-06 2024-09-15T00:11:45.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608385+0000", 2024-09-15T00:11:45.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8799999999999999e-06 2024-09-15T00:11:45.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626910+0000", 2024-09-15T00:11:45.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018525283 2024-09-15T00:11:45.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626995+0000", 2024-09-15T00:11:45.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4814000000000005e-05 2024-09-15T00:11:45.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627089+0000", 2024-09-15T00:11:45.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4201000000000002e-05 2024-09-15T00:11:45.667 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.667 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.667 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.668 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48894 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.668 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.608392+0000", 2024-09-15T00:11:45.668 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.0224345379999999, 2024-09-15T00:11:45.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018892147000000001, 2024-09-15T00:11:45.668 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.668 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.668 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:45.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.668 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48894 2024-09-15T00:11:45.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.669 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608392+0000", 2024-09-15T00:11:45.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608392+0000", 2024-09-15T00:11:45.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608392+0000", 2024-09-15T00:11:45.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-15T00:11:45.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608394+0000", 2024-09-15T00:11:45.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2910000000000002e-06 2024-09-15T00:11:45.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608402+0000", 2024-09-15T00:11:45.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3379999999999999e-06 2024-09-15T00:11:45.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608406+0000", 2024-09-15T00:11:45.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7500000000000003e-06 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627127+0000", 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018721115999999999 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627215+0000", 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7294999999999997e-05 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627284+0000", 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8992000000000006e-05 2024-09-15T00:11:45.671 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.672 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.672 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.672 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48895 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.672 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.608413+0000", 2024-09-15T00:11:45.672 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.022413003, 2024-09-15T00:11:45.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019021870999999999, 2024-09-15T00:11:45.672 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.672 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.672 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:45.672 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.672 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.672 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.673 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48895 2024-09-15T00:11:45.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.673 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608413+0000", 2024-09-15T00:11:45.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608413+0000", 2024-09-15T00:11:45.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608413+0000", 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3599999999999999e-07 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608415+0000", 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0310000000000001e-06 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608422+0000", 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1260000000000003e-06 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608427+0000", 2024-09-15T00:11:45.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9599999999999999e-06 2024-09-15T00:11:45.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627325+0000", 2024-09-15T00:11:45.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018898609 2024-09-15T00:11:45.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627373+0000", 2024-09-15T00:11:45.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8134999999999999e-05 2024-09-15T00:11:45.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627435+0000", 2024-09-15T00:11:45.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1674000000000006e-05 2024-09-15T00:11:45.676 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.676 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.676 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.676 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48896 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.676 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.608433+0000", 2024-09-15T00:11:45.676 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.022393466, 2024-09-15T00:11:45.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019236590000000001, 2024-09-15T00:11:45.676 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.676 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.676 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:45.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.677 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.677 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.677 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48896 2024-09-15T00:11:45.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.677 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608433+0000", 2024-09-15T00:11:45.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608433+0000", 2024-09-15T00:11:45.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608433+0000", 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.46e-07 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608435+0000", 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608441+0000", 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8969999999999996e-06 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608446+0000", 2024-09-15T00:11:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.921e-06 2024-09-15T00:11:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627487+0000", 2024-09-15T00:11:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019040973999999999 2024-09-15T00:11:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627572+0000", 2024-09-15T00:11:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4846999999999996e-05 2024-09-15T00:11:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627669+0000", 2024-09-15T00:11:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7381999999999997e-05 2024-09-15T00:11:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48980 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:11:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.0216344230000001, 2024-09-15T00:11:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019125406000000001, 2024-09-15T00:11:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48980 2024-09-15T00:11:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:11:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609193+0000", 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8999999999999995e-07 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609197+0000", 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2320000000000003e-06 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609208+0000", 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1484000000000001e-05 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609216+0000", 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3839999999999999e-06 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627694+0000", 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018477970999999999 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627738+0000", 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4801e-05 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628317+0000", 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00057864400000000001 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48981 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 2752512~73728] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609227+0000", 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.0215991280000001, 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019843092, 2024-09-15T00:11:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48981 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609227+0000", 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609227+0000", 2024-09-15T00:11:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609227+0000", 2024-09-15T00:11:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5900000000000002e-07 2024-09-15T00:11:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609230+0000", 2024-09-15T00:11:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2089999999999999e-06 2024-09-15T00:11:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609236+0000", 2024-09-15T00:11:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4919999999999998e-06 2024-09-15T00:11:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609241+0000", 2024-09-15T00:11:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0440000000000003e-06 2024-09-15T00:11:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628352+0000", 2024-09-15T00:11:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019111145999999999 2024-09-15T00:11:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628433+0000", 2024-09-15T00:11:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0887e-05 2024-09-15T00:11:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629070+0000", 2024-09-15T00:11:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00063685499999999997 2024-09-15T00:11:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48982 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 4587520~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609275+0000", 2024-09-15T00:11:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.021550661, 2024-09-15T00:11:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020353955999999999, 2024-09-15T00:11:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48982 2024-09-15T00:11:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609275+0000", 2024-09-15T00:11:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609275+0000", 2024-09-15T00:11:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609276+0000", 2024-09-15T00:11:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5799999999999999e-07 2024-09-15T00:11:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609279+0000", 2024-09-15T00:11:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5830000000000001e-06 2024-09-15T00:11:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609285+0000", 2024-09-15T00:11:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8240000000000002e-06 2024-09-15T00:11:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609291+0000", 2024-09-15T00:11:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1399999999999999e-06 2024-09-15T00:11:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629107+0000", 2024-09-15T00:11:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019816674999999999 2024-09-15T00:11:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629158+0000", 2024-09-15T00:11:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1005999999999999e-05 2024-09-15T00:11:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629629+0000", 2024-09-15T00:11:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047116999999999998 2024-09-15T00:11:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48983 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609298+0000", 2024-09-15T00:11:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.021527962, 2024-09-15T00:11:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020455086000000001, 2024-09-15T00:11:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48983 2024-09-15T00:11:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609298+0000", 2024-09-15T00:11:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609298+0000", 2024-09-15T00:11:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609299+0000", 2024-09-15T00:11:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7899999999999998e-07 2024-09-15T00:11:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609301+0000", 2024-09-15T00:11:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0379999999999998e-06 2024-09-15T00:11:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609307+0000", 2024-09-15T00:11:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2600000000000002e-06 2024-09-15T00:11:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609312+0000", 2024-09-15T00:11:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4009999999999996e-06 2024-09-15T00:11:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629651+0000", 2024-09-15T00:11:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020338438 2024-09-15T00:11:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629696+0000", 2024-09-15T00:11:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5123000000000001e-05 2024-09-15T00:11:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629753+0000", 2024-09-15T00:11:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7247000000000003e-05 2024-09-15T00:11:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48984 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609319+0000", 2024-09-15T00:11:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.021507025, 2024-09-15T00:11:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020570343000000001, 2024-09-15T00:11:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48984 2024-09-15T00:11:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609319+0000", 2024-09-15T00:11:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609319+0000", 2024-09-15T00:11:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609319+0000", 2024-09-15T00:11:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3099999999999998e-07 2024-09-15T00:11:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609321+0000", 2024-09-15T00:11:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:11:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609328+0000", 2024-09-15T00:11:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2149999999999998e-06 2024-09-15T00:11:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609333+0000", 2024-09-15T00:11:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.22e-06 2024-09-15T00:11:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629783+0000", 2024-09-15T00:11:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020450342 2024-09-15T00:11:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629829+0000", 2024-09-15T00:11:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5782999999999999e-05 2024-09-15T00:11:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629889+0000", 2024-09-15T00:11:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0393999999999998e-05 2024-09-15T00:11:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48985 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609339+0000", 2024-09-15T00:11:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.021487051, 2024-09-15T00:11:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020671998, 2024-09-15T00:11:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48985 2024-09-15T00:11:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609339+0000", 2024-09-15T00:11:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609339+0000", 2024-09-15T00:11:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609339+0000", 2024-09-15T00:11:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:11:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609341+0000", 2024-09-15T00:11:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9769999999999999e-06 2024-09-15T00:11:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609351+0000", 2024-09-15T00:11:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6220000000000004e-06 2024-09-15T00:11:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609356+0000", 2024-09-15T00:11:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1660000000000002e-06 2024-09-15T00:11:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629914+0000", 2024-09-15T00:11:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020558205 2024-09-15T00:11:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629954+0000", 2024-09-15T00:11:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9619999999999997e-05 2024-09-15T00:11:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630011+0000", 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7058999999999997e-05 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48986 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609363+0000", 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.021463182, 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020764550999999999, 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48986 2024-09-15T00:11:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609363+0000", 2024-09-15T00:11:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609363+0000", 2024-09-15T00:11:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609363+0000", 2024-09-15T00:11:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:11:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609366+0000", 2024-09-15T00:11:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.21e-06 2024-09-15T00:11:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609372+0000", 2024-09-15T00:11:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6390000000000003e-06 2024-09-15T00:11:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609377+0000", 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000002e-06 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630035+0000", 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020657847 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630076+0000", 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1597000000000002e-05 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630127+0000", 2024-09-15T00:11:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1232e-05 2024-09-15T00:11:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48987 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 13762560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609383+0000", 2024-09-15T00:11:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.021442714, 2024-09-15T00:11:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020904947, 2024-09-15T00:11:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48987 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609383+0000", 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609383+0000", 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609384+0000", 2024-09-15T00:11:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:11:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609386+0000", 2024-09-15T00:11:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1619999999999998e-06 2024-09-15T00:11:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609392+0000", 2024-09-15T00:11:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-15T00:11:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609399+0000", 2024-09-15T00:11:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3570000000000003e-06 2024-09-15T00:11:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630147+0000", 2024-09-15T00:11:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020748195000000001 2024-09-15T00:11:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630201+0000", 2024-09-15T00:11:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3838000000000002e-05 2024-09-15T00:11:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630288+0000", 2024-09-15T00:11:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7484999999999993e-05 2024-09-15T00:11:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49030 3.b 3:d0a51340:::rbd_data.113916cf513.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.662376+0000", 2024-09-15T00:11:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.96844989100000001, 2024-09-15T00:11:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.028039436000000001, 2024-09-15T00:11:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49030 2024-09-15T00:11:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.662376+0000", 2024-09-15T00:11:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.662376+0000", 2024-09-15T00:11:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.662379+0000", 2024-09-15T00:11:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.168e-06 2024-09-15T00:11:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663508+0000", 2024-09-15T00:11:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011290370000000001 2024-09-15T00:11:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663539+0000", 2024-09-15T00:11:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1006000000000001e-05 2024-09-15T00:11:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663551+0000", 2024-09-15T00:11:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1708e-05 2024-09-15T00:11:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663628+0000", 2024-09-15T00:11:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6633999999999998e-05 2024-09-15T00:11:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663783+0000", 2024-09-15T00:11:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015484099999999999 2024-09-15T00:11:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.664331+0000", 2024-09-15T00:11:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00054832600000000002 2024-09-15T00:11:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.690059+0000", 2024-09-15T00:11:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025727612 2024-09-15T00:11:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.690300+0000", 2024-09-15T00:11:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024122600000000001 2024-09-15T00:11:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.690360+0000", 2024-09-15T00:11:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0458000000000003e-05 2024-09-15T00:11:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.690416+0000", 2024-09-15T00:11:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5420000000000001e-05 2024-09-15T00:11:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:49026 3.19 e47/23 3:9c3e88bf:::rbd_data.113916cf513.0000000000000002:head v 47'92, mlcod=45'89)", 2024-09-15T00:11:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.665261+0000", 2024-09-15T00:11:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.96556483500000001, 2024-09-15T00:11:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019832186000000002, 2024-09-15T00:11:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.665261+0000", 2024-09-15T00:11:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.665261+0000", 2024-09-15T00:11:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.665265+0000", 2024-09-15T00:11:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4570000000000001e-06 2024-09-15T00:11:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667479+0000", 2024-09-15T00:11:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022139260000000002 2024-09-15T00:11:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667492+0000", 2024-09-15T00:11:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3084e-05 2024-09-15T00:11:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667500+0000", 2024-09-15T00:11:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7609999999999996e-06 2024-09-15T00:11:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667569+0000", 2024-09-15T00:11:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8359000000000001e-05 2024-09-15T00:11:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667607+0000", 2024-09-15T00:11:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7716999999999998e-05 2024-09-15T00:11:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.685060+0000", 2024-09-15T00:11:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017453344999999999 2024-09-15T00:11:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.685093+0000", 2024-09-15T00:11:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3537000000000001e-05 2024-09-15T00:11:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:45.724 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:45.726 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:45.726 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:45.724+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:45.785 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:11:45.826 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:45.824+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:45.834 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:45.834 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:45.834 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:45.834 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:45.843 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:45.905 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:45.906 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:45.922 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:45.923 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:45.927 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:45.925+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:45.938 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:45.938 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:45.954 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:45.954 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:45.970 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:45.970 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:45.987 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:45.987 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:45.988 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:45.988 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:45.996 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:46.005 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:46.005 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:46.021 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:46.022 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:46.038 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:46.036+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:46.112 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.6 to 1 (10000) 2024-09-15T00:11:46.126 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-15T00:11:46.128 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:46.126+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:46.144 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:46.144 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:46.154 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:46.229 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:46.227+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:46.258 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:46.258 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:46.267 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:46.330 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:46.328+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:46.362 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-15T00:11:46.427 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:46.427 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:46.431 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:46.429+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:46.436 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:46.474 INFO:teuthology.orchestra.run.smithi171.stdout:1842 write 0x5f8f503 thru 0x5f99573 (0xa071 bytes) 2024-09-15T00:11:46.504 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:46.504 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:46.504 INFO:teuthology.orchestra.run.smithi171.stdout:1848 writesame 0x757c97a thru 0x758144a data_size 0x1(0x4ad1 bytes) 2024-09-15T00:11:46.532 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:46.530+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:46.540 INFO:teuthology.orchestra.run.smithi171.stdout:1849 read 0x1ac2d5 thru 0x1ad0fb (0xe27 bytes) 2024-09-15T00:11:46.552 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:46.552 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:46.552 INFO:teuthology.orchestra.run.smithi171.stdout:1850 trunc from 0xd242d32 to 0x3b0fe29 2024-09-15T00:11:46.562 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:11:46.593 INFO:teuthology.orchestra.run.smithi171.stdout:1851 writesame 0x481b779 thru 0x481cab4 data_size 0x2(0x133c bytes) 2024-09-15T00:11:46.593 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:46.602 INFO:teuthology.orchestra.run.smithi171.stdout:1853 write 0x790a9ee thru 0x79137c7 (0x8dda bytes) 2024-09-15T00:11:46.607 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:46.607 INFO:teuthology.orchestra.run.smithi171.stdout:1855 read 0x5aaee1f thru 0x5aaf7e9 (0x9cb bytes) 2024-09-15T00:11:46.607 INFO:teuthology.orchestra.run.smithi171.stdout:1856 write 0x74c4b0f thru 0x74d4346 (0xf838 bytes) 2024-09-15T00:11:46.626 INFO:teuthology.orchestra.run.smithi171.stdout:1860 write 0xe3c6086 thru 0xe3ca8c8 (0x4843 bytes) 2024-09-15T00:11:46.629 INFO:teuthology.orchestra.run.smithi171.stdout:1861 write 0xb72d06 thru 0xb817bc (0xeab7 bytes) 2024-09-15T00:11:46.633 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:46.631+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:46.670 INFO:teuthology.orchestra.run.smithi171.stdout:1862 read 0x22d7313 thru 0x22ddda2 (0x6a90 bytes) 2024-09-15T00:11:46.671 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:46.671 INFO:teuthology.orchestra.run.smithi171.stdout:1866 write 0xa3edf86 thru 0xa3f9eb4 (0xbf2f bytes) 2024-09-15T00:11:46.676 INFO:teuthology.orchestra.run.smithi171.stdout:1867 read 0x10e1e2b thru 0x10eee16 (0xcfec bytes) 2024-09-15T00:11:46.677 INFO:teuthology.orchestra.run.smithi171.stdout:1869 writesame 0x9c3d449 thru 0x9c3f6cf data_size 0x1(0x2287 bytes) 2024-09-15T00:11:46.686 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:46.686 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:46.687 INFO:teuthology.orchestra.run.smithi171.stdout:1870 trunc from 0xe3ca8c9 to 0xa59963c 2024-09-15T00:11:46.695 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:11:46.699 INFO:teuthology.orchestra.run.smithi171.stdout:1871 punch from 0x5d2be9c to 0x5d32179, (0x62dd bytes) 2024-09-15T00:11:46.699 INFO:teuthology.orchestra.run.smithi171.stdout:1872 read 0x3f3d32a thru 0x3f46e4c (0x9b23 bytes) 2024-09-15T00:11:46.699 INFO:teuthology.orchestra.run.smithi171.stdout:1875 write 0x98e65fd thru 0x98e7cbd (0x16c1 bytes) 2024-09-15T00:11:46.709 INFO:teuthology.orchestra.run.smithi171.stdout:1876 trunc from 0xa59963c to 0x45b09ec 2024-09-15T00:11:46.737 INFO:teuthology.orchestra.run.smithi171.stdout:1878 writesame 0xd37f12b thru 0xd38a1c7 data_size 0x7(0xb09d bytes) 2024-09-15T00:11:46.737 INFO:teuthology.orchestra.run.smithi171.stdout:1879 write 0xc100b07 thru 0xc110096 (0xf590 bytes) 2024-09-15T00:11:46.740 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:46.739+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:46.746 INFO:teuthology.orchestra.run.smithi171.stdout:1880 write 0x9211af1 thru 0x921de86 (0xc396 bytes) 2024-09-15T00:11:46.756 INFO:teuthology.orchestra.run.smithi171.stdout:1882 write 0x11ba620 thru 0x11bcd55 (0x2736 bytes) 2024-09-15T00:11:46.769 INFO:teuthology.orchestra.run.smithi171.stdout:1884 read 0x9674bf0 thru 0x96753a6 (0x7b7 bytes) 2024-09-15T00:11:46.769 INFO:teuthology.orchestra.run.smithi171.stdout:1888 read 0x190eb7e thru 0x191d1e4 (0xe667 bytes) 2024-09-15T00:11:46.770 INFO:teuthology.orchestra.run.smithi171.stdout:1889 write 0xb82d346 thru 0xb83aa7f (0xd73a bytes) 2024-09-15T00:11:46.780 INFO:teuthology.orchestra.run.smithi171.stdout:1891 write 0x2a80cba thru 0x2a8b2ca (0xa611 bytes) 2024-09-15T00:11:46.811 INFO:teuthology.orchestra.run.smithi171.stdout:1892 trunc from 0xd38a1c8 to 0x2d49f11 2024-09-15T00:11:46.811 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:46.811 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:46.823 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:11:46.834 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:46.833+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:46.845 INFO:teuthology.orchestra.run.smithi171.stdout:1893 writesame 0xc1aede1 thru 0xc1bcce7 data_size 0x13(0xdf07 bytes) 2024-09-15T00:11:46.851 INFO:teuthology.orchestra.run.smithi171.stdout:1894 write 0xdaf11f5 thru 0xdaf42c2 (0x30ce bytes) 2024-09-15T00:11:46.858 INFO:teuthology.orchestra.run.smithi171.stdout:1895 writesame 0xda92c96 thru 0xda9a96c data_size 0x9(0x7cd7 bytes) 2024-09-15T00:11:46.869 INFO:teuthology.orchestra.run.smithi171.stdout:1896 trunc from 0xdaf42c3 to 0x7995236 2024-09-15T00:11:46.880 INFO:teuthology.orchestra.run.smithi171.stdout:1897 trunc from 0x7995236 to 0x8a4ef88 2024-09-15T00:11:46.885 INFO:teuthology.orchestra.run.smithi171.stdout:1899 read 0x3dc1d96 thru 0x3dc8602 (0x686d bytes) 2024-09-15T00:11:46.885 INFO:teuthology.orchestra.run.smithi171.stdout:1900 read 0x3a8aff6 thru 0x3a8f7b1 (0x47bc bytes) 2024-09-15T00:11:46.885 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:46.885 INFO:teuthology.orchestra.run.smithi171.stdout:1902 read 0x52c15a4 thru 0x52c8ca0 (0x76fd bytes) 2024-09-15T00:11:46.885 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:46.894 INFO:teuthology.orchestra.run.smithi171.stdout:1904 write 0xca1a7ae thru 0xca21459 (0x6cac bytes) 2024-09-15T00:11:46.899 INFO:teuthology.orchestra.run.smithi171.stdout:1905 read 0xba9424a thru 0xba9cdd2 (0x8b89 bytes) 2024-09-15T00:11:46.899 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:46.899 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:46.899 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:46.899 INFO:teuthology.orchestra.run.smithi171.stdout:1909 read 0x4d871ff thru 0x4d8ca15 (0x5817 bytes) 2024-09-15T00:11:46.899 INFO:teuthology.orchestra.run.smithi171.stdout:1910 punch from 0x19aaabe to 0x19b37a8, (0x8cea bytes) 2024-09-15T00:11:46.899 INFO:teuthology.orchestra.run.smithi171.stdout:1911 clone 14 order 22 su 1048576 sc 5 2024-09-15T00:11:46.935 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:46.934+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:46.967 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:46.967 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:46.976 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:11:47.036 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:47.035+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:47.113 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.7 to 1 (10000) 2024-09-15T00:11:47.120 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:47.120 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:47.129 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T00:11:47.129 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:11:47.131 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:11:47.172 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone13 from 0xca2145a (overlap 0x2d49f11) to 0x48764e 2024-09-15T00:11:47.176 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:47.173+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:47.215 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:11:47.260 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:47.258+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:47.266 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:47.266 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:47.278 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:11:47.315 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-15T00:11:47.338 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:47.337+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:47.422 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:47.422 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:47.431 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:47.475 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:47.474+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:47.527 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:11:47.528 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:11:47.540 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-15T00:11:47.113200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":438650011,"num_objects":723,"num_object_clones":237,"num_object_copies":1446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":723,"num_whiteouts":328,"num_read":62263,"num_read_kb":214823,"num_write":5936,"num_write_kb":132355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":5861053,"num_keys_recovered":56,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4946,"ondisk_log_size":4946,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":223912,"kb_used_data":20108,"kb_used_omap":9,"kb_used_meta":203638,"kb_avail":566007128,"statfs":{"total":579820584960,"available":579591299072,"internally_reserved":0,"allocated":20590592,"data_stored":230894541,"data_compressed":3396081,"data_compressed_allocated":14336000,"data_compressed_original":226983631,"omap_allocated":9561,"internal_metadata":208525991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":256973818,"num_objects":479,"num_object_clones":162,"num_object_copies":958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":479,"num_whiteouts":284,"num_read":34538,"num_read_kb":130296,"num_write":2056,"num_write_kb":75095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.028104"},"pg_stats":[{"pgid":"3.1f","version":"45'91","reported_seq":1633,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.092728+0000","last_change":"2024-09-15T00:11:37.280007+0000","last_active":"2024-09-15T00:11:44.092728+0000","last_peered":"2024-09-15T00:11:44.092728+0000","last_clean":"2024-09-15T00:11:44.092728+0000","last_became_active":"2024-09-15T00:11:37.236921+0000","last_became_peered":"2024-09-15T00:11:37.236921+0000","last_unstale":"2024-09-15T00:11:44.092728+0000","last_undegraded":"2024-09-15T00:11:44.092728+0000","last_fullsized":"2024-09-15T00:11:44.092728+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:55:58.976122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012175910000000001,"stat_sum":{"num_bytes":8456510,"num_objects":25,"num_object_clones":10,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":1471,"num_read_kb":7916,"num_write":80,"num_write_kb":2945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2779327,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"47'310","reported_seq":2389,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.526711+0000","last_change":"2024-09-15T00:11:21.594579+0000","last_active":"2024-09-15T00:11:44.526711+0000","last_peered":"2024-09-15T00:11:44.526711+0000","last_clean":"2024-09-15T00:11:44.526711+0000","last_became_active":"2024-09-15T00:11:19.598640+0000","last_became_peered":"2024-09-15T00:11:19.598640+0000","last_unstale":"2024-09-15T00:11:44.526711+0000","last_undegraded":"2024-09-15T00:11:44.526711+0000","last_fullsized":"2024-09-15T00:11:44.526711+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:22:07.222651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065049699999999999,"stat_sum":{"num_bytes":24845748,"num_objects":35,"num_object_clones":13,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":3134,"num_read_kb":9582,"num_write":467,"num_write_kb":6173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"46'68","reported_seq":972,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.092725+0000","last_change":"2024-09-15T00:11:21.603457+0000","last_active":"2024-09-15T00:11:44.092725+0000","last_peered":"2024-09-15T00:11:44.092725+0000","last_clean":"2024-09-15T00:11:44.092725+0000","last_became_active":"2024-09-15T00:11:19.595954+0000","last_became_peered":"2024-09-15T00:11:19.595954+0000","last_unstale":"2024-09-15T00:11:44.092725+0000","last_undegraded":"2024-09-15T00:11:44.092725+0000","last_fullsized":"2024-09-15T00:11:44.092725+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:51:19.403323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017890510000000001,"stat_sum":{"num_bytes":10020915,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":852,"num_read_kb":7479,"num_write":61,"num_write_kb":4225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"47'76","reported_seq":1120,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.308309+0000","last_change":"2024-09-15T00:11:21.602612+0000","last_active":"2024-09-15T00:11:44.308309+0000","last_peered":"2024-09-15T00:11:44.308309+0000","last_clean":"2024-09-15T00:11:44.308309+0000","last_became_active":"2024-09-15T00:11:19.602583+0000","last_became_peered":"2024-09-15T00:11:19.602583+0000","last_unstale":"2024-09-15T00:11:44.308309+0000","last_undegraded":"2024-09-15T00:11:44.308309+0000","last_fullsized":"2024-09-15T00:11:44.308309+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:10:23.534910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011501599999999999,"stat_sum":{"num_bytes":8319040,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":7,"num_read":1052,"num_read_kb":3323,"num_write":75,"num_write_kb":401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"46'104","reported_seq":3062,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T00:11:45.111792+0000","last_change":"2024-09-15T00:11:25.010259+0000","last_active":"2024-09-15T00:11:45.111792+0000","last_peered":"2024-09-15T00:11:45.111792+0000","last_clean":"2024-09-15T00:11:45.111792+0000","last_became_active":"2024-09-15T00:11:25.002839+0000","last_became_peered":"2024-09-15T00:11:25.002839+0000","last_unstale":"2024-09-15T00:11:45.111792+0000","last_undegraded":"2024-09-15T00:11:45.111792+0000","last_fullsized":"2024-09-15T00:11:45.111792+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:56:34.435479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074638899999999997,"stat_sum":{"num_bytes":11308867,"num_objects":31,"num_object_clones":12,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":15,"num_read":2930,"num_read_kb":11281,"num_write":88,"num_write_kb":3006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"47'191","reported_seq":2549,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T00:11:45.292940+0000","last_change":"2024-09-15T00:11:21.595070+0000","last_active":"2024-09-15T00:11:45.292940+0000","last_peered":"2024-09-15T00:11:45.292940+0000","last_clean":"2024-09-15T00:11:45.292940+0000","last_became_active":"2024-09-15T00:11:19.600452+0000","last_became_peered":"2024-09-15T00:11:19.600452+0000","last_unstale":"2024-09-15T00:11:45.292940+0000","last_undegraded":"2024-09-15T00:11:45.292940+0000","last_fullsized":"2024-09-15T00:11:45.292940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:00:15.375072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048699700000000003,"stat_sum":{"num_bytes":54326418,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":6,"num_read":3400,"num_read_kb":23253,"num_write":214,"num_write_kb":19892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"45'91","reported_seq":435,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.098084+0000","last_change":"2024-09-15T00:11:21.595707+0000","last_active":"2024-09-15T00:11:44.098084+0000","last_peered":"2024-09-15T00:11:44.098084+0000","last_clean":"2024-09-15T00:11:44.098084+0000","last_became_active":"2024-09-15T00:11:19.595311+0000","last_became_peered":"2024-09-15T00:11:19.595311+0000","last_unstale":"2024-09-15T00:11:44.098084+0000","last_undegraded":"2024-09-15T00:11:44.098084+0000","last_fullsized":"2024-09-15T00:11:44.098084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:19:38.818816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015502809999999999,"stat_sum":{"num_bytes":3074765,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":10,"num_read":305,"num_read_kb":1980,"num_write":87,"num_write_kb":1074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"45'189","reported_seq":2107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.096342+0000","last_change":"2024-09-15T00:11:25.014633+0000","last_active":"2024-09-15T00:11:44.096342+0000","last_peered":"2024-09-15T00:11:44.096342+0000","last_clean":"2024-09-15T00:11:44.096342+0000","last_became_active":"2024-09-15T00:11:25.006382+0000","last_became_peered":"2024-09-15T00:11:25.006382+0000","last_unstale":"2024-09-15T00:11:44.096342+0000","last_undegraded":"2024-09-15T00:11:44.096342+0000","last_fullsized":"2024-09-15T00:11:44.096342+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:13:08.720593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081289999999999997,"stat_sum":{"num_bytes":11347362,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":13,"num_read":2492,"num_read_kb":6424,"num_write":269,"num_write_kb":860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":52356,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"47'164","reported_seq":1732,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.225473+0000","last_change":"2024-09-15T00:11:25.011470+0000","last_active":"2024-09-15T00:11:44.225473+0000","last_peered":"2024-09-15T00:11:44.225473+0000","last_clean":"2024-09-15T00:11:44.225473+0000","last_became_active":"2024-09-15T00:11:25.011058+0000","last_became_peered":"2024-09-15T00:11:25.011058+0000","last_unstale":"2024-09-15T00:11:44.225473+0000","last_undegraded":"2024-09-15T00:11:44.225473+0000","last_fullsized":"2024-09-15T00:11:44.225473+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:46:07.728585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00163887,"stat_sum":{"num_bytes":36380221,"num_objects":26,"num_object_clones":11,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":9,"num_read":2228,"num_read_kb":2556,"num_write":196,"num_write_kb":11489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"45'320","reported_seq":585,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.092711+0000","last_change":"2024-09-15T00:11:21.602933+0000","last_active":"2024-09-15T00:11:44.092711+0000","last_peered":"2024-09-15T00:11:44.092711+0000","last_clean":"2024-09-15T00:11:44.092711+0000","last_became_active":"2024-09-15T00:11:19.597034+0000","last_became_peered":"2024-09-15T00:11:19.597034+0000","last_unstale":"2024-09-15T00:11:44.092711+0000","last_undegraded":"2024-09-15T00:11:44.092711+0000","last_fullsized":"2024-09-15T00:11:44.092711+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:36:46.679816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014330530000000001,"stat_sum":{"num_bytes":3455201,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":11,"num_read":1526,"num_read_kb":1717,"num_write":488,"num_write_kb":1010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"47'136","reported_seq":505,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.099416+0000","last_change":"2024-09-15T00:11:21.613456+0000","last_active":"2024-09-15T00:11:44.099416+0000","last_peered":"2024-09-15T00:11:44.099416+0000","last_clean":"2024-09-15T00:11:44.099416+0000","last_became_active":"2024-09-15T00:11:19.597821+0000","last_became_peered":"2024-09-15T00:11:19.597821+0000","last_unstale":"2024-09-15T00:11:44.099416+0000","last_undegraded":"2024-09-15T00:11:44.099416+0000","last_fullsized":"2024-09-15T00:11:44.099416+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:55:45.484712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085960799999999999,"stat_sum":{"num_bytes":26667310,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":524,"num_read_kb":3756,"num_write":156,"num_write_kb":7756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"48'141","reported_seq":1839,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T00:11:45.315662+0000","last_change":"2024-09-15T00:11:21.595646+0000","last_active":"2024-09-15T00:11:45.315662+0000","last_peered":"2024-09-15T00:11:45.315662+0000","last_clean":"2024-09-15T00:11:45.315662+0000","last_became_active":"2024-09-15T00:11:19.600830+0000","last_became_peered":"2024-09-15T00:11:19.600830+0000","last_unstale":"2024-09-15T00:11:45.315662+0000","last_undegraded":"2024-09-15T00:11:45.315662+0000","last_fullsized":"2024-09-15T00:11:45.315662+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:46:59.199108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077114700000000002,"stat_sum":{"num_bytes":12888218,"num_objects":29,"num_object_clones":10,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":2075,"num_read_kb":8977,"num_write":127,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.181553+0000","last_change":"2024-09-15T00:11:11.641988+0000","last_active":"2024-09-15T00:11:44.181553+0000","last_peered":"2024-09-15T00:11:44.181553+0000","last_clean":"2024-09-15T00:11:44.181553+0000","last_became_active":"2024-09-15T00:11:09.534794+0000","last_became_peered":"2024-09-15T00:11:09.534794+0000","last_unstale":"2024-09-15T00:11:44.181553+0000","last_undegraded":"2024-09-15T00:11:44.181553+0000","last_fullsized":"2024-09-15T00:11:44.181553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:55:52.135410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"47'83","reported_seq":1789,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.182732+0000","last_change":"2024-09-15T00:11:25.006414+0000","last_active":"2024-09-15T00:11:44.182732+0000","last_peered":"2024-09-15T00:11:44.182732+0000","last_clean":"2024-09-15T00:11:44.182732+0000","last_became_active":"2024-09-15T00:11:25.006069+0000","last_became_peered":"2024-09-15T00:11:25.006069+0000","last_unstale":"2024-09-15T00:11:44.182732+0000","last_undegraded":"2024-09-15T00:11:44.182732+0000","last_fullsized":"2024-09-15T00:11:44.182732+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:46:37.184304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014370050000000001,"stat_sum":{"num_bytes":20935839,"num_objects":21,"num_object_clones":8,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":11,"num_read":1676,"num_read_kb":5732,"num_write":94,"num_write_kb":5739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.098066+0000","last_change":"2024-09-15T00:11:11.545630+0000","last_active":"2024-09-15T00:11:44.098066+0000","last_peered":"2024-09-15T00:11:44.098066+0000","last_clean":"2024-09-15T00:11:44.098066+0000","last_became_active":"2024-09-15T00:11:09.536615+0000","last_became_peered":"2024-09-15T00:11:09.536615+0000","last_unstale":"2024-09-15T00:11:44.098066+0000","last_undegraded":"2024-09-15T00:11:44.098066+0000","last_fullsized":"2024-09-15T00:11:44.098066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:51:38.340257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035555,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"47'143","reported_seq":623,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.096833+0000","last_change":"2024-09-15T00:11:21.594822+0000","last_active":"2024-09-15T00:11:44.096833+0000","last_peered":"2024-09-15T00:11:44.096833+0000","last_clean":"2024-09-15T00:11:44.096833+0000","last_became_active":"2024-09-15T00:11:19.596547+0000","last_became_peered":"2024-09-15T00:11:19.596547+0000","last_unstale":"2024-09-15T00:11:44.096833+0000","last_undegraded":"2024-09-15T00:11:44.096833+0000","last_fullsized":"2024-09-15T00:11:44.096833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:17:32.416494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058512600000000005,"stat_sum":{"num_bytes":14717462,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":12,"num_read":678,"num_read_kb":2174,"num_write":177,"num_write_kb":4512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.099457+0000","last_change":"2024-09-15T00:11:25.003710+0000","last_active":"2024-09-15T00:11:44.099457+0000","last_peered":"2024-09-15T00:11:44.099457+0000","last_clean":"2024-09-15T00:11:44.099457+0000","last_became_active":"2024-09-15T00:11:25.003420+0000","last_became_peered":"2024-09-15T00:11:25.003420+0000","last_unstale":"2024-09-15T00:11:44.099457+0000","last_undegraded":"2024-09-15T00:11:44.099457+0000","last_fullsized":"2024-09-15T00:11:44.099457+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:12:12.038285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"45'180","reported_seq":6409,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.178571+0000","last_change":"2024-09-15T00:11:40.073377+0000","last_active":"2024-09-15T00:11:44.178571+0000","last_peered":"2024-09-15T00:11:44.178571+0000","last_clean":"2024-09-15T00:11:44.178571+0000","last_became_active":"2024-09-15T00:11:40.072803+0000","last_became_peered":"2024-09-15T00:11:40.072803+0000","last_unstale":"2024-09-15T00:11:44.178571+0000","last_undegraded":"2024-09-15T00:11:44.178571+0000","last_fullsized":"2024-09-15T00:11:44.178571+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:31:32.018193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001489752,"stat_sum":{"num_bytes":22961456,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":7909,"num_read_kb":19637,"num_write":175,"num_write_kb":4464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":482924,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.098144+0000","last_change":"2024-09-15T00:11:25.006347+0000","last_active":"2024-09-15T00:11:44.098144+0000","last_peered":"2024-09-15T00:11:44.098144+0000","last_clean":"2024-09-15T00:11:44.098144+0000","last_became_active":"2024-09-15T00:11:25.006068+0000","last_became_peered":"2024-09-15T00:11:25.006068+0000","last_unstale":"2024-09-15T00:11:44.098144+0000","last_undegraded":"2024-09-15T00:11:44.098144+0000","last_fullsized":"2024-09-15T00:11:44.098144+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:53:52.316019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"46'84","reported_seq":1527,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.156156+0000","last_change":"2024-09-15T00:11:21.603338+0000","last_active":"2024-09-15T00:11:44.156156+0000","last_peered":"2024-09-15T00:11:44.156156+0000","last_clean":"2024-09-15T00:11:44.156156+0000","last_became_active":"2024-09-15T00:11:19.595633+0000","last_became_peered":"2024-09-15T00:11:19.595633+0000","last_unstale":"2024-09-15T00:11:44.156156+0000","last_undegraded":"2024-09-15T00:11:44.156156+0000","last_fullsized":"2024-09-15T00:11:44.156156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:08:17.739592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017065330000000001,"stat_sum":{"num_bytes":5837178,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":1974,"num_read_kb":5624,"num_write":76,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"48'132","reported_seq":1477,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T00:11:45.311227+0000","last_change":"2024-09-15T00:11:24.999261+0000","last_active":"2024-09-15T00:11:45.311227+0000","last_peered":"2024-09-15T00:11:45.311227+0000","last_clean":"2024-09-15T00:11:45.311227+0000","last_became_active":"2024-09-15T00:11:24.998815+0000","last_became_peered":"2024-09-15T00:11:24.998815+0000","last_unstale":"2024-09-15T00:11:45.311227+0000","last_undegraded":"2024-09-15T00:11:45.311227+0000","last_fullsized":"2024-09-15T00:11:45.311227+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:32:51.945393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075923799999999997,"stat_sum":{"num_bytes":30775155,"num_objects":26,"num_object_clones":10,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":1335,"num_read_kb":10247,"num_write":162,"num_write_kb":17020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.092552+0000","last_change":"2024-09-15T00:11:11.543519+0000","last_active":"2024-09-15T00:11:44.092552+0000","last_peered":"2024-09-15T00:11:44.092552+0000","last_clean":"2024-09-15T00:11:44.092552+0000","last_became_active":"2024-09-15T00:11:09.533896+0000","last_became_peered":"2024-09-15T00:11:09.533896+0000","last_unstale":"2024-09-15T00:11:44.092552+0000","last_undegraded":"2024-09-15T00:11:44.092552+0000","last_fullsized":"2024-09-15T00:11:44.092552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:47.613931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"45'120","reported_seq":2190,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.098179+0000","last_change":"2024-09-15T00:11:37.317374+0000","last_active":"2024-09-15T00:11:44.098179+0000","last_peered":"2024-09-15T00:11:44.098179+0000","last_clean":"2024-09-15T00:11:44.098179+0000","last_became_active":"2024-09-15T00:11:37.238849+0000","last_became_peered":"2024-09-15T00:11:37.238849+0000","last_unstale":"2024-09-15T00:11:44.098179+0000","last_undegraded":"2024-09-15T00:11:44.098179+0000","last_fullsized":"2024-09-15T00:11:44.098179+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:15:12.120162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241242,"stat_sum":{"num_bytes":8083351,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":2182,"num_read_kb":6320,"num_write":127,"num_write_kb":2714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1846940,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.096448+0000","last_change":"2024-09-15T00:11:37.053880+0000","last_active":"2024-09-15T00:11:44.096448+0000","last_peered":"2024-09-15T00:11:44.096448+0000","last_clean":"2024-09-15T00:11:44.096448+0000","last_became_active":"2024-09-15T00:11:37.053552+0000","last_became_peered":"2024-09-15T00:11:37.053552+0000","last_unstale":"2024-09-15T00:11:44.096448+0000","last_undegraded":"2024-09-15T00:11:44.096448+0000","last_fullsized":"2024-09-15T00:11:44.096448+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:05:10.783349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"45'75","reported_seq":1607,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T00:11:45.111774+0000","last_change":"2024-09-15T00:11:25.036338+0000","last_active":"2024-09-15T00:11:45.111774+0000","last_peered":"2024-09-15T00:11:45.111774+0000","last_clean":"2024-09-15T00:11:45.111774+0000","last_became_active":"2024-09-15T00:11:25.005078+0000","last_became_peered":"2024-09-15T00:11:25.005078+0000","last_unstale":"2024-09-15T00:11:45.111774+0000","last_undegraded":"2024-09-15T00:11:45.111774+0000","last_fullsized":"2024-09-15T00:11:45.111774+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:39:59.833365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069791600000000003,"stat_sum":{"num_bytes":10597996,"num_objects":19,"num_object_clones":8,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":1483,"num_read_kb":8679,"num_write":71,"num_write_kb":3928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":64634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":70,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.099164+0000","last_change":"2024-09-15T00:11:11.641995+0000","last_active":"2024-09-15T00:11:44.099164+0000","last_peered":"2024-09-15T00:11:44.099164+0000","last_clean":"2024-09-15T00:11:44.099164+0000","last_became_active":"2024-09-15T00:11:09.533404+0000","last_became_peered":"2024-09-15T00:11:09.533404+0000","last_unstale":"2024-09-15T00:11:44.099164+0000","last_undegraded":"2024-09-15T00:11:44.099164+0000","last_fullsized":"2024-09-15T00:11:44.099164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:20.611551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046024999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"45'174","reported_seq":1439,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.096566+0000","last_change":"2024-09-15T00:11:21.595240+0000","last_active":"2024-09-15T00:11:44.096566+0000","last_peered":"2024-09-15T00:11:44.096566+0000","last_clean":"2024-09-15T00:11:44.096566+0000","last_became_active":"2024-09-15T00:11:19.594528+0000","last_became_peered":"2024-09-15T00:11:19.594528+0000","last_unstale":"2024-09-15T00:11:44.096566+0000","last_undegraded":"2024-09-15T00:11:44.096566+0000","last_fullsized":"2024-09-15T00:11:44.096566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:11.673492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080299099999999997,"stat_sum":{"num_bytes":5307592,"num_objects":23,"num_object_clones":8,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":1644,"num_read_kb":4121,"num_write":215,"num_write_kb":1530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.092542+0000","last_change":"2024-09-15T00:11:37.055535+0000","last_active":"2024-09-15T00:11:44.092542+0000","last_peered":"2024-09-15T00:11:44.092542+0000","last_clean":"2024-09-15T00:11:44.092542+0000","last_became_active":"2024-09-15T00:11:37.052915+0000","last_became_peered":"2024-09-15T00:11:37.052915+0000","last_unstale":"2024-09-15T00:11:44.092542+0000","last_undegraded":"2024-09-15T00:11:44.092542+0000","last_fullsized":"2024-09-15T00:11:44.092542+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:57:39.917591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":234,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.099167+0000","last_change":"2024-09-15T00:11:28.008530+0000","last_active":"2024-09-15T00:11:44.099167+0000","last_peered":"2024-09-15T00:11:44.099167+0000","last_clean":"2024-09-15T00:11:44.099167+0000","last_became_active":"2024-09-15T00:11:28.008130+0000","last_became_peered":"2024-09-15T00:11:28.008130+0000","last_unstale":"2024-09-15T00:11:44.099167+0000","last_undegraded":"2024-09-15T00:11:44.099167+0000","last_fullsized":"2024-09-15T00:11:44.099167+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_clean_scrub_stamp":"2024-09-15T00:11:04.453672+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:09:32.444877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"47'132","reported_seq":519,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.184490+0000","last_change":"2024-09-15T00:11:21.595379+0000","last_active":"2024-09-15T00:11:44.184490+0000","last_peered":"2024-09-15T00:11:44.184490+0000","last_clean":"2024-09-15T00:11:44.184490+0000","last_became_active":"2024-09-15T00:11:19.594476+0000","last_became_peered":"2024-09-15T00:11:19.594476+0000","last_unstale":"2024-09-15T00:11:44.184490+0000","last_undegraded":"2024-09-15T00:11:44.184490+0000","last_fullsized":"2024-09-15T00:11:44.184490+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:57:23.730620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089807799999999996,"stat_sum":{"num_bytes":8217457,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":372,"num_read_kb":2608,"num_write":129,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"46'655","reported_seq":3698,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T00:11:45.111717+0000","last_change":"2024-09-15T00:11:40.070277+0000","last_active":"2024-09-15T00:11:45.111717+0000","last_peered":"2024-09-15T00:11:45.111717+0000","last_clean":"2024-09-15T00:11:45.111717+0000","last_became_active":"2024-09-15T00:11:40.069596+0000","last_became_peered":"2024-09-15T00:11:40.069596+0000","last_unstale":"2024-09-15T00:11:45.111717+0000","last_undegraded":"2024-09-15T00:11:45.111717+0000","last_fullsized":"2024-09-15T00:11:45.111717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:56:41.942031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053468500000000002,"stat_sum":{"num_bytes":8639898,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":7084,"num_read_kb":12915,"num_write":994,"num_write_kb":1735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":44504,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"47'82","reported_seq":1081,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.181598+0000","last_change":"2024-09-15T00:11:21.608019+0000","last_active":"2024-09-15T00:11:44.181598+0000","last_peered":"2024-09-15T00:11:44.181598+0000","last_clean":"2024-09-15T00:11:44.181598+0000","last_became_active":"2024-09-15T00:11:19.602625+0000","last_became_peered":"2024-09-15T00:11:19.602625+0000","last_unstale":"2024-09-15T00:11:44.181598+0000","last_undegraded":"2024-09-15T00:11:44.181598+0000","last_fullsized":"2024-09-15T00:11:44.181598+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:41:02.109366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036072699999999998,"stat_sum":{"num_bytes":6197972,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":986,"num_read_kb":3418,"num_write":74,"num_write_kb":616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"47'74","reported_seq":1375,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.286196+0000","last_change":"2024-09-15T00:11:25.000430+0000","last_active":"2024-09-15T00:11:44.286196+0000","last_peered":"2024-09-15T00:11:44.286196+0000","last_clean":"2024-09-15T00:11:44.286196+0000","last_became_active":"2024-09-15T00:11:25.000142+0000","last_became_peered":"2024-09-15T00:11:25.000142+0000","last_unstale":"2024-09-15T00:11:44.286196+0000","last_undegraded":"2024-09-15T00:11:44.286196+0000","last_fullsized":"2024-09-15T00:11:44.286196+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:07:21.428434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082326600000000002,"stat_sum":{"num_bytes":8024095,"num_objects":27,"num_object_clones":10,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":1277,"num_read_kb":9041,"num_write":65,"num_write_kb":4558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"46'158","reported_seq":1085,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.317603+0000","last_change":"2024-09-15T00:11:21.603031+0000","last_active":"2024-09-15T00:11:44.317603+0000","last_peered":"2024-09-15T00:11:44.317603+0000","last_clean":"2024-09-15T00:11:44.317603+0000","last_became_active":"2024-09-15T00:11:19.595088+0000","last_became_peered":"2024-09-15T00:11:19.595088+0000","last_unstale":"2024-09-15T00:11:44.317603+0000","last_undegraded":"2024-09-15T00:11:44.317603+0000","last_fullsized":"2024-09-15T00:11:44.317603+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:12:25.782482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537407,"stat_sum":{"num_bytes":2997351,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":7,"num_read":1999,"num_read_kb":2649,"num_write":143,"num_write_kb":623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"47'80","reported_seq":1304,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.099456+0000","last_change":"2024-09-15T00:11:21.613267+0000","last_active":"2024-09-15T00:11:44.099456+0000","last_peered":"2024-09-15T00:11:44.099456+0000","last_clean":"2024-09-15T00:11:44.099456+0000","last_became_active":"2024-09-15T00:11:19.596135+0000","last_became_peered":"2024-09-15T00:11:19.596135+0000","last_unstale":"2024-09-15T00:11:44.099456+0000","last_undegraded":"2024-09-15T00:11:44.099456+0000","last_fullsized":"2024-09-15T00:11:44.099456+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:48:32.676111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069298099999999996,"stat_sum":{"num_bytes":22118400,"num_objects":22,"num_object_clones":9,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":1192,"num_read_kb":6054,"num_write":65,"num_write_kb":924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"47'139","reported_seq":278,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.233665+0000","last_change":"2024-09-15T00:11:21.613225+0000","last_active":"2024-09-15T00:11:44.233665+0000","last_peered":"2024-09-15T00:11:44.233665+0000","last_clean":"2024-09-15T00:11:44.233665+0000","last_became_active":"2024-09-15T00:11:19.602353+0000","last_became_peered":"2024-09-15T00:11:19.602353+0000","last_unstale":"2024-09-15T00:11:44.233665+0000","last_undegraded":"2024-09-15T00:11:44.233665+0000","last_fullsized":"2024-09-15T00:11:44.233665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:52:37.163814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076259299999999999,"stat_sum":{"num_bytes":1668636,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":9,"num_read":345,"num_read_kb":1574,"num_write":170,"num_write_kb":1270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"47'93","reported_seq":2018,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.182194+0000","last_change":"2024-09-15T00:11:21.613002+0000","last_active":"2024-09-15T00:11:44.182194+0000","last_peered":"2024-09-15T00:11:44.182194+0000","last_clean":"2024-09-15T00:11:44.182194+0000","last_became_active":"2024-09-15T00:11:19.601589+0000","last_became_peered":"2024-09-15T00:11:19.601589+0000","last_unstale":"2024-09-15T00:11:44.182194+0000","last_undegraded":"2024-09-15T00:11:44.182194+0000","last_fullsized":"2024-09-15T00:11:44.182194+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:55.433644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055547799999999998,"stat_sum":{"num_bytes":19433718,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":1966,"num_read_kb":4168,"num_write":90,"num_write_kb":765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"45'101","reported_seq":1165,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.181593+0000","last_change":"2024-09-15T00:11:25.005455+0000","last_active":"2024-09-15T00:11:44.181593+0000","last_peered":"2024-09-15T00:11:44.181593+0000","last_clean":"2024-09-15T00:11:44.181593+0000","last_became_active":"2024-09-15T00:11:25.001267+0000","last_became_peered":"2024-09-15T00:11:25.001267+0000","last_unstale":"2024-09-15T00:11:44.181593+0000","last_undegraded":"2024-09-15T00:11:44.181593+0000","last_fullsized":"2024-09-15T00:11:44.181593+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:55:43.914739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093430300000000004,"stat_sum":{"num_bytes":9567290,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":1225,"num_read_kb":4269,"num_write":91,"num_write_kb":3617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"45'170","reported_seq":1326,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.099400+0000","last_change":"2024-09-15T00:11:21.612759+0000","last_active":"2024-09-15T00:11:44.099400+0000","last_peered":"2024-09-15T00:11:44.099400+0000","last_clean":"2024-09-15T00:11:44.099400+0000","last_became_active":"2024-09-15T00:11:19.601674+0000","last_became_peered":"2024-09-15T00:11:19.601674+0000","last_unstale":"2024-09-15T00:11:44.099400+0000","last_undegraded":"2024-09-15T00:11:44.099400+0000","last_fullsized":"2024-09-15T00:11:44.099400+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:20:50.546699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042917200000000002,"stat_sum":{"num_bytes":4720846,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":15,"num_read":1671,"num_read_kb":5616,"num_write":236,"num_write_kb":1072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"45'138","reported_seq":1297,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.099367+0000","last_change":"2024-09-15T00:11:25.011332+0000","last_active":"2024-09-15T00:11:44.099367+0000","last_peered":"2024-09-15T00:11:44.099367+0000","last_clean":"2024-09-15T00:11:44.099367+0000","last_became_active":"2024-09-15T00:11:25.010982+0000","last_became_peered":"2024-09-15T00:11:25.010982+0000","last_unstale":"2024-09-15T00:11:44.099367+0000","last_undegraded":"2024-09-15T00:11:44.099367+0000","last_fullsized":"2024-09-15T00:11:44.099367+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:27.347827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042453400000000001,"stat_sum":{"num_bytes":9665530,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":8,"num_read":1416,"num_read_kb":6073,"num_write":196,"num_write_kb":4212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"45'132","reported_seq":1470,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.311610+0000","last_change":"2024-09-15T00:11:21.612992+0000","last_active":"2024-09-15T00:11:44.311610+0000","last_peered":"2024-09-15T00:11:44.311610+0000","last_clean":"2024-09-15T00:11:44.311610+0000","last_became_active":"2024-09-15T00:11:19.602532+0000","last_became_peered":"2024-09-15T00:11:19.602532+0000","last_unstale":"2024-09-15T00:11:44.311610+0000","last_undegraded":"2024-09-15T00:11:44.311610+0000","last_fullsized":"2024-09-15T00:11:44.311610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:40:28.510789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062135599999999999,"stat_sum":{"num_bytes":6501827,"num_objects":23,"num_object_clones":8,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":1744,"num_read_kb":5560,"num_write":117,"num_write_kb":2712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":438059624,"num_objects":719,"num_object_clones":237,"num_object_copies":1438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":719,"num_whiteouts":328,"num_read":62147,"num_read_kb":214723,"num_write":5775,"num_write_kb":129937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":5270685,"num_keys_recovered":56,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18046976,"data_stored":293987379,"data_compressed":3924277,"data_compressed_allocated":14880768,"data_compressed_original":290923514,"omap_allocated":0,"internal_metadata":0},"log_size":4826,"ondisk_log_size":4826,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":17376,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40700,"kb_used_data":5356,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331140,"statfs":{"total":96636764160,"available":96595087360,"internally_reserved":0,"allocated":5484544,"data_stored":86206756,"data_compressed":1109592,"data_compressed_allocated":4206592,"data_compressed_original":85224277,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542157,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39312,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332528,"statfs":{"total":96636764160,"available":96596508672,"internally_reserved":0,"allocated":4063232,"data_stored":46694748,"data_compressed":611500,"data_compressed_allocated":2895872,"data_compressed_original":45942125,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37872,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333968,"statfs":{"total":96636764160,"available":96597983232,"internally_reserved":0,"allocated":2588672,"data_stored":22477189,"data_compressed":422522,"data_compressed_allocated":1654784,"data_compressed_original":21975040,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542157,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33560,"kb_used_data":2364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338280,"statfs":{"total":96636764160,"available":96602398720,"internally_reserved":0,"allocated":2420736,"data_stored":21330515,"data_compressed":378688,"data_compressed_allocated":1560576,"data_compressed_original":20878701,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38156,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333684,"statfs":{"total":96636764160,"available":96597692416,"internally_reserved":0,"allocated":2834432,"data_stored":24850289,"data_compressed":465385,"data_compressed_allocated":1818624,"data_compressed_original":24266912,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":60129542156,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542155,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34312,"kb_used_data":3124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337528,"statfs":{"total":96636764160,"available":96601628672,"internally_reserved":0,"allocated":3198976,"data_stored":29335044,"data_compressed":408394,"data_compressed_allocated":2199552,"data_compressed_original":28696576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":60129542156,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":117006,"data_compressed":207,"data_compressed_allocated":8192,"data_compressed_original":110592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2691072,"data_stored":48539244,"data_compressed":589673,"data_compressed_allocated":2203648,"data_compressed_original":47944373,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":542788,"data_compressed":51639,"data_compressed_allocated":77824,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4141056,"data_stored":67742655,"data_compressed":794054,"data_compressed_allocated":3424256,"data_compressed_original":67029722,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4890624,"data_stored":86391384,"data_compressed":1081363,"data_compressed_allocated":4210688,"data_compressed_original":85697265,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2314240,"data_stored":34130101,"data_compressed":524318,"data_compressed_allocated":1818624,"data_compressed_original":33663136,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1859584,"data_stored":30046643,"data_compressed":451153,"data_compressed_allocated":1523712,"data_compressed_original":29760218,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1998848,"data_stored":26477558,"data_compressed":431870,"data_compressed_allocated":1613824,"data_compressed_original":26193920,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:11:47.542 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:11:47.543 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:47.538+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:47.543 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:47.543 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:47.550 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:11:47.553 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:47.554 INFO:teuthology.orchestra.run.smithi171.stdout:1913 clone 15 order 18 su 131072 sc 7 2024-09-15T00:11:47.641 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:47.639+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:47.658 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:47.659 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:47.668 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:11:47.741 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:47.740+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:47.773 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:47.773 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:47.782 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:47.836 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:11:47.837 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:11:47.843 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:47.841+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:47.848 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-15T00:11:47.113200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":438650011,"num_objects":723,"num_object_clones":237,"num_object_copies":1446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":723,"num_whiteouts":328,"num_read":62263,"num_read_kb":214823,"num_write":5936,"num_write_kb":132355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":5861053,"num_keys_recovered":56,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4946,"ondisk_log_size":4946,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":223912,"kb_used_data":20108,"kb_used_omap":9,"kb_used_meta":203638,"kb_avail":566007128,"statfs":{"total":579820584960,"available":579591299072,"internally_reserved":0,"allocated":20590592,"data_stored":230894541,"data_compressed":3396081,"data_compressed_allocated":14336000,"data_compressed_original":226983631,"omap_allocated":9561,"internal_metadata":208525991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":256973818,"num_objects":479,"num_object_clones":162,"num_object_copies":958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":479,"num_whiteouts":284,"num_read":34538,"num_read_kb":130296,"num_write":2056,"num_write_kb":75095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.028104"},"pg_stats":[{"pgid":"3.1f","version":"45'91","reported_seq":1633,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.092728+0000","last_change":"2024-09-15T00:11:37.280007+0000","last_active":"2024-09-15T00:11:44.092728+0000","last_peered":"2024-09-15T00:11:44.092728+0000","last_clean":"2024-09-15T00:11:44.092728+0000","last_became_active":"2024-09-15T00:11:37.236921+0000","last_became_peered":"2024-09-15T00:11:37.236921+0000","last_unstale":"2024-09-15T00:11:44.092728+0000","last_undegraded":"2024-09-15T00:11:44.092728+0000","last_fullsized":"2024-09-15T00:11:44.092728+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:55:58.976122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012175910000000001,"stat_sum":{"num_bytes":8456510,"num_objects":25,"num_object_clones":10,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":1471,"num_read_kb":7916,"num_write":80,"num_write_kb":2945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2779327,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"47'310","reported_seq":2389,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.526711+0000","last_change":"2024-09-15T00:11:21.594579+0000","last_active":"2024-09-15T00:11:44.526711+0000","last_peered":"2024-09-15T00:11:44.526711+0000","last_clean":"2024-09-15T00:11:44.526711+0000","last_became_active":"2024-09-15T00:11:19.598640+0000","last_became_peered":"2024-09-15T00:11:19.598640+0000","last_unstale":"2024-09-15T00:11:44.526711+0000","last_undegraded":"2024-09-15T00:11:44.526711+0000","last_fullsized":"2024-09-15T00:11:44.526711+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:22:07.222651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065049699999999999,"stat_sum":{"num_bytes":24845748,"num_objects":35,"num_object_clones":13,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":3134,"num_read_kb":9582,"num_write":467,"num_write_kb":6173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"46'68","reported_seq":972,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.092725+0000","last_change":"2024-09-15T00:11:21.603457+0000","last_active":"2024-09-15T00:11:44.092725+0000","last_peered":"2024-09-15T00:11:44.092725+0000","last_clean":"2024-09-15T00:11:44.092725+0000","last_became_active":"2024-09-15T00:11:19.595954+0000","last_became_peered":"2024-09-15T00:11:19.595954+0000","last_unstale":"2024-09-15T00:11:44.092725+0000","last_undegraded":"2024-09-15T00:11:44.092725+0000","last_fullsized":"2024-09-15T00:11:44.092725+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:51:19.403323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017890510000000001,"stat_sum":{"num_bytes":10020915,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":852,"num_read_kb":7479,"num_write":61,"num_write_kb":4225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"47'76","reported_seq":1120,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.308309+0000","last_change":"2024-09-15T00:11:21.602612+0000","last_active":"2024-09-15T00:11:44.308309+0000","last_peered":"2024-09-15T00:11:44.308309+0000","last_clean":"2024-09-15T00:11:44.308309+0000","last_became_active":"2024-09-15T00:11:19.602583+0000","last_became_peered":"2024-09-15T00:11:19.602583+0000","last_unstale":"2024-09-15T00:11:44.308309+0000","last_undegraded":"2024-09-15T00:11:44.308309+0000","last_fullsized":"2024-09-15T00:11:44.308309+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:10:23.534910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011501599999999999,"stat_sum":{"num_bytes":8319040,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":7,"num_read":1052,"num_read_kb":3323,"num_write":75,"num_write_kb":401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"46'104","reported_seq":3062,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T00:11:45.111792+0000","last_change":"2024-09-15T00:11:25.010259+0000","last_active":"2024-09-15T00:11:45.111792+0000","last_peered":"2024-09-15T00:11:45.111792+0000","last_clean":"2024-09-15T00:11:45.111792+0000","last_became_active":"2024-09-15T00:11:25.002839+0000","last_became_peered":"2024-09-15T00:11:25.002839+0000","last_unstale":"2024-09-15T00:11:45.111792+0000","last_undegraded":"2024-09-15T00:11:45.111792+0000","last_fullsized":"2024-09-15T00:11:45.111792+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:56:34.435479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074638899999999997,"stat_sum":{"num_bytes":11308867,"num_objects":31,"num_object_clones":12,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":15,"num_read":2930,"num_read_kb":11281,"num_write":88,"num_write_kb":3006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"47'191","reported_seq":2549,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T00:11:45.292940+0000","last_change":"2024-09-15T00:11:21.595070+0000","last_active":"2024-09-15T00:11:45.292940+0000","last_peered":"2024-09-15T00:11:45.292940+0000","last_clean":"2024-09-15T00:11:45.292940+0000","last_became_active":"2024-09-15T00:11:19.600452+0000","last_became_peered":"2024-09-15T00:11:19.600452+0000","last_unstale":"2024-09-15T00:11:45.292940+0000","last_undegraded":"2024-09-15T00:11:45.292940+0000","last_fullsized":"2024-09-15T00:11:45.292940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:00:15.375072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048699700000000003,"stat_sum":{"num_bytes":54326418,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":6,"num_read":3400,"num_read_kb":23253,"num_write":214,"num_write_kb":19892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"45'91","reported_seq":435,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.098084+0000","last_change":"2024-09-15T00:11:21.595707+0000","last_active":"2024-09-15T00:11:44.098084+0000","last_peered":"2024-09-15T00:11:44.098084+0000","last_clean":"2024-09-15T00:11:44.098084+0000","last_became_active":"2024-09-15T00:11:19.595311+0000","last_became_peered":"2024-09-15T00:11:19.595311+0000","last_unstale":"2024-09-15T00:11:44.098084+0000","last_undegraded":"2024-09-15T00:11:44.098084+0000","last_fullsized":"2024-09-15T00:11:44.098084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:19:38.818816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015502809999999999,"stat_sum":{"num_bytes":3074765,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":10,"num_read":305,"num_read_kb":1980,"num_write":87,"num_write_kb":1074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"45'189","reported_seq":2107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.096342+0000","last_change":"2024-09-15T00:11:25.014633+0000","last_active":"2024-09-15T00:11:44.096342+0000","last_peered":"2024-09-15T00:11:44.096342+0000","last_clean":"2024-09-15T00:11:44.096342+0000","last_became_active":"2024-09-15T00:11:25.006382+0000","last_became_peered":"2024-09-15T00:11:25.006382+0000","last_unstale":"2024-09-15T00:11:44.096342+0000","last_undegraded":"2024-09-15T00:11:44.096342+0000","last_fullsized":"2024-09-15T00:11:44.096342+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:13:08.720593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081289999999999997,"stat_sum":{"num_bytes":11347362,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":13,"num_read":2492,"num_read_kb":6424,"num_write":269,"num_write_kb":860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":52356,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"47'164","reported_seq":1732,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.225473+0000","last_change":"2024-09-15T00:11:25.011470+0000","last_active":"2024-09-15T00:11:44.225473+0000","last_peered":"2024-09-15T00:11:44.225473+0000","last_clean":"2024-09-15T00:11:44.225473+0000","last_became_active":"2024-09-15T00:11:25.011058+0000","last_became_peered":"2024-09-15T00:11:25.011058+0000","last_unstale":"2024-09-15T00:11:44.225473+0000","last_undegraded":"2024-09-15T00:11:44.225473+0000","last_fullsized":"2024-09-15T00:11:44.225473+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:46:07.728585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00163887,"stat_sum":{"num_bytes":36380221,"num_objects":26,"num_object_clones":11,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":9,"num_read":2228,"num_read_kb":2556,"num_write":196,"num_write_kb":11489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"45'320","reported_seq":585,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.092711+0000","last_change":"2024-09-15T00:11:21.602933+0000","last_active":"2024-09-15T00:11:44.092711+0000","last_peered":"2024-09-15T00:11:44.092711+0000","last_clean":"2024-09-15T00:11:44.092711+0000","last_became_active":"2024-09-15T00:11:19.597034+0000","last_became_peered":"2024-09-15T00:11:19.597034+0000","last_unstale":"2024-09-15T00:11:44.092711+0000","last_undegraded":"2024-09-15T00:11:44.092711+0000","last_fullsized":"2024-09-15T00:11:44.092711+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:36:46.679816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014330530000000001,"stat_sum":{"num_bytes":3455201,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":11,"num_read":1526,"num_read_kb":1717,"num_write":488,"num_write_kb":1010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"47'136","reported_seq":505,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.099416+0000","last_change":"2024-09-15T00:11:21.613456+0000","last_active":"2024-09-15T00:11:44.099416+0000","last_peered":"2024-09-15T00:11:44.099416+0000","last_clean":"2024-09-15T00:11:44.099416+0000","last_became_active":"2024-09-15T00:11:19.597821+0000","last_became_peered":"2024-09-15T00:11:19.597821+0000","last_unstale":"2024-09-15T00:11:44.099416+0000","last_undegraded":"2024-09-15T00:11:44.099416+0000","last_fullsized":"2024-09-15T00:11:44.099416+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:55:45.484712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085960799999999999,"stat_sum":{"num_bytes":26667310,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":524,"num_read_kb":3756,"num_write":156,"num_write_kb":7756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"48'141","reported_seq":1839,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T00:11:45.315662+0000","last_change":"2024-09-15T00:11:21.595646+0000","last_active":"2024-09-15T00:11:45.315662+0000","last_peered":"2024-09-15T00:11:45.315662+0000","last_clean":"2024-09-15T00:11:45.315662+0000","last_became_active":"2024-09-15T00:11:19.600830+0000","last_became_peered":"2024-09-15T00:11:19.600830+0000","last_unstale":"2024-09-15T00:11:45.315662+0000","last_undegraded":"2024-09-15T00:11:45.315662+0000","last_fullsized":"2024-09-15T00:11:45.315662+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:46:59.199108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077114700000000002,"stat_sum":{"num_bytes":12888218,"num_objects":29,"num_object_clones":10,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":2075,"num_read_kb":8977,"num_write":127,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.181553+0000","last_change":"2024-09-15T00:11:11.641988+0000","last_active":"2024-09-15T00:11:44.181553+0000","last_peered":"2024-09-15T00:11:44.181553+0000","last_clean":"2024-09-15T00:11:44.181553+0000","last_became_active":"2024-09-15T00:11:09.534794+0000","last_became_peered":"2024-09-15T00:11:09.534794+0000","last_unstale":"2024-09-15T00:11:44.181553+0000","last_undegraded":"2024-09-15T00:11:44.181553+0000","last_fullsized":"2024-09-15T00:11:44.181553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:55:52.135410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"47'83","reported_seq":1789,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.182732+0000","last_change":"2024-09-15T00:11:25.006414+0000","last_active":"2024-09-15T00:11:44.182732+0000","last_peered":"2024-09-15T00:11:44.182732+0000","last_clean":"2024-09-15T00:11:44.182732+0000","last_became_active":"2024-09-15T00:11:25.006069+0000","last_became_peered":"2024-09-15T00:11:25.006069+0000","last_unstale":"2024-09-15T00:11:44.182732+0000","last_undegraded":"2024-09-15T00:11:44.182732+0000","last_fullsized":"2024-09-15T00:11:44.182732+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:46:37.184304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014370050000000001,"stat_sum":{"num_bytes":20935839,"num_objects":21,"num_object_clones":8,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":11,"num_read":1676,"num_read_kb":5732,"num_write":94,"num_write_kb":5739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.098066+0000","last_change":"2024-09-15T00:11:11.545630+0000","last_active":"2024-09-15T00:11:44.098066+0000","last_peered":"2024-09-15T00:11:44.098066+0000","last_clean":"2024-09-15T00:11:44.098066+0000","last_became_active":"2024-09-15T00:11:09.536615+0000","last_became_peered":"2024-09-15T00:11:09.536615+0000","last_unstale":"2024-09-15T00:11:44.098066+0000","last_undegraded":"2024-09-15T00:11:44.098066+0000","last_fullsized":"2024-09-15T00:11:44.098066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:51:38.340257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035555,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"47'143","reported_seq":623,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.096833+0000","last_change":"2024-09-15T00:11:21.594822+0000","last_active":"2024-09-15T00:11:44.096833+0000","last_peered":"2024-09-15T00:11:44.096833+0000","last_clean":"2024-09-15T00:11:44.096833+0000","last_became_active":"2024-09-15T00:11:19.596547+0000","last_became_peered":"2024-09-15T00:11:19.596547+0000","last_unstale":"2024-09-15T00:11:44.096833+0000","last_undegraded":"2024-09-15T00:11:44.096833+0000","last_fullsized":"2024-09-15T00:11:44.096833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:17:32.416494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058512600000000005,"stat_sum":{"num_bytes":14717462,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":12,"num_read":678,"num_read_kb":2174,"num_write":177,"num_write_kb":4512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.099457+0000","last_change":"2024-09-15T00:11:25.003710+0000","last_active":"2024-09-15T00:11:44.099457+0000","last_peered":"2024-09-15T00:11:44.099457+0000","last_clean":"2024-09-15T00:11:44.099457+0000","last_became_active":"2024-09-15T00:11:25.003420+0000","last_became_peered":"2024-09-15T00:11:25.003420+0000","last_unstale":"2024-09-15T00:11:44.099457+0000","last_undegraded":"2024-09-15T00:11:44.099457+0000","last_fullsized":"2024-09-15T00:11:44.099457+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:12:12.038285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"45'180","reported_seq":6409,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.178571+0000","last_change":"2024-09-15T00:11:40.073377+0000","last_active":"2024-09-15T00:11:44.178571+0000","last_peered":"2024-09-15T00:11:44.178571+0000","last_clean":"2024-09-15T00:11:44.178571+0000","last_became_active":"2024-09-15T00:11:40.072803+0000","last_became_peered":"2024-09-15T00:11:40.072803+0000","last_unstale":"2024-09-15T00:11:44.178571+0000","last_undegraded":"2024-09-15T00:11:44.178571+0000","last_fullsized":"2024-09-15T00:11:44.178571+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:31:32.018193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001489752,"stat_sum":{"num_bytes":22961456,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":7909,"num_read_kb":19637,"num_write":175,"num_write_kb":4464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":482924,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.098144+0000","last_change":"2024-09-15T00:11:25.006347+0000","last_active":"2024-09-15T00:11:44.098144+0000","last_peered":"2024-09-15T00:11:44.098144+0000","last_clean":"2024-09-15T00:11:44.098144+0000","last_became_active":"2024-09-15T00:11:25.006068+0000","last_became_peered":"2024-09-15T00:11:25.006068+0000","last_unstale":"2024-09-15T00:11:44.098144+0000","last_undegraded":"2024-09-15T00:11:44.098144+0000","last_fullsized":"2024-09-15T00:11:44.098144+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:53:52.316019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"46'84","reported_seq":1527,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.156156+0000","last_change":"2024-09-15T00:11:21.603338+0000","last_active":"2024-09-15T00:11:44.156156+0000","last_peered":"2024-09-15T00:11:44.156156+0000","last_clean":"2024-09-15T00:11:44.156156+0000","last_became_active":"2024-09-15T00:11:19.595633+0000","last_became_peered":"2024-09-15T00:11:19.595633+0000","last_unstale":"2024-09-15T00:11:44.156156+0000","last_undegraded":"2024-09-15T00:11:44.156156+0000","last_fullsized":"2024-09-15T00:11:44.156156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:08:17.739592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017065330000000001,"stat_sum":{"num_bytes":5837178,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":1974,"num_read_kb":5624,"num_write":76,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"48'132","reported_seq":1477,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T00:11:45.311227+0000","last_change":"2024-09-15T00:11:24.999261+0000","last_active":"2024-09-15T00:11:45.311227+0000","last_peered":"2024-09-15T00:11:45.311227+0000","last_clean":"2024-09-15T00:11:45.311227+0000","last_became_active":"2024-09-15T00:11:24.998815+0000","last_became_peered":"2024-09-15T00:11:24.998815+0000","last_unstale":"2024-09-15T00:11:45.311227+0000","last_undegraded":"2024-09-15T00:11:45.311227+0000","last_fullsized":"2024-09-15T00:11:45.311227+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:32:51.945393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075923799999999997,"stat_sum":{"num_bytes":30775155,"num_objects":26,"num_object_clones":10,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":1335,"num_read_kb":10247,"num_write":162,"num_write_kb":17020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.092552+0000","last_change":"2024-09-15T00:11:11.543519+0000","last_active":"2024-09-15T00:11:44.092552+0000","last_peered":"2024-09-15T00:11:44.092552+0000","last_clean":"2024-09-15T00:11:44.092552+0000","last_became_active":"2024-09-15T00:11:09.533896+0000","last_became_peered":"2024-09-15T00:11:09.533896+0000","last_unstale":"2024-09-15T00:11:44.092552+0000","last_undegraded":"2024-09-15T00:11:44.092552+0000","last_fullsized":"2024-09-15T00:11:44.092552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:47.613931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"45'120","reported_seq":2190,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.098179+0000","last_change":"2024-09-15T00:11:37.317374+0000","last_active":"2024-09-15T00:11:44.098179+0000","last_peered":"2024-09-15T00:11:44.098179+0000","last_clean":"2024-09-15T00:11:44.098179+0000","last_became_active":"2024-09-15T00:11:37.238849+0000","last_became_peered":"2024-09-15T00:11:37.238849+0000","last_unstale":"2024-09-15T00:11:44.098179+0000","last_undegraded":"2024-09-15T00:11:44.098179+0000","last_fullsized":"2024-09-15T00:11:44.098179+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:15:12.120162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241242,"stat_sum":{"num_bytes":8083351,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":2182,"num_read_kb":6320,"num_write":127,"num_write_kb":2714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1846940,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.096448+0000","last_change":"2024-09-15T00:11:37.053880+0000","last_active":"2024-09-15T00:11:44.096448+0000","last_peered":"2024-09-15T00:11:44.096448+0000","last_clean":"2024-09-15T00:11:44.096448+0000","last_became_active":"2024-09-15T00:11:37.053552+0000","last_became_peered":"2024-09-15T00:11:37.053552+0000","last_unstale":"2024-09-15T00:11:44.096448+0000","last_undegraded":"2024-09-15T00:11:44.096448+0000","last_fullsized":"2024-09-15T00:11:44.096448+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:05:10.783349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"45'75","reported_seq":1607,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T00:11:45.111774+0000","last_change":"2024-09-15T00:11:25.036338+0000","last_active":"2024-09-15T00:11:45.111774+0000","last_peered":"2024-09-15T00:11:45.111774+0000","last_clean":"2024-09-15T00:11:45.111774+0000","last_became_active":"2024-09-15T00:11:25.005078+0000","last_became_peered":"2024-09-15T00:11:25.005078+0000","last_unstale":"2024-09-15T00:11:45.111774+0000","last_undegraded":"2024-09-15T00:11:45.111774+0000","last_fullsized":"2024-09-15T00:11:45.111774+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:39:59.833365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069791600000000003,"stat_sum":{"num_bytes":10597996,"num_objects":19,"num_object_clones":8,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":1483,"num_read_kb":8679,"num_write":71,"num_write_kb":3928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":64634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":70,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.099164+0000","last_change":"2024-09-15T00:11:11.641995+0000","last_active":"2024-09-15T00:11:44.099164+0000","last_peered":"2024-09-15T00:11:44.099164+0000","last_clean":"2024-09-15T00:11:44.099164+0000","last_became_active":"2024-09-15T00:11:09.533404+0000","last_became_peered":"2024-09-15T00:11:09.533404+0000","last_unstale":"2024-09-15T00:11:44.099164+0000","last_undegraded":"2024-09-15T00:11:44.099164+0000","last_fullsized":"2024-09-15T00:11:44.099164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:20.611551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046024999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"45'174","reported_seq":1439,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.096566+0000","last_change":"2024-09-15T00:11:21.595240+0000","last_active":"2024-09-15T00:11:44.096566+0000","last_peered":"2024-09-15T00:11:44.096566+0000","last_clean":"2024-09-15T00:11:44.096566+0000","last_became_active":"2024-09-15T00:11:19.594528+0000","last_became_peered":"2024-09-15T00:11:19.594528+0000","last_unstale":"2024-09-15T00:11:44.096566+0000","last_undegraded":"2024-09-15T00:11:44.096566+0000","last_fullsized":"2024-09-15T00:11:44.096566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:11.673492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080299099999999997,"stat_sum":{"num_bytes":5307592,"num_objects":23,"num_object_clones":8,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":1644,"num_read_kb":4121,"num_write":215,"num_write_kb":1530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.092542+0000","last_change":"2024-09-15T00:11:37.055535+0000","last_active":"2024-09-15T00:11:44.092542+0000","last_peered":"2024-09-15T00:11:44.092542+0000","last_clean":"2024-09-15T00:11:44.092542+0000","last_became_active":"2024-09-15T00:11:37.052915+0000","last_became_peered":"2024-09-15T00:11:37.052915+0000","last_unstale":"2024-09-15T00:11:44.092542+0000","last_undegraded":"2024-09-15T00:11:44.092542+0000","last_fullsized":"2024-09-15T00:11:44.092542+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:57:39.917591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":234,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.099167+0000","last_change":"2024-09-15T00:11:28.008530+0000","last_active":"2024-09-15T00:11:44.099167+0000","last_peered":"2024-09-15T00:11:44.099167+0000","last_clean":"2024-09-15T00:11:44.099167+0000","last_became_active":"2024-09-15T00:11:28.008130+0000","last_became_peered":"2024-09-15T00:11:28.008130+0000","last_unstale":"2024-09-15T00:11:44.099167+0000","last_undegraded":"2024-09-15T00:11:44.099167+0000","last_fullsized":"2024-09-15T00:11:44.099167+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_clean_scrub_stamp":"2024-09-15T00:11:04.453672+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:09:32.444877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"47'132","reported_seq":519,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.184490+0000","last_change":"2024-09-15T00:11:21.595379+0000","last_active":"2024-09-15T00:11:44.184490+0000","last_peered":"2024-09-15T00:11:44.184490+0000","last_clean":"2024-09-15T00:11:44.184490+0000","last_became_active":"2024-09-15T00:11:19.594476+0000","last_became_peered":"2024-09-15T00:11:19.594476+0000","last_unstale":"2024-09-15T00:11:44.184490+0000","last_undegraded":"2024-09-15T00:11:44.184490+0000","last_fullsized":"2024-09-15T00:11:44.184490+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:57:23.730620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089807799999999996,"stat_sum":{"num_bytes":8217457,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":372,"num_read_kb":2608,"num_write":129,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"46'655","reported_seq":3698,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T00:11:45.111717+0000","last_change":"2024-09-15T00:11:40.070277+0000","last_active":"2024-09-15T00:11:45.111717+0000","last_peered":"2024-09-15T00:11:45.111717+0000","last_clean":"2024-09-15T00:11:45.111717+0000","last_became_active":"2024-09-15T00:11:40.069596+0000","last_became_peered":"2024-09-15T00:11:40.069596+0000","last_unstale":"2024-09-15T00:11:45.111717+0000","last_undegraded":"2024-09-15T00:11:45.111717+0000","last_fullsized":"2024-09-15T00:11:45.111717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:56:41.942031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053468500000000002,"stat_sum":{"num_bytes":8639898,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":7084,"num_read_kb":12915,"num_write":994,"num_write_kb":1735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":44504,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"47'82","reported_seq":1081,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.181598+0000","last_change":"2024-09-15T00:11:21.608019+0000","last_active":"2024-09-15T00:11:44.181598+0000","last_peered":"2024-09-15T00:11:44.181598+0000","last_clean":"2024-09-15T00:11:44.181598+0000","last_became_active":"2024-09-15T00:11:19.602625+0000","last_became_peered":"2024-09-15T00:11:19.602625+0000","last_unstale":"2024-09-15T00:11:44.181598+0000","last_undegraded":"2024-09-15T00:11:44.181598+0000","last_fullsized":"2024-09-15T00:11:44.181598+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:41:02.109366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036072699999999998,"stat_sum":{"num_bytes":6197972,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":986,"num_read_kb":3418,"num_write":74,"num_write_kb":616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"47'74","reported_seq":1375,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.286196+0000","last_change":"2024-09-15T00:11:25.000430+0000","last_active":"2024-09-15T00:11:44.286196+0000","last_peered":"2024-09-15T00:11:44.286196+0000","last_clean":"2024-09-15T00:11:44.286196+0000","last_became_active":"2024-09-15T00:11:25.000142+0000","last_became_peered":"2024-09-15T00:11:25.000142+0000","last_unstale":"2024-09-15T00:11:44.286196+0000","last_undegraded":"2024-09-15T00:11:44.286196+0000","last_fullsized":"2024-09-15T00:11:44.286196+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:07:21.428434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082326600000000002,"stat_sum":{"num_bytes":8024095,"num_objects":27,"num_object_clones":10,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":1277,"num_read_kb":9041,"num_write":65,"num_write_kb":4558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"46'158","reported_seq":1085,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.317603+0000","last_change":"2024-09-15T00:11:21.603031+0000","last_active":"2024-09-15T00:11:44.317603+0000","last_peered":"2024-09-15T00:11:44.317603+0000","last_clean":"2024-09-15T00:11:44.317603+0000","last_became_active":"2024-09-15T00:11:19.595088+0000","last_became_peered":"2024-09-15T00:11:19.595088+0000","last_unstale":"2024-09-15T00:11:44.317603+0000","last_undegraded":"2024-09-15T00:11:44.317603+0000","last_fullsized":"2024-09-15T00:11:44.317603+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:12:25.782482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537407,"stat_sum":{"num_bytes":2997351,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":7,"num_read":1999,"num_read_kb":2649,"num_write":143,"num_write_kb":623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"47'80","reported_seq":1304,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.099456+0000","last_change":"2024-09-15T00:11:21.613267+0000","last_active":"2024-09-15T00:11:44.099456+0000","last_peered":"2024-09-15T00:11:44.099456+0000","last_clean":"2024-09-15T00:11:44.099456+0000","last_became_active":"2024-09-15T00:11:19.596135+0000","last_became_peered":"2024-09-15T00:11:19.596135+0000","last_unstale":"2024-09-15T00:11:44.099456+0000","last_undegraded":"2024-09-15T00:11:44.099456+0000","last_fullsized":"2024-09-15T00:11:44.099456+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:48:32.676111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069298099999999996,"stat_sum":{"num_bytes":22118400,"num_objects":22,"num_object_clones":9,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":1192,"num_read_kb":6054,"num_write":65,"num_write_kb":924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"47'139","reported_seq":278,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.233665+0000","last_change":"2024-09-15T00:11:21.613225+0000","last_active":"2024-09-15T00:11:44.233665+0000","last_peered":"2024-09-15T00:11:44.233665+0000","last_clean":"2024-09-15T00:11:44.233665+0000","last_became_active":"2024-09-15T00:11:19.602353+0000","last_became_peered":"2024-09-15T00:11:19.602353+0000","last_unstale":"2024-09-15T00:11:44.233665+0000","last_undegraded":"2024-09-15T00:11:44.233665+0000","last_fullsized":"2024-09-15T00:11:44.233665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:52:37.163814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076259299999999999,"stat_sum":{"num_bytes":1668636,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":9,"num_read":345,"num_read_kb":1574,"num_write":170,"num_write_kb":1270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"47'93","reported_seq":2018,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.182194+0000","last_change":"2024-09-15T00:11:21.613002+0000","last_active":"2024-09-15T00:11:44.182194+0000","last_peered":"2024-09-15T00:11:44.182194+0000","last_clean":"2024-09-15T00:11:44.182194+0000","last_became_active":"2024-09-15T00:11:19.601589+0000","last_became_peered":"2024-09-15T00:11:19.601589+0000","last_unstale":"2024-09-15T00:11:44.182194+0000","last_undegraded":"2024-09-15T00:11:44.182194+0000","last_fullsized":"2024-09-15T00:11:44.182194+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:55.433644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055547799999999998,"stat_sum":{"num_bytes":19433718,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":1966,"num_read_kb":4168,"num_write":90,"num_write_kb":765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"45'101","reported_seq":1165,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.181593+0000","last_change":"2024-09-15T00:11:25.005455+0000","last_active":"2024-09-15T00:11:44.181593+0000","last_peered":"2024-09-15T00:11:44.181593+0000","last_clean":"2024-09-15T00:11:44.181593+0000","last_became_active":"2024-09-15T00:11:25.001267+0000","last_became_peered":"2024-09-15T00:11:25.001267+0000","last_unstale":"2024-09-15T00:11:44.181593+0000","last_undegraded":"2024-09-15T00:11:44.181593+0000","last_fullsized":"2024-09-15T00:11:44.181593+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:55:43.914739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093430300000000004,"stat_sum":{"num_bytes":9567290,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":1225,"num_read_kb":4269,"num_write":91,"num_write_kb":3617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"45'170","reported_seq":1326,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.099400+0000","last_change":"2024-09-15T00:11:21.612759+0000","last_active":"2024-09-15T00:11:44.099400+0000","last_peered":"2024-09-15T00:11:44.099400+0000","last_clean":"2024-09-15T00:11:44.099400+0000","last_became_active":"2024-09-15T00:11:19.601674+0000","last_became_peered":"2024-09-15T00:11:19.601674+0000","last_unstale":"2024-09-15T00:11:44.099400+0000","last_undegraded":"2024-09-15T00:11:44.099400+0000","last_fullsized":"2024-09-15T00:11:44.099400+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:20:50.546699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042917200000000002,"stat_sum":{"num_bytes":4720846,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":15,"num_read":1671,"num_read_kb":5616,"num_write":236,"num_write_kb":1072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"45'138","reported_seq":1297,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.099367+0000","last_change":"2024-09-15T00:11:25.011332+0000","last_active":"2024-09-15T00:11:44.099367+0000","last_peered":"2024-09-15T00:11:44.099367+0000","last_clean":"2024-09-15T00:11:44.099367+0000","last_became_active":"2024-09-15T00:11:25.010982+0000","last_became_peered":"2024-09-15T00:11:25.010982+0000","last_unstale":"2024-09-15T00:11:44.099367+0000","last_undegraded":"2024-09-15T00:11:44.099367+0000","last_fullsized":"2024-09-15T00:11:44.099367+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:27.347827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042453400000000001,"stat_sum":{"num_bytes":9665530,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":8,"num_read":1416,"num_read_kb":6073,"num_write":196,"num_write_kb":4212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"45'132","reported_seq":1470,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T00:11:44.311610+0000","last_change":"2024-09-15T00:11:21.612992+0000","last_active":"2024-09-15T00:11:44.311610+0000","last_peered":"2024-09-15T00:11:44.311610+0000","last_clean":"2024-09-15T00:11:44.311610+0000","last_became_active":"2024-09-15T00:11:19.602532+0000","last_became_peered":"2024-09-15T00:11:19.602532+0000","last_unstale":"2024-09-15T00:11:44.311610+0000","last_undegraded":"2024-09-15T00:11:44.311610+0000","last_fullsized":"2024-09-15T00:11:44.311610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:40:28.510789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062135599999999999,"stat_sum":{"num_bytes":6501827,"num_objects":23,"num_object_clones":8,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":1744,"num_read_kb":5560,"num_write":117,"num_write_kb":2712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":438059624,"num_objects":719,"num_object_clones":237,"num_object_copies":1438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":719,"num_whiteouts":328,"num_read":62147,"num_read_kb":214723,"num_write":5775,"num_write_kb":129937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":5270685,"num_keys_recovered":56,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18046976,"data_stored":293987379,"data_compressed":3924277,"data_compressed_allocated":14880768,"data_compressed_original":290923514,"omap_allocated":0,"internal_metadata":0},"log_size":4826,"ondisk_log_size":4826,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":17376,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40700,"kb_used_data":5356,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331140,"statfs":{"total":96636764160,"available":96595087360,"internally_reserved":0,"allocated":5484544,"data_stored":86206756,"data_compressed":1109592,"data_compressed_allocated":4206592,"data_compressed_original":85224277,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542157,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39312,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332528,"statfs":{"total":96636764160,"available":96596508672,"internally_reserved":0,"allocated":4063232,"data_stored":46694748,"data_compressed":611500,"data_compressed_allocated":2895872,"data_compressed_original":45942125,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37872,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333968,"statfs":{"total":96636764160,"available":96597983232,"internally_reserved":0,"allocated":2588672,"data_stored":22477189,"data_compressed":422522,"data_compressed_allocated":1654784,"data_compressed_original":21975040,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542157,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33560,"kb_used_data":2364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338280,"statfs":{"total":96636764160,"available":96602398720,"internally_reserved":0,"allocated":2420736,"data_stored":21330515,"data_compressed":378688,"data_compressed_allocated":1560576,"data_compressed_original":20878701,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38156,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333684,"statfs":{"total":96636764160,"available":96597692416,"internally_reserved":0,"allocated":2834432,"data_stored":24850289,"data_compressed":465385,"data_compressed_allocated":1818624,"data_compressed_original":24266912,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":60129542156,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542155,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34312,"kb_used_data":3124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337528,"statfs":{"total":96636764160,"available":96601628672,"internally_reserved":0,"allocated":3198976,"data_stored":29335044,"data_compressed":408394,"data_compressed_allocated":2199552,"data_compressed_original":28696576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":60129542156,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":117006,"data_compressed":207,"data_compressed_allocated":8192,"data_compressed_original":110592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2691072,"data_stored":48539244,"data_compressed":589673,"data_compressed_allocated":2203648,"data_compressed_original":47944373,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":542788,"data_compressed":51639,"data_compressed_allocated":77824,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4141056,"data_stored":67742655,"data_compressed":794054,"data_compressed_allocated":3424256,"data_compressed_original":67029722,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4890624,"data_stored":86391384,"data_compressed":1081363,"data_compressed_allocated":4210688,"data_compressed_original":85697265,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2314240,"data_stored":34130101,"data_compressed":524318,"data_compressed_allocated":1818624,"data_compressed_original":33663136,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1859584,"data_stored":30046643,"data_compressed":451153,"data_compressed_allocated":1523712,"data_compressed_original":29760218,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1998848,"data_stored":26477558,"data_compressed":431870,"data_compressed_allocated":1613824,"data_compressed_original":26193920,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:11:47.850 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T00:11:47.851 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:11:47.851 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [1, 3, 4, 5, 6, 7] 2024-09-15T00:11:47.851 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 7 2024-09-15T00:11:47.891 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:47.891 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:47.900 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:47.943 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:47.942+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:48.033 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:11:48.045 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:48.043+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:48.047 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:48.047 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:48.057 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:48.124 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is unset 2024-09-15T00:11:48.145 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:48.144+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:48.162 INFO:tasks.ceph.mon.a.smithi171.stderr:2024-09-15T00:11:48.160+0000 7f0425d35640 -1 mon.a@0(leader).osd e51 definitely_dead 0 2024-09-15T00:11:48.163 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:48.163 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:48.184 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:48.184 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:48.201 INFO:teuthology.orchestra.run.smithi171.stdout:flattening image image_client.0-clone14 2024-09-15T00:11:48.203 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:48.203 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:48.239 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:48.240 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:48.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:48.240 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:48.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.240 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1183 3.4 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:48.240 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.947741+0000", 2024-09-15T00:11:48.240 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.288171534, 2024-09-15T00:11:48.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024684798000000001, 2024-09-15T00:11:48.240 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.240 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.240 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.241 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1183 2024-09-15T00:11:48.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.241 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947741+0000", 2024-09-15T00:11:48.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947741+0000", 2024-09-15T00:11:48.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947743+0000", 2024-09-15T00:11:48.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.446e-06 2024-09-15T00:11:48.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947750+0000", 2024-09-15T00:11:48.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7640000000000005e-06 2024-09-15T00:11:48.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947774+0000", 2024-09-15T00:11:48.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3958e-05 2024-09-15T00:11:48.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947783+0000", 2024-09-15T00:11:48.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8410000000000006e-06 2024-09-15T00:11:48.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947840+0000", 2024-09-15T00:11:48.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6784999999999999e-05 2024-09-15T00:11:48.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.947947+0000", 2024-09-15T00:11:48.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010688000000000001 2024-09-15T00:11:48.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:48.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.948963+0000", 2024-09-15T00:11:48.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001016482 2024-09-15T00:11:48.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:48.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972328+0000", 2024-09-15T00:11:48.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023364843999999999 2024-09-15T00:11:48.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:48.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972341+0000", 2024-09-15T00:11:48.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2777e-05 2024-09-15T00:11:48.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972378+0000", 2024-09-15T00:11:48.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6749999999999999e-05 2024-09-15T00:11:48.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972426+0000", 2024-09-15T00:11:48.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8270999999999999e-05 2024-09-15T00:11:48.246 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.246 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.246 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.246 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:1183 3.4 e29/23 ondisk, result = 0)", 2024-09-15T00:11:48.246 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.951923+0000", 2024-09-15T00:11:48.246 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.283989350999999, 2024-09-15T00:11:48.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020411286000000001, 2024-09-15T00:11:48.246 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.247 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.247 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:48.247 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951923+0000", 2024-09-15T00:11:48.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951923+0000", 2024-09-15T00:11:48.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951924+0000", 2024-09-15T00:11:48.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8700000000000004e-07 2024-09-15T00:11:48.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951928+0000", 2024-09-15T00:11:48.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1110000000000001e-06 2024-09-15T00:11:48.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951939+0000", 2024-09-15T00:11:48.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0752e-05 2024-09-15T00:11:48.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.951945+0000", 2024-09-15T00:11:48.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1009999999999997e-06 2024-09-15T00:11:48.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972316+0000", 2024-09-15T00:11:48.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020370335999999999 2024-09-15T00:11:48.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972323+0000", 2024-09-15T00:11:48.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9189999999999999e-06 2024-09-15T00:11:48.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.972335+0000", 2024-09-15T00:11:48.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.118e-05 2024-09-15T00:11:48.250 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.250 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.250 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.250 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:48.250 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:11:48.251 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.056087999999999, 2024-09-15T00:11:48.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:11:48.251 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.251 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.251 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.251 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.251 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.251 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.251 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:11:48.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.251 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:11:48.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:11:48.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:11:48.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:11:48.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:11:48.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:11:48.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:11:48.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:11:48.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:11:48.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:11:48.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:48.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:11:48.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:11:48.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:48.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:11:48.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:11:48.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:11:48.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:11:48.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:11:48.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:11:48.256 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.256 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.256 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.256 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:48.256 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:11:48.256 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.052660091, 2024-09-15T00:11:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032661645000000003, 2024-09-15T00:11:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for ondisk", 2024-09-15T00:11:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:11:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:11:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:11:48.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:11:48.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5300000000000004e-07 2024-09-15T00:11:48.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183257+0000", 2024-09-15T00:11:48.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9380000000000002e-06 2024-09-15T00:11:48.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183265+0000", 2024-09-15T00:11:48.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6599999999999995e-06 2024-09-15T00:11:48.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183270+0000", 2024-09-15T00:11:48.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6489999999999996e-06 2024-09-15T00:11:48.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215767+0000", 2024-09-15T00:11:48.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032496490000000003 2024-09-15T00:11:48.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for ondisk", 2024-09-15T00:11:48.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215819+0000", 2024-09-15T00:11:48.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1647999999999997e-05 2024-09-15T00:11:48.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215914+0000", 2024-09-15T00:11:48.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5706999999999997e-05 2024-09-15T00:11:48.260 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.260 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.260 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.261 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:1287 3.6 e30/28 ondisk, result = 0)", 2024-09-15T00:11:48.261 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.186816+0000", 2024-09-15T00:11:48.261 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.049096505000001, 2024-09-15T00:11:48.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.029151897999999999, 2024-09-15T00:11:48.261 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.261 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.261 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:48.261 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186816+0000", 2024-09-15T00:11:48.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186816+0000", 2024-09-15T00:11:48.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186817+0000", 2024-09-15T00:11:48.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0729999999999999e-06 2024-09-15T00:11:48.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186822+0000", 2024-09-15T00:11:48.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.006e-06 2024-09-15T00:11:48.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186837+0000", 2024-09-15T00:11:48.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4819e-05 2024-09-15T00:11:48.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.186846+0000", 2024-09-15T00:11:48.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1519999999999998e-06 2024-09-15T00:11:48.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215852+0000", 2024-09-15T00:11:48.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.029005882 2024-09-15T00:11:48.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215859+0000", 2024-09-15T00:11:48.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7499999999999997e-06 2024-09-15T00:11:48.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215968+0000", 2024-09-15T00:11:48.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010921600000000001 2024-09-15T00:11:48.265 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.265 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.265 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.265 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:48.265 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:11:48.265 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.326246821, 2024-09-15T00:11:48.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:11:48.266 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.266 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.266 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.266 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.266 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.266 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.266 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:11:48.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.266 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:11:48.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:11:48.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:11:48.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:11:48.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:11:48.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:11:48.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:11:48.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:11:48.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:11:48.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:11:48.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:11:48.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:11:48.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:11:48.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:11:48.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:48.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:11:48.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:11:48.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:48.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:11:48.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:11:48.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:48.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:11:48.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:11:48.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:11:48.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:11:48.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:11:48.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:11:48.271 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.271 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.271 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.271 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:11:48.271 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:11:48.271 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.319288458000001, 2024-09-15T00:11:48.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:11:48.272 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.272 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.272 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:48.272 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:11:48.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:11:48.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:11:48.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:11:48.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:11:48.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:11:48.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:11:48.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:11:48.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:11:48.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:11:48.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:11:48.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:11:48.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:11:48.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:11:48.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:11:48.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:11:48.275 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.275 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.275 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.276 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48879 3.6 3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:48.276 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.608209+0000", 2024-09-15T00:11:48.276 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.6277035399999997, 2024-09-15T00:11:48.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017763616999999999, 2024-09-15T00:11:48.276 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.276 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.276 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:48.276 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.276 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.276 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.276 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48879 2024-09-15T00:11:48.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.277 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608209+0000", 2024-09-15T00:11:48.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608209+0000", 2024-09-15T00:11:48.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608210+0000", 2024-09-15T00:11:48.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2000000000000001e-07 2024-09-15T00:11:48.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608211+0000", 2024-09-15T00:11:48.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.815e-06 2024-09-15T00:11:48.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608216+0000", 2024-09-15T00:11:48.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0749999999999997e-06 2024-09-15T00:11:48.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608220+0000", 2024-09-15T00:11:48.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.997e-06 2024-09-15T00:11:48.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.625837+0000", 2024-09-15T00:11:48.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017616817999999999 2024-09-15T00:11:48.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.625889+0000", 2024-09-15T00:11:48.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2163000000000002e-05 2024-09-15T00:11:48.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.625973+0000", 2024-09-15T00:11:48.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3429000000000004e-05 2024-09-15T00:11:48.280 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.280 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.280 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.280 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48880 3.6 3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:48.280 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.608225+0000", 2024-09-15T00:11:48.280 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.6276872600000001, 2024-09-15T00:11:48.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017978500000000001, 2024-09-15T00:11:48.280 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.280 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.281 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:48.281 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.281 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.281 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.281 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48880 2024-09-15T00:11:48.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.281 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608225+0000", 2024-09-15T00:11:48.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608225+0000", 2024-09-15T00:11:48.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608226+0000", 2024-09-15T00:11:48.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:11:48.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608228+0000", 2024-09-15T00:11:48.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9489999999999999e-06 2024-09-15T00:11:48.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608234+0000", 2024-09-15T00:11:48.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7960000000000001e-06 2024-09-15T00:11:48.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608238+0000", 2024-09-15T00:11:48.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7999999999999998e-06 2024-09-15T00:11:48.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626019+0000", 2024-09-15T00:11:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017780193999999999 2024-09-15T00:11:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626109+0000", 2024-09-15T00:11:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0785999999999994e-05 2024-09-15T00:11:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626204+0000", 2024-09-15T00:11:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4659000000000003e-05 2024-09-15T00:11:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.285 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48963 3.6 3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:48.285 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.608986+0000", 2024-09-15T00:11:48.285 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.6269268540000001, 2024-09-15T00:11:48.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017888314999999998, 2024-09-15T00:11:48.285 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.285 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.285 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:48.285 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.285 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.285 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.286 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48963 2024-09-15T00:11:48.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.286 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608986+0000", 2024-09-15T00:11:48.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608986+0000", 2024-09-15T00:11:48.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608987+0000", 2024-09-15T00:11:48.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3799999999999996e-07 2024-09-15T00:11:48.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608991+0000", 2024-09-15T00:11:48.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5739999999999997e-06 2024-09-15T00:11:48.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609004+0000", 2024-09-15T00:11:48.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2196e-05 2024-09-15T00:11:48.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609011+0000", 2024-09-15T00:11:48.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0469999999999996e-06 2024-09-15T00:11:48.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626240+0000", 2024-09-15T00:11:48.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017229429000000001 2024-09-15T00:11:48.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626292+0000", 2024-09-15T00:11:48.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1759999999999999e-05 2024-09-15T00:11:48.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626874+0000", 2024-09-15T00:11:48.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00058237100000000004 2024-09-15T00:11:48.289 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.289 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.289 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.289 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48964 3.6 3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:48.289 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609033+0000", 2024-09-15T00:11:48.289 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.6268798179999999, 2024-09-15T00:11:48.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018388555000000001, 2024-09-15T00:11:48.289 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.290 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.290 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:48.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.290 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48964 2024-09-15T00:11:48.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.290 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609033+0000", 2024-09-15T00:11:48.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609033+0000", 2024-09-15T00:11:48.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609033+0000", 2024-09-15T00:11:48.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1099999999999996e-07 2024-09-15T00:11:48.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609036+0000", 2024-09-15T00:11:48.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9270000000000001e-06 2024-09-15T00:11:48.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609044+0000", 2024-09-15T00:11:48.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1159999999999999e-06 2024-09-15T00:11:48.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609050+0000", 2024-09-15T00:11:48.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4949999999999999e-06 2024-09-15T00:11:48.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626902+0000", 2024-09-15T00:11:48.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017851812000000002 2024-09-15T00:11:48.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.626991+0000", 2024-09-15T00:11:48.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9234000000000007e-05 2024-09-15T00:11:48.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627421+0000", 2024-09-15T00:11:48.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00043046000000000002 2024-09-15T00:11:48.293 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.293 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.294 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.294 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48965 3.6 3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e [sparse-read 5111808~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:48.294 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609059+0000", 2024-09-15T00:11:48.294 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.626853916, 2024-09-15T00:11:48.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018948744, 2024-09-15T00:11:48.294 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.294 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.294 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:48.294 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.294 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.294 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.295 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48965 2024-09-15T00:11:48.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.295 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609059+0000", 2024-09-15T00:11:48.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609059+0000", 2024-09-15T00:11:48.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609059+0000", 2024-09-15T00:11:48.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9799999999999999e-07 2024-09-15T00:11:48.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609061+0000", 2024-09-15T00:11:48.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1090000000000001e-06 2024-09-15T00:11:48.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609068+0000", 2024-09-15T00:11:48.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1450000000000002e-06 2024-09-15T00:11:48.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609073+0000", 2024-09-15T00:11:48.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5950000000000001e-06 2024-09-15T00:11:48.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627457+0000", 2024-09-15T00:11:48.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018384167999999999 2024-09-15T00:11:48.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627526+0000", 2024-09-15T00:11:48.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8555999999999999e-05 2024-09-15T00:11:48.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628008+0000", 2024-09-15T00:11:48.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00048177300000000001 2024-09-15T00:11:48.298 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.298 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.298 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.298 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48966 3.6 3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e [sparse-read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:48.298 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609079+0000", 2024-09-15T00:11:48.299 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.626833795, 2024-09-15T00:11:48.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019065480999999999, 2024-09-15T00:11:48.299 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.299 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.299 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:48.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.299 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48966 2024-09-15T00:11:48.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.299 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609079+0000", 2024-09-15T00:11:48.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609079+0000", 2024-09-15T00:11:48.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609079+0000", 2024-09-15T00:11:48.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1899999999999998e-07 2024-09-15T00:11:48.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609081+0000", 2024-09-15T00:11:48.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9920000000000002e-06 2024-09-15T00:11:48.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609088+0000", 2024-09-15T00:11:48.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5459999999999997e-06 2024-09-15T00:11:48.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609092+0000", 2024-09-15T00:11:48.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5820000000000003e-06 2024-09-15T00:11:48.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628037+0000", 2024-09-15T00:11:48.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018944286000000001 2024-09-15T00:11:48.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628084+0000", 2024-09-15T00:11:48.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7787000000000002e-05 2024-09-15T00:11:48.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628144+0000", 2024-09-15T00:11:48.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9969000000000001e-05 2024-09-15T00:11:48.303 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.303 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.303 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.303 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48967 3.6 3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e [sparse-read 8781824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:48.303 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609097+0000", 2024-09-15T00:11:48.303 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.6268152360000001, 2024-09-15T00:11:48.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019190448999999998, 2024-09-15T00:11:48.303 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.303 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.303 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:48.303 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.304 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48967 2024-09-15T00:11:48.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.304 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609097+0000", 2024-09-15T00:11:48.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609097+0000", 2024-09-15T00:11:48.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609098+0000", 2024-09-15T00:11:48.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-15T00:11:48.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609100+0000", 2024-09-15T00:11:48.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.877e-06 2024-09-15T00:11:48.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609105+0000", 2024-09-15T00:11:48.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5720000000000002e-06 2024-09-15T00:11:48.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609110+0000", 2024-09-15T00:11:48.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4410000000000003e-06 2024-09-15T00:11:48.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628171+0000", 2024-09-15T00:11:48.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019061679000000002 2024-09-15T00:11:48.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628219+0000", 2024-09-15T00:11:48.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7852000000000002e-05 2024-09-15T00:11:48.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628288+0000", 2024-09-15T00:11:48.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8689000000000006e-05 2024-09-15T00:11:48.307 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.307 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.308 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.308 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48968 3.6 3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:48.308 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609114+0000", 2024-09-15T00:11:48.308 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.626798301, 2024-09-15T00:11:48.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019388655000000001, 2024-09-15T00:11:48.308 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.308 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.308 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:48.308 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.308 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.308 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.308 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48968 2024-09-15T00:11:48.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.309 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609114+0000", 2024-09-15T00:11:48.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609114+0000", 2024-09-15T00:11:48.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609115+0000", 2024-09-15T00:11:48.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3200000000000001e-07 2024-09-15T00:11:48.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609117+0000", 2024-09-15T00:11:48.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.869e-06 2024-09-15T00:11:48.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609123+0000", 2024-09-15T00:11:48.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5239999999999997e-06 2024-09-15T00:11:48.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609128+0000", 2024-09-15T00:11:48.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.374e-06 2024-09-15T00:11:48.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628329+0000", 2024-09-15T00:11:48.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019201840000000001 2024-09-15T00:11:48.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628407+0000", 2024-09-15T00:11:48.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7205000000000004e-05 2024-09-15T00:11:48.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628503+0000", 2024-09-15T00:11:48.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6510999999999998e-05 2024-09-15T00:11:48.312 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.312 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.312 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.313 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48969 3.6 3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e [sparse-read 12451840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:48.313 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609133+0000", 2024-09-15T00:11:48.313 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.626780122, 2024-09-15T00:11:48.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019527445000000001, 2024-09-15T00:11:48.313 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.313 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.313 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:48.313 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.313 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.313 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.313 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48969 2024-09-15T00:11:48.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.314 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609133+0000", 2024-09-15T00:11:48.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609133+0000", 2024-09-15T00:11:48.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609133+0000", 2024-09-15T00:11:48.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:11:48.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609135+0000", 2024-09-15T00:11:48.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.801e-06 2024-09-15T00:11:48.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609141+0000", 2024-09-15T00:11:48.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4500000000000001e-06 2024-09-15T00:11:48.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609146+0000", 2024-09-15T00:11:48.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3460000000000004e-06 2024-09-15T00:11:48.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628544+0000", 2024-09-15T00:11:48.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019398656 2024-09-15T00:11:48.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628597+0000", 2024-09-15T00:11:48.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2688000000000001e-05 2024-09-15T00:11:48.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628660+0000", 2024-09-15T00:11:48.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3188000000000006e-05 2024-09-15T00:11:48.317 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.317 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.317 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.317 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48970 3.6 3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:48.317 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609150+0000", 2024-09-15T00:11:48.317 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.6267624999999999, 2024-09-15T00:11:48.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019639139999999999, 2024-09-15T00:11:48.318 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.318 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.318 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:48.318 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.318 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.318 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.318 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48970 2024-09-15T00:11:48.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.318 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609150+0000", 2024-09-15T00:11:48.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609150+0000", 2024-09-15T00:11:48.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609151+0000", 2024-09-15T00:11:48.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8999999999999998e-07 2024-09-15T00:11:48.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609152+0000", 2024-09-15T00:11:48.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7799999999999999e-06 2024-09-15T00:11:48.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609164+0000", 2024-09-15T00:11:48.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1450999999999999e-05 2024-09-15T00:11:48.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609169+0000", 2024-09-15T00:11:48.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.31e-06 2024-09-15T00:11:48.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628681+0000", 2024-09-15T00:11:48.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019512150999999998 2024-09-15T00:11:48.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628727+0000", 2024-09-15T00:11:48.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6108e-05 2024-09-15T00:11:48.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628789+0000", 2024-09-15T00:11:48.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2050000000000004e-05 2024-09-15T00:11:48.321 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.321 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.322 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.322 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49026 3.19 3:9c3e88bf:::rbd_data.113916cf513.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:48.322 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.660548+0000", 2024-09-15T00:11:48.322 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.5753648579999999, 2024-09-15T00:11:48.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032745261999999997, 2024-09-15T00:11:48.322 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.322 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.322 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.322 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.322 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.322 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.323 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49026 2024-09-15T00:11:48.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.323 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.660548+0000", 2024-09-15T00:11:48.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.660548+0000", 2024-09-15T00:11:48.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.660550+0000", 2024-09-15T00:11:48.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3599999999999999e-06 2024-09-15T00:11:48.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663739+0000", 2024-09-15T00:11:48.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031890009999999999 2024-09-15T00:11:48.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663773+0000", 2024-09-15T00:11:48.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4048000000000003e-05 2024-09-15T00:11:48.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663787+0000", 2024-09-15T00:11:48.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.38e-05 2024-09-15T00:11:48.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663858+0000", 2024-09-15T00:11:48.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0883000000000006e-05 2024-09-15T00:11:48.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.664010+0000", 2024-09-15T00:11:48.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015168500000000001 2024-09-15T00:11:48.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:48.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.664511+0000", 2024-09-15T00:11:48.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00050103699999999997 2024-09-15T00:11:48.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:48.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693153+0000", 2024-09-15T00:11:48.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.028642203000000001 2024-09-15T00:11:48.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:48.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693197+0000", 2024-09-15T00:11:48.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4190000000000002e-05 2024-09-15T00:11:48.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693257+0000", 2024-09-15T00:11:48.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0299e-05 2024-09-15T00:11:48.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693293+0000", 2024-09-15T00:11:48.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5756000000000001e-05 2024-09-15T00:11:48.327 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.327 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.328 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:48.328 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:11:48.328 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.572099621, 2024-09-15T00:11:48.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:11:48.328 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.328 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.328 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.328 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.328 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.329 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:11:48.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.329 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:11:48.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:11:48.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:11:48.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:11:48.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:11:48.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:11:48.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:11:48.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:11:48.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:11:48.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:11:48.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:11:48.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:11:48.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:11:48.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:11:48.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:48.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:11:48.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:11:48.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:48.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:11:48.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:11:48.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:48.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:11:48.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:11:48.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:11:48.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:11:48.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:11:48.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:11:48.333 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.333 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.334 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.334 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49034 3.19 3:9c491a28:::rbd_data.113916cf513.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:48.334 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.673063+0000", 2024-09-15T00:11:48.334 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.5628500230000002, 2024-09-15T00:11:48.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020723139000000002, 2024-09-15T00:11:48.334 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.334 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.334 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:48.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:48.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:48.334 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49034 2024-09-15T00:11:48.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.335 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.673063+0000", 2024-09-15T00:11:48.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.673063+0000", 2024-09-15T00:11:48.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.673064+0000", 2024-09-15T00:11:48.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.733e-06 2024-09-15T00:11:48.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.673073+0000", 2024-09-15T00:11:48.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3250000000000008e-06 2024-09-15T00:11:48.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.673091+0000", 2024-09-15T00:11:48.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8227999999999999e-05 2024-09-15T00:11:48.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.673100+0000", 2024-09-15T00:11:48.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7690000000000002e-06 2024-09-15T00:11:48.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.692607+0000", 2024-09-15T00:11:48.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019507381000000001 2024-09-15T00:11:48.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.692721+0000", 2024-09-15T00:11:48.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000113548 2024-09-15T00:11:48.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:48.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.692858+0000", 2024-09-15T00:11:48.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013738699999999999 2024-09-15T00:11:48.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:48.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693379+0000", 2024-09-15T00:11:48.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052083100000000003 2024-09-15T00:11:48.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:48.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693710+0000", 2024-09-15T00:11:48.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033131600000000001 2024-09-15T00:11:48.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693743+0000", 2024-09-15T00:11:48.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2654000000000003e-05 2024-09-15T00:11:48.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693786+0000", 2024-09-15T00:11:48.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2966999999999997e-05 2024-09-15T00:11:48.339 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.339 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.339 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.340 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.340 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.340 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:48.341 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:48.341 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:48.342 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:48.342 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:48.343 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:48.343 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:48.343 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:48.343 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:48.344 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:48.344 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:48.344 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:48.344 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:48.339+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:48.442 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:48.441+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:48.450 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-15T00:11:48.464 INFO:teuthology.orchestra.run.smithi171.stdout:1914 punch from 0x1f2bf9c to 0x1f2cca8, (0xd0c bytes) 2024-09-15T00:11:48.469 INFO:teuthology.orchestra.run.smithi171.stdout:1915 trunc from 0xca2145a to 0xb6949ad 2024-09-15T00:11:48.487 INFO:teuthology.orchestra.run.smithi171.stdout:1916 write 0x42272c5 thru 0x4232b19 (0xb855 bytes) 2024-09-15T00:11:48.496 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:48.496 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:48.496 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:48.496 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:48.503 INFO:teuthology.orchestra.run.smithi171.stdout:1918 trunc from 0xb6949ad to 0x2139095 2024-09-15T00:11:48.509 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:48.530 INFO:teuthology.orchestra.run.smithi171.stdout:1919 write 0x6ebb338 thru 0x6ec6845 (0xb50e bytes) 2024-09-15T00:11:48.534 INFO:teuthology.orchestra.run.smithi171.stdout:1920 write 0x2bf3d37 thru 0x2c008a7 (0xcb71 bytes) 2024-09-15T00:11:48.539 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.543 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:48.541+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:48.549 INFO:teuthology.orchestra.run.smithi171.stdout:1923 writesame 0xb394975 thru 0xb3a058d data_size 0x7(0xbc19 bytes) 2024-09-15T00:11:48.556 INFO:teuthology.orchestra.run.smithi171.stdout:1924 trunc from 0xb3a058e to 0x5725035 2024-09-15T00:11:48.564 INFO:teuthology.orchestra.run.smithi171.stdout:1926 write 0xdc108b thru 0xdc7648 (0x65be bytes) 2024-09-15T00:11:48.578 INFO:teuthology.orchestra.run.smithi171.stdout:1927 write 0x94945f0 thru 0x9495068 (0xa79 bytes) 2024-09-15T00:11:48.585 INFO:teuthology.orchestra.run.smithi171.stdout:1930 trunc from 0x9495069 to 0xd243b95 2024-09-15T00:11:48.591 INFO:teuthology.orchestra.run.smithi171.stdout:1932 read 0x51dd70c thru 0x51e97c1 (0xc0b6 bytes) 2024-09-15T00:11:48.591 INFO:teuthology.orchestra.run.smithi171.stdout:1933 write 0x98f11a6 thru 0x9900da9 (0xfc04 bytes) 2024-09-15T00:11:48.598 INFO:teuthology.orchestra.run.smithi171.stdout:1934 trunc from 0xd243b95 to 0xe46940b 2024-09-15T00:11:48.612 INFO:teuthology.orchestra.run.smithi171.stdout:1936 trunc from 0xe46940b to 0x715b333 2024-09-15T00:11:48.621 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.621 INFO:teuthology.orchestra.run.smithi171.stdout:1940 read 0x5a84828 thru 0x5a915fe (0xcdd7 bytes) 2024-09-15T00:11:48.623 INFO:teuthology.orchestra.run.smithi171.stdout:1941 trunc from 0x715b333 to 0x52c3ec1 2024-09-15T00:11:48.629 INFO:teuthology.orchestra.run.smithi171.stdout:1942 writesame 0x5df6b8 thru 0x5ea5a4 data_size 0x21(0xaeed bytes) 2024-09-15T00:11:48.635 INFO:teuthology.orchestra.run.smithi171.stdout:1943 read 0x44643aa thru 0x446d808 (0x945f bytes) 2024-09-15T00:11:48.635 INFO:teuthology.orchestra.run.smithi171.stdout:1944 read 0x793f3c thru 0x7a3baa (0xfc6f bytes) 2024-09-15T00:11:48.644 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:48.642+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:48.648 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:48.648 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:48.648 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:48.648 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:48.648 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:48.650 INFO:teuthology.orchestra.run.smithi171.stdout:1945 writesame 0xc82ac69 thru 0xc832395 data_size 0x1(0x772d bytes) 2024-09-15T00:11:48.650 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.660 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:48.664 INFO:teuthology.orchestra.run.smithi171.stdout:1947 trunc from 0xc832396 to 0x17e7c53 2024-09-15T00:11:48.678 INFO:teuthology.orchestra.run.smithi171.stdout:1948 trunc from 0x17e7c53 to 0x1afd902 2024-09-15T00:11:48.683 INFO:teuthology.orchestra.run.smithi171.stdout:1949 read 0x17e4c4e thru 0x17f4253 (0xf606 bytes) 2024-09-15T00:11:48.683 INFO:teuthology.orchestra.run.smithi171.stdout:1950 punch from 0x10cd259 to 0x10d8107, (0xaeae bytes) 2024-09-15T00:11:48.684 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.700 INFO:teuthology.orchestra.run.smithi171.stdout:1954 write 0xaf8f8c1 thru 0xaf9bb43 (0xc283 bytes) 2024-09-15T00:11:48.703 INFO:teuthology.orchestra.run.smithi171.stdout:1955 read 0x5ade89a thru 0x5ae1567 (0x2cce bytes) 2024-09-15T00:11:48.703 INFO:teuthology.orchestra.run.smithi171.stdout:1956 writesame 0x98232b1 thru 0x9828e98 data_size 0x44(0x5be8 bytes) 2024-09-15T00:11:48.706 INFO:teuthology.orchestra.run.smithi171.stdout:1957 write 0x60dbc32 thru 0x60de0cd (0x249c bytes) 2024-09-15T00:11:48.709 INFO:teuthology.orchestra.run.smithi171.stdout:1959 write 0x75d1838 thru 0x75daf6d (0x9736 bytes) 2024-09-15T00:11:48.713 INFO:teuthology.orchestra.run.smithi171.stdout:1960 read 0x593b852 thru 0x59400b5 (0x4864 bytes) 2024-09-15T00:11:48.713 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.713 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.713 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.722 INFO:teuthology.orchestra.run.smithi171.stdout:1966 write 0xecf8601 thru 0xecfa57f (0x1f7f bytes) 2024-09-15T00:11:48.726 INFO:teuthology.orchestra.run.smithi171.stdout:1967 read 0x9c129e thru 0x9c8473 (0x71d6 bytes) 2024-09-15T00:11:48.727 INFO:teuthology.orchestra.run.smithi171.stdout:1968 read 0xea47c2b thru 0xea4c4f4 (0x48ca bytes) 2024-09-15T00:11:48.731 INFO:teuthology.orchestra.run.smithi171.stdout:1969 trunc from 0xecfa580 to 0xbbc42d8 2024-09-15T00:11:48.742 INFO:teuthology.orchestra.run.smithi171.stdout:1970 punch from 0x73a2bbb to 0x73b15dc, (0xea21 bytes) 2024-09-15T00:11:48.744 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:48.743+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:48.756 INFO:teuthology.orchestra.run.smithi171.stdout:1971 trunc from 0xbbc42d8 to 0xa12993 2024-09-15T00:11:48.776 INFO:teuthology.orchestra.run.smithi171.stdout:1972 read 0x9c0bdf thru 0x9c3f2a (0x334c bytes) 2024-09-15T00:11:48.777 INFO:teuthology.orchestra.run.smithi171.stdout:1973 punch from 0x17445f to 0x17a254, (0x5df5 bytes) 2024-09-15T00:11:48.787 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:48.787 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:48.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:48.787 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:48.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.787 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:462 3.0 e29/23 3:04479396:::rbd_id.image_client.0-clone2:head v 29'14, mlcod=27'12)", 2024-09-15T00:11:48.787 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:48.787 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.768014612999998, 2024-09-15T00:11:48.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030963029999999999, 2024-09-15T00:11:48.788 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.788 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.788 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.788 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:48.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:48.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:11:48.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:11:48.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016208+0000", 2024-09-15T00:11:48.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3669999999999997e-06 2024-09-15T00:11:48.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016250+0000", 2024-09-15T00:11:48.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1845999999999999e-05 2024-09-15T00:11:48.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016260+0000", 2024-09-15T00:11:48.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9750000000000002e-06 2024-09-15T00:11:48.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016342+0000", 2024-09-15T00:11:48.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1595999999999996e-05 2024-09-15T00:11:48.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016363+0000", 2024-09-15T00:11:48.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1073e-05 2024-09-15T00:11:48.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019246+0000", 2024-09-15T00:11:48.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028835240000000002 2024-09-15T00:11:48.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019296+0000", 2024-09-15T00:11:48.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9533999999999997e-05 2024-09-15T00:11:48.792 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.792 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.792 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.792 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:463 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'9, mlcod=27'7)", 2024-09-15T00:11:48.792 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:48.792 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.763401608999999, 2024-09-15T00:11:48.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040015709999999998, 2024-09-15T00:11:48.792 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.792 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.792 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.792 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:48.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:48.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:11:48.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7599999999999999e-07 2024-09-15T00:11:48.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020818+0000", 2024-09-15T00:11:48.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6609999999999999e-06 2024-09-15T00:11:48.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020826+0000", 2024-09-15T00:11:48.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4770000000000003e-06 2024-09-15T00:11:48.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020833+0000", 2024-09-15T00:11:48.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0809999999999996e-06 2024-09-15T00:11:48.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020878+0000", 2024-09-15T00:11:48.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4752000000000001e-05 2024-09-15T00:11:48.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020892+0000", 2024-09-15T00:11:48.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3899e-05 2024-09-15T00:11:48.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024767+0000", 2024-09-15T00:11:48.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038752259999999998 2024-09-15T00:11:48.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024814+0000", 2024-09-15T00:11:48.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7299000000000002e-05 2024-09-15T00:11:48.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.796 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.797 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:464 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'10, mlcod=27'8)", 2024-09-15T00:11:48.797 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:48.797 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.758481766999999, 2024-09-15T00:11:48.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048109290000000002, 2024-09-15T00:11:48.797 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.797 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.797 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.797 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:48.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:48.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:11:48.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-15T00:11:48.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025743+0000", 2024-09-15T00:11:48.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5789999999999993e-06 2024-09-15T00:11:48.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025752+0000", 2024-09-15T00:11:48.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8440000000000004e-06 2024-09-15T00:11:48.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025757+0000", 2024-09-15T00:11:48.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2610000000000001e-06 2024-09-15T00:11:48.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025796+0000", 2024-09-15T00:11:48.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9428999999999998e-05 2024-09-15T00:11:48.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025810+0000", 2024-09-15T00:11:48.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3129000000000001e-05 2024-09-15T00:11:48.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030514+0000", 2024-09-15T00:11:48.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047040099999999998 2024-09-15T00:11:48.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030544+0000", 2024-09-15T00:11:48.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9966000000000001e-05 2024-09-15T00:11:48.801 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.801 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.801 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.801 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:610 3.4 e29/23 3:255b0913:::rbd_data.11397bff333d.0000000000000039:head v 29'2, mlcod=0'0)", 2024-09-15T00:11:48.801 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:11:48.801 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.555738172000002, 2024-09-15T00:11:48.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003163014, 2024-09-15T00:11:48.801 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.801 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.801 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.802 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:11:48.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:11:48.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228477+0000", 2024-09-15T00:11:48.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1100000000000004e-07 2024-09-15T00:11:48.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228483+0000", 2024-09-15T00:11:48.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3720000000000001e-06 2024-09-15T00:11:48.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228497+0000", 2024-09-15T00:11:48.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3944000000000001e-05 2024-09-15T00:11:48.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228505+0000", 2024-09-15T00:11:48.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8140000000000001e-06 2024-09-15T00:11:48.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228549+0000", 2024-09-15T00:11:48.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4116000000000002e-05 2024-09-15T00:11:48.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228568+0000", 2024-09-15T00:11:48.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8473000000000001e-05 2024-09-15T00:11:48.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231599+0000", 2024-09-15T00:11:48.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030315590000000001 2024-09-15T00:11:48.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231639+0000", 2024-09-15T00:11:48.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9824999999999999e-05 2024-09-15T00:11:48.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.805 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.805 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:766 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'81, mlcod=29'78)", 2024-09-15T00:11:48.806 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:11:48.806 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.351189094999999, 2024-09-15T00:11:48.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029025880000000002, 2024-09-15T00:11:48.806 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.806 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.806 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.806 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:11:48.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:11:48.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433026+0000", 2024-09-15T00:11:48.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:11:48.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433030+0000", 2024-09-15T00:11:48.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6280000000000004e-06 2024-09-15T00:11:48.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433038+0000", 2024-09-15T00:11:48.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.072e-06 2024-09-15T00:11:48.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433045+0000", 2024-09-15T00:11:48.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1510000000000001e-06 2024-09-15T00:11:48.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433101+0000", 2024-09-15T00:11:48.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6336999999999999e-05 2024-09-15T00:11:48.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433113+0000", 2024-09-15T00:11:48.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1639e-05 2024-09-15T00:11:48.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435897+0000", 2024-09-15T00:11:48.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027845779999999998 2024-09-15T00:11:48.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435928+0000", 2024-09-15T00:11:48.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0689999999999999e-05 2024-09-15T00:11:48.810 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.810 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.810 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.810 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:832 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'100, mlcod=29'97)", 2024-09-15T00:11:48.810 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:48.810 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.258943595000002, 2024-09-15T00:11:48.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005498925, 2024-09-15T00:11:48.810 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.810 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.810 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.811 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:48.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:48.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:11:48.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7999999999999995e-07 2024-09-15T00:11:48.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525276+0000", 2024-09-15T00:11:48.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6739999999999996e-06 2024-09-15T00:11:48.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525287+0000", 2024-09-15T00:11:48.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0906e-05 2024-09-15T00:11:48.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525293+0000", 2024-09-15T00:11:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4409999999999998e-06 2024-09-15T00:11:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525329+0000", 2024-09-15T00:11:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5237000000000002e-05 2024-09-15T00:11:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525340+0000", 2024-09-15T00:11:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1782e-05 2024-09-15T00:11:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530710+0000", 2024-09-15T00:11:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053695460000000002 2024-09-15T00:11:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530770+0000", 2024-09-15T00:11:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9658999999999999e-05 2024-09-15T00:11:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:876 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'114, mlcod=29'111)", 2024-09-15T00:11:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.087904595000001, 2024-09-15T00:11:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028465830000000002, 2024-09-15T00:11:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:11:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2399999999999998e-07 2024-09-15T00:11:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696316+0000", 2024-09-15T00:11:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2800000000000003e-06 2024-09-15T00:11:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696326+0000", 2024-09-15T00:11:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0546e-05 2024-09-15T00:11:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696333+0000", 2024-09-15T00:11:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7660000000000001e-06 2024-09-15T00:11:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696369+0000", 2024-09-15T00:11:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5933000000000003e-05 2024-09-15T00:11:48.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696381+0000", 2024-09-15T00:11:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2417e-05 2024-09-15T00:11:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699124+0000", 2024-09-15T00:11:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027428000000000001 2024-09-15T00:11:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699156+0000", 2024-09-15T00:11:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2217e-05 2024-09-15T00:11:48.818 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1313 3.1f e30/28 3:fa9cde43:::rbd_data.11397bff333d.000000000000003a:head v 30'3, mlcod=26'1)", 2024-09-15T00:11:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:11:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.500064957999999, 2024-09-15T00:11:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044064359999999997, 2024-09-15T00:11:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:11:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:11:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284150+0000", 2024-09-15T00:11:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3799999999999999e-07 2024-09-15T00:11:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284228+0000", 2024-09-15T00:11:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:11:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284235+0000", 2024-09-15T00:11:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:11:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284239+0000", 2024-09-15T00:11:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3449999999999999e-06 2024-09-15T00:11:48.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284282+0000", 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2506000000000003e-05 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284297+0000", 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5523999999999999e-05 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288517+0000", 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042194019999999997 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288556+0000", 2024-09-15T00:11:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8952000000000002e-05 2024-09-15T00:11:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1321 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'7, mlcod=29'5)", 2024-09-15T00:11:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.475971698999999, 2024-09-15T00:11:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003131254, 2024-09-15T00:11:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:11:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:11:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308250+0000", 2024-09-15T00:11:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6109999999999999e-06 2024-09-15T00:11:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308264+0000", 2024-09-15T00:11:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4473e-05 2024-09-15T00:11:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308272+0000", 2024-09-15T00:11:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3540000000000002e-06 2024-09-15T00:11:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308340+0000", 2024-09-15T00:11:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8305000000000004e-05 2024-09-15T00:11:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308361+0000", 2024-09-15T00:11:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0409999999999999e-05 2024-09-15T00:11:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311332+0000", 2024-09-15T00:11:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029715950000000001 2024-09-15T00:11:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311374+0000", 2024-09-15T00:11:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1786000000000003e-05 2024-09-15T00:11:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.9 epoch 30/23 rep_tid 277 entries 30'217 (0'0) error 3:942069cf:::rbd_header.11397bff333d:head by client.4409.0:1373 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'214)", 2024-09-15T00:11:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:11:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.430419101999998, 2024-09-15T00:11:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034617960000000001, 2024-09-15T00:11:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:11:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:11:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:11:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353796+0000", 2024-09-15T00:11:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6699999999999999e-07 2024-09-15T00:11:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353803+0000", 2024-09-15T00:11:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.832e-06 2024-09-15T00:11:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353814+0000", 2024-09-15T00:11:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1815e-05 2024-09-15T00:11:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353822+0000", 2024-09-15T00:11:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7440000000000007e-06 2024-09-15T00:11:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353910+0000", 2024-09-15T00:11:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7491000000000006e-05 2024-09-15T00:11:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.357257+0000", 2024-09-15T00:11:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033474469999999999 2024-09-15T00:11:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1374 3.9 e30/23 3:942069cf:::rbd_header.11397bff333d:head v 30'218, mlcod=30'215)", 2024-09-15T00:11:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:11:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.429019459999999, 2024-09-15T00:11:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047299830000000001, 2024-09-15T00:11:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:11:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:11:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355196+0000", 2024-09-15T00:11:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3900000000000004e-07 2024-09-15T00:11:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355200+0000", 2024-09-15T00:11:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5700000000000003e-06 2024-09-15T00:11:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355210+0000", 2024-09-15T00:11:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6050000000000004e-06 2024-09-15T00:11:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355215+0000", 2024-09-15T00:11:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.933e-06 2024-09-15T00:11:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355249+0000", 2024-09-15T00:11:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4110999999999999e-05 2024-09-15T00:11:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355260+0000", 2024-09-15T00:11:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1235e-05 2024-09-15T00:11:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359895+0000", 2024-09-15T00:11:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046345550000000003 2024-09-15T00:11:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359925+0000", 2024-09-15T00:11:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0335e-05 2024-09-15T00:11:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1385 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'221, mlcod=30'218)", 2024-09-15T00:11:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:11:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.416562925000001, 2024-09-15T00:11:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030443950000000001, 2024-09-15T00:11:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:11:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:11:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367652+0000", 2024-09-15T00:11:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:11:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367657+0000", 2024-09-15T00:11:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5410000000000002e-06 2024-09-15T00:11:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367666+0000", 2024-09-15T00:11:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9280000000000005e-06 2024-09-15T00:11:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367671+0000", 2024-09-15T00:11:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7870000000000001e-06 2024-09-15T00:11:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367707+0000", 2024-09-15T00:11:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6102000000000001e-05 2024-09-15T00:11:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367719+0000", 2024-09-15T00:11:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1571e-05 2024-09-15T00:11:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370666+0000", 2024-09-15T00:11:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029474750000000002 2024-09-15T00:11:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370696+0000", 2024-09-15T00:11:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9402000000000001e-05 2024-09-15T00:11:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1662 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'300, mlcod=30'297)", 2024-09-15T00:11:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:11:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 21.996198287999999, 2024-09-15T00:11:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031296090000000002, 2024-09-15T00:11:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:11:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:11:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788017+0000", 2024-09-15T00:11:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8999999999999996e-07 2024-09-15T00:11:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788021+0000", 2024-09-15T00:11:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4970000000000003e-06 2024-09-15T00:11:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788027+0000", 2024-09-15T00:11:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1840000000000004e-06 2024-09-15T00:11:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788032+0000", 2024-09-15T00:11:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.135e-06 2024-09-15T00:11:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788070+0000", 2024-09-15T00:11:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7073999999999998e-05 2024-09-15T00:11:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788081+0000", 2024-09-15T00:11:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0987e-05 2024-09-15T00:11:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791116+0000", 2024-09-15T00:11:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030355209999999998 2024-09-15T00:11:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791146+0000", 2024-09-15T00:11:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9621e-05 2024-09-15T00:11:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1677 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'305, mlcod=30'302)", 2024-09-15T00:11:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:11:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 21.972469925999999, 2024-09-15T00:11:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031364320000000002, 2024-09-15T00:11:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:11:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811745+0000", 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811749+0000", 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9369999999999997e-06 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811756+0000", 2024-09-15T00:11:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:11:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811765+0000", 2024-09-15T00:11:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7930000000000003e-06 2024-09-15T00:11:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811797+0000", 2024-09-15T00:11:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2203999999999999e-05 2024-09-15T00:11:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811810+0000", 2024-09-15T00:11:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2359e-05 2024-09-15T00:11:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814852+0000", 2024-09-15T00:11:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003042276 2024-09-15T00:11:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814881+0000", 2024-09-15T00:11:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9037999999999999e-05 2024-09-15T00:11:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1724 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'9, mlcod=30'7)", 2024-09-15T00:11:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:11:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 21.903544241999999, 2024-09-15T00:11:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003115769, 2024-09-15T00:11:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880671+0000", 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4900000000000005e-07 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880678+0000", 2024-09-15T00:11:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0979999999999996e-06 2024-09-15T00:11:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880691+0000", 2024-09-15T00:11:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3288999999999999e-05 2024-09-15T00:11:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880698+0000", 2024-09-15T00:11:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9240000000000001e-06 2024-09-15T00:11:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880757+0000", 2024-09-15T00:11:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8866000000000002e-05 2024-09-15T00:11:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880790+0000", 2024-09-15T00:11:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2926999999999998e-05 2024-09-15T00:11:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883738+0000", 2024-09-15T00:11:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002948418 2024-09-15T00:11:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883786+0000", 2024-09-15T00:11:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7398000000000003e-05 2024-09-15T00:11:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1790 3.9 e30/23 3:9676e01b:::rbd_data.11397bff333d.0000000000000030:head v 30'338, mlcod=30'335)", 2024-09-15T00:11:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:11:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 21.824661872, 2024-09-15T00:11:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044496980000000002, 2024-09-15T00:11:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:11:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:11:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959553+0000", 2024-09-15T00:11:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3600000000000004e-07 2024-09-15T00:11:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959569+0000", 2024-09-15T00:11:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6152999999999999e-05 2024-09-15T00:11:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959577+0000", 2024-09-15T00:11:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5179999999999998e-06 2024-09-15T00:11:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959581+0000", 2024-09-15T00:11:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3669999999999999e-06 2024-09-15T00:11:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959619+0000", 2024-09-15T00:11:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8209e-05 2024-09-15T00:11:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959632+0000", 2024-09-15T00:11:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2405000000000001e-05 2024-09-15T00:11:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.963954+0000", 2024-09-15T00:11:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043226660000000002 2024-09-15T00:11:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.964002+0000", 2024-09-15T00:11:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7843999999999998e-05 2024-09-15T00:11:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:3177 3.4 e33/23 3:23818c00:::rbd_data.1139b531b717.0000000000000022:head v 33'58, mlcod=33'56)", 2024-09-15T00:11:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:11:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.126373306000001, 2024-09-15T00:11:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030564120000000001, 2024-09-15T00:11:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:11:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:11:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657842+0000", 2024-09-15T00:11:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.19e-06 2024-09-15T00:11:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657853+0000", 2024-09-15T00:11:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:11:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657871+0000", 2024-09-15T00:11:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8233e-05 2024-09-15T00:11:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657880+0000", 2024-09-15T00:11:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8319999999999995e-06 2024-09-15T00:11:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657949+0000", 2024-09-15T00:11:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8847e-05 2024-09-15T00:11:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657977+0000", 2024-09-15T00:11:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8308000000000001e-05 2024-09-15T00:11:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.660851+0000", 2024-09-15T00:11:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028743190000000002 2024-09-15T00:11:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.660897+0000", 2024-09-15T00:11:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6142e-05 2024-09-15T00:11:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:3602 3.1f e34/28 3:fd28af01:::rbd_data.1139b531b717.0000000000000032:head v 34'24, mlcod=33'22)", 2024-09-15T00:11:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:11:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.738975911000001, 2024-09-15T00:11:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034159709999999999, 2024-09-15T00:11:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:11:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:11:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045239+0000", 2024-09-15T00:11:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:11:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045252+0000", 2024-09-15T00:11:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2891000000000001e-05 2024-09-15T00:11:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045263+0000", 2024-09-15T00:11:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0875e-05 2024-09-15T00:11:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045271+0000", 2024-09-15T00:11:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3529999999999995e-06 2024-09-15T00:11:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045338+0000", 2024-09-15T00:11:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7377000000000002e-05 2024-09-15T00:11:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045359+0000", 2024-09-15T00:11:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1078e-05 2024-09-15T00:11:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.048621+0000", 2024-09-15T00:11:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003261543 2024-09-15T00:11:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.048654+0000", 2024-09-15T00:11:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3550999999999998e-05 2024-09-15T00:11:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:4253 3.4 e35/23 3:242f5431:::rbd_data.11396389f8f2.0000000000000000:head v 35'87, mlcod=35'85)", 2024-09-15T00:11:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:11:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 17.391487052999999, 2024-09-15T00:11:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002944997, 2024-09-15T00:11:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:11:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:11:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392728+0000", 2024-09-15T00:11:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.06e-07 2024-09-15T00:11:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392782+0000", 2024-09-15T00:11:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.448e-05 2024-09-15T00:11:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392794+0000", 2024-09-15T00:11:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1768e-05 2024-09-15T00:11:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392801+0000", 2024-09-15T00:11:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9589999999999998e-06 2024-09-15T00:11:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392845+0000", 2024-09-15T00:11:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3884999999999997e-05 2024-09-15T00:11:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392860+0000", 2024-09-15T00:11:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5364999999999999e-05 2024-09-15T00:11:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.395641+0000", 2024-09-15T00:11:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027807629999999999 2024-09-15T00:11:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.395672+0000", 2024-09-15T00:11:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1170999999999997e-05 2024-09-15T00:11:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:18791 3.0 e38/23 3:05ecfc97:::rbd_data.11398e1a091a.0000000000000030:head v 38'66, mlcod=38'64)", 2024-09-15T00:11:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:11:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.748571517, 2024-09-15T00:11:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029447940000000001, 2024-09-15T00:11:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:11:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:11:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035644+0000", 2024-09-15T00:11:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4e-07 2024-09-15T00:11:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035684+0000", 2024-09-15T00:11:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9857999999999997e-05 2024-09-15T00:11:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035694+0000", 2024-09-15T00:11:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0259e-05 2024-09-15T00:11:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035699+0000", 2024-09-15T00:11:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6389999999999997e-06 2024-09-15T00:11:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035745+0000", 2024-09-15T00:11:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5640999999999999e-05 2024-09-15T00:11:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035764+0000", 2024-09-15T00:11:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9368000000000002e-05 2024-09-15T00:11:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.038551+0000", 2024-09-15T00:11:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027860879999999999 2024-09-15T00:11:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.038588+0000", 2024-09-15T00:11:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7101000000000003e-05 2024-09-15T00:11:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:48.877 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:48.879 INFO:teuthology.orchestra.run.smithi171.stdout:1974 writesame 0x8a73bcb thru 0x8a7a0f6 data_size 0x64(0x652c bytes) 2024-09-15T00:11:48.879 INFO:teuthology.orchestra.run.smithi171.stdout:1975 punch from 0x5ebf8d1 to 0x5ec554c, (0x5c7b bytes) 2024-09-15T00:11:48.879 INFO:teuthology.orchestra.run.smithi171.stdout:1978 read 0x4d846c2 thru 0x4d885cb (0x3f0a bytes) 2024-09-15T00:11:48.879 INFO:teuthology.orchestra.run.smithi171.stdout:1979 write 0xbdd022 thru 0xbec281 (0xf260 bytes) 2024-09-15T00:11:48.880 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:48.881 INFO:teuthology.orchestra.run.smithi171.stdout:1980 read 0x2d96bb8 thru 0x2d9b44d (0x4896 bytes) 2024-09-15T00:11:48.881 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.882 INFO:teuthology.orchestra.run.smithi171.stdout:1982 read 0x552f8da thru 0x553d692 (0xddb9 bytes) 2024-09-15T00:11:48.882 INFO:teuthology.orchestra.run.smithi171.stdout:1984 punch from 0x76e1d58 to 0x76e55b5, (0x385d bytes) 2024-09-15T00:11:48.882 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.882 INFO:teuthology.orchestra.run.smithi171.stdout:1988 punch from 0x606e07c to 0x607a236, (0xc1ba bytes) 2024-09-15T00:11:48.882 INFO:teuthology.orchestra.run.smithi171.stdout:1989 writesame 0xc3abbef thru 0xc3b5e09 data_size 0x57(0xa21b bytes) 2024-09-15T00:11:48.882 INFO:teuthology.orchestra.run.smithi171.stdout:1990 write 0xb82b6e5 thru 0xb836921 (0xb23d bytes) 2024-09-15T00:11:48.882 INFO:teuthology.orchestra.run.smithi171.stdout:1991 trunc from 0xc3b5e0a to 0x3278736 2024-09-15T00:11:48.882 INFO:teuthology.orchestra.run.smithi171.stdout:1992 write 0xcfb1a81 thru 0xcfb9986 (0x7f06 bytes) 2024-09-15T00:11:48.882 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.882 INFO:teuthology.orchestra.run.smithi171.stdout:1994 punch from 0x29a19d9 to 0x29afefb, (0xe522 bytes) 2024-09-15T00:11:48.882 INFO:teuthology.orchestra.run.smithi171.stdout:1995 punch from 0x6ba1d51 to 0x6bb02fb, (0xe5aa bytes) 2024-09-15T00:11:48.882 INFO:teuthology.orchestra.run.smithi171.stdout:1996 read 0xbe8dcaa thru 0xbe961c5 (0x851c bytes) 2024-09-15T00:11:48.882 INFO:teuthology.orchestra.run.smithi171.stdout:1997 read 0xc9041b6 thru 0xc910315 (0xc160 bytes) 2024-09-15T00:11:48.882 INFO:teuthology.orchestra.run.smithi171.stdout:1999 trunc from 0xcfb9987 to 0x8421399 2024-09-15T00:11:48.883 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.883 INFO:teuthology.orchestra.run.smithi171.stdout:2001 punch from 0x119bda3 to 0x11a80b1, (0xc30e bytes) 2024-09-15T00:11:48.883 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.883 INFO:teuthology.orchestra.run.smithi171.stdout:2004 write 0x433fda8 thru 0x43437bd (0x3a16 bytes) 2024-09-15T00:11:48.883 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:48.877+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:48.883 INFO:teuthology.orchestra.run.smithi171.stdout:2005 write 0x25f01a thru 0x268374 (0x935b bytes) 2024-09-15T00:11:48.892 INFO:teuthology.orchestra.run.smithi171.stdout:2009 trunc from 0x8421399 to 0x7fb55e4 2024-09-15T00:11:48.896 INFO:teuthology.orchestra.run.smithi171.stdout:2010 writesame 0x1cd1eec thru 0x1cdcfa0 data_size 0x11(0xb0b5 bytes) 2024-09-15T00:11:48.903 INFO:teuthology.orchestra.run.smithi171.stdout:2011 read 0x2eccc3c thru 0x2edc1c9 (0xf58e bytes) 2024-09-15T00:11:48.904 INFO:teuthology.orchestra.run.smithi171.stdout:2012 punch from 0x5c7fe0a to 0x5c8ec12, (0xee08 bytes) 2024-09-15T00:11:48.904 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.911 INFO:teuthology.orchestra.run.smithi171.stdout:2014 writesame 0x9f56630 thru 0x9f5ea42 data_size 0x1(0x8413 bytes) 2024-09-15T00:11:48.911 INFO:teuthology.orchestra.run.smithi171.stdout:2015 write 0x5d1317 thru 0x5de659 (0xd343 bytes) 2024-09-15T00:11:48.914 INFO:teuthology.orchestra.run.smithi171.stdout:2016 write 0x19137b6 thru 0x1918d68 (0x55b3 bytes) 2024-09-15T00:11:48.928 INFO:teuthology.orchestra.run.smithi171.stdout:2017 write 0xe283619 thru 0xe28f910 (0xc2f8 bytes) 2024-09-15T00:11:48.931 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.931 INFO:teuthology.orchestra.run.smithi171.stdout:2019 read 0x15d8f4f thru 0x15de48b (0x553d bytes) 2024-09-15T00:11:48.931 INFO:teuthology.orchestra.run.smithi171.stdout:2021 punch from 0x821ce46 to 0x821f8ab, (0x2a65 bytes) 2024-09-15T00:11:48.931 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.931 INFO:teuthology.orchestra.run.smithi171.stdout:2024 read 0x61d4603 thru 0x61de6b5 (0xa0b3 bytes) 2024-09-15T00:11:48.931 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.931 INFO:teuthology.orchestra.run.smithi171.stdout:2027 punch from 0xa063730 to 0xa06977e, (0x604e bytes) 2024-09-15T00:11:48.931 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.931 INFO:teuthology.orchestra.run.smithi171.stdout:2031 read 0x60b55b2 thru 0x60bb192 (0x5be1 bytes) 2024-09-15T00:11:48.932 INFO:teuthology.orchestra.run.smithi171.stdout:2032 write 0x1755625 thru 0x175a91e (0x52fa bytes) 2024-09-15T00:11:48.934 INFO:teuthology.orchestra.run.smithi171.stdout:2034 writesame 0x8e7340f thru 0x8e74923 data_size 0x7(0x1515 bytes) 2024-09-15T00:11:48.937 INFO:teuthology.orchestra.run.smithi171.stdout:2036 write 0x980d228 thru 0x981c0fd (0xeed6 bytes) 2024-09-15T00:11:48.942 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.942 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.942 INFO:teuthology.orchestra.run.smithi171.stdout:2040 read 0x1f69e64 thru 0x1f71d54 (0x7ef1 bytes) 2024-09-15T00:11:48.942 INFO:teuthology.orchestra.run.smithi171.stdout:2041 writesame 0x2ebc202 thru 0x2ebf5ef data_size 0x22(0x33ee bytes) 2024-09-15T00:11:48.955 INFO:teuthology.orchestra.run.smithi171.stdout:2043 trunc from 0xe28f911 to 0x6e13617 2024-09-15T00:11:48.975 INFO:teuthology.orchestra.run.smithi171.stdout:2044 writesame 0xc16be2f thru 0xc17340e data_size 0x5c(0x75e0 bytes) 2024-09-15T00:11:48.980 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:48.978+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:48.980 INFO:teuthology.orchestra.run.smithi171.stdout:2045 write 0xc57e7a5 thru 0xc587819 (0x9075 bytes) 2024-09-15T00:11:48.983 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.984 INFO:teuthology.orchestra.run.smithi171.stdout:2048 writesame 0x2a31835 thru 0x2a3f350 data_size 0x25(0xdb1c bytes) 2024-09-15T00:11:48.989 INFO:teuthology.orchestra.run.smithi171.stdout:2049 read 0xb568ccd thru 0xb56e26a (0x559e bytes) 2024-09-15T00:11:48.989 INFO:teuthology.orchestra.run.smithi171.stdout:2050 punch from 0x47f231a to 0x47f77e9, (0x54cf bytes) 2024-09-15T00:11:48.989 INFO:teuthology.orchestra.run.smithi171.stdout:2051 read 0x9762843 thru 0x9763e99 (0x1657 bytes) 2024-09-15T00:11:48.989 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.990 INFO:teuthology.orchestra.run.smithi171.stdout:2053 read 0x5a85de5 thru 0x5a907f8 (0xaa14 bytes) 2024-09-15T00:11:48.990 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.990 INFO:teuthology.orchestra.run.smithi171.stdout:2055 read 0x4e1d245 thru 0x4e247ac (0x7568 bytes) 2024-09-15T00:11:48.990 INFO:teuthology.orchestra.run.smithi171.stdout:2056 read 0xae22062 thru 0xae23b79 (0x1b18 bytes) 2024-09-15T00:11:48.990 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:48.990 INFO:teuthology.orchestra.run.smithi171.stdout:2058 read 0x3cca679 thru 0x3ccab6c (0x4f4 bytes) 2024-09-15T00:11:48.994 INFO:teuthology.orchestra.run.smithi171.stdout:2061 write 0xcbdfcb8 thru 0xcbe180e (0x1b57 bytes) 2024-09-15T00:11:48.997 INFO:teuthology.orchestra.run.smithi171.stdout:2062 write 0x93fd1f5 thru 0x94015b2 (0x43be bytes) 2024-09-15T00:11:48.999 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:48.999 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:49.000 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:49.000 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:49.001 INFO:teuthology.orchestra.run.smithi171.stdout:2063 write 0x41279bc thru 0x412ad10 (0x3355 bytes) 2024-09-15T00:11:49.004 INFO:teuthology.orchestra.run.smithi171.stdout:2064 write 0xbfd76fd thru 0xbfe0f08 (0x980c bytes) 2024-09-15T00:11:49.008 INFO:teuthology.orchestra.run.smithi171.stdout:2065 read 0x424b8ed thru 0x424f6a1 (0x3db5 bytes) 2024-09-15T00:11:49.008 INFO:teuthology.orchestra.run.smithi171.stdout:2066 write 0x8933287 thru 0x8940eea (0xdc64 bytes) 2024-09-15T00:11:49.010 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:49.013 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:49.015 INFO:teuthology.orchestra.run.smithi171.stdout:2068 trunc from 0xcbe180f to 0xd70858e 2024-09-15T00:11:49.024 INFO:teuthology.orchestra.run.smithi171.stdout:2069 write 0xe4bc5b1 thru 0xe4c6ecb (0xa91b bytes) 2024-09-15T00:11:49.034 INFO:teuthology.orchestra.run.smithi171.stdout:2071 trunc from 0xe4c6ecc to 0xa6f2be7 2024-09-15T00:11:49.047 INFO:teuthology.orchestra.run.smithi171.stdout:2072 read 0x3e34093 thru 0x3e39ce5 (0x5c53 bytes) 2024-09-15T00:11:49.047 INFO:teuthology.orchestra.run.smithi171.stdout:2073 read 0x5e822c4 thru 0x5e8d396 (0xb0d3 bytes) 2024-09-15T00:11:49.047 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:49.048 INFO:teuthology.orchestra.run.smithi171.stdout:2075 clone 16 order 21 su 8192 sc 4 2024-09-15T00:11:49.081 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:49.079+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:49.119 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:49.119 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:49.129 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:49.132 INFO:teuthology.orchestra.run.smithi171.stderr:marked out osd.7. 2024-09-15T00:11:49.188 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:49.187+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:49.189 INFO:teuthology.orchestra.run.smithi171.stdout:flattening image image_client.0-clone15 2024-09-15T00:11:49.245 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.245 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:47798 3.7 3:e6004524:::rbd_data.113916cf513.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1789239~44328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.387407+0000", 2024-09-15T00:11:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8548271349999998, 2024-09-15T00:11:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033697973999999999, 2024-09-15T00:11:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 47798 2024-09-15T00:11:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.246 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.387407+0000", 2024-09-15T00:11:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.387407+0000", 2024-09-15T00:11:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.387408+0000", 2024-09-15T00:11:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2580000000000001e-06 2024-09-15T00:11:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389094+0000", 2024-09-15T00:11:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001685572 2024-09-15T00:11:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389116+0000", 2024-09-15T00:11:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2379999999999999e-05 2024-09-15T00:11:49.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389126+0000", 2024-09-15T00:11:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0044000000000001e-05 2024-09-15T00:11:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389203+0000", 2024-09-15T00:11:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6945000000000003e-05 2024-09-15T00:11:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389381+0000", 2024-09-15T00:11:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000177908 2024-09-15T00:11:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389890+0000", 2024-09-15T00:11:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00050844599999999996 2024-09-15T00:11:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.420647+0000", 2024-09-15T00:11:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030757201000000001 2024-09-15T00:11:49.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.420970+0000", 2024-09-15T00:11:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032290200000000002 2024-09-15T00:11:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.421037+0000", 2024-09-15T00:11:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6883000000000003e-05 2024-09-15T00:11:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.421105+0000", 2024-09-15T00:11:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8435000000000005e-05 2024-09-15T00:11:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:47892 3.12 e47/28 3:4ef0cbcd:::rbd_data.113916cf513.0000000000000000:head v 47'139, mlcod=45'137)", 2024-09-15T00:11:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.437387+0000", 2024-09-15T00:11:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8048473459999999, 2024-09-15T00:11:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021482557999999999, 2024-09-15T00:11:49.252 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.437387+0000", 2024-09-15T00:11:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.437387+0000", 2024-09-15T00:11:49.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.437389+0000", 2024-09-15T00:11:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6390000000000001e-06 2024-09-15T00:11:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.440729+0000", 2024-09-15T00:11:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033401490000000002 2024-09-15T00:11:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.440750+0000", 2024-09-15T00:11:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1376e-05 2024-09-15T00:11:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.440767+0000", 2024-09-15T00:11:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7127000000000001e-05 2024-09-15T00:11:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.440847+0000", 2024-09-15T00:11:49.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689000000000002e-05 2024-09-15T00:11:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.440876+0000", 2024-09-15T00:11:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8663e-05 2024-09-15T00:11:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.458828+0000", 2024-09-15T00:11:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017952633999999999 2024-09-15T00:11:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.458869+0000", 2024-09-15T00:11:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1281e-05 2024-09-15T00:11:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48090 3.b 3:d42b6c47:::rbd_data.113916cf513.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 13893632~5352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.534140+0000", 2024-09-15T00:11:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.7080943619999998, 2024-09-15T00:11:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022685002999999999, 2024-09-15T00:11:49.257 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48090 2024-09-15T00:11:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.534140+0000", 2024-09-15T00:11:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.534140+0000", 2024-09-15T00:11:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.534141+0000", 2024-09-15T00:11:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2699999999999999e-06 2024-09-15T00:11:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.534970+0000", 2024-09-15T00:11:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00082854499999999998 2024-09-15T00:11:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.534991+0000", 2024-09-15T00:11:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1058999999999999e-05 2024-09-15T00:11:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.535004+0000", 2024-09-15T00:11:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2830000000000001e-05 2024-09-15T00:11:49.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.535078+0000", 2024-09-15T00:11:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4695000000000002e-05 2024-09-15T00:11:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.535209+0000", 2024-09-15T00:11:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013081900000000001 2024-09-15T00:11:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:49.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.535595+0000", 2024-09-15T00:11:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000385993 2024-09-15T00:11:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.553922+0000", 2024-09-15T00:11:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018327011000000001 2024-09-15T00:11:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.556695+0000", 2024-09-15T00:11:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00277261 2024-09-15T00:11:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.556785+0000", 2024-09-15T00:11:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0649000000000006e-05 2024-09-15T00:11:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.556825+0000", 2024-09-15T00:11:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9521999999999999e-05 2024-09-15T00:11:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.263 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48896 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.608433+0000", 2024-09-15T00:11:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6338015920000002, 2024-09-15T00:11:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019236590000000001, 2024-09-15T00:11:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.264 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48896 2024-09-15T00:11:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608433+0000", 2024-09-15T00:11:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608433+0000", 2024-09-15T00:11:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608433+0000", 2024-09-15T00:11:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.46e-07 2024-09-15T00:11:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608435+0000", 2024-09-15T00:11:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:11:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608441+0000", 2024-09-15T00:11:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8969999999999996e-06 2024-09-15T00:11:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608446+0000", 2024-09-15T00:11:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.921e-06 2024-09-15T00:11:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627487+0000", 2024-09-15T00:11:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019040973999999999 2024-09-15T00:11:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627572+0000", 2024-09-15T00:11:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4846999999999996e-05 2024-09-15T00:11:49.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627669+0000", 2024-09-15T00:11:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7381999999999997e-05 2024-09-15T00:11:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48981 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 2752512~73728] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.269 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609227+0000", 2024-09-15T00:11:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6330072539999998, 2024-09-15T00:11:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019843092, 2024-09-15T00:11:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48981 2024-09-15T00:11:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609227+0000", 2024-09-15T00:11:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609227+0000", 2024-09-15T00:11:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609227+0000", 2024-09-15T00:11:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5900000000000002e-07 2024-09-15T00:11:49.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609230+0000", 2024-09-15T00:11:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2089999999999999e-06 2024-09-15T00:11:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609236+0000", 2024-09-15T00:11:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4919999999999998e-06 2024-09-15T00:11:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609241+0000", 2024-09-15T00:11:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0440000000000003e-06 2024-09-15T00:11:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628352+0000", 2024-09-15T00:11:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019111145999999999 2024-09-15T00:11:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628433+0000", 2024-09-15T00:11:49.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0887e-05 2024-09-15T00:11:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629070+0000", 2024-09-15T00:11:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00063685499999999997 2024-09-15T00:11:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48982 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 4587520~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609275+0000", 2024-09-15T00:11:49.274 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6329587869999997, 2024-09-15T00:11:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020353955999999999, 2024-09-15T00:11:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48982 2024-09-15T00:11:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609275+0000", 2024-09-15T00:11:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609275+0000", 2024-09-15T00:11:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609276+0000", 2024-09-15T00:11:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5799999999999999e-07 2024-09-15T00:11:49.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609279+0000", 2024-09-15T00:11:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5830000000000001e-06 2024-09-15T00:11:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609285+0000", 2024-09-15T00:11:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8240000000000002e-06 2024-09-15T00:11:49.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609291+0000", 2024-09-15T00:11:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1399999999999999e-06 2024-09-15T00:11:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629107+0000", 2024-09-15T00:11:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019816674999999999 2024-09-15T00:11:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629158+0000", 2024-09-15T00:11:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1005999999999999e-05 2024-09-15T00:11:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629629+0000", 2024-09-15T00:11:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047116999999999998 2024-09-15T00:11:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48983 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609298+0000", 2024-09-15T00:11:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6329360880000001, 2024-09-15T00:11:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020455086000000001, 2024-09-15T00:11:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48983 2024-09-15T00:11:49.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609298+0000", 2024-09-15T00:11:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609298+0000", 2024-09-15T00:11:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609299+0000", 2024-09-15T00:11:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7899999999999998e-07 2024-09-15T00:11:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609301+0000", 2024-09-15T00:11:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0379999999999998e-06 2024-09-15T00:11:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609307+0000", 2024-09-15T00:11:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2600000000000002e-06 2024-09-15T00:11:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609312+0000", 2024-09-15T00:11:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4009999999999996e-06 2024-09-15T00:11:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629651+0000", 2024-09-15T00:11:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020338438 2024-09-15T00:11:49.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629696+0000", 2024-09-15T00:11:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5123000000000001e-05 2024-09-15T00:11:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629753+0000", 2024-09-15T00:11:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7247000000000003e-05 2024-09-15T00:11:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48984 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609319+0000", 2024-09-15T00:11:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6329151509999997, 2024-09-15T00:11:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020570343000000001, 2024-09-15T00:11:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.285 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48984 2024-09-15T00:11:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609319+0000", 2024-09-15T00:11:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609319+0000", 2024-09-15T00:11:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609319+0000", 2024-09-15T00:11:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3099999999999998e-07 2024-09-15T00:11:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609321+0000", 2024-09-15T00:11:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:11:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609328+0000", 2024-09-15T00:11:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2149999999999998e-06 2024-09-15T00:11:49.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609333+0000", 2024-09-15T00:11:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.22e-06 2024-09-15T00:11:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629783+0000", 2024-09-15T00:11:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020450342 2024-09-15T00:11:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629829+0000", 2024-09-15T00:11:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5782999999999999e-05 2024-09-15T00:11:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629889+0000", 2024-09-15T00:11:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0393999999999998e-05 2024-09-15T00:11:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48985 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609339+0000", 2024-09-15T00:11:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.632895177, 2024-09-15T00:11:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020671998, 2024-09-15T00:11:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.290 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48985 2024-09-15T00:11:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609339+0000", 2024-09-15T00:11:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609339+0000", 2024-09-15T00:11:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609339+0000", 2024-09-15T00:11:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:11:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609341+0000", 2024-09-15T00:11:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9769999999999999e-06 2024-09-15T00:11:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609351+0000", 2024-09-15T00:11:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6220000000000004e-06 2024-09-15T00:11:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609356+0000", 2024-09-15T00:11:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1660000000000002e-06 2024-09-15T00:11:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629914+0000", 2024-09-15T00:11:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020558205 2024-09-15T00:11:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629954+0000", 2024-09-15T00:11:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9619999999999997e-05 2024-09-15T00:11:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630011+0000", 2024-09-15T00:11:49.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7058999999999997e-05 2024-09-15T00:11:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48986 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609363+0000", 2024-09-15T00:11:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6328713080000004, 2024-09-15T00:11:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020764550999999999, 2024-09-15T00:11:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.295 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48986 2024-09-15T00:11:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609363+0000", 2024-09-15T00:11:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609363+0000", 2024-09-15T00:11:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609363+0000", 2024-09-15T00:11:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:11:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609366+0000", 2024-09-15T00:11:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.21e-06 2024-09-15T00:11:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609372+0000", 2024-09-15T00:11:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6390000000000003e-06 2024-09-15T00:11:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609377+0000", 2024-09-15T00:11:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000002e-06 2024-09-15T00:11:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630035+0000", 2024-09-15T00:11:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020657847 2024-09-15T00:11:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630076+0000", 2024-09-15T00:11:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1597000000000002e-05 2024-09-15T00:11:49.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630127+0000", 2024-09-15T00:11:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1232e-05 2024-09-15T00:11:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48987 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 13762560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609383+0000", 2024-09-15T00:11:49.300 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6328508399999997, 2024-09-15T00:11:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020904947, 2024-09-15T00:11:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48987 2024-09-15T00:11:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609383+0000", 2024-09-15T00:11:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609383+0000", 2024-09-15T00:11:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609384+0000", 2024-09-15T00:11:49.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:11:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609386+0000", 2024-09-15T00:11:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1619999999999998e-06 2024-09-15T00:11:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609392+0000", 2024-09-15T00:11:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-15T00:11:49.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609399+0000", 2024-09-15T00:11:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3570000000000003e-06 2024-09-15T00:11:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630147+0000", 2024-09-15T00:11:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020748195000000001 2024-09-15T00:11:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630201+0000", 2024-09-15T00:11:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3838000000000002e-05 2024-09-15T00:11:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630288+0000", 2024-09-15T00:11:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7484999999999993e-05 2024-09-15T00:11:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49030 3.b 3:d0a51340:::rbd_data.113916cf513.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.662376+0000", 2024-09-15T00:11:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5798580170000003, 2024-09-15T00:11:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.028039436000000001, 2024-09-15T00:11:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49030 2024-09-15T00:11:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.662376+0000", 2024-09-15T00:11:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.662376+0000", 2024-09-15T00:11:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.662379+0000", 2024-09-15T00:11:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.168e-06 2024-09-15T00:11:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663508+0000", 2024-09-15T00:11:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011290370000000001 2024-09-15T00:11:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663539+0000", 2024-09-15T00:11:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1006000000000001e-05 2024-09-15T00:11:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663551+0000", 2024-09-15T00:11:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1708e-05 2024-09-15T00:11:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663628+0000", 2024-09-15T00:11:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6633999999999998e-05 2024-09-15T00:11:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663783+0000", 2024-09-15T00:11:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015484099999999999 2024-09-15T00:11:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.664331+0000", 2024-09-15T00:11:49.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00054832600000000002 2024-09-15T00:11:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.690059+0000", 2024-09-15T00:11:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025727612 2024-09-15T00:11:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.690300+0000", 2024-09-15T00:11:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024122600000000001 2024-09-15T00:11:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.690360+0000", 2024-09-15T00:11:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0458000000000003e-05 2024-09-15T00:11:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.690416+0000", 2024-09-15T00:11:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5420000000000001e-05 2024-09-15T00:11:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:49026 3.19 e47/23 3:9c3e88bf:::rbd_data.113916cf513.0000000000000002:head v 47'92, mlcod=45'89)", 2024-09-15T00:11:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.665261+0000", 2024-09-15T00:11:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5769729610000001, 2024-09-15T00:11:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019832186000000002, 2024-09-15T00:11:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.665261+0000", 2024-09-15T00:11:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.665261+0000", 2024-09-15T00:11:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.665265+0000", 2024-09-15T00:11:49.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4570000000000001e-06 2024-09-15T00:11:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667479+0000", 2024-09-15T00:11:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022139260000000002 2024-09-15T00:11:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667492+0000", 2024-09-15T00:11:49.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3084e-05 2024-09-15T00:11:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667500+0000", 2024-09-15T00:11:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7609999999999996e-06 2024-09-15T00:11:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667569+0000", 2024-09-15T00:11:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8359000000000001e-05 2024-09-15T00:11:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667607+0000", 2024-09-15T00:11:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7716999999999998e-05 2024-09-15T00:11:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.685060+0000", 2024-09-15T00:11:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017453344999999999 2024-09-15T00:11:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.685093+0000", 2024-09-15T00:11:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3537000000000001e-05 2024-09-15T00:11:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56198 3.2 3:44ae5a28:::rbd_data.1139167e758e.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:11:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.238999+0000", 2024-09-15T00:11:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.0032355660000001, 2024-09-15T00:11:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037097333000000003, 2024-09-15T00:11:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.318 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56198 2024-09-15T00:11:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.238999+0000", 2024-09-15T00:11:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.238999+0000", 2024-09-15T00:11:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.239000+0000", 2024-09-15T00:11:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.359e-06 2024-09-15T00:11:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242065+0000", 2024-09-15T00:11:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030647230000000001 2024-09-15T00:11:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242079+0000", 2024-09-15T00:11:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3801999999999999e-05 2024-09-15T00:11:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242108+0000", 2024-09-15T00:11:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9889000000000001e-05 2024-09-15T00:11:49.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242177+0000", 2024-09-15T00:11:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9072000000000005e-05 2024-09-15T00:11:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242332+0000", 2024-09-15T00:11:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000154816 2024-09-15T00:11:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:49.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.243189+0000", 2024-09-15T00:11:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00085691200000000004 2024-09-15T00:11:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.267650+0000", 2024-09-15T00:11:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024461146999999999 2024-09-15T00:11:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.275977+0000", 2024-09-15T00:11:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0083269729999999997 2024-09-15T00:11:49.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276040+0000", 2024-09-15T00:11:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2497999999999997e-05 2024-09-15T00:11:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276096+0000", 2024-09-15T00:11:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6141999999999999e-05 2024-09-15T00:11:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.324 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56199 3.2 3:44ae5a28:::rbd_data.1139167e758e.0000000000000000:head [write 0~0] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:11:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242772+0000", 2024-09-15T00:11:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.99946216300000001, 2024-09-15T00:11:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033898029000000003, 2024-09-15T00:11:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:49.325 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56199 2024-09-15T00:11:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242772+0000", 2024-09-15T00:11:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242772+0000", 2024-09-15T00:11:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242774+0000", 2024-09-15T00:11:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.483e-06 2024-09-15T00:11:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242782+0000", 2024-09-15T00:11:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9740000000000002e-06 2024-09-15T00:11:49.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242806+0000", 2024-09-15T00:11:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4689e-05 2024-09-15T00:11:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242815+0000", 2024-09-15T00:11:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3410000000000003e-06 2024-09-15T00:11:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.265144+0000", 2024-09-15T00:11:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022329244000000002 2024-09-15T00:11:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.265237+0000", 2024-09-15T00:11:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3154999999999998e-05 2024-09-15T00:11:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.265426+0000", 2024-09-15T00:11:49.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018955600000000001 2024-09-15T00:11:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.267912+0000", 2024-09-15T00:11:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024857880000000001 2024-09-15T00:11:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276583+0000", 2024-09-15T00:11:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0086705689999999995 2024-09-15T00:11:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276634+0000", 2024-09-15T00:11:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1465000000000003e-05 2024-09-15T00:11:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276670+0000", 2024-09-15T00:11:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5765000000000001e-05 2024-09-15T00:11:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:11:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:11:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.99939011700000002, 2024-09-15T00:11:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:11:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.332 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:11:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:11:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:11:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:11:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:11:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:11:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:11:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:11:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:11:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:11:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:11:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:11:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:11:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:11:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:11:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:11:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:11:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:11:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:11:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:11:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:11:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:11:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:11:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:11:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:11:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56207 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [write 0~0] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:11:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.245178+0000", 2024-09-15T00:11:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.99705641700000003, 2024-09-15T00:11:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038624626000000002, 2024-09-15T00:11:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56207 2024-09-15T00:11:49.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245178+0000", 2024-09-15T00:11:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245178+0000", 2024-09-15T00:11:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245180+0000", 2024-09-15T00:11:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-15T00:11:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245185+0000", 2024-09-15T00:11:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1780000000000002e-06 2024-09-15T00:11:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245192+0000", 2024-09-15T00:11:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6420000000000007e-06 2024-09-15T00:11:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245198+0000", 2024-09-15T00:11:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5729999999999998e-06 2024-09-15T00:11:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.271149+0000", 2024-09-15T00:11:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025950534000000001 2024-09-15T00:11:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.271238+0000", 2024-09-15T00:11:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9364999999999996e-05 2024-09-15T00:11:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.271398+0000", 2024-09-15T00:11:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015962600000000001 2024-09-15T00:11:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.277582+0000", 2024-09-15T00:11:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061847669999999999 2024-09-15T00:11:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283748+0000", 2024-09-15T00:11:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061654170000000003 2024-09-15T00:11:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283782+0000", 2024-09-15T00:11:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4245000000000002e-05 2024-09-15T00:11:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283802+0000", 2024-09-15T00:11:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-05 2024-09-15T00:11:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:56200 3.c e51/23 3:347f8d70:::rbd_data.1139167e758e.0000000000000002:head v 51'165, mlcod=50'162)", 2024-09-15T00:11:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.248115+0000", 2024-09-15T00:11:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.99411955500000004, 2024-09-15T00:11:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025900706999999999, 2024-09-15T00:11:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.248115+0000", 2024-09-15T00:11:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.248115+0000", 2024-09-15T00:11:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.248115+0000", 2024-09-15T00:11:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9999999999999997e-07 2024-09-15T00:11:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.253554+0000", 2024-09-15T00:11:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054384120000000001 2024-09-15T00:11:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.253569+0000", 2024-09-15T00:11:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5460000000000001e-05 2024-09-15T00:11:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.253579+0000", 2024-09-15T00:11:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0261000000000001e-05 2024-09-15T00:11:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.253642+0000", 2024-09-15T00:11:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2482000000000002e-05 2024-09-15T00:11:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.253665+0000", 2024-09-15T00:11:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.298e-05 2024-09-15T00:11:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273983+0000", 2024-09-15T00:11:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020318340000000001 2024-09-15T00:11:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274015+0000", 2024-09-15T00:11:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2172000000000003e-05 2024-09-15T00:11:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56253 3.b 3:d22b8105:::rbd_data.1139167e758e.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:11:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.272929+0000", 2024-09-15T00:11:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.96930526699999997, 2024-09-15T00:11:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020968238, 2024-09-15T00:11:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.351 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56253 2024-09-15T00:11:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.272929+0000", 2024-09-15T00:11:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.272929+0000", 2024-09-15T00:11:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.272932+0000", 2024-09-15T00:11:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6919999999999998e-06 2024-09-15T00:11:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273886+0000", 2024-09-15T00:11:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00095420599999999998 2024-09-15T00:11:49.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273902+0000", 2024-09-15T00:11:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5699e-05 2024-09-15T00:11:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273910+0000", 2024-09-15T00:11:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0590000000000003e-06 2024-09-15T00:11:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273966+0000", 2024-09-15T00:11:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6604999999999998e-05 2024-09-15T00:11:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274076+0000", 2024-09-15T00:11:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000109617 2024-09-15T00:11:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274959+0000", 2024-09-15T00:11:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088268500000000002 2024-09-15T00:11:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.290999+0000", 2024-09-15T00:11:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016040868999999999 2024-09-15T00:11:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.293799+0000", 2024-09-15T00:11:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027999159999999999 2024-09-15T00:11:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.293853+0000", 2024-09-15T00:11:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3783999999999999e-05 2024-09-15T00:11:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.293897+0000", 2024-09-15T00:11:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4106000000000001e-05 2024-09-15T00:11:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56254 3.b 3:d22b8105:::rbd_data.1139167e758e.0000000000000005:head [write 0~0] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:11:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.273938+0000", 2024-09-15T00:11:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.96829666999999997, 2024-09-15T00:11:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020239258, 2024-09-15T00:11:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56254 2024-09-15T00:11:49.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273938+0000", 2024-09-15T00:11:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273938+0000", 2024-09-15T00:11:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273939+0000", 2024-09-15T00:11:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5600000000000001e-06 2024-09-15T00:11:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273942+0000", 2024-09-15T00:11:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7669999999999999e-06 2024-09-15T00:11:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273947+0000", 2024-09-15T00:11:49.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0220000000000004e-06 2024-09-15T00:11:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273955+0000", 2024-09-15T00:11:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4999999999999999e-06 2024-09-15T00:11:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.289470+0000", 2024-09-15T00:11:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015514422999999999 2024-09-15T00:11:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.289523+0000", 2024-09-15T00:11:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3267999999999999e-05 2024-09-15T00:11:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.289629+0000", 2024-09-15T00:11:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000105623 2024-09-15T00:11:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.291337+0000", 2024-09-15T00:11:49.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001708289 2024-09-15T00:11:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.294125+0000", 2024-09-15T00:11:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027884530000000002 2024-09-15T00:11:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.294155+0000", 2024-09-15T00:11:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9183000000000002e-05 2024-09-15T00:11:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:49.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.294177+0000", 2024-09-15T00:11:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2169999999999999e-05 2024-09-15T00:11:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:49.364 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:49.364 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:49.368 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:49.368 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:49.364+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:49.466 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:49.464+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:49.471 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:49.471 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:49.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.471 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:56945 3.9 3.f58cc569 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-15T00:11:49.471 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:49.198223+0000", 2024-09-15T00:11:49.471 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.27212819900000002, 2024-09-15T00:11:49.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.27214908300000001, 2024-09-15T00:11:49.471 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.472 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.472 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "waiting for peered", 2024-09-15T00:11:49.472 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.472 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.472 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.472 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 56945 2024-09-15T00:11:49.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.472 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.198223+0000", 2024-09-15T00:11:49.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.198223+0000", 2024-09-15T00:11:49.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.198224+0000", 2024-09-15T00:11:49.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6000000000000003e-07 2024-09-15T00:11:49.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.198227+0000", 2024-09-15T00:11:49.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2780000000000002e-06 2024-09-15T00:11:49.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.198236+0000", 2024-09-15T00:11:49.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3859999999999996e-06 2024-09-15T00:11:49.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.198241+0000", 2024-09-15T00:11:49.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7299999999999996e-06 2024-09-15T00:11:49.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.198283+0000", 2024-09-15T00:11:49.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1863000000000002e-05 2024-09-15T00:11:49.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:11:49.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.198291+0000", 2024-09-15T00:11:49.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3690000000000007e-06 2024-09-15T00:11:49.475 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.475 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.475 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.475 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:56949 3.9 3.f58cc569 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-15T00:11:49.475 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:49.204003+0000", 2024-09-15T00:11:49.475 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.26634827900000002, 2024-09-15T00:11:49.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.26640840799999999, 2024-09-15T00:11:49.475 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.475 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.476 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "waiting for peered", 2024-09-15T00:11:49.476 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.476 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.476 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.476 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 56949 2024-09-15T00:11:49.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.476 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.204003+0000", 2024-09-15T00:11:49.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.204003+0000", 2024-09-15T00:11:49.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.204004+0000", 2024-09-15T00:11:49.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6300000000000004e-07 2024-09-15T00:11:49.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.204008+0000", 2024-09-15T00:11:49.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0029999999999996e-06 2024-09-15T00:11:49.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.204018+0000", 2024-09-15T00:11:49.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0496000000000001e-05 2024-09-15T00:11:49.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.204023+0000", 2024-09-15T00:11:49.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9529999999999998e-06 2024-09-15T00:11:49.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.204080+0000", 2024-09-15T00:11:49.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6433e-05 2024-09-15T00:11:49.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:11:49.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:49.204089+0000", 2024-09-15T00:11:49.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5589999999999994e-06 2024-09-15T00:11:49.479 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.479 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.479 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.479 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.479 INFO:teuthology.orchestra.run.smithi184.stdout: ], 2024-09-15T00:11:49.479 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 2 2024-09-15T00:11:49.479 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:49.480 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:49.567 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:49.565+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:49.584 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:49.584 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:49.593 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:49.668 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:49.667+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:49.697 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:11:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.122580991, 2024-09-15T00:11:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:11:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:11:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:11:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:11:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:11:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:11:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:11:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:11:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:11:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:11:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:11:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:11:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:11:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:11:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:11:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:11:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:11:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:11:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:11:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:11:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:11:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:11:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:11:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:11:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:11:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:11:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.122546299, 2024-09-15T00:11:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:11:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:11:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:11:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:11:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:11:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:11:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:11:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:11:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:11:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:11:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:11:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:11:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:11:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:11:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:11:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:11:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:11:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48937 3.1a 3:5e7e71d0:::rbd_data.11395b9829f3.0000000000000000:e [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:11:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0824533440000002, 2024-09-15T00:11:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072322336000000001, 2024-09-15T00:11:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48937 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612043+0000", 2024-09-15T00:11:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:11:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612045+0000", 2024-09-15T00:11:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2749999999999998e-06 2024-09-15T00:11:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612053+0000", 2024-09-15T00:11:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0439999999999996e-06 2024-09-15T00:11:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612062+0000", 2024-09-15T00:11:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6829999999999997e-06 2024-09-15T00:11:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684078+0000", 2024-09-15T00:11:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072016017000000002 2024-09-15T00:11:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684274+0000", 2024-09-15T00:11:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019636600000000001 2024-09-15T00:11:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684364+0000", 2024-09-15T00:11:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0420000000000005e-05 2024-09-15T00:11:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:11:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0818726109999997, 2024-09-15T00:11:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:11:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:11:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:11:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:11:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:11:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:11:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:11:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:11:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:11:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:11:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:11:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:11:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:11:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:11:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:11:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:11:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:11:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:11:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0818560799999997, 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:11:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:11:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:11:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:11:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:11:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:11:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:11:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:11:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:11:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:11:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:11:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:11:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:11:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:11:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:11:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:11:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0818393009999996, 2024-09-15T00:11:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:11:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:11:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:11:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:11:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:11:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:11:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:11:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:11:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:11:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:11:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:11:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:11:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:11:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:11:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:11:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:11:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:11:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:11:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:11:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0818215709999999, 2024-09-15T00:11:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:11:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:11:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:11:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:11:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:11:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:11:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:11:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:11:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:11:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:11:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:11:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:11:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:11:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:11:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:11:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:11:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:11:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:11:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:11:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0818044069999999, 2024-09-15T00:11:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:11:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:11:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:11:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:11:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:11:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:11:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:11:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:11:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:11:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:11:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:11:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:11:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:11:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:11:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:11:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:11:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:11:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:11:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:11:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0817827309999997, 2024-09-15T00:11:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:11:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:11:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:11:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:11:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:11:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:11:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:11:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:11:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:11:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:11:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:11:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:11:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:11:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0817366069999999, 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:11:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:11:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:11:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:11:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:11:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:11:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:11:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:11:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:11:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:11:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:11:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:11:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:11:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0817068499999998, 2024-09-15T00:11:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:11:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:11:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:11:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:11:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:11:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:11:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:11:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:11:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:11:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:11:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:11:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:11:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:11:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:11:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:11:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0816805309999999, 2024-09-15T00:11:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:11:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:11:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:11:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:11:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:11:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:11:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:11:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:11:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:11:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:11:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:11:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:11:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:11:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0816446690000001, 2024-09-15T00:11:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:11:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:11:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:11:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:11:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:11:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:11:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.754 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.754 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:11:49.754 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:11:49.754 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.754 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.754 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.754 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:11:49.754 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:11:49.754 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.754 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.754 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.755 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:11:49.755 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:11:49.755 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.755 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.755 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.755 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:11:49.755 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:11:49.755 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.755 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.755 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.755 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:11:49.755 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:11:49.755 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.756 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.756 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.756 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:11:49.756 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:11:49.756 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.756 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.756 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.756 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.756 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.756 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.756 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:11:49.756 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0816229509999999, 2024-09-15T00:11:49.756 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:11:49.757 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.757 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.757 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.757 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.757 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.757 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.757 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:11:49.757 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.757 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.757 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.757 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.758 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:11:49.758 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.758 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.758 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.758 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.758 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:11:49.758 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.758 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.758 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.758 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.758 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:11:49.758 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:11:49.759 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.759 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.759 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.759 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:11:49.759 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:11:49.759 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.759 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.759 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.759 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:11:49.759 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:11:49.759 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.759 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.759 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.759 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:11:49.760 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:11:49.760 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.760 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.760 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.760 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:11:49.760 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:11:49.760 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.760 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.760 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.760 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:11:49.760 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:11:49.760 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.760 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.761 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.761 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:11:49.761 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:11:49.761 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.761 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.761 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.761 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.761 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.761 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.761 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:11:49.761 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0814264839999996, 2024-09-15T00:11:49.761 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:11:49.761 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.761 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.761 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.762 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.762 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.762 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.762 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:11:49.762 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.762 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.762 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.762 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.762 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:11:49.762 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.762 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.762 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.762 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.762 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:11:49.762 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.763 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.763 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.763 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.763 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:11:49.763 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:11:49.763 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.763 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.763 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.763 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:11:49.763 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:11:49.763 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.763 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.763 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.763 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:11:49.763 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:11:49.764 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.764 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.764 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.764 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:11:49.764 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:11:49.764 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.764 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.764 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.764 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:11:49.764 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:11:49.764 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.764 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.764 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.764 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:11:49.765 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:11:49.765 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.765 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.765 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.765 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:11:49.765 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:11:49.765 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.765 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.765 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.765 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.765 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.765 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.765 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:11:49.765 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0814006450000004, 2024-09-15T00:11:49.766 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:11:49.766 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.766 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.766 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.766 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.766 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.766 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.766 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:11:49.766 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.766 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.766 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.766 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.766 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:11:49.767 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.767 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.767 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.767 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.767 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:11:49.767 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.767 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.767 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.767 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.767 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:11:49.767 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:11:49.767 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.768 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.768 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.768 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:11:49.768 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:11:49.768 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.768 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.768 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.768 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:11:49.768 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:11:49.768 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.768 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.768 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.768 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:11:49.769 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:11:49.769 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.769 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.769 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.769 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:11:49.769 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:11:49.769 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.769 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.769 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.769 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:11:49.769 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:11:49.769 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.769 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.770 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.770 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:11:49.770 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:11:49.770 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.770 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.770 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.770 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.770 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.770 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.770 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:11:49.770 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0813389989999997, 2024-09-15T00:11:49.770 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:11:49.770 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.770 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.771 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.771 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.771 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.771 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.771 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:11:49.771 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.771 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.771 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.771 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.771 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:11:49.771 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.771 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.771 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.771 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.772 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:11:49.772 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.772 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.772 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.772 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:11:49.772 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:11:49.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.772 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.772 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.772 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:11:49.772 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:11:49.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.772 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.773 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.773 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:11:49.773 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:11:49.773 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.773 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.773 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.773 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:11:49.773 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:11:49.773 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.773 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.773 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.773 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:11:49.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:11:49.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.774 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:11:49.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:11:49.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.774 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:11:49.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:11:49.774 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.775 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.775 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.775 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.775 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.775 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:11:49.775 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.081299671, 2024-09-15T00:11:49.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:11:49.775 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.775 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.775 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.775 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.775 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.775 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.776 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:11:49.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.776 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:11:49.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:11:49.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.777 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:11:49.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:11:49.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.777 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:11:49.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:11:49.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:11:49.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:11:49.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:11:49.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:11:49.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:11:49.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:11:49.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.780 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.780 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:11:49.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:11:49.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.780 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:11:49.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:11:49.781 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.781 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.781 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.781 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.781 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:11:49.781 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0795742710000003, 2024-09-15T00:11:49.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:11:49.781 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.781 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.781 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.782 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.782 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.782 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.782 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:11:49.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.782 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.782 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.782 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:11:49.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.782 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.782 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:11:49.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:11:49.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:11:49.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:11:49.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:11:49.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:11:49.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:11:49.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:11:49.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:11:49.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:11:49.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:11:49.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:11:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:11:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:11:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:11:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:11:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0795521099999998, 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:49.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:11:49.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:49.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:49.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:11:49.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:11:49.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:49.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:11:49.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:11:49.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:49.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:11:49.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:11:49.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:49.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:11:49.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:11:49.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:49.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:11:49.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:11:49.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:49.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:11:49.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:11:49.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:49.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:49.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:49.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:11:49.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:11:49.789 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.789 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.789 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.789 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:49.789 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:49.789 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:49.791 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:49.791 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:49.789+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:49.892 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:49.890+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:49.894 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:49.894 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:49.894 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:49.894 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:49.903 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:49.992 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:49.991+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:50.009 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:50.010 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:50.010 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:50.010 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:50.010 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:50.021 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:50.105 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:50.104+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:50.147 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:11:50.167 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:47986 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:11:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.6932190629999999, 2024-09-15T00:11:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045287635, 2024-09-15T00:11:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 47986 2024-09-15T00:11:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:11:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:11:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:11:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8400000000000004e-07 2024-09-15T00:11:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474928+0000", 2024-09-15T00:11:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033009430000000002 2024-09-15T00:11:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474943+0000", 2024-09-15T00:11:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5172e-05 2024-09-15T00:11:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474951+0000", 2024-09-15T00:11:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1779999999999997e-06 2024-09-15T00:11:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475058+0000", 2024-09-15T00:11:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000107595 2024-09-15T00:11:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475200+0000", 2024-09-15T00:11:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014180400000000001 2024-09-15T00:11:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.478046+0000", 2024-09-15T00:11:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028464150000000001 2024-09-15T00:11:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516790+0000", 2024-09-15T00:11:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038743278999999999 2024-09-15T00:11:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516832+0000", 2024-09-15T00:11:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1974000000000002e-05 2024-09-15T00:11:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516872+0000", 2024-09-15T00:11:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9928999999999997e-05 2024-09-15T00:11:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516914+0000", 2024-09-15T00:11:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2762000000000002e-05 2024-09-15T00:11:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48888 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13762560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.608340+0000", 2024-09-15T00:11:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5565060989999999, 2024-09-15T00:11:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043750791999999997, 2024-09-15T00:11:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48888 2024-09-15T00:11:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608340+0000", 2024-09-15T00:11:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608340+0000", 2024-09-15T00:11:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608340+0000", 2024-09-15T00:11:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3099999999999998e-07 2024-09-15T00:11:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608342+0000", 2024-09-15T00:11:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9759999999999998e-06 2024-09-15T00:11:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608349+0000", 2024-09-15T00:11:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1010000000000003e-06 2024-09-15T00:11:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608354+0000", 2024-09-15T00:11:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4669999999999998e-06 2024-09-15T00:11:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.651990+0000", 2024-09-15T00:11:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043636767999999999 2024-09-15T00:11:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.652034+0000", 2024-09-15T00:11:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4048000000000002e-05 2024-09-15T00:11:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.652090+0000", 2024-09-15T00:11:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6001000000000001e-05 2024-09-15T00:11:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48956 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 2621440~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609044+0000", 2024-09-15T00:11:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5558021110000002, 2024-09-15T00:11:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044014043000000003, 2024-09-15T00:11:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48956 2024-09-15T00:11:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609044+0000", 2024-09-15T00:11:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609044+0000", 2024-09-15T00:11:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609044+0000", 2024-09-15T00:11:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6100000000000001e-07 2024-09-15T00:11:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609046+0000", 2024-09-15T00:11:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2350000000000002e-06 2024-09-15T00:11:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609053+0000", 2024-09-15T00:11:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1750000000000002e-06 2024-09-15T00:11:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609057+0000", 2024-09-15T00:11:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7190000000000001e-06 2024-09-15T00:11:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.652645+0000", 2024-09-15T00:11:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043587689999999998 2024-09-15T00:11:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.652700+0000", 2024-09-15T00:11:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.507e-05 2024-09-15T00:11:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653058+0000", 2024-09-15T00:11:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035769300000000002 2024-09-15T00:11:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48957 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 4456448~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:11:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5557805509999998, 2024-09-15T00:11:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044512426000000001, 2024-09-15T00:11:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48957 2024-09-15T00:11:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:11:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:11:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609066+0000", 2024-09-15T00:11:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:11:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609068+0000", 2024-09-15T00:11:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9700000000000002e-06 2024-09-15T00:11:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609073+0000", 2024-09-15T00:11:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7939999999999999e-06 2024-09-15T00:11:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609078+0000", 2024-09-15T00:11:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3209999999999998e-06 2024-09-15T00:11:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653082+0000", 2024-09-15T00:11:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044003852000000003 2024-09-15T00:11:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653136+0000", 2024-09-15T00:11:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4024000000000003e-05 2024-09-15T00:11:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653578+0000", 2024-09-15T00:11:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044211599999999999 2024-09-15T00:11:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48958 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:11:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5557629120000005, 2024-09-15T00:11:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044724939999999998, 2024-09-15T00:11:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48958 2024-09-15T00:11:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:11:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:11:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:11:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:11:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609085+0000", 2024-09-15T00:11:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6610000000000001e-06 2024-09-15T00:11:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609090+0000", 2024-09-15T00:11:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2719999999999997e-06 2024-09-15T00:11:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609094+0000", 2024-09-15T00:11:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.907e-06 2024-09-15T00:11:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653618+0000", 2024-09-15T00:11:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044523759000000003 2024-09-15T00:11:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653703+0000", 2024-09-15T00:11:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5550000000000006e-05 2024-09-15T00:11:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653808+0000", 2024-09-15T00:11:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010445700000000001 2024-09-15T00:11:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48959 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:11:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5557469729999998, 2024-09-15T00:11:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044884741999999998, 2024-09-15T00:11:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48959 2024-09-15T00:11:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:11:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:11:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:11:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2899999999999999e-07 2024-09-15T00:11:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609101+0000", 2024-09-15T00:11:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.826e-06 2024-09-15T00:11:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609108+0000", 2024-09-15T00:11:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.165e-06 2024-09-15T00:11:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609113+0000", 2024-09-15T00:11:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7849999999999999e-06 2024-09-15T00:11:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653842+0000", 2024-09-15T00:11:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044728954000000001 2024-09-15T00:11:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653896+0000", 2024-09-15T00:11:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4593999999999999e-05 2024-09-15T00:11:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653984+0000", 2024-09-15T00:11:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7089000000000006e-05 2024-09-15T00:11:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48960 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 9961472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:11:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5557264540000002, 2024-09-15T00:11:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045084194000000001, 2024-09-15T00:11:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48960 2024-09-15T00:11:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:11:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:11:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609120+0000", 2024-09-15T00:11:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7099999999999997e-07 2024-09-15T00:11:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609122+0000", 2024-09-15T00:11:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:11:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609129+0000", 2024-09-15T00:11:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4159999999999998e-06 2024-09-15T00:11:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609134+0000", 2024-09-15T00:11:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7210000000000002e-06 2024-09-15T00:11:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654023+0000", 2024-09-15T00:11:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044888979000000002 2024-09-15T00:11:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654104+0000", 2024-09-15T00:11:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1108999999999999e-05 2024-09-15T00:11:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654204+0000", 2024-09-15T00:11:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9630000000000007e-05 2024-09-15T00:11:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48961 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:11:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5557056329999996, 2024-09-15T00:11:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045197821999999999, 2024-09-15T00:11:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48961 2024-09-15T00:11:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:11:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:11:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609141+0000", 2024-09-15T00:11:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:11:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609143+0000", 2024-09-15T00:11:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-15T00:11:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609150+0000", 2024-09-15T00:11:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2420000000000003e-06 2024-09-15T00:11:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609155+0000", 2024-09-15T00:11:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.028e-06 2024-09-15T00:11:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654232+0000", 2024-09-15T00:11:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045076702000000003 2024-09-15T00:11:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654281+0000", 2024-09-15T00:11:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9798e-05 2024-09-15T00:11:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654338+0000", 2024-09-15T00:11:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6498e-05 2024-09-15T00:11:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48962 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 13631488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:11:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5556845829999997, 2024-09-15T00:11:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045312458, 2024-09-15T00:11:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48962 2024-09-15T00:11:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:11:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:11:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609162+0000", 2024-09-15T00:11:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:11:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609164+0000", 2024-09-15T00:11:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0099999999999998e-06 2024-09-15T00:11:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609171+0000", 2024-09-15T00:11:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7530000000000001e-06 2024-09-15T00:11:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609178+0000", 2024-09-15T00:11:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1709999999999999e-06 2024-09-15T00:11:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654365+0000", 2024-09-15T00:11:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045187867999999999 2024-09-15T00:11:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654420+0000", 2024-09-15T00:11:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4376000000000003e-05 2024-09-15T00:11:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654474+0000", 2024-09-15T00:11:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3879999999999999e-05 2024-09-15T00:11:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48971 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:11:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5556535890000003, 2024-09-15T00:11:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045896151000000003, 2024-09-15T00:11:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48971 2024-09-15T00:11:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:11:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:11:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609193+0000", 2024-09-15T00:11:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7599999999999997e-07 2024-09-15T00:11:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609195+0000", 2024-09-15T00:11:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:11:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609210+0000", 2024-09-15T00:11:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4956999999999999e-05 2024-09-15T00:11:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609505+0000", 2024-09-15T00:11:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000295078 2024-09-15T00:11:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654498+0000", 2024-09-15T00:11:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044992277999999997 2024-09-15T00:11:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654541+0000", 2024-09-15T00:11:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3525000000000001e-05 2024-09-15T00:11:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655088+0000", 2024-09-15T00:11:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054726799999999995 2024-09-15T00:11:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48972 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 2097152~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:11:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5553270149999996, 2024-09-15T00:11:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046150885000000003, 2024-09-15T00:11:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48972 2024-09-15T00:11:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:11:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:11:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:11:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6400000000000002e-07 2024-09-15T00:11:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609523+0000", 2024-09-15T00:11:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5300000000000001e-06 2024-09-15T00:11:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609533+0000", 2024-09-15T00:11:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0023999999999999e-05 2024-09-15T00:11:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609539+0000", 2024-09-15T00:11:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3119999999999999e-06 2024-09-15T00:11:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655115+0000", 2024-09-15T00:11:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045576094999999997 2024-09-15T00:11:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655197+0000", 2024-09-15T00:11:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1129000000000002e-05 2024-09-15T00:11:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655670+0000", 2024-09-15T00:11:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00047313100000000001 2024-09-15T00:11:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:11:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5552987529999998, 2024-09-15T00:11:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:11:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:11:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:11:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:11:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:11:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:11:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:11:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:11:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:11:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:11:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:11:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:11:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:11:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:11:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:11:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:11:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:11:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:11:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:11:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5552775240000001, 2024-09-15T00:11:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:11:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:11:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:11:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:11:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:11:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:11:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:11:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:11:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:11:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:11:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:11:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:11:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:11:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:11:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:11:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:11:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:11:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:11:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:11:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5552519829999998, 2024-09-15T00:11:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:11:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:11:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:11:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:11:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:11:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:11:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:11:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:11:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:11:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:11:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:11:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:11:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:11:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:11:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:11:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:11:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:11:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:11:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:11:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.555229508, 2024-09-15T00:11:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:11:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:11:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:11:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:11:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:11:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:11:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:11:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:11:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:11:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:11:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:11:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:11:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:11:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:11:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:11:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:11:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:11:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:11:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:11:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5552089980000003, 2024-09-15T00:11:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:11:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:11:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:11:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:11:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:11:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:11:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:11:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:11:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:11:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:11:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:11:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:11:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:11:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:11:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:11:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:11:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:11:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:11:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:11:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5551886829999999, 2024-09-15T00:11:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:11:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:11:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:11:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:11:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:11:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:11:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:11:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:11:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:11:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:11:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:11:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:11:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:11:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:11:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:11:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:11:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:11:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:11:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:11:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5551639069999998, 2024-09-15T00:11:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:11:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:11:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:11:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:11:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:11:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:11:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:11:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:11:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:11:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:11:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:11:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:11:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:11:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:11:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:11:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:11:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:11:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:11:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:11:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3758225199999998, 2024-09-15T00:11:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:11:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:11:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:11:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:11:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:11:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:11:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:11:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:11:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:11:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:11:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:11:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:11:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:11:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:11:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:11:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:11:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:11:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:11:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:11:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:11:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:11:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:11:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:11:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:11:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:11:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:11:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:11:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:11:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3588690679999997, 2024-09-15T00:11:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:11:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:11:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:11:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:11:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:11:50.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:50.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:11:50.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:11:50.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:50.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:11:50.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:11:50.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:50.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:11:50.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:11:50.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:50.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:11:50.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:11:50.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:50.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:11:50.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:11:50.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:50.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:50.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:11:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:11:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:50.261 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:50.263 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:11:50.263 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:50.261+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:50.345 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:11:50.392 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:50.388+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:50.406 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:50.406 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:50.406 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:50.406 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:50.415 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:11:50.464 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:50.462+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:50.467 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:50.467 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:50.483 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:50.484 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:50.499 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:50.499 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:50.515 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:50.515 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:50.517 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:50.517 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:50.517 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:50.517 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:50.517 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:50.527 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:11:50.531 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:50.531 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:50.548 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:50.548 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:50.565 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:50.563+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:50.565 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:50.565 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:50.583 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:50.583 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:50.632 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:50.632 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:50.641 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:11:50.666 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:50.664+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:50.743 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:50.743 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:50.752 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:11:50.766 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:50.764+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:50.854 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:50.855 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:50.863 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:11:50.867 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:50.866+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:50.965 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:50.965 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:50.968 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:50.967+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:50.974 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:51.000 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is unset 2024-09-15T00:11:51.069 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:51.067+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:51.106 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:51.106 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:51.117 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:11:51.197 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-15T00:11:51.197 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:51.195+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:51.271 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:51.270+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:51.280 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:51.280 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:51.296 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:11:51.379 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:51.378+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:51.451 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:51.452 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:51.456 INFO:teuthology.orchestra.run.smithi171.stdout:2076 trunc from 0xa6f2be7 to 0xa94a8cc 2024-09-15T00:11:51.462 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:51.466 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:51.466 INFO:teuthology.orchestra.run.smithi171.stdout:2079 read 0x7499a3d thru 0x749aaeb (0x10af bytes) 2024-09-15T00:11:51.466 INFO:teuthology.orchestra.run.smithi171.stdout:2081 read 0x7c3c694 thru 0x7c42e41 (0x67ae bytes) 2024-09-15T00:11:51.466 INFO:teuthology.orchestra.run.smithi171.stdout:2082 writesame 0x6f4e88f thru 0x6f5ac71 data_size 0x1(0xc3e3 bytes) 2024-09-15T00:11:51.473 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:51.472+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:51.508 INFO:teuthology.orchestra.run.smithi171.stdout:2086 trunc from 0xa94a8cc to 0xaa0e27e 2024-09-15T00:11:51.513 INFO:teuthology.orchestra.run.smithi171.stdout:2087 write 0x4a53715 thru 0x4a5379e (0x8a bytes) 2024-09-15T00:11:51.516 INFO:teuthology.orchestra.run.smithi171.stdout:2090 read 0x7117617 thru 0x7118f48 (0x1932 bytes) 2024-09-15T00:11:51.516 INFO:teuthology.orchestra.run.smithi171.stdout:2091 read 0x88ce57 thru 0x8983af (0xb559 bytes) 2024-09-15T00:11:51.516 INFO:teuthology.orchestra.run.smithi171.stdout:2092 write 0x27158ce thru 0x271701c (0x174f bytes) 2024-09-15T00:11:51.530 INFO:teuthology.orchestra.run.smithi171.stdout:2093 writesame 0xda77b29 thru 0xda875a3 data_size 0x1(0xfa7b bytes) 2024-09-15T00:11:51.530 INFO:teuthology.orchestra.run.smithi171.stdout:2094 read 0x6858ce4 thru 0x6867b7e (0xee9b bytes) 2024-09-15T00:11:51.530 INFO:teuthology.orchestra.run.smithi171.stdout:2095 read 0xc765b66 thru 0xc770253 (0xa6ee bytes) 2024-09-15T00:11:51.530 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:51.531 INFO:teuthology.orchestra.run.smithi171.stdout:2097 writesame 0x43354c0 thru 0x4342a7f data_size 0xa(0xd5c0 bytes) 2024-09-15T00:11:51.546 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:51.546 INFO:teuthology.orchestra.run.smithi171.stdout:2100 punch from 0x359ff49 to 0x35a35ff, (0x36b6 bytes) 2024-09-15T00:11:51.547 INFO:teuthology.orchestra.run.smithi171.stdout:2101 write 0x178d619 thru 0x1795848 (0x8230 bytes) 2024-09-15T00:11:51.556 INFO:teuthology.orchestra.run.smithi171.stdout:2102 read 0xaed6191 thru 0xaed8513 (0x2383 bytes) 2024-09-15T00:11:51.556 INFO:teuthology.orchestra.run.smithi171.stdout:2103 writesame 0x8436600 thru 0x843ed28 data_size 0x1(0x8729 bytes) 2024-09-15T00:11:51.571 INFO:teuthology.orchestra.run.smithi171.stdout:2104 read 0x994a0b3 thru 0x994c32b (0x2279 bytes) 2024-09-15T00:11:51.572 INFO:teuthology.orchestra.run.smithi171.stdout:2106 writesame 0x879efcd thru 0x87aba9a data_size 0x33(0xcace bytes) 2024-09-15T00:11:51.573 INFO:teuthology.orchestra.run.smithi171.stdout:2107 trunc from 0xda875a4 to 0xda81d13 2024-09-15T00:11:51.574 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:51.572+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:51.579 INFO:teuthology.orchestra.run.smithi171.stdout:2108 write 0x24cf5bb thru 0x24d79b9 (0x83ff bytes) 2024-09-15T00:11:51.584 INFO:teuthology.orchestra.run.smithi171.stdout:2109 read 0x6a52c23 thru 0x6a5e13d (0xb51b bytes) 2024-09-15T00:11:51.584 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:51.585 INFO:teuthology.orchestra.run.smithi171.stdout:2111 punch from 0x4095a4b to 0x409c87b, (0x6e30 bytes) 2024-09-15T00:11:51.588 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:51.588 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:51.591 INFO:teuthology.orchestra.run.smithi171.stdout:2112 trunc from 0xda81d13 to 0x8a33632 2024-09-15T00:11:51.601 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:51.606 INFO:teuthology.orchestra.run.smithi171.stdout:2113 writesame 0x2e1d637 thru 0x2e2d61e data_size 0x13(0xffe8 bytes) 2024-09-15T00:11:51.621 INFO:teuthology.orchestra.run.smithi171.stdout:2114 writesame 0x856b9f4 thru 0x856d373 data_size 0x20(0x1980 bytes) 2024-09-15T00:11:51.623 INFO:teuthology.orchestra.run.smithi171.stdout:2116 write 0x88fc165 thru 0x8906f8a (0xae26 bytes) 2024-09-15T00:11:51.625 INFO:teuthology.orchestra.run.smithi171.stdout:2117 read 0x6238e59 thru 0x6245817 (0xc9bf bytes) 2024-09-15T00:11:51.625 INFO:teuthology.orchestra.run.smithi171.stdout:2118 punch from 0x580a8bc to 0x58140b2, (0x97f6 bytes) 2024-09-15T00:11:51.626 INFO:teuthology.orchestra.run.smithi171.stdout:2119 write 0x2723e48 thru 0x2724f1a (0x10d3 bytes) 2024-09-15T00:11:51.628 INFO:teuthology.orchestra.run.smithi171.stdout:2120 trunc from 0x8a33632 to 0x72725f0 2024-09-15T00:11:51.667 INFO:teuthology.orchestra.run.smithi171.stdout:2121 write 0xe9e3bd0 thru 0xe9edc5d (0xa08e bytes) 2024-09-15T00:11:51.674 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:51.673+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:51.676 INFO:teuthology.orchestra.run.smithi171.stdout:2122 read 0x8e0a1f2 thru 0x8e180c3 (0xded2 bytes) 2024-09-15T00:11:51.684 INFO:teuthology.orchestra.run.smithi171.stdout:2124 trunc from 0xe9edc5e to 0x795de3a 2024-09-15T00:11:51.698 INFO:teuthology.orchestra.run.smithi171.stdout:2125 trunc from 0x795de3a to 0x48f965f 2024-09-15T00:11:51.717 INFO:teuthology.orchestra.run.smithi171.stdout:2126 trunc from 0x48f965f to 0x2f8dd38 2024-09-15T00:11:51.724 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:51.724 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:51.734 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:51.735 INFO:teuthology.orchestra.run.smithi171.stdout:2127 write 0x3ab4202 thru 0x3ab9b26 (0x5925 bytes) 2024-09-15T00:11:51.758 INFO:teuthology.orchestra.run.smithi171.stdout:2128 write 0xb430868 thru 0xb430baa (0x343 bytes) 2024-09-15T00:11:51.761 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:51.770 INFO:teuthology.orchestra.run.smithi171.stdout:2132 write 0xda619da thru 0xda6597e (0x3fa5 bytes) 2024-09-15T00:11:51.775 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:51.773+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:51.780 INFO:teuthology.orchestra.run.smithi171.stdout:2133 read 0x94a070 thru 0x94fbef (0x5b80 bytes) 2024-09-15T00:11:51.781 INFO:teuthology.orchestra.run.smithi171.stdout:2134 punch from 0xc7e597d to 0xc7f1e82, (0xc505 bytes) 2024-09-15T00:11:51.781 INFO:teuthology.orchestra.run.smithi171.stdout:2136 read 0xd45cd33 thru 0xd465ab8 (0x8d86 bytes) 2024-09-15T00:11:51.781 INFO:teuthology.orchestra.run.smithi171.stdout:2140 punch from 0x2911933 to 0x2913ee7, (0x25b4 bytes) 2024-09-15T00:11:51.781 INFO:teuthology.orchestra.run.smithi171.stdout:2141 write 0xb31aae0 thru 0xb3262f6 (0xb817 bytes) 2024-09-15T00:11:51.788 INFO:teuthology.orchestra.run.smithi171.stdout:2142 write 0x32527a5 thru 0x3252bbf (0x41b bytes) 2024-09-15T00:11:51.792 INFO:teuthology.orchestra.run.smithi171.stdout:2143 punch from 0x8fe0d8c to 0x8fe3966, (0x2bda bytes) 2024-09-15T00:11:51.792 INFO:teuthology.orchestra.run.smithi171.stdout:2144 write 0xb6b36b2 thru 0xb6b901d (0x596c bytes) 2024-09-15T00:11:51.794 INFO:teuthology.orchestra.run.smithi171.stdout:2145 writesame 0x62856fb thru 0x628cb11 data_size 0x1(0x7417 bytes) 2024-09-15T00:11:51.820 INFO:teuthology.orchestra.run.smithi171.stdout:2146 read 0x645c6d8 thru 0x6462fa7 (0x68d0 bytes) 2024-09-15T00:11:51.821 INFO:teuthology.orchestra.run.smithi171.stdout:2149 write 0x19bce81 thru 0x19c9003 (0xc183 bytes) 2024-09-15T00:11:51.830 INFO:teuthology.orchestra.run.smithi171.stdout:2150 read 0x190058c thru 0x190fa10 (0xf485 bytes) 2024-09-15T00:11:51.844 INFO:teuthology.orchestra.run.smithi171.stdout:2151 trunc from 0xda6597f to 0x426b4ed 2024-09-15T00:11:51.851 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:51.851 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:51.857 INFO:teuthology.orchestra.run.smithi171.stdout:2152 read 0x277b0fd thru 0x277bf0d (0xe11 bytes) 2024-09-15T00:11:51.857 INFO:teuthology.orchestra.run.smithi171.stdout:2154 write 0x15a209b thru 0x15a57b4 (0x371a bytes) 2024-09-15T00:11:51.860 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:51.867 INFO:teuthology.orchestra.run.smithi171.stdout:2155 write 0x6fcb778 thru 0x6fcc51c (0xda5 bytes) 2024-09-15T00:11:51.870 INFO:teuthology.orchestra.run.smithi171.stdout:2156 writesame 0x36c8479 thru 0x36c8a6d data_size 0x5(0x5f5 bytes) 2024-09-15T00:11:51.873 INFO:teuthology.orchestra.run.smithi171.stdout:2157 write 0x1447f1e thru 0x144cc8c (0x4d6f bytes) 2024-09-15T00:11:51.875 INFO:teuthology.orchestra.run.smithi171.stdout:2158 write 0x28f366d thru 0x28f8d57 (0x56eb bytes) 2024-09-15T00:11:51.876 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:51.874+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:51.877 INFO:teuthology.orchestra.run.smithi171.stdout:2159 write 0x4d6d3eb thru 0x4d71ac9 (0x46df bytes) 2024-09-15T00:11:51.882 INFO:teuthology.orchestra.run.smithi171.stdout:2160 read 0x4037f36 thru 0x403b17b (0x3246 bytes) 2024-09-15T00:11:51.893 INFO:teuthology.orchestra.run.smithi171.stdout:2161 write 0xb751e1a thru 0xb7564db (0x46c2 bytes) 2024-09-15T00:11:51.899 INFO:teuthology.orchestra.run.smithi171.stdout:2163 read 0x70c3ea2 thru 0x70d0d70 (0xcecf bytes) 2024-09-15T00:11:51.899 INFO:teuthology.orchestra.run.smithi171.stdout:2164 write 0x1384a8d thru 0x139157a (0xcaee bytes) 2024-09-15T00:11:51.902 INFO:teuthology.orchestra.run.smithi171.stdout:2167 write 0x82f9e92 thru 0x8303630 (0x979f bytes) 2024-09-15T00:11:51.907 INFO:teuthology.orchestra.run.smithi171.stdout:2168 writesame 0x367d625 thru 0x3688028 data_size 0x9c(0xaa04 bytes) 2024-09-15T00:11:51.912 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:51.912 INFO:teuthology.orchestra.run.smithi171.stdout:2170 read 0x2eb8437 thru 0x2ec2150 (0x9d1a bytes) 2024-09-15T00:11:51.912 INFO:teuthology.orchestra.run.smithi171.stdout:2171 writesame 0xa64bb17 thru 0xa64cf5d data_size 0x1(0x1447 bytes) 2024-09-15T00:11:51.922 INFO:teuthology.orchestra.run.smithi171.stdout:2173 trunc from 0xb7564dc to 0x7202da4 2024-09-15T00:11:51.933 INFO:teuthology.orchestra.run.smithi171.stdout:2174 read 0x593f664 thru 0x594efb8 (0xf955 bytes) 2024-09-15T00:11:51.933 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:51.933 INFO:teuthology.orchestra.run.smithi171.stdout:2177 write 0x2b2203e thru 0x2b2365e (0x1621 bytes) 2024-09-15T00:11:51.934 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:51.934 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:51.934 INFO:teuthology.orchestra.run.smithi171.stdout:2181 writesame 0x3b23db4 thru 0x3b2bfa1 data_size 0x1(0x81ee bytes) 2024-09-15T00:11:51.958 INFO:teuthology.orchestra.run.smithi171.stdout:2185 write 0x7594f22 thru 0x7597086 (0x2165 bytes) 2024-09-15T00:11:51.961 INFO:teuthology.orchestra.run.smithi171.stdout:2186 punch from 0x69b75f4 to 0x69bed66, (0x7772 bytes) 2024-09-15T00:11:51.961 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:51.962 INFO:teuthology.orchestra.run.smithi171.stdout:2188 writesame 0x7041989 thru 0x704e665 data_size 0x11(0xccdd bytes) 2024-09-15T00:11:51.968 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:51.975 INFO:teuthology.orchestra.run.smithi171.stdout:2192 write 0x8b3883d thru 0x8b4092a (0x80ee bytes) 2024-09-15T00:11:51.976 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:51.975+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:51.982 INFO:teuthology.orchestra.run.smithi171.stdout:2193 read 0x7559f8 thru 0x764da0 (0xf3a9 bytes) 2024-09-15T00:11:51.982 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:51.982 INFO:teuthology.orchestra.run.smithi171.stdout:2195 write 0xd7d1c6 thru 0xd8c296 (0xf0d1 bytes) 2024-09-15T00:11:51.986 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:51.986 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:51.999 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:52.068 INFO:teuthology.orchestra.run.smithi171.stdout:2196 write 0xbea7677 thru 0xbeaef4d (0x78d7 bytes) 2024-09-15T00:11:52.111 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:52.110+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:52.172 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:52.173 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:52.178 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:52.176+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:52.181 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:52.279 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:52.277+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:52.328 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:52.329 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:52.336 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:52.380 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:52.378+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:52.480 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:52.478+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:52.481 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:52.481 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:52.490 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:52.581 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:52.580+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:52.595 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:11:52.637 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:52.637 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:52.645 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:52.682 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:52.681+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:52.717 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:52.717 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:52.734 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:52.734 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:52.751 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:52.751 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:52.767 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:52.767 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:52.783 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:52.781+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:52.786 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:52.786 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:52.794 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:47798 3.7 3:e6004524:::rbd_data.113916cf513.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1789239~44328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.387407+0000", 2024-09-15T00:11:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.403448418, 2024-09-15T00:11:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033697973999999999, 2024-09-15T00:11:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 47798 2024-09-15T00:11:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.387407+0000", 2024-09-15T00:11:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.387407+0000", 2024-09-15T00:11:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.387408+0000", 2024-09-15T00:11:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2580000000000001e-06 2024-09-15T00:11:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389094+0000", 2024-09-15T00:11:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001685572 2024-09-15T00:11:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389116+0000", 2024-09-15T00:11:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2379999999999999e-05 2024-09-15T00:11:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389126+0000", 2024-09-15T00:11:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0044000000000001e-05 2024-09-15T00:11:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389203+0000", 2024-09-15T00:11:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6945000000000003e-05 2024-09-15T00:11:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389381+0000", 2024-09-15T00:11:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000177908 2024-09-15T00:11:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389890+0000", 2024-09-15T00:11:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00050844599999999996 2024-09-15T00:11:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.420647+0000", 2024-09-15T00:11:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030757201000000001 2024-09-15T00:11:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.420970+0000", 2024-09-15T00:11:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032290200000000002 2024-09-15T00:11:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.421037+0000", 2024-09-15T00:11:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6883000000000003e-05 2024-09-15T00:11:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.421105+0000", 2024-09-15T00:11:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8435000000000005e-05 2024-09-15T00:11:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:47892 3.12 e47/28 3:4ef0cbcd:::rbd_data.113916cf513.0000000000000000:head v 47'139, mlcod=45'137)", 2024-09-15T00:11:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.437387+0000", 2024-09-15T00:11:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.353468629, 2024-09-15T00:11:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021482557999999999, 2024-09-15T00:11:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.437387+0000", 2024-09-15T00:11:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.437387+0000", 2024-09-15T00:11:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.437389+0000", 2024-09-15T00:11:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6390000000000001e-06 2024-09-15T00:11:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.440729+0000", 2024-09-15T00:11:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033401490000000002 2024-09-15T00:11:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.440750+0000", 2024-09-15T00:11:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1376e-05 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.440767+0000", 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7127000000000001e-05 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.440847+0000", 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689000000000002e-05 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.440876+0000", 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8663e-05 2024-09-15T00:11:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.458828+0000", 2024-09-15T00:11:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017952633999999999 2024-09-15T00:11:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.458869+0000", 2024-09-15T00:11:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1281e-05 2024-09-15T00:11:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48090 3.b 3:d42b6c47:::rbd_data.113916cf513.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 13893632~5352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.534140+0000", 2024-09-15T00:11:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2567156449999999, 2024-09-15T00:11:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022685002999999999, 2024-09-15T00:11:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48090 2024-09-15T00:11:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.534140+0000", 2024-09-15T00:11:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.534140+0000", 2024-09-15T00:11:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.534141+0000", 2024-09-15T00:11:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2699999999999999e-06 2024-09-15T00:11:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.534970+0000", 2024-09-15T00:11:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00082854499999999998 2024-09-15T00:11:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.534991+0000", 2024-09-15T00:11:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1058999999999999e-05 2024-09-15T00:11:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.535004+0000", 2024-09-15T00:11:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2830000000000001e-05 2024-09-15T00:11:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.535078+0000", 2024-09-15T00:11:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4695000000000002e-05 2024-09-15T00:11:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.535209+0000", 2024-09-15T00:11:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013081900000000001 2024-09-15T00:11:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.535595+0000", 2024-09-15T00:11:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000385993 2024-09-15T00:11:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.553922+0000", 2024-09-15T00:11:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018327011000000001 2024-09-15T00:11:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.556695+0000", 2024-09-15T00:11:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00277261 2024-09-15T00:11:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.556785+0000", 2024-09-15T00:11:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0649000000000006e-05 2024-09-15T00:11:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.556825+0000", 2024-09-15T00:11:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9521999999999999e-05 2024-09-15T00:11:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48896 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.608433+0000", 2024-09-15T00:11:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1824228750000003, 2024-09-15T00:11:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019236590000000001, 2024-09-15T00:11:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48896 2024-09-15T00:11:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608433+0000", 2024-09-15T00:11:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608433+0000", 2024-09-15T00:11:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608433+0000", 2024-09-15T00:11:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.46e-07 2024-09-15T00:11:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608435+0000", 2024-09-15T00:11:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:11:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608441+0000", 2024-09-15T00:11:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8969999999999996e-06 2024-09-15T00:11:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.608446+0000", 2024-09-15T00:11:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.921e-06 2024-09-15T00:11:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627487+0000", 2024-09-15T00:11:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019040973999999999 2024-09-15T00:11:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627572+0000", 2024-09-15T00:11:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4846999999999996e-05 2024-09-15T00:11:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.627669+0000", 2024-09-15T00:11:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7381999999999997e-05 2024-09-15T00:11:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48981 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 2752512~73728] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609227+0000", 2024-09-15T00:11:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1816285369999999, 2024-09-15T00:11:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019843092, 2024-09-15T00:11:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48981 2024-09-15T00:11:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609227+0000", 2024-09-15T00:11:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609227+0000", 2024-09-15T00:11:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609227+0000", 2024-09-15T00:11:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5900000000000002e-07 2024-09-15T00:11:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609230+0000", 2024-09-15T00:11:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2089999999999999e-06 2024-09-15T00:11:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609236+0000", 2024-09-15T00:11:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4919999999999998e-06 2024-09-15T00:11:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609241+0000", 2024-09-15T00:11:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0440000000000003e-06 2024-09-15T00:11:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628352+0000", 2024-09-15T00:11:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019111145999999999 2024-09-15T00:11:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.628433+0000", 2024-09-15T00:11:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0887e-05 2024-09-15T00:11:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629070+0000", 2024-09-15T00:11:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00063685499999999997 2024-09-15T00:11:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48982 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 4587520~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609275+0000", 2024-09-15T00:11:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1815800700000008, 2024-09-15T00:11:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020353955999999999, 2024-09-15T00:11:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48982 2024-09-15T00:11:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609275+0000", 2024-09-15T00:11:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609275+0000", 2024-09-15T00:11:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609276+0000", 2024-09-15T00:11:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5799999999999999e-07 2024-09-15T00:11:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609279+0000", 2024-09-15T00:11:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5830000000000001e-06 2024-09-15T00:11:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609285+0000", 2024-09-15T00:11:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8240000000000002e-06 2024-09-15T00:11:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609291+0000", 2024-09-15T00:11:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1399999999999999e-06 2024-09-15T00:11:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629107+0000", 2024-09-15T00:11:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019816674999999999 2024-09-15T00:11:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629158+0000", 2024-09-15T00:11:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1005999999999999e-05 2024-09-15T00:11:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629629+0000", 2024-09-15T00:11:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047116999999999998 2024-09-15T00:11:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48983 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609298+0000", 2024-09-15T00:11:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1815573710000002, 2024-09-15T00:11:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020455086000000001, 2024-09-15T00:11:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48983 2024-09-15T00:11:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609298+0000", 2024-09-15T00:11:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609298+0000", 2024-09-15T00:11:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609299+0000", 2024-09-15T00:11:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7899999999999998e-07 2024-09-15T00:11:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609301+0000", 2024-09-15T00:11:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0379999999999998e-06 2024-09-15T00:11:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609307+0000", 2024-09-15T00:11:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2600000000000002e-06 2024-09-15T00:11:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609312+0000", 2024-09-15T00:11:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4009999999999996e-06 2024-09-15T00:11:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629651+0000", 2024-09-15T00:11:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020338438 2024-09-15T00:11:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629696+0000", 2024-09-15T00:11:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5123000000000001e-05 2024-09-15T00:11:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629753+0000", 2024-09-15T00:11:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7247000000000003e-05 2024-09-15T00:11:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48984 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609319+0000", 2024-09-15T00:11:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1815364339999999, 2024-09-15T00:11:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020570343000000001, 2024-09-15T00:11:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48984 2024-09-15T00:11:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609319+0000", 2024-09-15T00:11:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609319+0000", 2024-09-15T00:11:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609319+0000", 2024-09-15T00:11:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3099999999999998e-07 2024-09-15T00:11:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609321+0000", 2024-09-15T00:11:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:11:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609328+0000", 2024-09-15T00:11:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2149999999999998e-06 2024-09-15T00:11:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609333+0000", 2024-09-15T00:11:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.22e-06 2024-09-15T00:11:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629783+0000", 2024-09-15T00:11:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020450342 2024-09-15T00:11:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629829+0000", 2024-09-15T00:11:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5782999999999999e-05 2024-09-15T00:11:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629889+0000", 2024-09-15T00:11:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0393999999999998e-05 2024-09-15T00:11:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48985 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609339+0000", 2024-09-15T00:11:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1815164599999992, 2024-09-15T00:11:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020671998, 2024-09-15T00:11:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.833 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:52.833 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.833 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.833 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.833 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48985 2024-09-15T00:11:52.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.833 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609339+0000", 2024-09-15T00:11:52.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609339+0000", 2024-09-15T00:11:52.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609339+0000", 2024-09-15T00:11:52.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:11:52.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609341+0000", 2024-09-15T00:11:52.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9769999999999999e-06 2024-09-15T00:11:52.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609351+0000", 2024-09-15T00:11:52.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6220000000000004e-06 2024-09-15T00:11:52.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609356+0000", 2024-09-15T00:11:52.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1660000000000002e-06 2024-09-15T00:11:52.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629914+0000", 2024-09-15T00:11:52.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020558205 2024-09-15T00:11:52.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.629954+0000", 2024-09-15T00:11:52.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9619999999999997e-05 2024-09-15T00:11:52.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630011+0000", 2024-09-15T00:11:52.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7058999999999997e-05 2024-09-15T00:11:52.836 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.836 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.836 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.836 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48986 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609363+0000", 2024-09-15T00:11:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1814925909999996, 2024-09-15T00:11:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020764550999999999, 2024-09-15T00:11:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48986 2024-09-15T00:11:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609363+0000", 2024-09-15T00:11:52.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609363+0000", 2024-09-15T00:11:52.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609363+0000", 2024-09-15T00:11:52.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:11:52.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609366+0000", 2024-09-15T00:11:52.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.21e-06 2024-09-15T00:11:52.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609372+0000", 2024-09-15T00:11:52.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6390000000000003e-06 2024-09-15T00:11:52.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609377+0000", 2024-09-15T00:11:52.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6190000000000002e-06 2024-09-15T00:11:52.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630035+0000", 2024-09-15T00:11:52.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020657847 2024-09-15T00:11:52.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630076+0000", 2024-09-15T00:11:52.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1597000000000002e-05 2024-09-15T00:11:52.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630127+0000", 2024-09-15T00:11:52.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1232e-05 2024-09-15T00:11:52.841 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.841 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.841 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.841 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:48987 3.1e 3:7a27b018:::rbd_data.11395b9829f3.0000000000000006:e [sparse-read 13762560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:52.841 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.609383+0000", 2024-09-15T00:11:52.841 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1814721230000007, 2024-09-15T00:11:52.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020904947, 2024-09-15T00:11:52.841 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.841 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.841 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:52.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.842 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.842 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 48987 2024-09-15T00:11:52.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.842 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609383+0000", 2024-09-15T00:11:52.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609383+0000", 2024-09-15T00:11:52.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609384+0000", 2024-09-15T00:11:52.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:11:52.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609386+0000", 2024-09-15T00:11:52.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1619999999999998e-06 2024-09-15T00:11:52.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609392+0000", 2024-09-15T00:11:52.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4999999999999996e-06 2024-09-15T00:11:52.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.609399+0000", 2024-09-15T00:11:52.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3570000000000003e-06 2024-09-15T00:11:52.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630147+0000", 2024-09-15T00:11:52.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020748195000000001 2024-09-15T00:11:52.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630201+0000", 2024-09-15T00:11:52.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3838000000000002e-05 2024-09-15T00:11:52.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.630288+0000", 2024-09-15T00:11:52.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7484999999999993e-05 2024-09-15T00:11:52.847 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.848 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.848 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.848 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49030 3.b 3:d0a51340:::rbd_data.113916cf513.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:52.848 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.662376+0000", 2024-09-15T00:11:52.848 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1284793000000004, 2024-09-15T00:11:52.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.028039436000000001, 2024-09-15T00:11:52.848 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.848 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.848 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:52.848 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.848 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.848 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.849 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49030 2024-09-15T00:11:52.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.849 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.662376+0000", 2024-09-15T00:11:52.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.662376+0000", 2024-09-15T00:11:52.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.662379+0000", 2024-09-15T00:11:52.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.168e-06 2024-09-15T00:11:52.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663508+0000", 2024-09-15T00:11:52.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011290370000000001 2024-09-15T00:11:52.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663539+0000", 2024-09-15T00:11:52.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1006000000000001e-05 2024-09-15T00:11:52.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663551+0000", 2024-09-15T00:11:52.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1708e-05 2024-09-15T00:11:52.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663628+0000", 2024-09-15T00:11:52.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6633999999999998e-05 2024-09-15T00:11:52.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663783+0000", 2024-09-15T00:11:52.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015484099999999999 2024-09-15T00:11:52.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:52.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.664331+0000", 2024-09-15T00:11:52.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00054832600000000002 2024-09-15T00:11:52.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:52.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.690059+0000", 2024-09-15T00:11:52.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025727612 2024-09-15T00:11:52.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:52.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.690300+0000", 2024-09-15T00:11:52.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024122600000000001 2024-09-15T00:11:52.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:52.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.690360+0000", 2024-09-15T00:11:52.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0458000000000003e-05 2024-09-15T00:11:52.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.690416+0000", 2024-09-15T00:11:52.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5420000000000001e-05 2024-09-15T00:11:52.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.853 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.854 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:49026 3.19 e47/23 3:9c3e88bf:::rbd_data.113916cf513.0000000000000002:head v 47'92, mlcod=45'89)", 2024-09-15T00:11:52.854 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.665261+0000", 2024-09-15T00:11:52.854 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1255942440000002, 2024-09-15T00:11:52.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019832186000000002, 2024-09-15T00:11:52.854 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.854 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.854 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:52.854 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.665261+0000", 2024-09-15T00:11:52.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.665261+0000", 2024-09-15T00:11:52.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.665265+0000", 2024-09-15T00:11:52.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4570000000000001e-06 2024-09-15T00:11:52.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667479+0000", 2024-09-15T00:11:52.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022139260000000002 2024-09-15T00:11:52.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667492+0000", 2024-09-15T00:11:52.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3084e-05 2024-09-15T00:11:52.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667500+0000", 2024-09-15T00:11:52.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7609999999999996e-06 2024-09-15T00:11:52.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667569+0000", 2024-09-15T00:11:52.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8359000000000001e-05 2024-09-15T00:11:52.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667607+0000", 2024-09-15T00:11:52.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7716999999999998e-05 2024-09-15T00:11:52.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:52.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.685060+0000", 2024-09-15T00:11:52.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017453344999999999 2024-09-15T00:11:52.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.685093+0000", 2024-09-15T00:11:52.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3537000000000001e-05 2024-09-15T00:11:52.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.858 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.858 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56198 3.2 3:44ae5a28:::rbd_data.1139167e758e.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:11:52.858 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.238999+0000", 2024-09-15T00:11:52.858 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.551856849, 2024-09-15T00:11:52.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037097333000000003, 2024-09-15T00:11:52.858 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.858 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.859 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:52.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.859 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56198 2024-09-15T00:11:52.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.859 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.238999+0000", 2024-09-15T00:11:52.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.238999+0000", 2024-09-15T00:11:52.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.239000+0000", 2024-09-15T00:11:52.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.359e-06 2024-09-15T00:11:52.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242065+0000", 2024-09-15T00:11:52.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030647230000000001 2024-09-15T00:11:52.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242079+0000", 2024-09-15T00:11:52.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3801999999999999e-05 2024-09-15T00:11:52.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242108+0000", 2024-09-15T00:11:52.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9889000000000001e-05 2024-09-15T00:11:52.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242177+0000", 2024-09-15T00:11:52.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9072000000000005e-05 2024-09-15T00:11:52.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242332+0000", 2024-09-15T00:11:52.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000154816 2024-09-15T00:11:52.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:52.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.243189+0000", 2024-09-15T00:11:52.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00085691200000000004 2024-09-15T00:11:52.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:52.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.267650+0000", 2024-09-15T00:11:52.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024461146999999999 2024-09-15T00:11:52.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:52.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.275977+0000", 2024-09-15T00:11:52.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0083269729999999997 2024-09-15T00:11:52.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:52.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276040+0000", 2024-09-15T00:11:52.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2497999999999997e-05 2024-09-15T00:11:52.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276096+0000", 2024-09-15T00:11:52.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6141999999999999e-05 2024-09-15T00:11:52.863 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.863 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.863 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.864 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56199 3.2 3:44ae5a28:::rbd_data.1139167e758e.0000000000000000:head [write 0~0] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:11:52.864 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242772+0000", 2024-09-15T00:11:52.864 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5480834459999997, 2024-09-15T00:11:52.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033898029000000003, 2024-09-15T00:11:52.864 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.864 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.864 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:52.864 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.864 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.864 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.864 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56199 2024-09-15T00:11:52.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.865 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242772+0000", 2024-09-15T00:11:52.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242772+0000", 2024-09-15T00:11:52.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242774+0000", 2024-09-15T00:11:52.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.483e-06 2024-09-15T00:11:52.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242782+0000", 2024-09-15T00:11:52.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9740000000000002e-06 2024-09-15T00:11:52.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242806+0000", 2024-09-15T00:11:52.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4689e-05 2024-09-15T00:11:52.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242815+0000", 2024-09-15T00:11:52.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3410000000000003e-06 2024-09-15T00:11:52.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.265144+0000", 2024-09-15T00:11:52.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022329244000000002 2024-09-15T00:11:52.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.265237+0000", 2024-09-15T00:11:52.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3154999999999998e-05 2024-09-15T00:11:52.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:52.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.265426+0000", 2024-09-15T00:11:52.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018955600000000001 2024-09-15T00:11:52.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:52.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.267912+0000", 2024-09-15T00:11:52.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024857880000000001 2024-09-15T00:11:52.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:52.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276583+0000", 2024-09-15T00:11:52.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0086705689999999995 2024-09-15T00:11:52.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:52.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276634+0000", 2024-09-15T00:11:52.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1465000000000003e-05 2024-09-15T00:11:52.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276670+0000", 2024-09-15T00:11:52.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5765000000000001e-05 2024-09-15T00:11:52.869 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.869 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.869 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.869 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:11:52.870 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:11:52.870 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5480114, 2024-09-15T00:11:52.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:11:52.870 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.870 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.870 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:52.870 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.870 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.870 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.870 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:11:52.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.870 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:11:52.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:11:52.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:11:52.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:11:52.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:11:52.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:11:52.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:11:52.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:11:52.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:11:52.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:11:52.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:11:52.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:11:52.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:11:52.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:11:52.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:52.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:11:52.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:11:52.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:52.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:11:52.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:11:52.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:52.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:11:52.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:11:52.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:52.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:11:52.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:11:52.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:11:52.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:11:52.875 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.875 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.875 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.875 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56207 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [write 0~0] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:11:52.875 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.245178+0000", 2024-09-15T00:11:52.875 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5456776999999997, 2024-09-15T00:11:52.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038624626000000002, 2024-09-15T00:11:52.875 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.876 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.876 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:52.876 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.876 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.876 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.876 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56207 2024-09-15T00:11:52.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.876 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245178+0000", 2024-09-15T00:11:52.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245178+0000", 2024-09-15T00:11:52.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245180+0000", 2024-09-15T00:11:52.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-15T00:11:52.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245185+0000", 2024-09-15T00:11:52.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1780000000000002e-06 2024-09-15T00:11:52.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245192+0000", 2024-09-15T00:11:52.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6420000000000007e-06 2024-09-15T00:11:52.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245198+0000", 2024-09-15T00:11:52.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5729999999999998e-06 2024-09-15T00:11:52.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.271149+0000", 2024-09-15T00:11:52.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025950534000000001 2024-09-15T00:11:52.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.271238+0000", 2024-09-15T00:11:52.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9364999999999996e-05 2024-09-15T00:11:52.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:52.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.271398+0000", 2024-09-15T00:11:52.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015962600000000001 2024-09-15T00:11:52.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:52.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.277582+0000", 2024-09-15T00:11:52.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061847669999999999 2024-09-15T00:11:52.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:52.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283748+0000", 2024-09-15T00:11:52.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061654170000000003 2024-09-15T00:11:52.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:52.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283782+0000", 2024-09-15T00:11:52.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4245000000000002e-05 2024-09-15T00:11:52.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283802+0000", 2024-09-15T00:11:52.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-05 2024-09-15T00:11:52.881 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.881 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.881 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.881 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:56200 3.c e51/23 3:347f8d70:::rbd_data.1139167e758e.0000000000000002:head v 51'165, mlcod=50'162)", 2024-09-15T00:11:52.881 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.248115+0000", 2024-09-15T00:11:52.881 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5427408380000003, 2024-09-15T00:11:52.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025900706999999999, 2024-09-15T00:11:52.881 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.881 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.881 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:52.881 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.248115+0000", 2024-09-15T00:11:52.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.248115+0000", 2024-09-15T00:11:52.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.248115+0000", 2024-09-15T00:11:52.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9999999999999997e-07 2024-09-15T00:11:52.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.253554+0000", 2024-09-15T00:11:52.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054384120000000001 2024-09-15T00:11:52.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.253569+0000", 2024-09-15T00:11:52.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5460000000000001e-05 2024-09-15T00:11:52.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.253579+0000", 2024-09-15T00:11:52.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0261000000000001e-05 2024-09-15T00:11:52.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.253642+0000", 2024-09-15T00:11:52.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2482000000000002e-05 2024-09-15T00:11:52.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.253665+0000", 2024-09-15T00:11:52.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.298e-05 2024-09-15T00:11:52.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:52.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273983+0000", 2024-09-15T00:11:52.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020318340000000001 2024-09-15T00:11:52.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274015+0000", 2024-09-15T00:11:52.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2172000000000003e-05 2024-09-15T00:11:52.885 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.885 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.885 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.885 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56253 3.b 3:d22b8105:::rbd_data.1139167e758e.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:11:52.885 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.272929+0000", 2024-09-15T00:11:52.886 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5179265500000003, 2024-09-15T00:11:52.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020968238, 2024-09-15T00:11:52.886 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.886 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.886 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:52.886 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.886 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.886 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.886 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56253 2024-09-15T00:11:52.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.886 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.272929+0000", 2024-09-15T00:11:52.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.272929+0000", 2024-09-15T00:11:52.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.272932+0000", 2024-09-15T00:11:52.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6919999999999998e-06 2024-09-15T00:11:52.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273886+0000", 2024-09-15T00:11:52.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00095420599999999998 2024-09-15T00:11:52.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273902+0000", 2024-09-15T00:11:52.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5699e-05 2024-09-15T00:11:52.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273910+0000", 2024-09-15T00:11:52.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0590000000000003e-06 2024-09-15T00:11:52.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273966+0000", 2024-09-15T00:11:52.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6604999999999998e-05 2024-09-15T00:11:52.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274076+0000", 2024-09-15T00:11:52.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000109617 2024-09-15T00:11:52.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:52.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274959+0000", 2024-09-15T00:11:52.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088268500000000002 2024-09-15T00:11:52.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:52.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.290999+0000", 2024-09-15T00:11:52.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016040868999999999 2024-09-15T00:11:52.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:52.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.293799+0000", 2024-09-15T00:11:52.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027999159999999999 2024-09-15T00:11:52.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:52.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.293853+0000", 2024-09-15T00:11:52.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3783999999999999e-05 2024-09-15T00:11:52.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.293897+0000", 2024-09-15T00:11:52.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4106000000000001e-05 2024-09-15T00:11:52.891 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.891 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.891 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.891 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56254 3.b 3:d22b8105:::rbd_data.1139167e758e.0000000000000005:head [write 0~0] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:11:52.891 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.273938+0000", 2024-09-15T00:11:52.891 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5169179530000001, 2024-09-15T00:11:52.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020239258, 2024-09-15T00:11:52.891 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:52.891 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:52.891 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:52.892 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:11:52.892 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:11:52.892 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:52.892 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56254 2024-09-15T00:11:52.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.892 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:52.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:52.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273938+0000", 2024-09-15T00:11:52.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:52.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273938+0000", 2024-09-15T00:11:52.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:52.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:52.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273939+0000", 2024-09-15T00:11:52.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5600000000000001e-06 2024-09-15T00:11:52.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:52.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273942+0000", 2024-09-15T00:11:52.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7669999999999999e-06 2024-09-15T00:11:52.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:52.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273947+0000", 2024-09-15T00:11:52.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0220000000000004e-06 2024-09-15T00:11:52.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:52.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.273955+0000", 2024-09-15T00:11:52.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4999999999999999e-06 2024-09-15T00:11:52.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:52.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.289470+0000", 2024-09-15T00:11:52.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015514422999999999 2024-09-15T00:11:52.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:52.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.289523+0000", 2024-09-15T00:11:52.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3267999999999999e-05 2024-09-15T00:11:52.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:52.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.289629+0000", 2024-09-15T00:11:52.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000105623 2024-09-15T00:11:52.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:11:52.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.291337+0000", 2024-09-15T00:11:52.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001708289 2024-09-15T00:11:52.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.294125+0000", 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027884530000000002 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.294155+0000", 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9183000000000002e-05 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.294177+0000", 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2169999999999999e-05 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.896 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.897 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:52.897 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:52.897 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:52.898 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:52.899 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:52.899 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:52.899 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:52.900 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:52.900 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:52.900 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:52.901 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:52.896+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:52.998 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:52.997+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:53.015 INFO:teuthology.orchestra.run.smithi171.stdout:2197 punch from 0xb1f8753 to 0xb200678, (0x7f25 bytes) 2024-09-15T00:11:53.015 INFO:teuthology.orchestra.run.smithi171.stdout:2198 read 0xb3fe017 thru 0xb406e65 (0x8e4f bytes) 2024-09-15T00:11:53.015 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:53.015 INFO:teuthology.orchestra.run.smithi171.stdout:2201 write 0x7ca5d11 thru 0x7ca8b07 (0x2df7 bytes) 2024-09-15T00:11:53.019 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:11:53.025 INFO:teuthology.orchestra.run.smithi171.stdout:2204 writesame 0x7a694a9 thru 0x7a69c1c data_size 0x3(0x774 bytes) 2024-09-15T00:11:53.025 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:53.025 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:53.026 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:53.026 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:53.026 INFO:teuthology.orchestra.run.smithi171.stdout:2206 write 0x7d5164b thru 0x7d5a57f (0x8f35 bytes) 2024-09-15T00:11:53.031 INFO:teuthology.orchestra.run.smithi171.stdout:2207 read 0x6e26106 thru 0x6e31ba4 (0xba9f bytes) 2024-09-15T00:11:53.032 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:53.032 INFO:teuthology.orchestra.run.smithi171.stdout:2211 write 0x4f49c49 thru 0x4f4eee4 (0x529c bytes) 2024-09-15T00:11:53.037 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:53.037 INFO:teuthology.orchestra.run.smithi171.stdout:2212 trunc from 0xbeaef4e to 0xc5153f2 2024-09-15T00:11:53.043 INFO:teuthology.orchestra.run.smithi171.stdout:2213 write 0x926f049 thru 0x92706ea (0x16a2 bytes) 2024-09-15T00:11:53.049 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:53.049 INFO:teuthology.orchestra.run.smithi171.stdout:2217 write 0x92ead93 thru 0x92ec623 (0x1891 bytes) 2024-09-15T00:11:53.055 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:53.056 INFO:teuthology.orchestra.run.smithi171.stdout:2221 read 0xa93f832 thru 0xa94c19a (0xc969 bytes) 2024-09-15T00:11:53.056 INFO:teuthology.orchestra.run.smithi171.stdout:2222 read 0x13f9355 thru 0x13fd877 (0x4523 bytes) 2024-09-15T00:11:53.063 INFO:teuthology.orchestra.run.smithi171.stdout:2223 write 0xd74e361 thru 0xd7580cb (0x9d6b bytes) 2024-09-15T00:11:53.072 INFO:teuthology.orchestra.run.smithi171.stdout:2224 trunc from 0xd7580cc to 0xdfefb9f 2024-09-15T00:11:53.077 INFO:teuthology.orchestra.run.smithi171.stdout:2225 read 0xd2df0c2 thru 0xd2e47aa (0x56e9 bytes) 2024-09-15T00:11:53.078 INFO:teuthology.orchestra.run.smithi171.stdout:2226 write 0x45e9bf8 thru 0x45f9612 (0xfa1b bytes) 2024-09-15T00:11:53.087 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:53.087 INFO:teuthology.orchestra.run.smithi171.stdout:2230 write 0x80e0f1f thru 0x80f06e3 (0xf7c5 bytes) 2024-09-15T00:11:53.096 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:53.096 INFO:teuthology.orchestra.run.smithi171.stdout:2234 write 0x542e8dd thru 0x543e73d (0xfe61 bytes) 2024-09-15T00:11:53.099 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:53.098+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:53.102 INFO:teuthology.orchestra.run.smithi171.stdout:2235 read 0x98331a2 thru 0x9834434 (0x1293 bytes) 2024-09-15T00:11:53.102 INFO:teuthology.orchestra.run.smithi171.stdout:2236 write 0x275d1cf thru 0x275dda6 (0xbd8 bytes) 2024-09-15T00:11:53.104 INFO:teuthology.orchestra.run.smithi171.stdout:2237 write 0xb12e3c3 thru 0xb13e30d (0xff4b bytes) 2024-09-15T00:11:53.115 INFO:teuthology.orchestra.run.smithi171.stdout:2238 write 0xea0060f thru 0xea0fe44 (0xf836 bytes) 2024-09-15T00:11:53.121 INFO:teuthology.orchestra.run.smithi171.stdout:2239 write 0x56971fb thru 0x56a1952 (0xa758 bytes) 2024-09-15T00:11:53.131 INFO:teuthology.orchestra.run.smithi171.stdout:2241 trunc from 0xea0fe45 to 0x861a74e 2024-09-15T00:11:53.156 INFO:teuthology.orchestra.run.smithi171.stdout:2242 write 0x9b8a912 thru 0x9b918a1 (0x6f90 bytes) 2024-09-15T00:11:53.157 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:53.157 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:53.158 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:53.158 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:53.158 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:53.162 INFO:teuthology.orchestra.run.smithi171.stdout:2243 punch from 0x34166bd to 0x341eb30, (0x8473 bytes) 2024-09-15T00:11:53.162 INFO:teuthology.orchestra.run.smithi171.stdout:2245 read 0x1535100 thru 0x1536595 (0x1496 bytes) 2024-09-15T00:11:53.164 INFO:teuthology.orchestra.run.smithi171.stdout:2246 trunc from 0x9b918a2 to 0xba9f13d 2024-09-15T00:11:53.167 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:53.170 INFO:teuthology.orchestra.run.smithi171.stdout:2247 read 0x33d8df7 thru 0x33de715 (0x591f bytes) 2024-09-15T00:11:53.170 INFO:teuthology.orchestra.run.smithi171.stdout:2248 punch from 0x9722a2f to 0x972c9a9, (0x9f7a bytes) 2024-09-15T00:11:53.170 INFO:teuthology.orchestra.run.smithi171.stdout:2249 read 0x8b16f20 thru 0x8b1d8be (0x699f bytes) 2024-09-15T00:11:53.179 INFO:teuthology.orchestra.run.smithi171.stdout:2251 write 0xe7e197a thru 0xe7ee910 (0xcf97 bytes) 2024-09-15T00:11:53.184 INFO:teuthology.orchestra.run.smithi171.stdout:2252 write 0x4a953b6 thru 0x4a9b11d (0x5d68 bytes) 2024-09-15T00:11:53.186 INFO:teuthology.orchestra.run.smithi171.stdout:2254 read 0x1dc664b thru 0x1dc7d49 (0x16ff bytes) 2024-09-15T00:11:53.186 INFO:teuthology.orchestra.run.smithi171.stdout:2255 write 0x67b0487 thru 0x67bb710 (0xb28a bytes) 2024-09-15T00:11:53.208 INFO:teuthology.orchestra.run.smithi171.stdout:2256 trunc from 0xe7ee911 to 0x46ace51 2024-09-15T00:11:53.209 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:53.207+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:53.243 INFO:teuthology.orchestra.run.smithi171.stdout:2257 write 0xe47b03a thru 0xe47b3e9 (0x3b0 bytes) 2024-09-15T00:11:53.246 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:53.246 INFO:teuthology.orchestra.run.smithi171.stdout:2260 write 0x7784e74 thru 0x7789695 (0x4822 bytes) 2024-09-15T00:11:53.252 INFO:teuthology.orchestra.run.smithi171.stdout:2261 write 0x78ebb75 thru 0x78fbb09 (0xff95 bytes) 2024-09-15T00:11:53.260 INFO:teuthology.orchestra.run.smithi171.stdout:2262 read 0x27a392e thru 0x27af632 (0xbd05 bytes) 2024-09-15T00:11:53.260 INFO:teuthology.orchestra.run.smithi171.stdout:2263 writesame 0x9f26dd0 thru 0x9f350db data_size 0x16(0xe30c bytes) 2024-09-15T00:11:53.270 INFO:teuthology.orchestra.run.smithi171.stdout:2264 writesame 0x562a49 thru 0x5703e7 data_size 0x1(0xd99f bytes) 2024-09-15T00:11:53.302 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:53.300+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:53.326 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:11:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.751414739000001, 2024-09-15T00:11:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:11:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:11:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:11:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:11:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:11:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:11:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:11:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:11:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:11:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:11:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:11:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:11:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:11:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:11:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:11:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:11:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:11:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:11:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:11:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:11:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:11:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:11:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:11:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:11:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:11:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:11:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:11:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.751380047000001, 2024-09-15T00:11:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:11:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:11:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:11:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:11:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:11:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:11:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:11:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:11:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:11:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:11:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:11:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:11:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:11:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:11:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:11:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:11:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:11:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:11:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:11:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:11:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:11:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:11:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:11:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:11:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48937 3.1a 3:5e7e71d0:::rbd_data.11395b9829f3.0000000000000000:e [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:11:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7112870919999992, 2024-09-15T00:11:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072322336000000001, 2024-09-15T00:11:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48937 2024-09-15T00:11:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:11:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:11:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612043+0000", 2024-09-15T00:11:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:11:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612045+0000", 2024-09-15T00:11:53.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2749999999999998e-06 2024-09-15T00:11:53.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612053+0000", 2024-09-15T00:11:53.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0439999999999996e-06 2024-09-15T00:11:53.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612062+0000", 2024-09-15T00:11:53.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6829999999999997e-06 2024-09-15T00:11:53.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684078+0000", 2024-09-15T00:11:53.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072016017000000002 2024-09-15T00:11:53.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684274+0000", 2024-09-15T00:11:53.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019636600000000001 2024-09-15T00:11:53.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684364+0000", 2024-09-15T00:11:53.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0420000000000005e-05 2024-09-15T00:11:53.343 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.343 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.343 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.343 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.343 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:11:53.343 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7107063589999996, 2024-09-15T00:11:53.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:11:53.343 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.343 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.343 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.343 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.343 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.344 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.344 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:11:53.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.344 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:11:53.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:11:53.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:11:53.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:11:53.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:11:53.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:11:53.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:11:53.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:11:53.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:11:53.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:11:53.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:11:53.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:11:53.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:11:53.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:11:53.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:11:53.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:11:53.347 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.347 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.347 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.347 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.348 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:11:53.348 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7106898279999996, 2024-09-15T00:11:53.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:11:53.348 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.348 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.348 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.348 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.348 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.348 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.348 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:11:53.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.348 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:11:53.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:11:53.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:11:53.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:11:53.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:11:53.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:11:53.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:11:53.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:11:53.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:11:53.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:11:53.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.351 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.352 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.352 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.352 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:11:53.352 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7106730490000004, 2024-09-15T00:11:53.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:11:53.352 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.352 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.352 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.353 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:11:53.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.353 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:11:53.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:11:53.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:11:53.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:11:53.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:11:53.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:11:53.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:11:53.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:11:53.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:11:53.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:11:53.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:11:53.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:11:53.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:11:53.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:11:53.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:11:53.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:11:53.356 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.356 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.356 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.356 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.356 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:11:53.357 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7106553190000007, 2024-09-15T00:11:53.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:11:53.357 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.357 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.357 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.357 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.357 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.357 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.357 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:11:53.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.357 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:11:53.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:11:53.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:11:53.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:11:53.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:11:53.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:11:53.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:11:53.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:11:53.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:11:53.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:11:53.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:11:53.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:11:53.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:11:53.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:11:53.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:11:53.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:11:53.361 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.361 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.361 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.361 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.361 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:11:53.361 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7106381549999998, 2024-09-15T00:11:53.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:11:53.361 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.361 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.361 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.362 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.362 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:11:53.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.362 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:11:53.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:11:53.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:11:53.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:11:53.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:11:53.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:11:53.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:11:53.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:11:53.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:11:53.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:11:53.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:11:53.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:11:53.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:11:53.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:11:53.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:11:53.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:11:53.365 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.365 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.365 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.365 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.365 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:11:53.366 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7106164790000005, 2024-09-15T00:11:53.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:11:53.366 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.366 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.366 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.366 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.366 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.366 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.366 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:11:53.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.366 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:11:53.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:11:53.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:11:53.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:11:53.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:11:53.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:11:53.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:11:53.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:11:53.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:11:53.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:11:53.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:11:53.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:11:53.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:11:53.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:11:53.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:11:53.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:11:53.370 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.370 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.370 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.370 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.370 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:11:53.370 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7105703549999998, 2024-09-15T00:11:53.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:11:53.370 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.370 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.370 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:11:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:11:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:11:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:11:53.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:11:53.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:11:53.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:11:53.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:11:53.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:11:53.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:11:53.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:11:53.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:11:53.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:11:53.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:11:53.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:11:53.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:11:53.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:11:53.374 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.374 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.374 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.374 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.374 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:11:53.374 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7105405979999997, 2024-09-15T00:11:53.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:11:53.375 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.375 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.375 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.375 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.375 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.375 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.375 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:11:53.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.375 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:11:53.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:11:53.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:11:53.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:11:53.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:11:53.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:11:53.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:11:53.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:11:53.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:11:53.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:11:53.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:11:53.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:11:53.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:11:53.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:11:53.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:11:53.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:11:53.379 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.379 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.379 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.379 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.379 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:11:53.379 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7105142789999999, 2024-09-15T00:11:53.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:11:53.379 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.379 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.379 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.380 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.380 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:11:53.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.380 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:11:53.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:11:53.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:11:53.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:11:53.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:11:53.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:11:53.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:11:53.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:11:53.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:11:53.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:11:53.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:11:53.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:11:53.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:11:53.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:11:53.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:11:53.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:11:53.383 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.383 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.383 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.383 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.384 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:11:53.384 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7104784170000009, 2024-09-15T00:11:53.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:11:53.384 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.384 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.384 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.384 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.384 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.384 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.384 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:11:53.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.384 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:11:53.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:11:53.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:11:53.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:11:53.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:11:53.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:11:53.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:11:53.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:11:53.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:11:53.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:11:53.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:11:53.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:11:53.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:11:53.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:11:53.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:11:53.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:11:53.388 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.388 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.388 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.388 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.388 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:11:53.388 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7104566989999999, 2024-09-15T00:11:53.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:11:53.388 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.388 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.388 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.389 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.389 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.389 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.389 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:11:53.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.389 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:11:53.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:11:53.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:11:53.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:11:53.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:11:53.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:11:53.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:11:53.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:11:53.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:11:53.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:11:53.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:11:53.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:11:53.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:11:53.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:11:53.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:11:53.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:11:53.392 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.392 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.392 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.393 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.393 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:11:53.393 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7102602319999995, 2024-09-15T00:11:53.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:11:53.393 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.393 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.393 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.393 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:11:53.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.393 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:11:53.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:11:53.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:11:53.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:11:53.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:11:53.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:11:53.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:11:53.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:11:53.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:11:53.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:11:53.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:11:53.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:11:53.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:11:53.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:11:53.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:11:53.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:11:53.397 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.397 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.397 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.397 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.397 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:11:53.397 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7102343930000004, 2024-09-15T00:11:53.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:11:53.397 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.397 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.397 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.398 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.398 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.398 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.398 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:11:53.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.398 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:11:53.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:11:53.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:11:53.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:11:53.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:11:53.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:11:53.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:11:53.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:11:53.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:11:53.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:11:53.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:11:53.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:11:53.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:11:53.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:11:53.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:11:53.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:11:53.401 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.401 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.401 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.402 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.402 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:11:53.402 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7101727469999997, 2024-09-15T00:11:53.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:11:53.402 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.402 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.402 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.402 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.402 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.402 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.402 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:11:53.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.403 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:11:53.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:11:53.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:11:53.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:11:53.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:11:53.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:11:53.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:11:53.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:11:53.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:11:53.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:11:53.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:11:53.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:11:53.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:11:53.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:11:53.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:11:53.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:11:53.406 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.406 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.406 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.406 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.406 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:11:53.406 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7101334189999999, 2024-09-15T00:11:53.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:11:53.406 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.406 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.407 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.407 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.407 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.407 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.407 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:11:53.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.407 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:11:53.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:11:53.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:11:53.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:11:53.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:11:53.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:11:53.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:11:53.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:11:53.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:11:53.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:11:53.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:11:53.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:11:53.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:11:53.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:11:53.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:11:53.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:11:53.410 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.410 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.410 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.411 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.411 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:11:53.411 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7084080190000002, 2024-09-15T00:11:53.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:11:53.411 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.411 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.411 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.411 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.411 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.411 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.411 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:11:53.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.411 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:11:53.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:11:53.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:11:53.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:11:53.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:11:53.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:11:53.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:11:53.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:11:53.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:11:53.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:11:53.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:11:53.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:11:53.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:11:53.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:11:53.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:11:53.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:11:53.415 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.415 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.415 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.415 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.415 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:11:53.415 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7083858579999998, 2024-09-15T00:11:53.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:11:53.415 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.416 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.416 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.416 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.416 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.416 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.416 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:11:53.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.416 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:11:53.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:11:53.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:11:53.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:11:53.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:11:53.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:11:53.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:11:53.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:11:53.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:11:53.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:11:53.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:11:53.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:11:53.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:11:53.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:11:53.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:11:53.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:11:53.419 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.419 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.420 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.420 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.420 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.420 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:53.422 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:53.422 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:53.419+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:53.423 INFO:teuthology.orchestra.run.smithi171.stdout:2265 punch from 0x706f7e0 to 0x7078054, (0x8874 bytes) 2024-09-15T00:11:53.423 INFO:teuthology.orchestra.run.smithi171.stdout:2266 write 0xfc8cac thru 0xfd10b3 (0x8408 bytes) 2024-09-15T00:11:53.423 INFO:teuthology.orchestra.run.smithi171.stdout:2267 punch from 0x60fbd38 to 0x6101a6b, (0x5d33 bytes) 2024-09-15T00:11:53.423 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:53.423 INFO:teuthology.orchestra.run.smithi171.stdout:2269 write 0x1124e8 thru 0x117380 (0x4e99 bytes) 2024-09-15T00:11:53.423 INFO:teuthology.orchestra.run.smithi171.stdout:2270 punch from 0x583f41c to 0x58474b0, (0x8094 bytes) 2024-09-15T00:11:53.423 INFO:teuthology.orchestra.run.smithi171.stdout:2271 trunc from 0xe47b3ea to 0x1bde8fd 2024-09-15T00:11:53.423 INFO:teuthology.orchestra.run.smithi171.stdout:2272 read 0x1694698 thru 0x169d51d (0x8e86 bytes) 2024-09-15T00:11:53.423 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:53.423 INFO:teuthology.orchestra.run.smithi171.stdout:2274 trunc from 0x1bde8fd to 0x577f9f5 2024-09-15T00:11:53.424 INFO:teuthology.orchestra.run.smithi171.stdout:2276 write 0x3a366a thru 0x3a4c6d (0x1604 bytes) 2024-09-15T00:11:53.424 INFO:teuthology.orchestra.run.smithi171.stdout:2277 write 0x382b94c thru 0x3833643 (0x7cf8 bytes) 2024-09-15T00:11:53.426 INFO:teuthology.orchestra.run.smithi171.stdout:2279 writesame 0x885d696 thru 0x885dc64 data_size 0x1(0x5cf bytes) 2024-09-15T00:11:53.427 INFO:teuthology.orchestra.run.smithi171.stdout:2280 writesame 0x30ddae8 thru 0x30df5c6 data_size 0x1(0x1adf bytes) 2024-09-15T00:11:53.444 INFO:teuthology.orchestra.run.smithi171.stdout:2281 write 0xe7124bb thru 0xe719494 (0x6fda bytes) 2024-09-15T00:11:53.457 INFO:teuthology.orchestra.run.smithi171.stdout:2283 write 0x2f64419 thru 0x2f6a26c (0x5e54 bytes) 2024-09-15T00:11:53.467 INFO:teuthology.orchestra.run.smithi171.stdout:2284 punch from 0x7a7e1f7 to 0x7a7edc8, (0xbd1 bytes) 2024-09-15T00:11:53.467 INFO:teuthology.orchestra.run.smithi171.stdout:2285 writesame 0xc36040c thru 0xc36ab1d data_size 0x2f(0xa712 bytes) 2024-09-15T00:11:53.472 INFO:teuthology.orchestra.run.smithi171.stdout:2287 write 0xe061787 thru 0xe0659b3 (0x422d bytes) 2024-09-15T00:11:53.474 INFO:teuthology.orchestra.run.smithi171.stdout:2288 read 0xe0f72d6 thru 0xe0fb036 (0x3d61 bytes) 2024-09-15T00:11:53.474 INFO:teuthology.orchestra.run.smithi171.stdout:2289 writesame 0xc4df03 thru 0xc533d7 data_size 0x39(0x54d5 bytes) 2024-09-15T00:11:53.476 INFO:teuthology.orchestra.run.smithi171.stdout:2290 punch from 0x8f5798e to 0x8f59ad7, (0x2149 bytes) 2024-09-15T00:11:53.476 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:53.476 INFO:teuthology.orchestra.run.smithi171.stdout:2292 writesame 0x6570feb thru 0x6580ab6 data_size 0xe(0xfacc bytes) 2024-09-15T00:11:53.482 INFO:teuthology.orchestra.run.smithi171.stdout:2294 read 0x4ead713 thru 0x4eb2656 (0x4f44 bytes) 2024-09-15T00:11:53.482 INFO:teuthology.orchestra.run.smithi171.stdout:2295 read 0xb5455df thru 0xb54684c (0x126e bytes) 2024-09-15T00:11:53.482 INFO:teuthology.orchestra.run.smithi171.stdout:2296 writesame 0x176f38 thru 0x1793fb data_size 0x1a(0x24c4 bytes) 2024-09-15T00:11:53.483 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:53.483 INFO:teuthology.orchestra.run.smithi171.stdout:2300 write 0xbbe463c thru 0xbbf42a3 (0xfc68 bytes) 2024-09-15T00:11:53.489 INFO:teuthology.orchestra.run.smithi171.stdout:2302 write 0x2aab26d thru 0x2ab5192 (0x9f26 bytes) 2024-09-15T00:11:53.492 INFO:teuthology.orchestra.run.smithi171.stdout:2303 trunc from 0xe719495 to 0xd460023 2024-09-15T00:11:53.507 INFO:teuthology.orchestra.run.smithi171.stdout:2304 trunc from 0xd460023 to 0x964d1a8 2024-09-15T00:11:53.518 INFO:teuthology.orchestra.run.smithi171.stdout:2305 punch from 0x80bd85c to 0x80bff96, (0x273a bytes) 2024-09-15T00:11:53.518 INFO:teuthology.orchestra.run.smithi171.stdout:2306 read 0x77ad471 thru 0x77bb449 (0xdfd9 bytes) 2024-09-15T00:11:53.522 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:53.520+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:53.528 INFO:teuthology.orchestra.run.smithi171.stdout:2307 write 0xca79eb0 thru 0xca7d92e (0x3a7f bytes) 2024-09-15T00:11:53.543 INFO:teuthology.orchestra.run.smithi171.stdout:2308 writesame 0xcf60ed7 thru 0xcf60fa6 data_size 0x2(0xd0 bytes) 2024-09-15T00:11:53.544 INFO:teuthology.orchestra.run.smithi171.stdout:2310 read 0x8e7f019 thru 0x8e89a89 (0xaa71 bytes) 2024-09-15T00:11:53.545 INFO:teuthology.orchestra.run.smithi171.stdout:2311 read 0x3fba93a thru 0x3fbd8e2 (0x2fa9 bytes) 2024-09-15T00:11:53.545 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:53.545 INFO:teuthology.orchestra.run.smithi171.stdout:2313 write 0x751de20 thru 0x75293b1 (0xb592 bytes) 2024-09-15T00:11:53.555 INFO:teuthology.orchestra.run.smithi171.stdout:2314 write 0x6a94b58 thru 0x6a98fb3 (0x445c bytes) 2024-09-15T00:11:53.562 INFO:teuthology.orchestra.run.smithi171.stdout:2316 punch from 0x26a270b to 0x26ac2b2, (0x9ba7 bytes) 2024-09-15T00:11:53.563 INFO:teuthology.orchestra.run.smithi171.stdout:2317 read 0x9c19b64 thru 0x9c1ee73 (0x5310 bytes) 2024-09-15T00:11:53.563 INFO:teuthology.orchestra.run.smithi171.stdout:2318 writesame 0x3af0077 thru 0x3af5738 data_size 0x5(0x56c2 bytes) 2024-09-15T00:11:53.565 INFO:teuthology.orchestra.run.smithi171.stdout:2320 write 0x90a94ec thru 0x90b5e0e (0xc923 bytes) 2024-09-15T00:11:53.575 INFO:teuthology.orchestra.run.smithi171.stdout:2321 read 0x2311df1 thru 0x231b2ae (0x94be bytes) 2024-09-15T00:11:53.575 INFO:teuthology.orchestra.run.smithi171.stdout:2323 read 0x30895f3 thru 0x30907e1 (0x71ef bytes) 2024-09-15T00:11:53.582 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:53.582 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:53.582 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:53.583 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:53.583 INFO:teuthology.orchestra.run.smithi171.stdout:2324 trunc from 0xcf60fa7 to 0x7611ac9 2024-09-15T00:11:53.593 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:53.595 INFO:teuthology.orchestra.run.smithi171.stdout:2325 read 0x2448fb1 thru 0x2454e81 (0xbed1 bytes) 2024-09-15T00:11:53.596 INFO:teuthology.orchestra.run.smithi171.stdout:2327 read 0xb3184 thru 0xb658e (0x340b bytes) 2024-09-15T00:11:53.596 INFO:teuthology.orchestra.run.smithi171.stdout:2328 read 0x688966a thru 0x6897895 (0xe22c bytes) 2024-09-15T00:11:53.596 INFO:teuthology.orchestra.run.smithi171.stdout:2329 read 0x3d057b1 thru 0x3d08f34 (0x3784 bytes) 2024-09-15T00:11:53.611 INFO:teuthology.orchestra.run.smithi171.stdout:2330 write 0xe4c34c6 thru 0xe4c5eca (0x2a05 bytes) 2024-09-15T00:11:53.615 INFO:teuthology.orchestra.run.smithi171.stdout:2333 clone 17 order 20 su 4096 sc 15 2024-09-15T00:11:53.634 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:53.632+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:53.723 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:53.722+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:53.736 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:53.736 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:53.736 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:53.736 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:53.736 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:53.744 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:53.825 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:53.823+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:53.889 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:53.889 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:47986 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:11:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.4154337780000006, 2024-09-15T00:11:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045287635, 2024-09-15T00:11:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 47986 2024-09-15T00:11:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:11:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:11:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:11:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8400000000000004e-07 2024-09-15T00:11:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474928+0000", 2024-09-15T00:11:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033009430000000002 2024-09-15T00:11:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474943+0000", 2024-09-15T00:11:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5172e-05 2024-09-15T00:11:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474951+0000", 2024-09-15T00:11:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1779999999999997e-06 2024-09-15T00:11:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475058+0000", 2024-09-15T00:11:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000107595 2024-09-15T00:11:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475200+0000", 2024-09-15T00:11:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014180400000000001 2024-09-15T00:11:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.478046+0000", 2024-09-15T00:11:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028464150000000001 2024-09-15T00:11:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516790+0000", 2024-09-15T00:11:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038743278999999999 2024-09-15T00:11:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516832+0000", 2024-09-15T00:11:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1974000000000002e-05 2024-09-15T00:11:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516872+0000", 2024-09-15T00:11:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9928999999999997e-05 2024-09-15T00:11:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516914+0000", 2024-09-15T00:11:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2762000000000002e-05 2024-09-15T00:11:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48888 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13762560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.608340+0000", 2024-09-15T00:11:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2787208139999997, 2024-09-15T00:11:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043750791999999997, 2024-09-15T00:11:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48888 2024-09-15T00:11:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608340+0000", 2024-09-15T00:11:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608340+0000", 2024-09-15T00:11:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608340+0000", 2024-09-15T00:11:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3099999999999998e-07 2024-09-15T00:11:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608342+0000", 2024-09-15T00:11:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9759999999999998e-06 2024-09-15T00:11:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608349+0000", 2024-09-15T00:11:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1010000000000003e-06 2024-09-15T00:11:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608354+0000", 2024-09-15T00:11:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4669999999999998e-06 2024-09-15T00:11:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.651990+0000", 2024-09-15T00:11:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043636767999999999 2024-09-15T00:11:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.652034+0000", 2024-09-15T00:11:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4048000000000002e-05 2024-09-15T00:11:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.652090+0000", 2024-09-15T00:11:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6001000000000001e-05 2024-09-15T00:11:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48956 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 2621440~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609044+0000", 2024-09-15T00:11:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.278016826, 2024-09-15T00:11:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044014043000000003, 2024-09-15T00:11:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48956 2024-09-15T00:11:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609044+0000", 2024-09-15T00:11:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609044+0000", 2024-09-15T00:11:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609044+0000", 2024-09-15T00:11:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6100000000000001e-07 2024-09-15T00:11:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609046+0000", 2024-09-15T00:11:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2350000000000002e-06 2024-09-15T00:11:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609053+0000", 2024-09-15T00:11:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1750000000000002e-06 2024-09-15T00:11:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609057+0000", 2024-09-15T00:11:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7190000000000001e-06 2024-09-15T00:11:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.652645+0000", 2024-09-15T00:11:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043587689999999998 2024-09-15T00:11:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.652700+0000", 2024-09-15T00:11:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.507e-05 2024-09-15T00:11:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653058+0000", 2024-09-15T00:11:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035769300000000002 2024-09-15T00:11:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48957 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 4456448~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:11:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2779952659999996, 2024-09-15T00:11:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044512426000000001, 2024-09-15T00:11:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48957 2024-09-15T00:11:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:11:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:11:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609066+0000", 2024-09-15T00:11:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:11:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609068+0000", 2024-09-15T00:11:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9700000000000002e-06 2024-09-15T00:11:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609073+0000", 2024-09-15T00:11:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7939999999999999e-06 2024-09-15T00:11:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609078+0000", 2024-09-15T00:11:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3209999999999998e-06 2024-09-15T00:11:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653082+0000", 2024-09-15T00:11:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044003852000000003 2024-09-15T00:11:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653136+0000", 2024-09-15T00:11:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4024000000000003e-05 2024-09-15T00:11:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653578+0000", 2024-09-15T00:11:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044211599999999999 2024-09-15T00:11:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48958 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:11:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2779776270000003, 2024-09-15T00:11:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044724939999999998, 2024-09-15T00:11:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48958 2024-09-15T00:11:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:11:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:11:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:11:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:11:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609085+0000", 2024-09-15T00:11:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6610000000000001e-06 2024-09-15T00:11:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609090+0000", 2024-09-15T00:11:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2719999999999997e-06 2024-09-15T00:11:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609094+0000", 2024-09-15T00:11:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.907e-06 2024-09-15T00:11:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653618+0000", 2024-09-15T00:11:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044523759000000003 2024-09-15T00:11:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653703+0000", 2024-09-15T00:11:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5550000000000006e-05 2024-09-15T00:11:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653808+0000", 2024-09-15T00:11:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010445700000000001 2024-09-15T00:11:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48959 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:11:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2779616879999995, 2024-09-15T00:11:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044884741999999998, 2024-09-15T00:11:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48959 2024-09-15T00:11:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:11:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:11:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:11:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2899999999999999e-07 2024-09-15T00:11:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609101+0000", 2024-09-15T00:11:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.826e-06 2024-09-15T00:11:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609108+0000", 2024-09-15T00:11:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.165e-06 2024-09-15T00:11:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609113+0000", 2024-09-15T00:11:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7849999999999999e-06 2024-09-15T00:11:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653842+0000", 2024-09-15T00:11:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044728954000000001 2024-09-15T00:11:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653896+0000", 2024-09-15T00:11:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4593999999999999e-05 2024-09-15T00:11:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653984+0000", 2024-09-15T00:11:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7089000000000006e-05 2024-09-15T00:11:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48960 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 9961472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:11:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.277941169, 2024-09-15T00:11:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045084194000000001, 2024-09-15T00:11:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48960 2024-09-15T00:11:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:11:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:11:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609120+0000", 2024-09-15T00:11:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7099999999999997e-07 2024-09-15T00:11:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609122+0000", 2024-09-15T00:11:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:11:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609129+0000", 2024-09-15T00:11:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4159999999999998e-06 2024-09-15T00:11:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609134+0000", 2024-09-15T00:11:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7210000000000002e-06 2024-09-15T00:11:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654023+0000", 2024-09-15T00:11:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044888979000000002 2024-09-15T00:11:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654104+0000", 2024-09-15T00:11:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1108999999999999e-05 2024-09-15T00:11:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654204+0000", 2024-09-15T00:11:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9630000000000007e-05 2024-09-15T00:11:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48961 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:11:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2779203480000003, 2024-09-15T00:11:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045197821999999999, 2024-09-15T00:11:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48961 2024-09-15T00:11:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:11:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:11:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609141+0000", 2024-09-15T00:11:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:11:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609143+0000", 2024-09-15T00:11:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-15T00:11:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609150+0000", 2024-09-15T00:11:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2420000000000003e-06 2024-09-15T00:11:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609155+0000", 2024-09-15T00:11:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.028e-06 2024-09-15T00:11:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654232+0000", 2024-09-15T00:11:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045076702000000003 2024-09-15T00:11:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654281+0000", 2024-09-15T00:11:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9798e-05 2024-09-15T00:11:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654338+0000", 2024-09-15T00:11:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6498e-05 2024-09-15T00:11:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48962 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 13631488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:11:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2778992979999995, 2024-09-15T00:11:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045312458, 2024-09-15T00:11:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48962 2024-09-15T00:11:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:11:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:11:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609162+0000", 2024-09-15T00:11:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:11:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609164+0000", 2024-09-15T00:11:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0099999999999998e-06 2024-09-15T00:11:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609171+0000", 2024-09-15T00:11:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7530000000000001e-06 2024-09-15T00:11:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609178+0000", 2024-09-15T00:11:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1709999999999999e-06 2024-09-15T00:11:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654365+0000", 2024-09-15T00:11:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045187867999999999 2024-09-15T00:11:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654420+0000", 2024-09-15T00:11:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4376000000000003e-05 2024-09-15T00:11:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654474+0000", 2024-09-15T00:11:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3879999999999999e-05 2024-09-15T00:11:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48971 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:11:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2778683040000001, 2024-09-15T00:11:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045896151000000003, 2024-09-15T00:11:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48971 2024-09-15T00:11:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:11:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:11:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609193+0000", 2024-09-15T00:11:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7599999999999997e-07 2024-09-15T00:11:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609195+0000", 2024-09-15T00:11:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:11:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609210+0000", 2024-09-15T00:11:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4956999999999999e-05 2024-09-15T00:11:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609505+0000", 2024-09-15T00:11:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000295078 2024-09-15T00:11:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654498+0000", 2024-09-15T00:11:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044992277999999997 2024-09-15T00:11:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654541+0000", 2024-09-15T00:11:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3525000000000001e-05 2024-09-15T00:11:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655088+0000", 2024-09-15T00:11:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054726799999999995 2024-09-15T00:11:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48972 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 2097152~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:11:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2775417299999994, 2024-09-15T00:11:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046150885000000003, 2024-09-15T00:11:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48972 2024-09-15T00:11:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:11:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:11:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:11:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6400000000000002e-07 2024-09-15T00:11:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609523+0000", 2024-09-15T00:11:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5300000000000001e-06 2024-09-15T00:11:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609533+0000", 2024-09-15T00:11:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0023999999999999e-05 2024-09-15T00:11:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609539+0000", 2024-09-15T00:11:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3119999999999999e-06 2024-09-15T00:11:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655115+0000", 2024-09-15T00:11:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045576094999999997 2024-09-15T00:11:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655197+0000", 2024-09-15T00:11:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1129000000000002e-05 2024-09-15T00:11:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655670+0000", 2024-09-15T00:11:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00047313100000000001 2024-09-15T00:11:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:11:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2775134680000004, 2024-09-15T00:11:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:11:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:11:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:11:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:11:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:11:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:11:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:11:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:11:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:11:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:11:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:11:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:11:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:11:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2774922390000008, 2024-09-15T00:11:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:11:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:11:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:11:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:11:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:11:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:11:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:11:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:11:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:11:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:11:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:11:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:11:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:11:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:11:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:11:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:11:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2774666979999996, 2024-09-15T00:11:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:11:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:11:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:11:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:11:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:11:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:11:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:11:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:11:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:11:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:11:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:11:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:11:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:11:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:11:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:11:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:11:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:11:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:11:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2774442229999998, 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:11:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:11:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:11:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:11:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:11:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:11:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:11:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:11:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:11:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:11:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:11:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:11:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:11:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:11:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:11:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:11:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:11:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2774237129999992, 2024-09-15T00:11:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:11:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:11:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:11:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:11:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:11:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:11:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:11:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:11:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:11:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:11:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:11:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:11:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:11:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:11:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:11:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:11:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:11:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:11:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:11:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2774033980000006, 2024-09-15T00:11:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:11:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:11:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:11:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:11:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:11:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:11:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:11:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:11:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:11:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:11:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:11:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:11:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:11:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:11:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:11:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:11:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:11:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:11:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:11:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2773786220000005, 2024-09-15T00:11:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:11:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:11:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:11:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:11:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:11:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:11:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:11:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:11:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:11:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:11:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:11:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:11:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:11:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:11:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:11:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:11:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:11:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:11:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:11:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.0980372349999996, 2024-09-15T00:11:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:11:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:11:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:11:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:11:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:11:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:11:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:11:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:11:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:11:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:11:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:11:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:11:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:11:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:11:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:11:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:11:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:11:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:11:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:11:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:11:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:11:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:11:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:11:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:11:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:11:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:11:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:11:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:11:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.0810837830000004, 2024-09-15T00:11:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:11:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:11:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:11:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:11:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:11:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:11:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:11:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:11:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:11:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:11:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:11:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:11:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:11:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:11:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:11:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:11:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:11:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:53.981 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:53.982 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:11:53.983 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:53.981+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:54.084 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:54.082+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:54.086 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:54.086 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:54.086 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:54.086 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:54.095 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:11:54.147 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6] out_osds: [0, 2, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T00:11:54.147 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:11:54.148 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-15T00:11:54.148 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T00:11:54.148 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T00:11:54.148 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-15T00:11:54.154 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is set 2024-09-15T00:11:54.211 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:54.210+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:54.216 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone16 from 0xe4c5ecb (overlap 0x1bde8fd) to 0x592e99 2024-09-15T00:11:54.260 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:54.260 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:54.260 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:54.260 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:54.260 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:54.270 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:11:54.296 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-15T00:11:54.296 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:54.296 INFO:teuthology.orchestra.run.smithi184.stdout: "success": "" 2024-09-15T00:11:54.296 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:54.297 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:54.295+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:54.384 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3037 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'86, mlcod=33'82)", 2024-09-15T00:11:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.497583+0000", 2024-09-15T00:11:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.884571981000001, 2024-09-15T00:11:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077497779999999997, 2024-09-15T00:11:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497583+0000", 2024-09-15T00:11:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497583+0000", 2024-09-15T00:11:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497584+0000", 2024-09-15T00:11:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6599999999999996e-07 2024-09-15T00:11:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497588+0000", 2024-09-15T00:11:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3529999999999997e-06 2024-09-15T00:11:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497598+0000", 2024-09-15T00:11:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4839999999999993e-06 2024-09-15T00:11:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497602+0000", 2024-09-15T00:11:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:11:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497643+0000", 2024-09-15T00:11:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0326e-05 2024-09-15T00:11:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.497662+0000", 2024-09-15T00:11:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9590999999999999e-05 2024-09-15T00:11:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505303+0000", 2024-09-15T00:11:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076408919999999998 2024-09-15T00:11:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505333+0000", 2024-09-15T00:11:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0071999999999999e-05 2024-09-15T00:11:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3038 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'87, mlcod=33'82)", 2024-09-15T00:11:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.883727469, 2024-09-15T00:11:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069253919999999998, 2024-09-15T00:11:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498428+0000", 2024-09-15T00:11:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6899999999999997e-07 2024-09-15T00:11:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498435+0000", 2024-09-15T00:11:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1330000000000004e-06 2024-09-15T00:11:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498444+0000", 2024-09-15T00:11:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6749999999999997e-06 2024-09-15T00:11:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498449+0000", 2024-09-15T00:11:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5449999999999997e-06 2024-09-15T00:11:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498485+0000", 2024-09-15T00:11:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5737e-05 2024-09-15T00:11:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.498501+0000", 2024-09-15T00:11:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6637e-05 2024-09-15T00:11:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505334+0000", 2024-09-15T00:11:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0068333100000000004 2024-09-15T00:11:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505353+0000", 2024-09-15T00:11:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8785999999999999e-05 2024-09-15T00:11:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3039 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'88, mlcod=33'83)", 2024-09-15T00:11:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.882936834999999, 2024-09-15T00:11:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0061577309999999996, 2024-09-15T00:11:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499219+0000", 2024-09-15T00:11:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3099999999999997e-07 2024-09-15T00:11:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499224+0000", 2024-09-15T00:11:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.916e-06 2024-09-15T00:11:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499233+0000", 2024-09-15T00:11:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7949999999999996e-06 2024-09-15T00:11:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499239+0000", 2024-09-15T00:11:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0090000000000004e-06 2024-09-15T00:11:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499292+0000", 2024-09-15T00:11:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3049e-05 2024-09-15T00:11:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499312+0000", 2024-09-15T00:11:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0574e-05 2024-09-15T00:11:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505355+0000", 2024-09-15T00:11:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060423930000000001 2024-09-15T00:11:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505376+0000", 2024-09-15T00:11:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1364000000000001e-05 2024-09-15T00:11:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3040 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'89, mlcod=33'83)", 2024-09-15T00:11:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.499752+0000", 2024-09-15T00:11:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.882403044, 2024-09-15T00:11:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0059059300000000002, 2024-09-15T00:11:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499752+0000", 2024-09-15T00:11:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499752+0000", 2024-09-15T00:11:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499753+0000", 2024-09-15T00:11:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1399999999999997e-07 2024-09-15T00:11:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499763+0000", 2024-09-15T00:11:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0365e-05 2024-09-15T00:11:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499774+0000", 2024-09-15T00:11:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0920999999999999e-05 2024-09-15T00:11:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499780+0000", 2024-09-15T00:11:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.02e-06 2024-09-15T00:11:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499829+0000", 2024-09-15T00:11:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9066000000000001e-05 2024-09-15T00:11:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.499849+0000", 2024-09-15T00:11:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9596999999999999e-05 2024-09-15T00:11:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505633+0000", 2024-09-15T00:11:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057841189999999999 2024-09-15T00:11:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.505658+0000", 2024-09-15T00:11:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5228e-05 2024-09-15T00:11:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3102 3.a 3:566166ea:::rbd_data.1139b531b717.000000000000000b:head [stat,truncate 385024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.812295642999999, 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060108410000000003, 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3102 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569860+0000", 2024-09-15T00:11:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:11:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569863+0000", 2024-09-15T00:11:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.819e-06 2024-09-15T00:11:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569871+0000", 2024-09-15T00:11:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6690000000000006e-06 2024-09-15T00:11:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.569882+0000", 2024-09-15T00:11:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1195999999999999e-05 2024-09-15T00:11:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.570647+0000", 2024-09-15T00:11:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00076547900000000001 2024-09-15T00:11:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.570734+0000", 2024-09-15T00:11:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7087000000000002e-05 2024-09-15T00:11:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.570863+0000", 2024-09-15T00:11:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012899799999999999 2024-09-15T00:11:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.573483+0000", 2024-09-15T00:11:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026192799999999999 2024-09-15T00:11:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.575796+0000", 2024-09-15T00:11:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0023136570000000002 2024-09-15T00:11:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.575827+0000", 2024-09-15T00:11:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0468e-05 2024-09-15T00:11:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.575871+0000", 2024-09-15T00:11:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3656999999999999e-05 2024-09-15T00:11:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3502 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'181, mlcod=33'177)", 2024-09-15T00:11:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.433840477, 2024-09-15T00:11:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0067634649999999998, 2024-09-15T00:11:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948315+0000", 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5199999999999997e-07 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948319+0000", 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0740000000000003e-06 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948329+0000", 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1409999999999994e-06 2024-09-15T00:11:54.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948335+0000", 2024-09-15T00:11:54.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6290000000000004e-06 2024-09-15T00:11:54.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948398+0000", 2024-09-15T00:11:54.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2422999999999995e-05 2024-09-15T00:11:54.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948412+0000", 2024-09-15T00:11:54.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4538999999999999e-05 2024-09-15T00:11:54.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.954988+0000", 2024-09-15T00:11:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.006575455 2024-09-15T00:11:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955078+0000", 2024-09-15T00:11:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0651999999999999e-05 2024-09-15T00:11:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3503 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'182, mlcod=33'177)", 2024-09-15T00:11:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.948857+0000", 2024-09-15T00:11:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.433297882000002, 2024-09-15T00:11:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0062792079999999997, 2024-09-15T00:11:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948857+0000", 2024-09-15T00:11:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948857+0000", 2024-09-15T00:11:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948858+0000", 2024-09-15T00:11:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6899999999999997e-07 2024-09-15T00:11:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948862+0000", 2024-09-15T00:11:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2429999999999999e-06 2024-09-15T00:11:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948872+0000", 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5340000000000005e-06 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948883+0000", 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1304999999999999e-05 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948917+0000", 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3510999999999998e-05 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.948929+0000", 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2473000000000001e-05 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955082+0000", 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0061525010000000003 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955137+0000", 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5071999999999997e-05 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:3504 3.1e e33/23 3:792e48de:::rbd_object_map.1139b531b717:head v 33'183, mlcod=33'178)", 2024-09-15T00:11:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.432542505000001, 2024-09-15T00:11:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058979369999999998, 2024-09-15T00:11:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949613+0000", 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8400000000000004e-07 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949620+0000", 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9739999999999996e-06 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949630+0000", 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6399999999999992e-06 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949635+0000", 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949688+0000", 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2667000000000002e-05 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.949701+0000", 2024-09-15T00:11:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.36e-05 2024-09-15T00:11:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955477+0000", 2024-09-15T00:11:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057763119999999996 2024-09-15T00:11:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:29.955511+0000", 2024-09-15T00:11:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3337000000000003e-05 2024-09-15T00:11:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:3737 3.a 3:53de29a7:::rbd_data.1139b531b717.000000000000000a:head [stat,truncate 270336] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:31.100151+0000", 2024-09-15T00:11:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.282004359999998, 2024-09-15T00:11:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076797300000000001, 2024-09-15T00:11:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3737 2024-09-15T00:11:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100151+0000", 2024-09-15T00:11:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100151+0000", 2024-09-15T00:11:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100153+0000", 2024-09-15T00:11:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6530000000000001e-06 2024-09-15T00:11:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100158+0000", 2024-09-15T00:11:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7100000000000004e-06 2024-09-15T00:11:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100173+0000", 2024-09-15T00:11:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4242e-05 2024-09-15T00:11:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100180+0000", 2024-09-15T00:11:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7680000000000008e-06 2024-09-15T00:11:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100248+0000", 2024-09-15T00:11:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7459000000000006e-05 2024-09-15T00:11:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100321+0000", 2024-09-15T00:11:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2854000000000004e-05 2024-09-15T00:11:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.100418+0000", 2024-09-15T00:11:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7827999999999999e-05 2024-09-15T00:11:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.105090+0000", 2024-09-15T00:11:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0046714349999999998 2024-09-15T00:11:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.107770+0000", 2024-09-15T00:11:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026799660000000002 2024-09-15T00:11:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.107801+0000", 2024-09-15T00:11:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1158000000000002e-05 2024-09-15T00:11:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:31.107831+0000", 2024-09-15T00:11:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9657000000000001e-05 2024-09-15T00:11:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:18294 3.3 e38/28 3:c1f28201:::rbd_data.11398e1a091a.0000000000000000:head v 38'44, mlcod=37'42)", 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.296284+0000", 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.085871097999998, 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013754903000000001, 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296284+0000", 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296284+0000", 2024-09-15T00:11:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296285+0000", 2024-09-15T00:11:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7300000000000005e-07 2024-09-15T00:11:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296638+0000", 2024-09-15T00:11:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035318399999999998 2024-09-15T00:11:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296653+0000", 2024-09-15T00:11:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.468e-05 2024-09-15T00:11:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296661+0000", 2024-09-15T00:11:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8460000000000008e-06 2024-09-15T00:11:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296724+0000", 2024-09-15T00:11:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2947999999999994e-05 2024-09-15T00:11:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.296744+0000", 2024-09-15T00:11:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0525000000000001e-05 2024-09-15T00:11:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.309995+0000", 2024-09-15T00:11:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013251284 2024-09-15T00:11:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.310039+0000", 2024-09-15T00:11:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3662999999999999e-05 2024-09-15T00:11:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:19308 3.10 3:0ee1524d:::rbd_data.11398e1a091a.0000000000000017:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-15T00:11:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:11:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.396974631000001, 2024-09-15T00:11:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02547147, 2024-09-15T00:11:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 19308 2024-09-15T00:11:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:11:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:11:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985182+0000", 2024-09-15T00:11:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6500000000000001e-06 2024-09-15T00:11:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985189+0000", 2024-09-15T00:11:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9949999999999999e-06 2024-09-15T00:11:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985205+0000", 2024-09-15T00:11:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6055000000000001e-05 2024-09-15T00:11:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985212+0000", 2024-09-15T00:11:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:11:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985261+0000", 2024-09-15T00:11:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8380000000000001e-05 2024-09-15T00:11:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985311+0000", 2024-09-15T00:11:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0763999999999997e-05 2024-09-15T00:11:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.986489+0000", 2024-09-15T00:11:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0011773420000000001 2024-09-15T00:11:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010514+0000", 2024-09-15T00:11:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024025392999999999 2024-09-15T00:11:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010535+0000", 2024-09-15T00:11:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0652000000000001e-05 2024-09-15T00:11:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010597+0000", 2024-09-15T00:11:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2411999999999998e-05 2024-09-15T00:11:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010652+0000", 2024-09-15T00:11:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5074999999999997e-05 2024-09-15T00:11:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:19308 3.10 e41/28 ondisk, result = 0)", 2024-09-15T00:11:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:38.990011+0000", 2024-09-15T00:11:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.392144786999999, 2024-09-15T00:11:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.020513610000000002, 2024-09-15T00:11:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.990011+0000", 2024-09-15T00:11:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.990011+0000", 2024-09-15T00:11:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.990011+0000", 2024-09-15T00:11:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6099999999999999e-07 2024-09-15T00:11:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.990015+0000", 2024-09-15T00:11:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.596e-06 2024-09-15T00:11:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.990026+0000", 2024-09-15T00:11:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1456e-05 2024-09-15T00:11:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.990034+0000", 2024-09-15T00:11:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6389999999999992e-06 2024-09-15T00:11:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010494+0000", 2024-09-15T00:11:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.020460177 2024-09-15T00:11:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010505+0000", 2024-09-15T00:11:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0944000000000001e-05 2024-09-15T00:11:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010524+0000", 2024-09-15T00:11:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8936999999999998e-05 2024-09-15T00:11:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20008 3.10 3:0e764489:::rbd_data.113917258088.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 2031616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:11:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.614630+0000", 2024-09-15T00:11:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.767525253000001, 2024-09-15T00:11:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0084570530000000008, 2024-09-15T00:11:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20008 2024-09-15T00:11:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.614630+0000", 2024-09-15T00:11:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.614630+0000", 2024-09-15T00:11:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.614631+0000", 2024-09-15T00:11:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0559999999999999e-06 2024-09-15T00:11:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.614636+0000", 2024-09-15T00:11:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8860000000000003e-06 2024-09-15T00:11:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.614649+0000", 2024-09-15T00:11:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2476999999999999e-05 2024-09-15T00:11:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.614655+0000", 2024-09-15T00:11:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8390000000000001e-06 2024-09-15T00:11:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.614719+0000", 2024-09-15T00:11:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3410000000000004e-05 2024-09-15T00:11:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.614850+0000", 2024-09-15T00:11:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013150099999999999 2024-09-15T00:11:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.615039+0000", 2024-09-15T00:11:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000188343 2024-09-15T00:11:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.615476+0000", 2024-09-15T00:11:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043787399999999999 2024-09-15T00:11:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.622990+0000", 2024-09-15T00:11:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0075133509999999997 2024-09-15T00:11:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.623045+0000", 2024-09-15T00:11:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5549e-05 2024-09-15T00:11:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.623087+0000", 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1767000000000002e-05 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20397 3.a 3:53738ef5:::rbd_data.113917258088.000000000000001d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:41.190264+0000", 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.191891269999999, 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077362220000000001, 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.447 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20397 2024-09-15T00:11:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.190264+0000", 2024-09-15T00:11:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.190264+0000", 2024-09-15T00:11:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.190265+0000", 2024-09-15T00:11:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2999999999999999e-07 2024-09-15T00:11:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.190270+0000", 2024-09-15T00:11:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5419999999999999e-06 2024-09-15T00:11:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.190281+0000", 2024-09-15T00:11:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119e-05 2024-09-15T00:11:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.190286+0000", 2024-09-15T00:11:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1109999999999999e-06 2024-09-15T00:11:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.190340+0000", 2024-09-15T00:11:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3984999999999999e-05 2024-09-15T00:11:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.190400+0000", 2024-09-15T00:11:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0429e-05 2024-09-15T00:11:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.190507+0000", 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000106672 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.195222+0000", 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047154550000000003 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.197945+0000", 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0027222610000000001 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.197975+0000", 2024-09-15T00:11:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9947e-05 2024-09-15T00:11:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.198000+0000", 2024-09-15T00:11:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5771000000000001e-05 2024-09-15T00:11:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20408 3.a 3:507e50ad:::rbd_data.113917258088.0000000000000068:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T00:11:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:41.200537+0000", 2024-09-15T00:11:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.181618456000001, 2024-09-15T00:11:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060821410000000001, 2024-09-15T00:11:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20408 2024-09-15T00:11:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.200537+0000", 2024-09-15T00:11:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.200537+0000", 2024-09-15T00:11:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.200538+0000", 2024-09-15T00:11:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.127e-06 2024-09-15T00:11:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.200543+0000", 2024-09-15T00:11:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9830000000000004e-06 2024-09-15T00:11:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.200557+0000", 2024-09-15T00:11:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3831e-05 2024-09-15T00:11:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.200564+0000", 2024-09-15T00:11:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.204e-06 2024-09-15T00:11:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.200620+0000", 2024-09-15T00:11:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6131000000000002e-05 2024-09-15T00:11:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.200689+0000", 2024-09-15T00:11:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8862999999999994e-05 2024-09-15T00:11:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.200848+0000", 2024-09-15T00:11:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015943700000000001 2024-09-15T00:11:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.203509+0000", 2024-09-15T00:11:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026603629999999998 2024-09-15T00:11:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.206532+0000", 2024-09-15T00:11:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030235240000000001 2024-09-15T00:11:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.206582+0000", 2024-09-15T00:11:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9345999999999998e-05 2024-09-15T00:11:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:41.206619+0000", 2024-09-15T00:11:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7332000000000001e-05 2024-09-15T00:11:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:48194 3.3 e47/28 3:c56220aa:::rbd_data.113916cf513.0000000000000009:head v 47'98, mlcod=46'95)", 2024-09-15T00:11:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.581757+0000", 2024-09-15T00:11:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.8003981449999991, 2024-09-15T00:11:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017430109999999999, 2024-09-15T00:11:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.581757+0000", 2024-09-15T00:11:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.581757+0000", 2024-09-15T00:11:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.581759+0000", 2024-09-15T00:11:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.375e-06 2024-09-15T00:11:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.583453+0000", 2024-09-15T00:11:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0016939990000000001 2024-09-15T00:11:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.583472+0000", 2024-09-15T00:11:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9283000000000002e-05 2024-09-15T00:11:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.583482+0000", 2024-09-15T00:11:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9890000000000004e-06 2024-09-15T00:11:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.583548+0000", 2024-09-15T00:11:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6328000000000006e-05 2024-09-15T00:11:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.583567+0000", 2024-09-15T00:11:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8901000000000001e-05 2024-09-15T00:11:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.599142+0000", 2024-09-15T00:11:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01557473 2024-09-15T00:11:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.599187+0000", 2024-09-15T00:11:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5504999999999999e-05 2024-09-15T00:11:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:11:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.7147976630000006, 2024-09-15T00:11:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:11:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:11:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:11:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:11:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:11:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:11:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:11:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:11:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:11:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:11:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:11:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:11:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:11:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:11:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:11:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:11:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:11:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:11:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:11:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:11:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:11:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:11:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:49038 3.3 e47/28 3:c57bf448:::rbd_data.113916cf513.000000000000000d:head v 47'100, mlcod=47'98)", 2024-09-15T00:11:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.705049+0000", 2024-09-15T00:11:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.6771065590000003, 2024-09-15T00:11:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015452862, 2024-09-15T00:11:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705049+0000", 2024-09-15T00:11:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705049+0000", 2024-09-15T00:11:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705050+0000", 2024-09-15T00:11:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.513e-06 2024-09-15T00:11:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705988+0000", 2024-09-15T00:11:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00093736299999999998 2024-09-15T00:11:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705997+0000", 2024-09-15T00:11:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0489999999999993e-06 2024-09-15T00:11:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.706008+0000", 2024-09-15T00:11:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1127999999999999e-05 2024-09-15T00:11:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.706056+0000", 2024-09-15T00:11:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8300000000000002e-05 2024-09-15T00:11:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.706074+0000", 2024-09-15T00:11:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7476999999999999e-05 2024-09-15T00:11:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.720467+0000", 2024-09-15T00:11:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014392894999999999 2024-09-15T00:11:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.720502+0000", 2024-09-15T00:11:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5136999999999999e-05 2024-09-15T00:11:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:59259 3.10 3:093d1973:::rbd_data.1139766160da.0000000000000007:head [stat,delete] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.217533+0000", 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.164622765, 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0054624360000000002, 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 59259 2024-09-15T00:11:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217533+0000", 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217533+0000", 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217535+0000", 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1019999999999999e-06 2024-09-15T00:11:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217543+0000", 2024-09-15T00:11:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2009999999999999e-06 2024-09-15T00:11:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217563+0000", 2024-09-15T00:11:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9678999999999999e-05 2024-09-15T00:11:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217573+0000", 2024-09-15T00:11:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0566e-05 2024-09-15T00:11:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217631+0000", 2024-09-15T00:11:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8053999999999997e-05 2024-09-15T00:11:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217711+0000", 2024-09-15T00:11:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9485000000000002e-05 2024-09-15T00:11:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.218686+0000", 2024-09-15T00:11:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00097544400000000001 2024-09-15T00:11:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.222169+0000", 2024-09-15T00:11:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034827019999999998 2024-09-15T00:11:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.222889+0000", 2024-09-15T00:11:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00072048200000000005 2024-09-15T00:11:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.222948+0000", 2024-09-15T00:11:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.222995+0000", 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.71e-05 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:59261 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:head [stat,delete] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.217597+0000", 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.16455846800000001, 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0056219119999999997, 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 59261 2024-09-15T00:11:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217597+0000", 2024-09-15T00:11:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217597+0000", 2024-09-15T00:11:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217597+0000", 2024-09-15T00:11:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2399999999999998e-07 2024-09-15T00:11:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217601+0000", 2024-09-15T00:11:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.416e-06 2024-09-15T00:11:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217609+0000", 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9950000000000005e-06 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217615+0000", 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9240000000000004e-06 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217668+0000", 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2828999999999999e-05 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217756+0000", 2024-09-15T00:11:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7941000000000004e-05 2024-09-15T00:11:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.217958+0000", 2024-09-15T00:11:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020202499999999999 2024-09-15T00:11:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.219171+0000", 2024-09-15T00:11:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0012134170000000001 2024-09-15T00:11:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.223117+0000", 2024-09-15T00:11:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039455580000000001 2024-09-15T00:11:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.223170+0000", 2024-09-15T00:11:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3128999999999999e-05 2024-09-15T00:11:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.223219+0000", 2024-09-15T00:11:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9054000000000002e-05 2024-09-15T00:11:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.486 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:54.487 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:11:54.487 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:54.485+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:54.495 INFO:teuthology.orchestra.run.smithi171.stdout:2336 write 0xa9fadde thru 0xaa05950 (0xab73 bytes) 2024-09-15T00:11:54.524 INFO:teuthology.orchestra.run.smithi171.stdout:2337 trunc from 0xe4c5ecb to 0x42db5e8 2024-09-15T00:11:54.588 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:54.586+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:54.611 INFO:teuthology.orchestra.run.smithi171.stdout:2338 trunc from 0x42db5e8 to 0xbcd1f19 2024-09-15T00:11:54.617 INFO:teuthology.orchestra.run.smithi171.stdout:2339 punch from 0x35e2269 to 0x35ebbf3, (0x998a bytes) 2024-09-15T00:11:54.617 INFO:teuthology.orchestra.run.smithi171.stdout:2341 trunc from 0xbcd1f19 to 0xbea0ced 2024-09-15T00:11:54.621 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:54.621 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:54.621 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:54.621 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:54.622 INFO:teuthology.orchestra.run.smithi171.stdout:2342 punch from 0xbc9da86 to 0xbca6b39, (0x90b3 bytes) 2024-09-15T00:11:54.622 INFO:teuthology.orchestra.run.smithi171.stdout:2343 write 0x61a63ab thru 0x61b295b (0xc5b1 bytes) 2024-09-15T00:11:54.631 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:11:54.637 INFO:teuthology.orchestra.run.smithi171.stdout:2344 write 0x30754cc thru 0x3080187 (0xacbc bytes) 2024-09-15T00:11:54.689 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:54.687+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:54.745 INFO:teuthology.orchestra.run.smithi171.stdout:2348 punch from 0xaf6679b to 0xaf6a843, (0x40a8 bytes) 2024-09-15T00:11:54.745 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:54.745 INFO:teuthology.orchestra.run.smithi171.stdout:2352 punch from 0xbfe980 to 0xc05049, (0x66c9 bytes) 2024-09-15T00:11:54.745 INFO:teuthology.orchestra.run.smithi171.stdout:2353 writesame 0x8faf236 thru 0x8fbe3c1 data_size 0x6(0xf18c bytes) 2024-09-15T00:11:54.773 INFO:teuthology.orchestra.run.smithi171.stdout:2354 trunc from 0xbea0ced to 0x32e1352 2024-09-15T00:11:54.789 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:54.788+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:54.817 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:54.817 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:54.817 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:54.817 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:54.818 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:54.826 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:11:54.842 INFO:teuthology.orchestra.run.smithi171.stdout:2355 writesame 0xee66202 thru 0xee6b27f data_size 0x1(0x507e bytes) 2024-09-15T00:11:54.842 INFO:teuthology.orchestra.run.smithi171.stdout:2356 punch from 0x13271cd to 0x132e2e0, (0x7113 bytes) 2024-09-15T00:11:54.842 INFO:teuthology.orchestra.run.smithi171.stdout:2359 punch from 0x9ceb570 to 0x9cec324, (0xdb4 bytes) 2024-09-15T00:11:54.842 INFO:teuthology.orchestra.run.smithi171.stdout:2360 read 0x6eed8aa thru 0x6ef1354 (0x3aab bytes) 2024-09-15T00:11:54.858 INFO:teuthology.orchestra.run.smithi171.stdout:2361 trunc from 0xee6b280 to 0x150e66f 2024-09-15T00:11:54.890 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:54.888+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:54.902 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:11:54.975 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:54.976 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:54.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:54.976 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:54.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.976 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:54.976 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:11:54.976 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.894367635000002, 2024-09-15T00:11:54.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:11:54.976 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.976 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.976 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.976 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:54.976 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:54.977 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:54.977 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:11:54.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.977 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:11:54.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:11:54.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:11:54.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:11:54.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:11:54.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:11:54.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:11:54.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:11:54.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:11:54.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:11:54.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:11:54.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:11:54.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:11:54.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:11:54.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:54.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:11:54.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:11:54.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:54.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:11:54.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:11:54.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:54.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:11:54.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:11:54.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:11:54.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:11:54.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:11:54.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:11:54.981 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.982 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.982 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.982 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:11:54.982 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:11:54.982 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.881627092999999, 2024-09-15T00:11:54.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:11:54.982 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.982 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.982 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:54.982 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:11:54.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:11:54.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:11:54.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:11:54.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:11:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:11:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:11:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:11:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:11:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:11:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:11:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:11:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:11:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:11:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:11:54.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:11:54.986 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.986 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.986 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.986 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:54.986 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:11:54.986 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.749271333999999, 2024-09-15T00:11:54.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:11:54.986 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.986 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:11:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:11:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:11:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:11:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:11:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:11:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:11:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:11:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:11:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:11:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:11:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:11:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:11:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:11:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:11:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:11:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:11:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:11:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:11:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:11:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:11:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:11:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:11:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:11:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:11:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:11:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:11:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.735809689, 2024-09-15T00:11:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:11:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:11:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:11:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:11:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:11:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:11:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:11:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:11:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:11:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:11:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:11:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:11:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:11:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:11:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:11:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:11:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:11:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:11:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.550808205999999, 2024-09-15T00:11:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:11:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:11:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:11:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:11:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:11:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:11:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:11:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:11:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:11:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:11:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:11:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:11:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:11:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:11:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:11:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:11:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:11:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:11:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:11:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:11:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:11:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:11:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:11:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:11:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:11:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:11:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:11:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:11:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.537931939, 2024-09-15T00:11:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:11:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:11:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:11:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:11:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:11:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:11:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:11:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:11:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:11:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:11:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:11:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:11:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:11:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:11:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:11:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:11:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:11:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:11:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:11:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.079416795, 2024-09-15T00:11:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:11:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:11:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:11:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:11:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:11:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:11:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:11:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:11:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:11:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:11:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:11:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:11:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:11:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:11:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:11:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:11:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:11:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:11:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:11:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:11:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:11:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:11:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:11:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:11:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:11:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:11:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:11:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.361906234999999, 2024-09-15T00:11:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:11:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:11:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:11:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:11:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:11:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:11:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:11:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:11:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:11:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:11:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:11:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:11:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:11:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:11:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:11:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:11:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:11:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:11:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:11:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.361887626, 2024-09-15T00:11:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:11:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:11:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:11:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:11:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:11:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:11:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:11:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:11:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:11:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:11:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:11:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:11:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:11:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:11:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:11:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:11:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:11:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:11:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:11:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.361862302, 2024-09-15T00:11:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:11:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:11:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:11:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:11:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:11:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:11:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:11:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:11:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:11:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:11:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:11:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:11:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:11:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:11:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:11:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:11:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:11:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:11:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:11:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.361843978, 2024-09-15T00:11:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:11:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:11:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:11:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:11:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:11:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:11:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:11:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:11:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:11:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:11:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:11:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:11:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:11:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:11:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:11:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:11:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:11:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:11:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:11:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.361661405, 2024-09-15T00:11:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:11:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:11:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:11:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:11:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:11:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:11:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:11:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:11:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:55.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:11:55.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:11:55.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:11:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:11:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:11:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:11:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:11:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:11:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:11:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:11:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:11:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.361632564000001, 2024-09-15T00:11:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:11:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:11:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:11:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:11:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:11:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:11:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:11:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:11:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:11:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:11:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:11:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:11:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:11:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:11:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:11:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:11:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:11:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:11:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:11:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.361610353, 2024-09-15T00:11:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:11:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:11:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:11:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:11:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:11:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:11:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:11:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:11:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:11:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:11:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:11:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:11:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:11:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:11:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:11:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:11:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:11:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:11:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:11:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.361589194, 2024-09-15T00:11:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:11:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:11:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:11:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:11:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:11:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:11:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:11:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:11:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:11:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:11:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:11:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:11:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:11:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:11:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:11:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:11:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:11:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:11:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:11:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.361559246000001, 2024-09-15T00:11:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:11:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:11:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:11:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:11:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:11:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:11:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:11:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:11:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:11:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:11:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:11:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:11:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:11:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:11:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:11:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:11:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:11:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:11:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.054 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.054 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:55.054 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:11:55.054 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.361537282, 2024-09-15T00:11:55.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:11:55.054 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:55.054 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:55.054 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:55.055 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:55.055 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:55.055 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:55.055 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:11:55.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.055 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:55.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:55.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:11:55.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:55.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:11:55.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:55.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:11:55.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:11:55.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:55.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:11:55.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:11:55.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:55.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:11:55.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:11:55.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:55.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:11:55.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:11:55.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:11:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:11:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:11:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:11:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:11:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:11:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:11:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.361519854999999, 2024-09-15T00:11:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:11:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:11:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:11:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:11:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:11:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:11:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:11:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:11:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:11:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:11:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:11:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:11:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:11:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:11:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:11:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:11:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:11:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:11:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:11:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.361498056, 2024-09-15T00:11:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:11:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:11:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:11:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:11:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:11:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:11:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:11:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:11:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:11:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:11:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:11:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:11:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:11:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:11:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:11:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:11:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:11:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:11:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:11:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.312269697, 2024-09-15T00:11:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:11:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:11:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:11:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:11:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:11:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:11:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:11:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:11:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:11:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:11:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:11:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:11:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:11:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:11:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:11:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:11:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:11:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:11:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:11:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:11:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:11:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:11:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:11:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:11:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:11:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:11:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:55.075 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:55.076 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:55.077 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:55.077 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:55.075+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:55.078 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:55.085 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:55.085 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:55.104 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:55.105 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:55.110 INFO:teuthology.orchestra.run.smithi171.stdout:2362 write 0x746fb38 thru 0x7478e20 (0x92e9 bytes) 2024-09-15T00:11:55.124 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:55.125 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:55.125 INFO:teuthology.orchestra.run.smithi171.stdout:2365 punch from 0x524e78c to 0x5256c64, (0x84d8 bytes) 2024-09-15T00:11:55.125 INFO:teuthology.orchestra.run.smithi171.stdout:2369 read 0x23358a3 thru 0x2339938 (0x4096 bytes) 2024-09-15T00:11:55.131 INFO:teuthology.orchestra.run.smithi171.stdout:2371 writesame 0x7b77501 thru 0x7b7f335 data_size 0x1(0x7e35 bytes) 2024-09-15T00:11:55.141 INFO:teuthology.orchestra.run.smithi171.stdout:2374 writesame 0xa57d86e thru 0xa5875da data_size 0x1(0x9d6d bytes) 2024-09-15T00:11:55.141 INFO:teuthology.orchestra.run.smithi171.stdout:2375 write 0x8ab121f thru 0x8aba7a7 (0x9589 bytes) 2024-09-15T00:11:55.142 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:55.142 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:55.157 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:55.157 INFO:teuthology.orchestra.run.smithi171.stdout:2378 write 0x4c23854 thru 0x4c2db7a (0xa327 bytes) 2024-09-15T00:11:55.163 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:55.164 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:55.173 INFO:teuthology.orchestra.run.smithi171.stdout:2379 read 0x4693c71 thru 0x4694fb0 (0x1340 bytes) 2024-09-15T00:11:55.173 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:55.177 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:55.176+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:55.182 INFO:teuthology.orchestra.run.smithi171.stdout:2383 write 0xc85b5f6 thru 0xc8630a9 (0x7ab4 bytes) 2024-09-15T00:11:55.183 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:55.183 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:55.201 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:55.202 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:55.202 INFO:teuthology.orchestra.run.smithi171.stdout:2384 write 0xd50313a thru 0xd50e17f (0xb046 bytes) 2024-09-15T00:11:55.207 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:55.207 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:55.217 INFO:teuthology.orchestra.run.smithi171.stdout:2385 read 0x93399f2 thru 0x934877f (0xed8e bytes) 2024-09-15T00:11:55.217 INFO:teuthology.orchestra.run.smithi171.stdout:2386 read 0x34f4bd2 thru 0x34fc85d (0x7c8c bytes) 2024-09-15T00:11:55.217 INFO:teuthology.orchestra.run.smithi171.stdout:2388 write 0xacc9134 thru 0xaccedb6 (0x5c83 bytes) 2024-09-15T00:11:55.219 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:11:55.224 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:55.224 INFO:teuthology.orchestra.run.smithi171.stdout:2391 read 0x249df5d thru 0x24ad9ea (0xfa8e bytes) 2024-09-15T00:11:55.224 INFO:teuthology.orchestra.run.smithi171.stdout:2393 writesame 0xa213c50 thru 0xa21ec0a data_size 0x1(0xafbb bytes) 2024-09-15T00:11:55.240 INFO:teuthology.orchestra.run.smithi171.stdout:2394 write 0x17f6800 thru 0x1802283 (0xba84 bytes) 2024-09-15T00:11:55.246 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:55.246 INFO:teuthology.orchestra.run.smithi171.stdout:2397 punch from 0x2bc934d to 0x2bcb55f, (0x2212 bytes) 2024-09-15T00:11:55.246 INFO:teuthology.orchestra.run.smithi171.stdout:2399 writesame 0x911de13 thru 0x91249a4 data_size 0x31(0x6b92 bytes) 2024-09-15T00:11:55.252 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:55.253 INFO:teuthology.orchestra.run.smithi171.stdout:2403 trunc from 0xd50e180 to 0xc12e660 2024-09-15T00:11:55.266 INFO:teuthology.orchestra.run.smithi171.stdout:2404 write 0x2f7566b thru 0x2f80b42 (0xb4d8 bytes) 2024-09-15T00:11:55.279 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:55.277+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:55.280 INFO:teuthology.orchestra.run.smithi171.stdout:2405 trunc from 0xc12e660 to 0xd4e54c0 2024-09-15T00:11:55.299 INFO:teuthology.orchestra.run.smithi171.stdout:2406 trunc from 0xd4e54c0 to 0x2aac28e 2024-09-15T00:11:55.336 INFO:teuthology.orchestra.run.smithi171.stdout:2407 writesame 0x7ce41a7 thru 0x7cf185f data_size 0x1(0xd6b9 bytes) 2024-09-15T00:11:55.336 INFO:teuthology.orchestra.run.smithi171.stdout:2408 write 0x2422718 thru 0x2422845 (0x12e bytes) 2024-09-15T00:11:55.348 INFO:teuthology.orchestra.run.smithi171.stdout:2409 write 0x94ca972 thru 0x94cc519 (0x1ba8 bytes) 2024-09-15T00:11:55.354 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:55.354 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:55.367 INFO:teuthology.orchestra.run.smithi171.stdout:2410 trunc from 0x94cc51a to 0x9e2bb0 2024-09-15T00:11:55.374 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:11:55.379 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:55.378+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:55.480 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:55.479+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:55.578 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:55.578 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:55.581 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:55.580+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:55.590 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:55.656 INFO:teuthology.orchestra.run.smithi171.stdout:2411 write 0xc4dcc9a thru 0xc4dec43 (0x1faa bytes) 2024-09-15T00:11:55.665 INFO:teuthology.orchestra.run.smithi171.stdout:2413 write 0x893067b thru 0x893d80e (0xd194 bytes) 2024-09-15T00:11:55.682 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:55.681+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:55.683 INFO:teuthology.orchestra.run.smithi171.stdout:2414 trunc from 0xc4dec44 to 0x91c3077 2024-09-15T00:11:55.698 INFO:teuthology.orchestra.run.smithi171.stdout:2416 punch from 0x5473cb5 to 0x5482042, (0xe38d bytes) 2024-09-15T00:11:55.698 INFO:teuthology.orchestra.run.smithi171.stdout:2417 write 0x3515797 thru 0x35194d5 (0x3d3f bytes) 2024-09-15T00:11:55.705 INFO:teuthology.orchestra.run.smithi171.stdout:2418 writesame 0x9110938 thru 0x9119818 data_size 0x1(0x8ee1 bytes) 2024-09-15T00:11:55.713 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:55.713 INFO:teuthology.orchestra.run.smithi171.stdout:2421 write 0x42f689b thru 0x42ffda2 (0x9508 bytes) 2024-09-15T00:11:55.736 INFO:teuthology.orchestra.run.smithi171.stdout:2423 writesame 0xaaaf842 thru 0xaab16de data_size 0x1(0x1e9d bytes) 2024-09-15T00:11:55.736 INFO:teuthology.orchestra.run.smithi171.stdout:2424 write 0x4661906 thru 0x466a14a (0x8845 bytes) 2024-09-15T00:11:55.750 INFO:teuthology.orchestra.run.smithi171.stdout:2425 write 0xbfb9646 thru 0xbfbf373 (0x5d2e bytes) 2024-09-15T00:11:55.754 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:55.754 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:55.761 INFO:teuthology.orchestra.run.smithi171.stdout:2427 write 0xac950a7 thru 0xac9558a (0x4e4 bytes) 2024-09-15T00:11:55.763 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:55.766 INFO:teuthology.orchestra.run.smithi171.stdout:2428 trunc from 0xbfbf374 to 0xde8b3f4 2024-09-15T00:11:55.773 INFO:teuthology.orchestra.run.smithi171.stdout:2429 read 0x9913352 thru 0x99169e6 (0x3695 bytes) 2024-09-15T00:11:55.773 INFO:teuthology.orchestra.run.smithi171.stdout:2430 trunc from 0xde8b3f4 to 0xe7b26e2 2024-09-15T00:11:55.782 INFO:teuthology.orchestra.run.smithi171.stdout:2431 trunc from 0xe7b26e2 to 0xc3caea9 2024-09-15T00:11:55.783 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:55.781+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:55.787 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:55.788 INFO:teuthology.orchestra.run.smithi171.stdout:2433 trunc from 0xc3caea9 to 0xbdced6e 2024-09-15T00:11:55.798 INFO:teuthology.orchestra.run.smithi171.stdout:2434 write 0xb8016db thru 0xb81155f (0xfe85 bytes) 2024-09-15T00:11:55.810 INFO:teuthology.orchestra.run.smithi171.stdout:2435 read 0x556ca92 thru 0x5573395 (0x6904 bytes) 2024-09-15T00:11:55.810 INFO:teuthology.orchestra.run.smithi171.stdout:2437 write 0x5e73970 thru 0x5e777eb (0x3e7c bytes) 2024-09-15T00:11:55.821 INFO:teuthology.orchestra.run.smithi171.stdout:2438 write 0x21f057c thru 0x21ff015 (0xea9a bytes) 2024-09-15T00:11:55.836 INFO:teuthology.orchestra.run.smithi171.stdout:2439 writesame 0x6f905ad thru 0x6f9364b data_size 0x9(0x309f bytes) 2024-09-15T00:11:55.842 INFO:teuthology.orchestra.run.smithi171.stdout:2441 write 0x385a34f thru 0x3861b07 (0x77b9 bytes) 2024-09-15T00:11:55.846 INFO:teuthology.orchestra.run.smithi171.stdout:2443 read 0x5b1056b thru 0x5b1f08d (0xeb23 bytes) 2024-09-15T00:11:55.847 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:55.847 INFO:teuthology.orchestra.run.smithi171.stdout:2446 read 0x9618961 thru 0x96249f8 (0xc098 bytes) 2024-09-15T00:11:55.847 INFO:teuthology.orchestra.run.smithi171.stdout:2447 punch from 0x69bbe55 to 0x69c957f, (0xd72a bytes) 2024-09-15T00:11:55.847 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:55.847 INFO:teuthology.orchestra.run.smithi171.stdout:2451 write 0x985b004 thru 0x985ed9a (0x3d97 bytes) 2024-09-15T00:11:55.851 INFO:teuthology.orchestra.run.smithi171.stdout:2455 write 0x7fdd7de thru 0x7fe5bd1 (0x83f4 bytes) 2024-09-15T00:11:55.861 INFO:teuthology.orchestra.run.smithi171.stdout:2456 punch from 0x6b84c5b to 0x6b937c3, (0xeb68 bytes) 2024-09-15T00:11:55.861 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:55.867 INFO:teuthology.orchestra.run.smithi171.stdout:2458 write 0xc0edf97 thru 0xc0f5abd (0x7b27 bytes) 2024-09-15T00:11:55.869 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:55.869 INFO:teuthology.orchestra.run.smithi171.stdout:2460 read 0x4a9919 thru 0x4ad35f (0x3a47 bytes) 2024-09-15T00:11:55.870 INFO:teuthology.orchestra.run.smithi171.stdout:2461 write 0xbf8365a thru 0xbf8c2c3 (0x8c6a bytes) 2024-09-15T00:11:55.873 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:55.873 INFO:teuthology.orchestra.run.smithi171.stdout:2464 punch from 0x28ffb1e to 0x2902b95, (0x3077 bytes) 2024-09-15T00:11:55.878 INFO:teuthology.orchestra.run.smithi171.stdout:2465 trunc from 0xc0f5abe to 0x7e0b361 2024-09-15T00:11:55.883 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:55.881+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:55.887 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:55.888 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:55.898 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:55.903 INFO:teuthology.orchestra.run.smithi171.stdout:2466 trunc from 0x7e0b361 to 0x5344de6 2024-09-15T00:11:55.917 INFO:teuthology.orchestra.run.smithi171.stdout:2467 read 0x2007a29 thru 0x2007a5f (0x37 bytes) 2024-09-15T00:11:55.917 INFO:teuthology.orchestra.run.smithi171.stdout:2468 write 0x3cab530 thru 0x3cb0180 (0x4c51 bytes) 2024-09-15T00:11:55.921 INFO:teuthology.orchestra.run.smithi171.stdout:2469 write 0x266c3b3 thru 0x266dca7 (0x18f5 bytes) 2024-09-15T00:11:55.932 INFO:teuthology.orchestra.run.smithi171.stdout:2470 write 0x82532e0 thru 0x825dc18 (0xa939 bytes) 2024-09-15T00:11:55.954 INFO:teuthology.orchestra.run.smithi171.stdout:2471 write 0x8ce8f92 thru 0x8cf7ea3 (0xef12 bytes) 2024-09-15T00:11:55.982 INFO:teuthology.orchestra.run.smithi171.stdout:2472 write 0xbcc1c96 thru 0xbcc3556 (0x18c1 bytes) 2024-09-15T00:11:55.984 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:55.982+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:55.991 INFO:teuthology.orchestra.run.smithi171.stdout:2473 read 0x8f152b6 thru 0x8f167fd (0x1548 bytes) 2024-09-15T00:11:55.992 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:55.998 INFO:teuthology.orchestra.run.smithi171.stdout:2476 writesame 0xbd726c2 thru 0xbd7455d data_size 0x6(0x1e9c bytes) 2024-09-15T00:11:56.008 INFO:teuthology.orchestra.run.smithi171.stdout:2477 trunc from 0xbd7455e to 0x3bc1e83 2024-09-15T00:11:56.026 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:56.026 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:56.037 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:56.037 INFO:teuthology.orchestra.run.smithi171.stdout:2478 write 0x3ffe7a6 thru 0x4007855 (0x90b0 bytes) 2024-09-15T00:11:56.046 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:56.046 INFO:teuthology.orchestra.run.smithi171.stdout:2481 punch from 0x28af31f to 0x28afc1b, (0x8fc bytes) 2024-09-15T00:11:56.046 INFO:teuthology.orchestra.run.smithi171.stdout:2482 punch from 0x3d46d8b to 0x3d52fe3, (0xc258 bytes) 2024-09-15T00:11:56.053 INFO:teuthology.orchestra.run.smithi171.stdout:2483 write 0x4aa0494 thru 0x4aafd77 (0xf8e4 bytes) 2024-09-15T00:11:56.071 INFO:teuthology.orchestra.run.smithi171.stdout:2484 write 0x5161ae9 thru 0x516b97e (0x9e96 bytes) 2024-09-15T00:11:56.084 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:56.083+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:56.085 INFO:teuthology.orchestra.run.smithi171.stdout:2485 read 0x2052c53 thru 0x205f49f (0xc84d bytes) 2024-09-15T00:11:56.098 INFO:teuthology.orchestra.run.smithi171.stdout:2486 write 0xaf918c7 thru 0xaf9f3d0 (0xdb0a bytes) 2024-09-15T00:11:56.118 INFO:teuthology.orchestra.run.smithi171.stdout:2487 write 0xad5a262 thru 0xad660ee (0xbe8d bytes) 2024-09-15T00:11:56.121 INFO:teuthology.orchestra.run.smithi171.stdout:2488 read 0x980db62 thru 0x981ab5e (0xcffd bytes) 2024-09-15T00:11:56.134 INFO:teuthology.orchestra.run.smithi171.stdout:2490 trunc from 0xaf9f3d1 to 0x745127 2024-09-15T00:11:56.168 INFO:teuthology.orchestra.run.smithi171.stdout:2493 trunc from 0x745127 to 0x5d75b8f 2024-09-15T00:11:56.171 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:11:56.172 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:56.172 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:56.185 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:56.183+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:56.186 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:56.190 INFO:teuthology.orchestra.run.smithi171.stdout:2494 write 0xeda5d72 thru 0xeda8eac (0x313b bytes) 2024-09-15T00:11:56.195 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:56.195 INFO:teuthology.orchestra.run.smithi171.stdout:2497 write 0xc0470ef thru 0xc04d6fd (0x660f bytes) 2024-09-15T00:11:56.217 INFO:teuthology.orchestra.run.smithi171.stdout:2500 trunc from 0xeda8ead to 0x4102a35 2024-09-15T00:11:56.228 INFO:teuthology.orchestra.run.smithi171.stdout:2501 read 0x38c0abb thru 0x38cb1f1 (0xa737 bytes) 2024-09-15T00:11:56.228 INFO:teuthology.orchestra.run.smithi171.stdout:2502 write 0x2307d thru 0x241cc (0x1150 bytes) 2024-09-15T00:11:56.230 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:56.230 INFO:teuthology.orchestra.run.smithi171.stdout:2506 read 0x1604919 thru 0x16057a2 (0xe8a bytes) 2024-09-15T00:11:56.230 INFO:teuthology.orchestra.run.smithi171.stdout:2507 read 0x3ea952f thru 0x3ea960e (0xe0 bytes) 2024-09-15T00:11:56.234 INFO:teuthology.orchestra.run.smithi171.stdout:2508 trunc from 0x4102a35 to 0x10e0406 2024-09-15T00:11:56.239 INFO:teuthology.orchestra.run.smithi171.stdout:2509 read 0xc967ca thru 0xc9830e (0x1b45 bytes) 2024-09-15T00:11:56.254 INFO:teuthology.orchestra.run.smithi171.stdout:2511 writesame 0x69eac3a thru 0x69ee654 data_size 0x5(0x3a1b bytes) 2024-09-15T00:11:56.260 INFO:teuthology.orchestra.run.smithi171.stdout:2512 write 0x7548709 thru 0x754d9a9 (0x52a1 bytes) 2024-09-15T00:11:56.267 INFO:teuthology.orchestra.run.smithi171.stdout:2513 write 0x44fe13f thru 0x4501c60 (0x3b22 bytes) 2024-09-15T00:11:56.272 INFO:teuthology.orchestra.run.smithi171.stdout:2514 writesame 0x691c8d8 thru 0x69227c4 data_size 0x1(0x5eed bytes) 2024-09-15T00:11:56.281 INFO:teuthology.orchestra.run.smithi171.stdout:2515 punch from 0x231ad03 to 0x231cf54, (0x2251 bytes) 2024-09-15T00:11:56.281 INFO:teuthology.orchestra.run.smithi171.stdout:2516 read 0x8bd9fd thru 0x8c22a4 (0x48a8 bytes) 2024-09-15T00:11:56.281 INFO:teuthology.orchestra.run.smithi171.stdout:2518 punch from 0x2c4ee8f to 0x2c5b707, (0xc878 bytes) 2024-09-15T00:11:56.286 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:56.285+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:56.292 INFO:teuthology.orchestra.run.smithi171.stdout:2522 write 0xb7f3fe1 thru 0xb8033f3 (0xf413 bytes) 2024-09-15T00:11:56.310 INFO:teuthology.orchestra.run.smithi171.stdout:2523 write 0x35bcdea thru 0x35ca772 (0xd989 bytes) 2024-09-15T00:11:56.316 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:56.316 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:56.327 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:56.328 INFO:teuthology.orchestra.run.smithi171.stdout:2524 read 0x57afcef thru 0x57afd3c (0x4e bytes) 2024-09-15T00:11:56.339 INFO:teuthology.orchestra.run.smithi171.stdout:2525 write 0xe4859ad thru 0xe4932d3 (0xd927 bytes) 2024-09-15T00:11:56.358 INFO:teuthology.orchestra.run.smithi171.stdout:2527 read 0x49b86bb thru 0x49ba6ad (0x1ff3 bytes) 2024-09-15T00:11:56.358 INFO:teuthology.orchestra.run.smithi171.stdout:2528 read 0x83433d4 thru 0x834760a (0x4237 bytes) 2024-09-15T00:11:56.359 INFO:teuthology.orchestra.run.smithi171.stdout:2529 write 0xa97e459 thru 0xa989a79 (0xb621 bytes) 2024-09-15T00:11:56.375 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:56.375 INFO:teuthology.orchestra.run.smithi171.stdout:2531 write 0xd17f0df thru 0xd18be03 (0xcd25 bytes) 2024-09-15T00:11:56.387 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:56.385+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:56.395 INFO:teuthology.orchestra.run.smithi171.stdout:2532 read 0x3ceb316 thru 0x3cee836 (0x3521 bytes) 2024-09-15T00:11:56.395 INFO:teuthology.orchestra.run.smithi171.stdout:2533 read 0x967a037 thru 0x967afb9 (0xf83 bytes) 2024-09-15T00:11:56.395 INFO:teuthology.orchestra.run.smithi171.stdout:2535 writesame 0xc14c04e thru 0xc150798 data_size 0x1(0x474b bytes) 2024-09-15T00:11:56.399 INFO:teuthology.orchestra.run.smithi171.stdout:2536 write 0x579b768 thru 0x57a3730 (0x7fc9 bytes) 2024-09-15T00:11:56.411 INFO:teuthology.orchestra.run.smithi171.stdout:2537 read 0xcc97d2c thru 0xcca5c57 (0xdf2c bytes) 2024-09-15T00:11:56.422 INFO:teuthology.orchestra.run.smithi171.stdout:2538 trunc from 0xe4932d4 to 0x682892b 2024-09-15T00:11:56.446 INFO:teuthology.orchestra.run.smithi171.stdout:2539 writesame 0x5e9fd5d thru 0x5eaf419 data_size 0x5(0xf6bd bytes) 2024-09-15T00:11:56.458 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:56.458 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:56.464 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:56.464 INFO:teuthology.orchestra.run.smithi171.stdout:2543 write 0x26cff01 thru 0x26d4d5e (0x4e5e bytes) 2024-09-15T00:11:56.470 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:56.472 INFO:teuthology.orchestra.run.smithi171.stdout:2544 read 0x1674ec2 thru 0x1680c09 (0xbd48 bytes) 2024-09-15T00:11:56.473 INFO:teuthology.orchestra.run.smithi171.stdout:2545 read 0x31d4b7e thru 0x31dbf67 (0x73ea bytes) 2024-09-15T00:11:56.473 INFO:teuthology.orchestra.run.smithi171.stdout:2547 write 0x8bbd5d thru 0x8bceb3 (0x1157 bytes) 2024-09-15T00:11:56.474 INFO:teuthology.orchestra.run.smithi171.stdout:2548 read 0x25502c0 thru 0x2552c29 (0x296a bytes) 2024-09-15T00:11:56.484 INFO:teuthology.orchestra.run.smithi171.stdout:2550 write 0x947b3f6 thru 0x94815dd (0x61e8 bytes) 2024-09-15T00:11:56.488 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:56.486+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:56.496 INFO:teuthology.orchestra.run.smithi171.stdout:2553 read 0x8b0e5a4 thru 0x8b0fd81 (0x17de bytes) 2024-09-15T00:11:56.496 INFO:teuthology.orchestra.run.smithi171.stdout:2555 write 0x5933c73 thru 0x593bd79 (0x8107 bytes) 2024-09-15T00:11:56.507 INFO:teuthology.orchestra.run.smithi171.stdout:2556 write 0x93e8a26 thru 0x93f7084 (0xe65f bytes) 2024-09-15T00:11:56.520 INFO:teuthology.orchestra.run.smithi171.stdout:2557 read 0x8d73700 thru 0x8d7d9f6 (0xa2f7 bytes) 2024-09-15T00:11:56.520 INFO:teuthology.orchestra.run.smithi171.stdout:2558 read 0x3fa6633 thru 0x3fad2d7 (0x6ca5 bytes) 2024-09-15T00:11:56.521 INFO:teuthology.orchestra.run.smithi171.stdout:2559 write 0x399fd9d thru 0x39a9dec (0xa050 bytes) 2024-09-15T00:11:56.526 INFO:teuthology.orchestra.run.smithi171.stdout:2560 read 0x5405a83 thru 0x5405cee (0x26c bytes) 2024-09-15T00:11:56.526 INFO:teuthology.orchestra.run.smithi171.stdout:2562 write 0x31c5894 thru 0x31cbc76 (0x63e3 bytes) 2024-09-15T00:11:56.528 INFO:teuthology.orchestra.run.smithi171.stdout:2565 writesame 0x553dfce thru 0x5547c24 data_size 0x3(0x9c57 bytes) 2024-09-15T00:11:56.540 INFO:teuthology.orchestra.run.smithi171.stdout:2566 write 0x91ca6ca thru 0x91d414d (0x9a84 bytes) 2024-09-15T00:11:56.543 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:56.543 INFO:teuthology.orchestra.run.smithi171.stdout:2569 read 0x1fdaa5d thru 0x1fe2707 (0x7cab bytes) 2024-09-15T00:11:56.544 INFO:teuthology.orchestra.run.smithi171.stdout:2570 read 0x8a09e04 thru 0x8a12b6b (0x8d68 bytes) 2024-09-15T00:11:56.551 INFO:teuthology.orchestra.run.smithi171.stdout:2571 trunc from 0x94815de to 0x4a3446d 2024-09-15T00:11:56.588 INFO:teuthology.orchestra.run.smithi171.stdout:2572 write 0xd0afb1a thru 0xd0b5f4f (0x6436 bytes) 2024-09-15T00:11:56.589 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:56.587+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:56.600 INFO:teuthology.orchestra.run.smithi171.stdout:2573 read 0xc1d61a0 thru 0xc1d9df9 (0x3c5a bytes) 2024-09-15T00:11:56.609 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:56.609 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:56.614 INFO:teuthology.orchestra.run.smithi171.stdout:2575 trunc from 0xd0b5f50 to 0x35aed3d 2024-09-15T00:11:56.623 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:56.636 INFO:teuthology.orchestra.run.smithi171.stdout:2576 write 0x4a20e61 thru 0x4a2b80a (0xa9aa bytes) 2024-09-15T00:11:56.652 INFO:teuthology.orchestra.run.smithi171.stdout:2579 punch from 0x3786e72 to 0x3792e24, (0xbfb2 bytes) 2024-09-15T00:11:56.667 INFO:teuthology.orchestra.run.smithi171.stdout:2580 write 0xc524fb1 thru 0xc5349af (0xf9ff bytes) 2024-09-15T00:11:56.685 INFO:teuthology.orchestra.run.smithi171.stdout:2582 read 0x214ee4f thru 0x215901f (0xa1d1 bytes) 2024-09-15T00:11:56.685 INFO:teuthology.orchestra.run.smithi171.stdout:2583 write 0xae5dae thru 0xae756b (0x17be bytes) 2024-09-15T00:11:56.687 INFO:teuthology.orchestra.run.smithi171.stdout:2584 writesame 0x4e11285 thru 0x4e1f70d data_size 0x1(0xe489 bytes) 2024-09-15T00:11:56.690 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:56.688+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:56.706 INFO:teuthology.orchestra.run.smithi171.stdout:2586 write 0x6ae0fbe thru 0x6ae7fb6 (0x6ff9 bytes) 2024-09-15T00:11:56.726 INFO:teuthology.orchestra.run.smithi171.stdout:2587 writesame 0xde43441 thru 0xde446be data_size 0x3(0x127e bytes) 2024-09-15T00:11:56.726 INFO:teuthology.orchestra.run.smithi171.stdout:2588 punch from 0x4a7afb0 to 0x4a88304, (0xd354 bytes) 2024-09-15T00:11:56.732 INFO:teuthology.orchestra.run.smithi171.stdout:2589 trunc from 0xde446bf to 0x8ccc1c9 2024-09-15T00:11:56.746 INFO:teuthology.orchestra.run.smithi171.stdout:2590 read 0x1234c57 thru 0x1238c3f (0x3fe9 bytes) 2024-09-15T00:11:56.747 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:56.747 INFO:teuthology.orchestra.run.smithi171.stdout:2592 write 0x4101f62 thru 0x410da75 (0xbb14 bytes) 2024-09-15T00:11:56.761 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:56.762 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:56.766 INFO:teuthology.orchestra.run.smithi171.stdout:2593 write 0xbc12f96 thru 0xbc19c2d (0x6c98 bytes) 2024-09-15T00:11:56.771 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:56.781 INFO:teuthology.orchestra.run.smithi171.stdout:2594 read 0x30c59ee thru 0x30c8dd3 (0x33e6 bytes) 2024-09-15T00:11:56.790 INFO:teuthology.orchestra.run.smithi171.stdout:2595 write 0xdee4c8a thru 0xdee4ce1 (0x58 bytes) 2024-09-15T00:11:56.791 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:56.789+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:56.796 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:56.796 INFO:teuthology.orchestra.run.smithi171.stdout:2597 write 0xd7ed15 thru 0xd8e785 (0xfa71 bytes) 2024-09-15T00:11:56.818 INFO:teuthology.orchestra.run.smithi171.stdout:2598 trunc from 0xdee4ce2 to 0x1f6f1d3 2024-09-15T00:11:56.857 INFO:teuthology.orchestra.run.smithi171.stdout:2601 write 0x93e9323 thru 0x93f034f (0x702d bytes) 2024-09-15T00:11:56.872 INFO:teuthology.orchestra.run.smithi171.stdout:2602 writesame 0xa4951ee thru 0xa49b0ad data_size 0x20(0x5ec0 bytes) 2024-09-15T00:11:56.877 INFO:teuthology.orchestra.run.smithi171.stdout:2603 trunc from 0xa49b0ae to 0xed6e238 2024-09-15T00:11:56.883 INFO:teuthology.orchestra.run.smithi171.stdout:2604 read 0x31e94a6 thru 0x31f137f (0x7eda bytes) 2024-09-15T00:11:56.883 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:56.883 INFO:teuthology.orchestra.run.smithi171.stdout:2606 read 0xc6240b8 thru 0xc629696 (0x55df bytes) 2024-09-15T00:11:56.883 INFO:teuthology.orchestra.run.smithi171.stdout:2607 writesame 0xe97c16d thru 0xe9844de data_size 0x19(0x8372 bytes) 2024-09-15T00:11:56.891 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:56.890+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:56.896 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:56.897 INFO:teuthology.orchestra.run.smithi171.stdout:2609 write 0x8dd524e thru 0x8dd5c2a (0x9dd bytes) 2024-09-15T00:11:56.898 INFO:teuthology.orchestra.run.smithi171.stdout:2611 read 0x60954f6 thru 0x609af83 (0x5a8e bytes) 2024-09-15T00:11:56.898 INFO:teuthology.orchestra.run.smithi171.stdout:2612 write 0x873d6f9 thru 0x87445fc (0x6f04 bytes) 2024-09-15T00:11:56.904 INFO:teuthology.orchestra.run.smithi171.stdout:2613 read 0x5f909fb thru 0x5f9ff2e (0xf534 bytes) 2024-09-15T00:11:56.904 INFO:teuthology.orchestra.run.smithi171.stdout:2616 write 0x5adf070 thru 0x5ae50ff (0x6090 bytes) 2024-09-15T00:11:56.906 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:56.906 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:56.917 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:56.917 INFO:teuthology.orchestra.run.smithi171.stdout:2617 read 0x7fa035a thru 0x7faebca (0xe871 bytes) 2024-09-15T00:11:56.917 INFO:teuthology.orchestra.run.smithi171.stdout:2618 writesame 0x2bdbda5 thru 0x2be3ad6 data_size 0x19(0x7d32 bytes) 2024-09-15T00:11:56.927 INFO:teuthology.orchestra.run.smithi171.stdout:2619 punch from 0x7475130 to 0x747f662, (0xa532 bytes) 2024-09-15T00:11:56.927 INFO:teuthology.orchestra.run.smithi171.stdout:2621 write 0xdc04755 thru 0xdc082c6 (0x3b72 bytes) 2024-09-15T00:11:56.933 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:56.933 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:56.933 INFO:teuthology.orchestra.run.smithi171.stdout:2626 read 0x75c15cb thru 0x75cf4b9 (0xdeef bytes) 2024-09-15T00:11:56.938 INFO:teuthology.orchestra.run.smithi171.stdout:2627 trunc from 0xed6e238 to 0xad412a3 2024-09-15T00:11:56.955 INFO:teuthology.orchestra.run.smithi171.stdout:2629 writesame 0x79aafe0 thru 0x79ad473 data_size 0x2(0x2494 bytes) 2024-09-15T00:11:56.970 INFO:teuthology.orchestra.run.smithi171.stdout:2630 write 0xd7c36b4 thru 0xd7c58da (0x2227 bytes) 2024-09-15T00:11:56.980 INFO:teuthology.orchestra.run.smithi171.stdout:2631 write 0xe3c8c12 thru 0xe3cbc7a (0x3069 bytes) 2024-09-15T00:11:56.987 INFO:teuthology.orchestra.run.smithi171.stdout:2632 punch from 0xceaf38d to 0xceb6b30, (0x77a3 bytes) 2024-09-15T00:11:56.987 INFO:teuthology.orchestra.run.smithi171.stdout:2635 write 0x6e53a5e thru 0x6e5c9b8 (0x8f5b bytes) 2024-09-15T00:11:56.992 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:56.990+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:56.998 INFO:teuthology.orchestra.run.smithi171.stdout:2636 write 0x138129c thru 0x1387337 (0x609c bytes) 2024-09-15T00:11:57.007 INFO:teuthology.orchestra.run.smithi171.stdout:2637 write 0x16910f3 thru 0x1697d38 (0x6c46 bytes) 2024-09-15T00:11:57.014 INFO:teuthology.orchestra.run.smithi171.stdout:2638 punch from 0xae40a18 to 0xae47c13, (0x71fb bytes) 2024-09-15T00:11:57.014 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:57.014 INFO:teuthology.orchestra.run.smithi171.stdout:2641 clone 18 order 20 su 65536 sc 15 2024-09-15T00:11:57.035 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:57.035 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:57.043 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:57.093 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:57.091+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:57.147 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:57.147 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:57.155 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:57.163 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is set 2024-09-15T00:11:57.216 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:11:57.219 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:57.218+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:57.227 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone17 from 0xe3cbc7b (overlap 0x745127) to 0x45ca4a 2024-09-15T00:11:57.295 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:57.293+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:57.316 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:57.316 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:57.325 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:57.347 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-15T00:11:57.354 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:57.354 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:57.372 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:57.373 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:57.373 INFO:teuthology.orchestra.run.smithi171.stdout:2642 trunc from 0xe3cbc7b to 0x20e6f74 2024-09-15T00:11:57.401 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:57.401 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:57.409 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:57.408+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:57.441 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:57.441 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:57.471 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:57.471 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:57.494 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:57.494 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:57.497 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:57.496+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:57.498 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:57.498 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:57.504 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:57.523 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:57.523 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:57.549 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:57.549 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:57.572 INFO:teuthology.orchestra.run.smithi171.stdout:2643 write 0xe634144 thru 0xe6378d3 (0x3790 bytes) 2024-09-15T00:11:57.575 INFO:teuthology.orchestra.run.smithi171.stdout:2644 read 0xe03dad7 thru 0xe04cc5c (0xf186 bytes) 2024-09-15T00:11:57.585 INFO:teuthology.orchestra.run.smithi171.stdout:2645 trunc from 0xe6378d4 to 0x602ab91 2024-09-15T00:11:57.599 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:11:57.597+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:11:57.599 INFO:teuthology.orchestra.run.smithi171.stdout:2646 trunc from 0x602ab91 to 0x2759ee3 2024-09-15T00:11:57.617 INFO:teuthology.orchestra.run.smithi171.stdout:2647 write 0x51954ff thru 0x519ecf0 (0x97f2 bytes) 2024-09-15T00:11:57.630 INFO:teuthology.orchestra.run.smithi171.stdout:2648 trunc from 0x519ecf1 to 0xd81315b 2024-09-15T00:11:57.637 INFO:teuthology.orchestra.run.smithi171.stdout:2649 punch from 0x1b08141 to 0x1b0c680, (0x453f bytes) 2024-09-15T00:11:57.637 INFO:teuthology.orchestra.run.smithi171.stdout:2651 read 0x7ec1c24 thru 0x7ec4460 (0x283d bytes) 2024-09-15T00:11:57.637 INFO:teuthology.orchestra.run.smithi171.stdout:2652 punch from 0x85b8129 to 0x85c662d, (0xe504 bytes) 2024-09-15T00:11:57.637 INFO:teuthology.orchestra.run.smithi171.stdout:2653 writesame 0x851588 thru 0x8588b2 data_size 0x1(0x732b bytes) 2024-09-15T00:11:57.660 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:57.660 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:57.660 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:57.660 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:57.660 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:57.670 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:57.699 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:57.698+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:57.725 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:57.725 INFO:teuthology.orchestra.run.smithi171.stdout:2656 punch from 0xc01a771 to 0xc020405, (0x5c94 bytes) 2024-09-15T00:11:57.725 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:57.731 INFO:teuthology.orchestra.run.smithi171.stdout:2659 write 0xe212cb5 thru 0xe2130f6 (0x442 bytes) 2024-09-15T00:11:57.734 INFO:teuthology.orchestra.run.smithi171.stdout:2660 read 0x88d1cdf thru 0x88d5b03 (0x3e25 bytes) 2024-09-15T00:11:57.734 INFO:teuthology.orchestra.run.smithi171.stdout:2662 write 0x333445 thru 0x33c1b4 (0x8d70 bytes) 2024-09-15T00:11:57.763 INFO:teuthology.orchestra.run.smithi171.stdout:2665 trunc from 0xe2130f7 to 0x8bd649f 2024-09-15T00:11:57.777 INFO:teuthology.orchestra.run.smithi171.stdout:2666 trunc from 0x8bd649f to 0x4fe293b 2024-09-15T00:11:57.787 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:57.787 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:57.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:57.787 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:57.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.787 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:47986 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.787 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:11:57.787 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.312978398, 2024-09-15T00:11:57.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045287635, 2024-09-15T00:11:57.788 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.788 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.788 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:57.788 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.788 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.788 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.788 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 47986 2024-09-15T00:11:57.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.788 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:11:57.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:11:57.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:11:57.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8400000000000004e-07 2024-09-15T00:11:57.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474928+0000", 2024-09-15T00:11:57.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033009430000000002 2024-09-15T00:11:57.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474943+0000", 2024-09-15T00:11:57.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5172e-05 2024-09-15T00:11:57.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474951+0000", 2024-09-15T00:11:57.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1779999999999997e-06 2024-09-15T00:11:57.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.791 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475058+0000", 2024-09-15T00:11:57.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000107595 2024-09-15T00:11:57.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.791 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.791 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475200+0000", 2024-09-15T00:11:57.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014180400000000001 2024-09-15T00:11:57.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.791 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:57.791 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.478046+0000", 2024-09-15T00:11:57.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028464150000000001 2024-09-15T00:11:57.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.791 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:57.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516790+0000", 2024-09-15T00:11:57.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038743278999999999 2024-09-15T00:11:57.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:57.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516832+0000", 2024-09-15T00:11:57.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1974000000000002e-05 2024-09-15T00:11:57.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:57.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516872+0000", 2024-09-15T00:11:57.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9928999999999997e-05 2024-09-15T00:11:57.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516914+0000", 2024-09-15T00:11:57.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2762000000000002e-05 2024-09-15T00:11:57.793 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.793 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.793 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.793 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48888 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13762560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.793 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.608340+0000", 2024-09-15T00:11:57.793 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.176265433999999, 2024-09-15T00:11:57.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043750791999999997, 2024-09-15T00:11:57.793 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.793 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.794 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.794 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.794 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.794 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.794 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48888 2024-09-15T00:11:57.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.794 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608340+0000", 2024-09-15T00:11:57.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608340+0000", 2024-09-15T00:11:57.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608340+0000", 2024-09-15T00:11:57.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3099999999999998e-07 2024-09-15T00:11:57.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608342+0000", 2024-09-15T00:11:57.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9759999999999998e-06 2024-09-15T00:11:57.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608349+0000", 2024-09-15T00:11:57.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1010000000000003e-06 2024-09-15T00:11:57.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.608354+0000", 2024-09-15T00:11:57.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4669999999999998e-06 2024-09-15T00:11:57.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.651990+0000", 2024-09-15T00:11:57.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043636767999999999 2024-09-15T00:11:57.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.652034+0000", 2024-09-15T00:11:57.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4048000000000002e-05 2024-09-15T00:11:57.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.652090+0000", 2024-09-15T00:11:57.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6001000000000001e-05 2024-09-15T00:11:57.797 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.797 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.798 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.798 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48956 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 2621440~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.798 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609044+0000", 2024-09-15T00:11:57.798 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.175561446, 2024-09-15T00:11:57.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044014043000000003, 2024-09-15T00:11:57.798 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.798 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.798 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.798 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.798 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.798 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.798 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48956 2024-09-15T00:11:57.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.799 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609044+0000", 2024-09-15T00:11:57.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609044+0000", 2024-09-15T00:11:57.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609044+0000", 2024-09-15T00:11:57.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6100000000000001e-07 2024-09-15T00:11:57.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609046+0000", 2024-09-15T00:11:57.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2350000000000002e-06 2024-09-15T00:11:57.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609053+0000", 2024-09-15T00:11:57.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1750000000000002e-06 2024-09-15T00:11:57.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609057+0000", 2024-09-15T00:11:57.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7190000000000001e-06 2024-09-15T00:11:57.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.652645+0000", 2024-09-15T00:11:57.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043587689999999998 2024-09-15T00:11:57.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.652700+0000", 2024-09-15T00:11:57.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.507e-05 2024-09-15T00:11:57.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653058+0000", 2024-09-15T00:11:57.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035769300000000002 2024-09-15T00:11:57.802 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.802 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.802 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.802 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48957 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 4456448~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.802 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:11:57.803 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.175539885999999, 2024-09-15T00:11:57.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044512426000000001, 2024-09-15T00:11:57.803 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.803 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.803 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.803 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.803 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.803 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.803 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48957 2024-09-15T00:11:57.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.803 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:11:57.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:11:57.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609066+0000", 2024-09-15T00:11:57.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:11:57.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609068+0000", 2024-09-15T00:11:57.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9700000000000002e-06 2024-09-15T00:11:57.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609073+0000", 2024-09-15T00:11:57.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7939999999999999e-06 2024-09-15T00:11:57.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609078+0000", 2024-09-15T00:11:57.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3209999999999998e-06 2024-09-15T00:11:57.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653082+0000", 2024-09-15T00:11:57.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044003852000000003 2024-09-15T00:11:57.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653136+0000", 2024-09-15T00:11:57.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4024000000000003e-05 2024-09-15T00:11:57.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653578+0000", 2024-09-15T00:11:57.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044211599999999999 2024-09-15T00:11:57.807 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.807 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.807 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.807 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48958 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.807 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:11:57.807 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.175522247, 2024-09-15T00:11:57.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044724939999999998, 2024-09-15T00:11:57.807 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.807 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.807 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.807 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.807 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.808 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.808 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48958 2024-09-15T00:11:57.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.808 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:11:57.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:11:57.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:11:57.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:11:57.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609085+0000", 2024-09-15T00:11:57.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6610000000000001e-06 2024-09-15T00:11:57.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609090+0000", 2024-09-15T00:11:57.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2719999999999997e-06 2024-09-15T00:11:57.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609094+0000", 2024-09-15T00:11:57.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.907e-06 2024-09-15T00:11:57.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653618+0000", 2024-09-15T00:11:57.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044523759000000003 2024-09-15T00:11:57.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653703+0000", 2024-09-15T00:11:57.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5550000000000006e-05 2024-09-15T00:11:57.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653808+0000", 2024-09-15T00:11:57.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010445700000000001 2024-09-15T00:11:57.811 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.811 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.811 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.811 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48959 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.812 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:11:57.812 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.175506307999999, 2024-09-15T00:11:57.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044884741999999998, 2024-09-15T00:11:57.812 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.812 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.812 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.812 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.812 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.812 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.812 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48959 2024-09-15T00:11:57.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.812 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:11:57.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:11:57.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:11:57.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2899999999999999e-07 2024-09-15T00:11:57.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609101+0000", 2024-09-15T00:11:57.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.826e-06 2024-09-15T00:11:57.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609108+0000", 2024-09-15T00:11:57.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.165e-06 2024-09-15T00:11:57.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609113+0000", 2024-09-15T00:11:57.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7849999999999999e-06 2024-09-15T00:11:57.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653842+0000", 2024-09-15T00:11:57.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044728954000000001 2024-09-15T00:11:57.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653896+0000", 2024-09-15T00:11:57.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4593999999999999e-05 2024-09-15T00:11:57.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653984+0000", 2024-09-15T00:11:57.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7089000000000006e-05 2024-09-15T00:11:57.816 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.816 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.816 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.816 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48960 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 9961472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.817 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:11:57.817 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.175485789, 2024-09-15T00:11:57.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045084194000000001, 2024-09-15T00:11:57.817 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.817 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.817 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.817 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.817 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.817 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.817 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48960 2024-09-15T00:11:57.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.817 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:11:57.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:11:57.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609120+0000", 2024-09-15T00:11:57.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7099999999999997e-07 2024-09-15T00:11:57.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609122+0000", 2024-09-15T00:11:57.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:11:57.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609129+0000", 2024-09-15T00:11:57.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4159999999999998e-06 2024-09-15T00:11:57.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609134+0000", 2024-09-15T00:11:57.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7210000000000002e-06 2024-09-15T00:11:57.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654023+0000", 2024-09-15T00:11:57.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044888979000000002 2024-09-15T00:11:57.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654104+0000", 2024-09-15T00:11:57.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1108999999999999e-05 2024-09-15T00:11:57.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654204+0000", 2024-09-15T00:11:57.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9630000000000007e-05 2024-09-15T00:11:57.821 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.821 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.821 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.821 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48961 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.821 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:11:57.821 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.175464968, 2024-09-15T00:11:57.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045197821999999999, 2024-09-15T00:11:57.821 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.821 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.821 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.821 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.822 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.822 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.822 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48961 2024-09-15T00:11:57.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.822 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:11:57.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:11:57.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609141+0000", 2024-09-15T00:11:57.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:11:57.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609143+0000", 2024-09-15T00:11:57.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-15T00:11:57.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609150+0000", 2024-09-15T00:11:57.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2420000000000003e-06 2024-09-15T00:11:57.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609155+0000", 2024-09-15T00:11:57.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.028e-06 2024-09-15T00:11:57.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654232+0000", 2024-09-15T00:11:57.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045076702000000003 2024-09-15T00:11:57.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654281+0000", 2024-09-15T00:11:57.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9798e-05 2024-09-15T00:11:57.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654338+0000", 2024-09-15T00:11:57.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6498e-05 2024-09-15T00:11:57.825 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.825 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.825 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.825 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48962 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 13631488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.825 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:11:57.825 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.175443917999999, 2024-09-15T00:11:57.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045312458, 2024-09-15T00:11:57.826 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.826 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.826 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.826 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.826 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.826 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.826 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48962 2024-09-15T00:11:57.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.826 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:11:57.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:11:57.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609162+0000", 2024-09-15T00:11:57.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:11:57.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609164+0000", 2024-09-15T00:11:57.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0099999999999998e-06 2024-09-15T00:11:57.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609171+0000", 2024-09-15T00:11:57.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7530000000000001e-06 2024-09-15T00:11:57.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609178+0000", 2024-09-15T00:11:57.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1709999999999999e-06 2024-09-15T00:11:57.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654365+0000", 2024-09-15T00:11:57.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045187867999999999 2024-09-15T00:11:57.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654420+0000", 2024-09-15T00:11:57.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4376000000000003e-05 2024-09-15T00:11:57.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654474+0000", 2024-09-15T00:11:57.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3879999999999999e-05 2024-09-15T00:11:57.829 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.829 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.829 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.830 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48971 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.830 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:11:57.830 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.175412924, 2024-09-15T00:11:57.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045896151000000003, 2024-09-15T00:11:57.830 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.830 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.830 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.830 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.830 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.830 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.830 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48971 2024-09-15T00:11:57.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.830 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:11:57.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:11:57.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609193+0000", 2024-09-15T00:11:57.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7599999999999997e-07 2024-09-15T00:11:57.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609195+0000", 2024-09-15T00:11:57.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:11:57.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609210+0000", 2024-09-15T00:11:57.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4956999999999999e-05 2024-09-15T00:11:57.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609505+0000", 2024-09-15T00:11:57.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000295078 2024-09-15T00:11:57.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654498+0000", 2024-09-15T00:11:57.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044992277999999997 2024-09-15T00:11:57.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654541+0000", 2024-09-15T00:11:57.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3525000000000001e-05 2024-09-15T00:11:57.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655088+0000", 2024-09-15T00:11:57.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054726799999999995 2024-09-15T00:11:57.834 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.834 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.834 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.834 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48972 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 2097152~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.834 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:11:57.834 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.175086350000001, 2024-09-15T00:11:57.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046150885000000003, 2024-09-15T00:11:57.834 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.834 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.834 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.834 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.834 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48972 2024-09-15T00:11:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:11:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:11:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:11:57.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6400000000000002e-07 2024-09-15T00:11:57.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609523+0000", 2024-09-15T00:11:57.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5300000000000001e-06 2024-09-15T00:11:57.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609533+0000", 2024-09-15T00:11:57.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0023999999999999e-05 2024-09-15T00:11:57.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609539+0000", 2024-09-15T00:11:57.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3119999999999999e-06 2024-09-15T00:11:57.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655115+0000", 2024-09-15T00:11:57.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045576094999999997 2024-09-15T00:11:57.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655197+0000", 2024-09-15T00:11:57.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1129000000000002e-05 2024-09-15T00:11:57.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655670+0000", 2024-09-15T00:11:57.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00047313100000000001 2024-09-15T00:11:57.838 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.838 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.838 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.838 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.838 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:11:57.838 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.175058088, 2024-09-15T00:11:57.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:11:57.838 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.839 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.839 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.839 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.839 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.839 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.839 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:11:57.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.839 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:11:57.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:11:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:11:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:11:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:11:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:11:57.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:11:57.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:11:57.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:11:57.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:11:57.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:11:57.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:11:57.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:11:57.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:11:57.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:11:57.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:11:57.842 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.842 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.842 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.843 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.843 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:11:57.843 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.175036859, 2024-09-15T00:11:57.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:11:57.843 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.843 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.843 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.843 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.843 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.843 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.843 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:11:57.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.843 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:11:57.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:11:57.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:11:57.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:11:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:11:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:11:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:11:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:11:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:11:57.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:11:57.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:11:57.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:11:57.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:11:57.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:11:57.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:11:57.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:11:57.847 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.847 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.847 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.847 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.847 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:11:57.848 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.175011317999999, 2024-09-15T00:11:57.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:11:57.848 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.848 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.848 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.848 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:11:57.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.848 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:11:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:11:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:11:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:11:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:11:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:11:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:11:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:11:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:11:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:11:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:11:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:11:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:11:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:11:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:11:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:11:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.852 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.852 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.852 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:11:57.852 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.174988843, 2024-09-15T00:11:57.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:11:57.852 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.852 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.852 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.852 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.852 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.852 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.852 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:11:57.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.853 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:11:57.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:11:57.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:11:57.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:11:57.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:11:57.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:11:57.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:11:57.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:11:57.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:11:57.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:11:57.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:11:57.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:11:57.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:11:57.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:11:57.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:11:57.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:11:57.856 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.856 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.856 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.856 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.856 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:11:57.856 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.174968333000001, 2024-09-15T00:11:57.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:11:57.856 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.856 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.856 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.857 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.857 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.857 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.857 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:11:57.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.857 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:11:57.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:11:57.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:11:57.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:11:57.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:11:57.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:11:57.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:11:57.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:11:57.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:11:57.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:11:57.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:11:57.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:11:57.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:11:57.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:11:57.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:11:57.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:11:57.861 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.861 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.861 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.861 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.861 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:11:57.861 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.174948018, 2024-09-15T00:11:57.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:11:57.861 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.861 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.861 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.861 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.861 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.862 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:11:57.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.862 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:11:57.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:11:57.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:11:57.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:11:57.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:11:57.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:11:57.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:11:57.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:11:57.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:11:57.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:11:57.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:11:57.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:11:57.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:11:57.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:11:57.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:11:57.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:11:57.865 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.865 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.865 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.865 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.865 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:11:57.865 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.174923242, 2024-09-15T00:11:57.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:11:57.866 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.866 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.866 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.866 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:11:57.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.866 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:11:57.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:11:57.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:11:57.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:11:57.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:11:57.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:11:57.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:11:57.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:11:57.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:11:57.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:11:57.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:11:57.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:11:57.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:11:57.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:11:57.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:11:57.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:11:57.869 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.869 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.869 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.869 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:57.870 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:11:57.870 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.995581854999999, 2024-09-15T00:11:57.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:11:57.870 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.870 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.870 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:57.870 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:57.870 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:57.870 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:57.870 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:11:57.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.870 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:11:57.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:11:57.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:11:57.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:11:57.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:11:57.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:11:57.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:11:57.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:11:57.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:11:57.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:11:57.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:11:57.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:11:57.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:11:57.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:11:57.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:11:57.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:11:57.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:11:57.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:57.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:11:57.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:11:57.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:57.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:11:57.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:11:57.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:57.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:11:57.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:11:57.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:11:57.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:11:57.875 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.875 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.875 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.875 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:11:57.875 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:11:57.876 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.978628403, 2024-09-15T00:11:57.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:11:57.876 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:57.876 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:57.876 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:57.876 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:57.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:57.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:11:57.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:57.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:11:57.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:57.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:57.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:11:57.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:11:57.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:57.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:11:57.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:11:57.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:57.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:11:57.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:11:57.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:57.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:11:57.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:11:57.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:57.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:11:57.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:11:57.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:57.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:11:57.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:11:57.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:57.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:57.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:57.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:11:57.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:11:57.879 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.879 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.879 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.879 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:57.879 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:57.879 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:57.881 INFO:teuthology.orchestra.run.smithi171.stdout:2667 writesame 0x7b52505 thru 0x7b536c6 data_size 0x1(0x11c2 bytes) 2024-09-15T00:11:57.881 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:57.881 INFO:teuthology.orchestra.run.smithi171.stdout:2669 trunc from 0x7b536c7 to 0x125fc88 2024-09-15T00:11:57.881 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:57.879+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:57.882 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:11:57.981 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:57.980+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:57.984 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:57.985 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:57.993 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:11:58.083 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:58.081+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:58.090 INFO:teuthology.orchestra.run.smithi171.stdout:2670 writesame 0x63e8013 thru 0x63ebdc0 data_size 0x5(0x3dae bytes) 2024-09-15T00:11:58.092 INFO:teuthology.orchestra.run.smithi171.stdout:2671 trunc from 0x63ebdc1 to 0x8411d89 2024-09-15T00:11:58.101 INFO:teuthology.orchestra.run.smithi171.stdout:2672 writesame 0x1967e02 thru 0x1977174 data_size 0x1(0xf373 bytes) 2024-09-15T00:11:58.113 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:58.113 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:58.113 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:58.113 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:58.113 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:58.124 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:11:58.180 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:58.180 INFO:teuthology.orchestra.run.smithi171.stdout:2675 write 0x77d04fa thru 0x77d12d9 (0xde0 bytes) 2024-09-15T00:11:58.183 INFO:teuthology.orchestra.run.smithi171.stdout:2677 read 0x42ccf8f thru 0x42da4cb (0xd53d bytes) 2024-09-15T00:11:58.183 INFO:teuthology.orchestra.run.smithi171.stdout:2678 read 0x1c2ba18 thru 0x1c375fb (0xbbe4 bytes) 2024-09-15T00:11:58.191 INFO:teuthology.orchestra.run.smithi171.stdout:2679 writesame 0x8596375 thru 0x8599c6d data_size 0x1(0x38f9 bytes) 2024-09-15T00:11:58.191 INFO:teuthology.orchestra.run.smithi171.stdout:2680 read 0xd58086 thru 0xd62d6a (0xace5 bytes) 2024-09-15T00:11:58.205 INFO:teuthology.orchestra.run.smithi171.stdout:2681 write 0xecc6951 thru 0xeccd999 (0x7049 bytes) 2024-09-15T00:11:58.208 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:58.207+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:58.209 INFO:teuthology.orchestra.run.smithi171.stdout:2683 write 0x6935e6d thru 0x6936939 (0xacd bytes) 2024-09-15T00:11:58.212 INFO:teuthology.orchestra.run.smithi171.stdout:2684 read 0xd29486f thru 0xd29c9ba (0x814c bytes) 2024-09-15T00:11:58.212 INFO:teuthology.orchestra.run.smithi171.stdout:2686 write 0xa82d594 thru 0xa83a8c5 (0xd332 bytes) 2024-09-15T00:11:58.219 INFO:teuthology.orchestra.run.smithi171.stdout:2687 punch from 0x4a5f338 to 0x4a6eb4a, (0xf812 bytes) 2024-09-15T00:11:58.225 INFO:teuthology.orchestra.run.smithi171.stdout:2688 trunc from 0xeccd99a to 0x9da26d1 2024-09-15T00:11:58.235 INFO:teuthology.orchestra.run.smithi171.stdout:2690 read 0x3354bac thru 0x335b762 (0x6bb7 bytes) 2024-09-15T00:11:58.235 INFO:teuthology.orchestra.run.smithi171.stdout:2693 write 0x6f6e5da thru 0x6f75f81 (0x79a8 bytes) 2024-09-15T00:11:58.246 INFO:teuthology.orchestra.run.smithi171.stdout:2694 write 0xa012ebf thru 0xa0135ab (0x6ed bytes) 2024-09-15T00:11:58.259 INFO:teuthology.orchestra.run.smithi171.stdout:2695 write 0xcbba790 thru 0xcbbb92b (0x119c bytes) 2024-09-15T00:11:58.262 INFO:teuthology.orchestra.run.smithi171.stdout:2696 write 0xd4d2b1 thru 0xd5935c (0xc0ac bytes) 2024-09-15T00:11:58.284 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:58.283+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:58.285 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:58.286 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:58.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:58.286 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:58.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.286 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:19308 3.10 3:0ee1524d:::rbd_data.11398e1a091a.0000000000000017:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-15T00:11:58.286 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:11:58.286 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.298193786999999, 2024-09-15T00:11:58.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02547147, 2024-09-15T00:11:58.286 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.286 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.286 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:58.286 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.287 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.287 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.287 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 19308 2024-09-15T00:11:58.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.287 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:11:58.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:11:58.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985182+0000", 2024-09-15T00:11:58.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6500000000000001e-06 2024-09-15T00:11:58.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985189+0000", 2024-09-15T00:11:58.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9949999999999999e-06 2024-09-15T00:11:58.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985205+0000", 2024-09-15T00:11:58.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6055000000000001e-05 2024-09-15T00:11:58.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985212+0000", 2024-09-15T00:11:58.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:11:58.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985261+0000", 2024-09-15T00:11:58.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8380000000000001e-05 2024-09-15T00:11:58.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.290 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.290 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985311+0000", 2024-09-15T00:11:58.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0763999999999997e-05 2024-09-15T00:11:58.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.290 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 5", 2024-09-15T00:11:58.290 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.986489+0000", 2024-09-15T00:11:58.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0011773420000000001 2024-09-15T00:11:58.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.290 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:58.290 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010514+0000", 2024-09-15T00:11:58.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024025392999999999 2024-09-15T00:11:58.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:58.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010535+0000", 2024-09-15T00:11:58.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0652000000000001e-05 2024-09-15T00:11:58.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:58.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010597+0000", 2024-09-15T00:11:58.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2411999999999998e-05 2024-09-15T00:11:58.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010652+0000", 2024-09-15T00:11:58.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5074999999999997e-05 2024-09-15T00:11:58.292 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.292 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.292 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.292 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:58.292 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:11:58.292 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.616016819, 2024-09-15T00:11:58.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:11:58.292 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.292 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.292 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:58.292 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.292 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.293 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.293 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:11:58.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.293 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:11:58.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:11:58.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:11:58.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:11:58.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:11:58.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:11:58.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:11:58.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:11:58.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:11:58.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:11:58.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:11:58.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:11:58.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:11:58.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:11:58.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:58.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:11:58.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:11:58.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:58.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:11:58.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:11:58.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:58.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:11:58.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:11:58.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:58.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:11:58.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:11:58.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:11:58.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:11:58.297 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.297 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.297 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.298 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65519 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.298 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882394+0000", 2024-09-15T00:11:58.298 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.4009802439999999, 2024-09-15T00:11:58.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024506363999999999, 2024-09-15T00:11:58.298 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.298 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.298 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:11:58.298 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.298 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.298 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.298 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65519 2024-09-15T00:11:58.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.298 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882394+0000", 2024-09-15T00:11:58.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882394+0000", 2024-09-15T00:11:58.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882395+0000", 2024-09-15T00:11:58.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7799999999999997e-07 2024-09-15T00:11:58.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882396+0000", 2024-09-15T00:11:58.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8789999999999999e-06 2024-09-15T00:11:58.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882402+0000", 2024-09-15T00:11:58.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9159999999999998e-06 2024-09-15T00:11:58.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882407+0000", 2024-09-15T00:11:58.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6469999999999997e-06 2024-09-15T00:11:58.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906894+0000", 2024-09-15T00:11:58.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024486535 2024-09-15T00:11:58.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906901+0000", 2024-09-15T00:11:58.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.109e-06 2024-09-15T00:11:58.301 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.301 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.301 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.302 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65520 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.302 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:11:58.302 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.4009617639999998, 2024-09-15T00:11:58.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024513282000000001, 2024-09-15T00:11:58.302 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.302 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.302 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:11:58.302 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.302 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.302 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.302 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65520 2024-09-15T00:11:58.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.302 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:11:58.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:11:58.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:11:58.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.39e-07 2024-09-15T00:11:58.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882415+0000", 2024-09-15T00:11:58.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8020000000000001e-06 2024-09-15T00:11:58.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882421+0000", 2024-09-15T00:11:58.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7119999999999997e-06 2024-09-15T00:11:58.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882425+0000", 2024-09-15T00:11:58.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2660000000000003e-06 2024-09-15T00:11:58.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906919+0000", 2024-09-15T00:11:58.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024494438 2024-09-15T00:11:58.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906926+0000", 2024-09-15T00:11:58.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.725e-06 2024-09-15T00:11:58.305 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.305 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.305 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.306 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65521 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.306 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882444+0000", 2024-09-15T00:11:58.306 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.4009308109999998, 2024-09-15T00:11:58.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024504620000000001, 2024-09-15T00:11:58.306 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.306 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.306 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:11:58.306 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.306 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.306 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.306 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65521 2024-09-15T00:11:58.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.306 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882444+0000", 2024-09-15T00:11:58.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882444+0000", 2024-09-15T00:11:58.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882444+0000", 2024-09-15T00:11:58.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.06e-07 2024-09-15T00:11:58.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882447+0000", 2024-09-15T00:11:58.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3199999999999998e-06 2024-09-15T00:11:58.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882453+0000", 2024-09-15T00:11:58.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.968e-06 2024-09-15T00:11:58.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882459+0000", 2024-09-15T00:11:58.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0239999999999997e-06 2024-09-15T00:11:58.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906942+0000", 2024-09-15T00:11:58.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024483847 2024-09-15T00:11:58.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906948+0000", 2024-09-15T00:11:58.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9549999999999997e-06 2024-09-15T00:11:58.309 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.309 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.309 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.310 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65522 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.310 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882465+0000", 2024-09-15T00:11:58.310 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.4009093379999999, 2024-09-15T00:11:58.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024507998, 2024-09-15T00:11:58.310 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.310 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.310 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:11:58.310 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.310 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.310 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.310 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65522 2024-09-15T00:11:58.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.310 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882465+0000", 2024-09-15T00:11:58.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882465+0000", 2024-09-15T00:11:58.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882466+0000", 2024-09-15T00:11:58.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2399999999999999e-07 2024-09-15T00:11:58.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882472+0000", 2024-09-15T00:11:58.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7819999999999996e-06 2024-09-15T00:11:58.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882479+0000", 2024-09-15T00:11:58.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2859999999999996e-06 2024-09-15T00:11:58.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882484+0000", 2024-09-15T00:11:58.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0329999999999999e-06 2024-09-15T00:11:58.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906966+0000", 2024-09-15T00:11:58.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024481975 2024-09-15T00:11:58.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906973+0000", 2024-09-15T00:11:58.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.498e-06 2024-09-15T00:11:58.314 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.314 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.314 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.314 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65523 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.314 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882490+0000", 2024-09-15T00:11:58.314 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.400884102, 2024-09-15T00:11:58.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024506971999999998, 2024-09-15T00:11:58.314 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.314 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.314 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:11:58.315 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.315 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.315 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.315 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65523 2024-09-15T00:11:58.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.315 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882490+0000", 2024-09-15T00:11:58.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882490+0000", 2024-09-15T00:11:58.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882491+0000", 2024-09-15T00:11:58.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:11:58.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882493+0000", 2024-09-15T00:11:58.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9439999999999999e-06 2024-09-15T00:11:58.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882499+0000", 2024-09-15T00:11:58.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1489999999999999e-06 2024-09-15T00:11:58.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882503+0000", 2024-09-15T00:11:58.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.617e-06 2024-09-15T00:11:58.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906991+0000", 2024-09-15T00:11:58.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024487143999999999 2024-09-15T00:11:58.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906997+0000", 2024-09-15T00:11:58.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:11:58.318 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.318 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.318 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.318 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65524 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.318 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882509+0000", 2024-09-15T00:11:58.318 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.4008650380000001, 2024-09-15T00:11:58.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024511689999999999, 2024-09-15T00:11:58.318 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.318 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.319 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:11:58.319 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.319 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.319 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.319 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65524 2024-09-15T00:11:58.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.319 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882509+0000", 2024-09-15T00:11:58.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882509+0000", 2024-09-15T00:11:58.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882510+0000", 2024-09-15T00:11:58.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:11:58.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882512+0000", 2024-09-15T00:11:58.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.818e-06 2024-09-15T00:11:58.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882518+0000", 2024-09-15T00:11:58.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8919999999999997e-06 2024-09-15T00:11:58.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882522+0000", 2024-09-15T00:11:58.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3939999999999998e-06 2024-09-15T00:11:58.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907015+0000", 2024-09-15T00:11:58.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024492726999999999 2024-09-15T00:11:58.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907021+0000", 2024-09-15T00:11:58.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5250000000000002e-06 2024-09-15T00:11:58.322 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.322 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.322 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.322 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65525 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.322 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:11:58.322 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.400846896, 2024-09-15T00:11:58.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024517753, 2024-09-15T00:11:58.322 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.322 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.322 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:11:58.323 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.323 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.323 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.323 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65525 2024-09-15T00:11:58.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.323 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:11:58.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:11:58.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:11:58.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:11:58.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882530+0000", 2024-09-15T00:11:58.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.804e-06 2024-09-15T00:11:58.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882535+0000", 2024-09-15T00:11:58.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5500000000000002e-06 2024-09-15T00:11:58.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882540+0000", 2024-09-15T00:11:58.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6909999999999996e-06 2024-09-15T00:11:58.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907039+0000", 2024-09-15T00:11:58.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024498809 2024-09-15T00:11:58.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907045+0000", 2024-09-15T00:11:58.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5459999999999997e-06 2024-09-15T00:11:58.326 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.326 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.326 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.326 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66541 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1449984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.326 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:11:58.326 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.369239307, 2024-09-15T00:11:58.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024574262999999999, 2024-09-15T00:11:58.326 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66541 2024-09-15T00:11:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:11:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:11:58.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914136+0000", 2024-09-15T00:11:58.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3599999999999999e-07 2024-09-15T00:11:58.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914137+0000", 2024-09-15T00:11:58.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9309999999999998e-06 2024-09-15T00:11:58.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914143+0000", 2024-09-15T00:11:58.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0120000000000002e-06 2024-09-15T00:11:58.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914148+0000", 2024-09-15T00:11:58.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.707e-06 2024-09-15T00:11:58.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938536+0000", 2024-09-15T00:11:58.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024387671999999999 2024-09-15T00:11:58.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938618+0000", 2024-09-15T00:11:58.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1811999999999995e-05 2024-09-15T00:11:58.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938709+0000", 2024-09-15T00:11:58.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1792999999999993e-05 2024-09-15T00:11:58.330 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.330 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.330 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.331 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66542 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1560576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.331 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:11:58.331 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.3692201659999998, 2024-09-15T00:11:58.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024740410000000001, 2024-09-15T00:11:58.331 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.331 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.331 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.331 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.331 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.331 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.331 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66542 2024-09-15T00:11:58.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.331 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:11:58.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:11:58.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914155+0000", 2024-09-15T00:11:58.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:11:58.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914157+0000", 2024-09-15T00:11:58.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:11:58.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914162+0000", 2024-09-15T00:11:58.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:11:58.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914167+0000", 2024-09-15T00:11:58.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3789999999999999e-06 2024-09-15T00:11:58.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938730+0000", 2024-09-15T00:11:58.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024562694 2024-09-15T00:11:58.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938803+0000", 2024-09-15T00:11:58.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3325999999999996e-05 2024-09-15T00:11:58.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938895+0000", 2024-09-15T00:11:58.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1884000000000003e-05 2024-09-15T00:11:58.335 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.335 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.335 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.335 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66543 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1572864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.335 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:11:58.335 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.369189907, 2024-09-15T00:11:58.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024893976000000002, 2024-09-15T00:11:58.335 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.335 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.335 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.336 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.336 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.336 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.336 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66543 2024-09-15T00:11:58.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.336 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:11:58.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:11:58.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:11:58.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:11:58.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914188+0000", 2024-09-15T00:11:58.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:11:58.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914193+0000", 2024-09-15T00:11:58.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4129999999999997e-06 2024-09-15T00:11:58.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914198+0000", 2024-09-15T00:11:58.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9309999999999999e-06 2024-09-15T00:11:58.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938914+0000", 2024-09-15T00:11:58.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024716380999999999 2024-09-15T00:11:58.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938993+0000", 2024-09-15T00:11:58.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8763000000000005e-05 2024-09-15T00:11:58.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939079+0000", 2024-09-15T00:11:58.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5372999999999997e-05 2024-09-15T00:11:58.339 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.339 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.339 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.339 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66544 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1683456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.339 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:11:58.340 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.369167504, 2024-09-15T00:11:58.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025051225, 2024-09-15T00:11:58.340 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.340 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.340 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.340 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.340 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.340 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.340 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66544 2024-09-15T00:11:58.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.340 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:11:58.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:11:58.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:11:58.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:11:58.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914210+0000", 2024-09-15T00:11:58.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2450000000000001e-06 2024-09-15T00:11:58.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914215+0000", 2024-09-15T00:11:58.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7869999999999998e-06 2024-09-15T00:11:58.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914220+0000", 2024-09-15T00:11:58.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7899999999999999e-06 2024-09-15T00:11:58.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939105+0000", 2024-09-15T00:11:58.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024884381000000001 2024-09-15T00:11:58.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939177+0000", 2024-09-15T00:11:58.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2138999999999995e-05 2024-09-15T00:11:58.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939258+0000", 2024-09-15T00:11:58.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1462e-05 2024-09-15T00:11:58.343 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66545 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1695744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:11:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.36914523, 2024-09-15T00:11:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025209788, 2024-09-15T00:11:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.345 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66545 2024-09-15T00:11:58.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.345 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:11:58.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:11:58.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914230+0000", 2024-09-15T00:11:58.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1099999999999996e-07 2024-09-15T00:11:58.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914232+0000", 2024-09-15T00:11:58.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3080000000000002e-06 2024-09-15T00:11:58.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914238+0000", 2024-09-15T00:11:58.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.02e-06 2024-09-15T00:11:58.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914243+0000", 2024-09-15T00:11:58.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1200000000000001e-06 2024-09-15T00:11:58.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939285+0000", 2024-09-15T00:11:58.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025042212000000001 2024-09-15T00:11:58.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939361+0000", 2024-09-15T00:11:58.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5558999999999999e-05 2024-09-15T00:11:58.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939439+0000", 2024-09-15T00:11:58.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8058000000000004e-05 2024-09-15T00:11:58.348 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.348 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.348 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.348 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66546 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.348 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:11:58.348 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.3691252550000002, 2024-09-15T00:11:58.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025375960999999999, 2024-09-15T00:11:58.349 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.349 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.349 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.349 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.349 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.349 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.349 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66546 2024-09-15T00:11:58.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.349 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:11:58.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:11:58.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914250+0000", 2024-09-15T00:11:58.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:11:58.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914252+0000", 2024-09-15T00:11:58.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.052e-06 2024-09-15T00:11:58.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914258+0000", 2024-09-15T00:11:58.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9039999999999997e-06 2024-09-15T00:11:58.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914262+0000", 2024-09-15T00:11:58.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4719999999999997e-06 2024-09-15T00:11:58.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939466+0000", 2024-09-15T00:11:58.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025203550000000002 2024-09-15T00:11:58.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939537+0000", 2024-09-15T00:11:58.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1780999999999996e-05 2024-09-15T00:11:58.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939625+0000", 2024-09-15T00:11:58.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7828e-05 2024-09-15T00:11:58.353 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.353 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.353 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.353 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66547 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1818624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.353 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:11:58.353 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.369105582, 2024-09-15T00:11:58.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025552638999999999, 2024-09-15T00:11:58.353 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.353 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.353 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.354 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.354 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.354 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.354 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66547 2024-09-15T00:11:58.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.354 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:11:58.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:11:58.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:11:58.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:11:58.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914271+0000", 2024-09-15T00:11:58.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8920000000000001e-06 2024-09-15T00:11:58.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914277+0000", 2024-09-15T00:11:58.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:11:58.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914282+0000", 2024-09-15T00:11:58.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7190000000000001e-06 2024-09-15T00:11:58.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939654+0000", 2024-09-15T00:11:58.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025372372000000001 2024-09-15T00:11:58.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939734+0000", 2024-09-15T00:11:58.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9946999999999999e-05 2024-09-15T00:11:58.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939822+0000", 2024-09-15T00:11:58.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7680000000000001e-05 2024-09-15T00:11:58.357 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.357 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.357 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.358 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66548 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.358 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:11:58.358 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.3690751950000002, 2024-09-15T00:11:58.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025712727000000001, 2024-09-15T00:11:58.358 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.358 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.358 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.358 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.358 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.358 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.358 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66548 2024-09-15T00:11:58.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.358 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:11:58.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:11:58.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914300+0000", 2024-09-15T00:11:58.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3599999999999999e-07 2024-09-15T00:11:58.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914302+0000", 2024-09-15T00:11:58.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3369999999999998e-06 2024-09-15T00:11:58.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914313+0000", 2024-09-15T00:11:58.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1399999999999999e-05 2024-09-15T00:11:58.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914319+0000", 2024-09-15T00:11:58.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9100000000000002e-06 2024-09-15T00:11:58.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939850+0000", 2024-09-15T00:11:58.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025531087000000001 2024-09-15T00:11:58.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939932+0000", 2024-09-15T00:11:58.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1534999999999998e-05 2024-09-15T00:11:58.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940012+0000", 2024-09-15T00:11:58.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0022000000000001e-05 2024-09-15T00:11:58.362 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.362 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.362 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.362 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66549 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1941504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.362 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:11:58.362 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.369010104, 2024-09-15T00:11:58.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025813836999999999, 2024-09-15T00:11:58.362 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.362 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.362 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.362 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.362 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.362 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.363 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66549 2024-09-15T00:11:58.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.363 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:11:58.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:11:58.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914365+0000", 2024-09-15T00:11:58.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2799999999999995e-07 2024-09-15T00:11:58.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914368+0000", 2024-09-15T00:11:58.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.92e-06 2024-09-15T00:11:58.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914376+0000", 2024-09-15T00:11:58.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8110000000000003e-06 2024-09-15T00:11:58.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914381+0000", 2024-09-15T00:11:58.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5119999999999999e-06 2024-09-15T00:11:58.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940040+0000", 2024-09-15T00:11:58.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025658271999999999 2024-09-15T00:11:58.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940103+0000", 2024-09-15T00:11:58.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2891000000000005e-05 2024-09-15T00:11:58.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940178+0000", 2024-09-15T00:11:58.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5703000000000003e-05 2024-09-15T00:11:58.366 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.366 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.368 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.368 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66550 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.368 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:11:58.368 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.368866524, 2024-09-15T00:11:58.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025820701000000001, 2024-09-15T00:11:58.369 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.369 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.369 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.369 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.369 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.369 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.369 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66550 2024-09-15T00:11:58.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.369 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:11:58.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:11:58.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914509+0000", 2024-09-15T00:11:58.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1999999999999999e-07 2024-09-15T00:11:58.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914513+0000", 2024-09-15T00:11:58.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1640000000000003e-06 2024-09-15T00:11:58.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914522+0000", 2024-09-15T00:11:58.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7639999999999994e-06 2024-09-15T00:11:58.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914527+0000", 2024-09-15T00:11:58.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9170000000000003e-06 2024-09-15T00:11:58.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940199+0000", 2024-09-15T00:11:58.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025672037000000002 2024-09-15T00:11:58.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940269+0000", 2024-09-15T00:11:58.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9208999999999994e-05 2024-09-15T00:11:58.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940329+0000", 2024-09-15T00:11:58.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.999e-05 2024-09-15T00:11:58.372 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.372 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.372 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.373 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66551 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:11:58.373 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:11:58.373 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.3688097770000001, 2024-09-15T00:11:58.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025917345000000001, 2024-09-15T00:11:58.373 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.373 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.373 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.373 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.373 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.373 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.373 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66551 2024-09-15T00:11:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:11:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:11:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:11:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1699999999999997e-07 2024-09-15T00:11:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914569+0000", 2024-09-15T00:11:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.225e-06 2024-09-15T00:11:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914582+0000", 2024-09-15T00:11:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3648999999999999e-05 2024-09-15T00:11:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914588+0000", 2024-09-15T00:11:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8449999999999999e-06 2024-09-15T00:11:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940355+0000", 2024-09-15T00:11:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025767110999999999 2024-09-15T00:11:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940423+0000", 2024-09-15T00:11:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7574e-05 2024-09-15T00:11:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940482+0000", 2024-09-15T00:11:58.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9224000000000001e-05 2024-09-15T00:11:58.377 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.377 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.377 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.377 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.377 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.377 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:58.379 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:11:58.380 INFO:teuthology.orchestra.run.smithi171.stdout:2697 write 0x35e0cfd thru 0x35e91c8 (0x84cc bytes) 2024-09-15T00:11:58.380 INFO:teuthology.orchestra.run.smithi171.stdout:2699 writesame 0xdad84d7 thru 0xdad91e5 data_size 0x1(0xd0f bytes) 2024-09-15T00:11:58.380 INFO:teuthology.orchestra.run.smithi171.stdout:2700 write 0xad918f0 thru 0xada1379 (0xfa8a bytes) 2024-09-15T00:11:58.380 INFO:teuthology.orchestra.run.smithi171.stdout:2701 write 0x9017f5f thru 0x9022aa0 (0xab42 bytes) 2024-09-15T00:11:58.380 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:58.380 INFO:teuthology.orchestra.run.smithi171.stdout:2703 write 0x30960c6 thru 0x309920c (0x3147 bytes) 2024-09-15T00:11:58.380 INFO:teuthology.orchestra.run.smithi171.stdout:2704 trunc from 0xdad91e6 to 0x412a631 2024-09-15T00:11:58.380 INFO:teuthology.orchestra.run.smithi171.stdout:2705 punch from 0x192a25b to 0x1931970, (0x7715 bytes) 2024-09-15T00:11:58.380 INFO:teuthology.orchestra.run.smithi171.stdout:2706 write 0x646d0af thru 0x6474309 (0x725b bytes) 2024-09-15T00:11:58.380 INFO:teuthology.orchestra.run.smithi171.stdout:2707 write 0x108b28 thru 0x10e8bc (0x5d95 bytes) 2024-09-15T00:11:58.383 INFO:teuthology.orchestra.run.smithi171.stdout:2708 write 0x2438f0 thru 0x2536ce (0xfddf bytes) 2024-09-15T00:11:58.402 INFO:teuthology.orchestra.run.smithi171.stdout:2709 read 0x2bde53a thru 0x2be9768 (0xb22f bytes) 2024-09-15T00:11:58.403 INFO:teuthology.orchestra.run.smithi171.stdout:2710 read 0x5ec33a9 thru 0x5ec3efa (0xb52 bytes) 2024-09-15T00:11:58.403 INFO:teuthology.orchestra.run.smithi171.stdout:2711 punch from 0x5149b48 to 0x5157fd9, (0xe491 bytes) 2024-09-15T00:11:58.410 INFO:teuthology.orchestra.run.smithi171.stdout:2712 write 0x760ada4 thru 0x7618c08 (0xde65 bytes) 2024-09-15T00:11:58.420 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:58.418+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:58.427 INFO:teuthology.orchestra.run.smithi171.stdout:2713 writesame 0xcd81cc2 thru 0xcd90a45 data_size 0x36(0xed84 bytes) 2024-09-15T00:11:58.427 INFO:teuthology.orchestra.run.smithi171.stdout:2714 read 0x59649cd thru 0x5969652 (0x4c86 bytes) 2024-09-15T00:11:58.427 INFO:teuthology.orchestra.run.smithi171.stdout:2715 write 0x5778137 thru 0x577be77 (0x3d41 bytes) 2024-09-15T00:11:58.430 INFO:teuthology.orchestra.run.smithi171.stdout:2716 write 0x1456543 thru 0x14650e0 (0xeb9e bytes) 2024-09-15T00:11:58.433 INFO:teuthology.orchestra.run.smithi171.stdout:2717 read 0x7a6c88f thru 0x7a72d9c (0x650e bytes) 2024-09-15T00:11:58.444 INFO:teuthology.orchestra.run.smithi171.stdout:2718 trunc from 0xcd90a46 to 0x3773dbf 2024-09-15T00:11:58.455 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:58.468 INFO:teuthology.orchestra.run.smithi171.stdout:2722 write 0x935010c thru 0x9356747 (0x663c bytes) 2024-09-15T00:11:58.478 INFO:teuthology.orchestra.run.smithi171.stdout:2723 write 0xa30e61b thru 0xa314c59 (0x663f bytes) 2024-09-15T00:11:58.484 INFO:teuthology.orchestra.run.smithi171.stdout:2724 read 0x22c26ad thru 0x22c5948 (0x329c bytes) 2024-09-15T00:11:58.484 INFO:teuthology.orchestra.run.smithi171.stdout:2726 punch from 0x8e0f6f4 to 0x8e0f806, (0x112 bytes) 2024-09-15T00:11:58.484 INFO:teuthology.orchestra.run.smithi171.stdout:2727 read 0x6375f0c thru 0x637e0e9 (0x81de bytes) 2024-09-15T00:11:58.484 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:58.484 INFO:teuthology.orchestra.run.smithi171.stdout:2730 punch from 0x72ea2ca to 0x72f9067, (0xed9d bytes) 2024-09-15T00:11:58.486 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:58.485+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:58.489 INFO:teuthology.orchestra.run.smithi171.stdout:2731 trunc from 0xa314c5a to 0x5b749c3 2024-09-15T00:11:58.511 INFO:teuthology.orchestra.run.smithi171.stdout:2732 writesame 0x8780040 thru 0x8782da9 data_size 0x1(0x2d6a bytes) 2024-09-15T00:11:58.511 INFO:teuthology.orchestra.run.smithi171.stdout:2733 punch from 0x48d31bb to 0x48d3313, (0x158 bytes) 2024-09-15T00:11:58.511 INFO:teuthology.orchestra.run.smithi171.stdout:2734 read 0x4d64107 thru 0x4d6425e (0x158 bytes) 2024-09-15T00:11:58.511 INFO:teuthology.orchestra.run.smithi171.stdout:2735 punch from 0x19d0a81 to 0x19d779d, (0x6d1c bytes) 2024-09-15T00:11:58.519 INFO:teuthology.orchestra.run.smithi171.stdout:2737 trunc from 0x8782daa to 0x17dfa21 2024-09-15T00:11:58.532 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:58.532 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:58.532 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:11:58.533 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:58.534 INFO:teuthology.orchestra.run.smithi171.stdout:2738 read 0xca2d21 thru 0xcb29b7 (0xfc97 bytes) 2024-09-15T00:11:58.542 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:11:58.543 INFO:teuthology.orchestra.run.smithi171.stdout:2739 writesame 0x37d8874 thru 0x37d9c2c data_size 0x21(0x13b9 bytes) 2024-09-15T00:11:58.553 INFO:teuthology.orchestra.run.smithi171.stdout:2740 writesame 0x659bb0d thru 0x65aa982 data_size 0x83(0xee76 bytes) 2024-09-15T00:11:58.553 INFO:teuthology.orchestra.run.smithi171.stdout:2741 punch from 0x48c6fd3 to 0x48cd66a, (0x6697 bytes) 2024-09-15T00:11:58.553 INFO:teuthology.orchestra.run.smithi171.stdout:2742 punch from 0x22a7c7f to 0x22b4060, (0xc3e1 bytes) 2024-09-15T00:11:58.553 INFO:teuthology.orchestra.run.smithi171.stdout:2743 punch from 0x213cb7a to 0x213ed9f, (0x2225 bytes) 2024-09-15T00:11:58.559 INFO:teuthology.orchestra.run.smithi171.stdout:2744 write 0x718c837 thru 0x718c8a2 (0x6c bytes) 2024-09-15T00:11:58.562 INFO:teuthology.orchestra.run.smithi171.stdout:2745 writesame 0x66813bf thru 0x6685dcb data_size 0x47(0x4a0d bytes) 2024-09-15T00:11:58.566 INFO:teuthology.orchestra.run.smithi171.stdout:2746 clone 19 order 23 su 65536 sc 9 2024-09-15T00:11:58.587 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:58.585+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:58.645 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:58.645 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:11:58.645 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:11:58.646 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:11:58.646 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:58.653 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:11:58.695 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:11:58.693+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:11:58.789 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:11:58.788+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:11:58.799 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:11:58.799 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:11:58.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:11:58.799 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:11:58.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.799 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:58.799 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:11:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.718081955999999, 2024-09-15T00:11:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:11:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:11:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:11:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:11:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:11:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:11:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:11:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:11:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:11:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:11:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:11:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:11:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:11:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:11:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:11:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:11:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:11:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:11:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:11:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:11:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:11:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:11:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:11:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:11:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:11:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:11:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:11:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:11:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.705341414000003, 2024-09-15T00:11:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:11:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:11:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:11:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:11:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:11:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:11:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:11:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:11:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:11:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:11:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:11:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:11:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:11:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:11:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:11:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:11:58.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:11:58.810 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.810 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.810 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.810 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:11:58.810 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:11:58.810 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.572985654999997, 2024-09-15T00:11:58.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:11:58.810 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.810 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.810 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:58.810 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.810 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:11:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:11:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:11:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:11:58.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:11:58.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:11:58.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:11:58.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:11:58.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:11:58.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:11:58.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:11:58.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:11:58.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:11:58.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:11:58.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:11:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:11:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:11:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:11:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:11:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:11:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:11:58.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:58.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:11:58.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:11:58.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:11:58.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:11:58.815 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.815 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.815 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.816 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:11:58.816 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:11:58.816 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.559524009999997, 2024-09-15T00:11:58.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:11:58.816 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.816 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.816 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.816 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:11:58.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:11:58.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:11:58.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:11:58.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:11:58.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:11:58.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:11:58.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:11:58.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:11:58.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:11:58.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:11:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:11:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:11:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:11:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:11:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:11:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:11:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:11:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.374522527, 2024-09-15T00:11:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:11:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.821 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:11:58.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.821 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:11:58.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:11:58.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:11:58.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:11:58.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:11:58.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:11:58.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:11:58.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:11:58.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:11:58.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:11:58.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:11:58.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:11:58.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:11:58.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:11:58.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:58.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:11:58.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:11:58.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:58.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:11:58.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:11:58.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:58.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:11:58.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:11:58.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:58.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:11:58.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:11:58.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:11:58.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:11:58.825 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.825 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.825 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.826 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:11:58.826 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:11:58.826 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.361646260000001, 2024-09-15T00:11:58.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:11:58.826 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.826 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.826 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.826 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:11:58.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:11:58.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:11:58.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:11:58.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:11:58.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:11:58.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:11:58.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:11:58.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:11:58.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:11:58.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:11:58.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:11:58.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:11:58.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:11:58.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:11:58.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:11:58.829 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.829 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.829 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.830 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:11:58.830 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:11:58.830 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.903131116000001, 2024-09-15T00:11:58.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:11:58.830 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.830 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.830 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:58.830 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.830 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.830 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.830 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:11:58.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.830 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:11:58.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:11:58.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:11:58.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:11:58.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:11:58.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:11:58.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:11:58.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:11:58.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:11:58.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:11:58.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:11:58.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:11:58.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:11:58.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:11:58.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:58.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:11:58.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:11:58.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:58.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:11:58.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:11:58.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:58.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:11:58.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:11:58.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:58.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:11:58.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:11:58.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:11:58.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:11:58.835 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.835 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.835 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.835 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:58.835 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:11:58.835 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.185620556, 2024-09-15T00:11:58.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:11:58.836 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.836 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.836 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.836 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.836 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.836 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.836 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:11:58.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.836 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:11:58.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:11:58.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:11:58.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:11:58.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:11:58.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:11:58.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:11:58.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:11:58.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:11:58.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:11:58.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:11:58.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:11:58.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:11:58.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:11:58.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:11:58.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:11:58.839 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.839 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.839 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.840 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:58.840 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:11:58.840 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.185601947, 2024-09-15T00:11:58.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:11:58.840 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.840 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.840 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.840 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.840 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.840 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.840 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:11:58.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.841 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:11:58.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:11:58.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:11:58.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:11:58.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:11:58.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:11:58.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:11:58.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:11:58.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:11:58.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:11:58.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:11:58.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:11:58.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:11:58.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:11:58.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:11:58.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:11:58.844 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.844 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.844 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.844 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:58.844 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:11:58.844 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.185576622999999, 2024-09-15T00:11:58.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:11:58.845 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.845 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.845 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.845 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.845 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.845 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.845 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:11:58.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.845 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:11:58.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:11:58.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:11:58.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:11:58.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:11:58.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:11:58.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:11:58.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:11:58.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:11:58.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:11:58.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:11:58.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:11:58.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:11:58.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:11:58.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:11:58.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:11:58.849 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.849 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.849 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.849 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:58.849 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:11:58.849 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.185558299, 2024-09-15T00:11:58.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:11:58.849 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.849 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.849 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.849 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.850 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.850 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.850 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:11:58.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.850 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:11:58.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:11:58.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:11:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:11:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:11:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:11:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:11:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:11:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:11:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:11:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:11:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.185375726, 2024-09-15T00:11:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:11:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:11:58.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:11:58.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:11:58.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:11:58.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:11:58.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:11:58.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:11:58.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:11:58.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:11:58.857 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.857 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.857 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.857 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:58.857 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:11:58.857 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.185346885, 2024-09-15T00:11:58.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:11:58.858 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.858 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.858 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.858 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:11:58.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.858 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:11:58.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:11:58.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:11:58.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:11:58.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:11:58.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:11:58.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:11:58.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:11:58.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:11:58.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:11:58.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:11:58.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:11:58.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:11:58.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:11:58.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:11:58.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:11:58.861 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.861 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.861 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.861 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:58.862 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:11:58.862 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.185324674, 2024-09-15T00:11:58.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:11:58.862 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.862 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.862 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.862 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:11:58.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.862 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:11:58.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:11:58.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:11:58.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:11:58.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:11:58.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:11:58.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:11:58.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:11:58.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:11:58.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:11:58.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:11:58.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:11:58.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:11:58.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:11:58.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:11:58.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:11:58.866 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.866 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.866 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.867 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:58.867 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:11:58.867 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.185303514999999, 2024-09-15T00:11:58.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:11:58.867 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.867 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.867 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.867 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:11:58.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.867 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:11:58.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:11:58.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:11:58.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:11:58.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:11:58.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:11:58.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:11:58.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:11:58.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:11:58.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:11:58.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:11:58.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:11:58.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:11:58.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:11:58.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:11:58.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:11:58.871 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.871 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.871 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.871 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:58.871 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:11:58.871 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.185273566999999, 2024-09-15T00:11:58.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:11:58.871 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.871 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.871 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.872 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:11:58.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.872 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:11:58.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:11:58.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:11:58.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:11:58.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:11:58.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:11:58.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:11:58.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:11:58.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:11:58.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:11:58.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:11:58.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:11:58.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:11:58.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:11:58.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:11:58.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:11:58.875 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.875 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.875 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.875 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:58.875 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:11:58.875 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.185251602999999, 2024-09-15T00:11:58.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:11:58.876 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.876 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.876 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.876 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.876 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.876 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.876 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:11:58.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.876 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:11:58.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:11:58.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:11:58.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:11:58.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:11:58.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:11:58.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:11:58.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:11:58.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:11:58.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:11:58.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:11:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.185234176, 2024-09-15T00:11:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:11:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:11:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:11:58.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:11:58.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:11:58.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:11:58.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:11:58.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:11:58.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:11:58.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:11:58.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:11:58.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:11:58.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:11:58.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:11:58.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:11:58.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:11:58.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:11:58.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:11:58.883 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.884 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.884 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.884 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:58.884 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:11:58.884 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.185212376999999, 2024-09-15T00:11:58.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:11:58.884 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.884 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.884 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:11:58.884 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.884 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.884 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:11:58.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:11:58.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:11:58.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:11:58.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:11:58.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:11:58.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:11:58.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:11:58.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:11:58.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:11:58.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:11:58.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:11:58.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:11:58.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:11:58.888 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.888 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.888 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.888 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:11:58.888 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:11:58.888 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.135984018, 2024-09-15T00:11:58.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:11:58.888 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:11:58.888 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:11:58.888 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:58.889 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:11:58.889 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:11:58.889 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:11:58.889 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:11:58.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.889 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:11:58.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:11:58.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:11:58.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:11:58.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:11:58.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:11:58.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:11:58.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:11:58.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:11:58.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:11:58.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:11:58.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:11:58.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:11:58.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:11:58.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:11:58.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:11:58.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:11:58.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:11:58.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:11:58.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:11:58.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:11:58.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:11:58.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:11:58.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:11:58.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:11:58.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:11:58.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:11:58.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:11:58.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:11:58.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:11:58.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:58.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:11:58.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:11:58.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:11:58.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:11:58.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:11:58.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:11:58.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:11:58.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:11:58.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:11:58.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:11:58.894 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.894 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.894 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.894 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:11:58.894 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:11:58.894 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:11:58.895 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:58.894+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:58.896 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:58.996 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:11:58.995+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:11:59.001 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:11:59.002 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:59.010 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:11:59.097 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:59.096+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:59.113 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:59.113 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:59.121 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:11:59.179 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:11:59.198 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:59.196+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:59.248 INFO:teuthology.orchestra.run.smithi171.stdout:flattening image image_client.0-clone18 2024-09-15T00:11:59.300 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:59.300 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:11:59.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:11:59.300 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:59.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.300 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-15T00:11:59.300 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:11:59.300 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.760841225, 2024-09-15T00:11:59.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0035574980000000001, 2024-09-15T00:11:59.300 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.301 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.301 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.301 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:11:59.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:11:59.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535370+0000", 2024-09-15T00:11:59.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-15T00:11:59.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535438+0000", 2024-09-15T00:11:59.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199000000000002e-05 2024-09-15T00:11:59.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535446+0000", 2024-09-15T00:11:59.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1640000000000001e-06 2024-09-15T00:11:59.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535450+0000", 2024-09-15T00:11:59.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1520000000000002e-06 2024-09-15T00:11:59.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535491+0000", 2024-09-15T00:11:59.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0969e-05 2024-09-15T00:11:59.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535510+0000", 2024-09-15T00:11:59.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9031000000000001e-05 2024-09-15T00:11:59.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.538893+0000", 2024-09-15T00:11:59.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033823870000000002 2024-09-15T00:11:59.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.538927+0000", 2024-09-15T00:11:59.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4085999999999999e-05 2024-09-15T00:11:59.304 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.304 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.305 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.305 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:70 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-09-15T00:11:59.305 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:11:59.305 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.760101005000003, 2024-09-15T00:11:59.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057871490000000001, 2024-09-15T00:11:59.305 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.305 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.305 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.305 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:11:59.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:11:59.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536110+0000", 2024-09-15T00:11:59.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-15T00:11:59.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536115+0000", 2024-09-15T00:11:59.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8960000000000002e-06 2024-09-15T00:11:59.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536123+0000", 2024-09-15T00:11:59.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3979999999999999e-06 2024-09-15T00:11:59.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536128+0000", 2024-09-15T00:11:59.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5569999999999997e-06 2024-09-15T00:11:59.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537011+0000", 2024-09-15T00:11:59.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088276299999999995 2024-09-15T00:11:59.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537034+0000", 2024-09-15T00:11:59.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3448999999999999e-05 2024-09-15T00:11:59.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541867+0000", 2024-09-15T00:11:59.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048332949999999996 2024-09-15T00:11:59.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541896+0000", 2024-09-15T00:11:59.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9179999999999998e-05 2024-09-15T00:11:59.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.310 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.310 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-15T00:11:59.310 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:59.310 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.732482552, 2024-09-15T00:11:59.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027567730000000001, 2024-09-15T00:11:59.311 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.311 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.311 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.311 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:59.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:59.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:11:59.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5599999999999995e-07 2024-09-15T00:11:59.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563787+0000", 2024-09-15T00:11:59.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9092000000000003e-05 2024-09-15T00:11:59.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563796+0000", 2024-09-15T00:11:59.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0059999999999998e-06 2024-09-15T00:11:59.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563801+0000", 2024-09-15T00:11:59.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3290000000000004e-06 2024-09-15T00:11:59.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563830+0000", 2024-09-15T00:11:59.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9369999999999998e-05 2024-09-15T00:11:59.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563842+0000", 2024-09-15T00:11:59.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1646e-05 2024-09-15T00:11:59.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.566459+0000", 2024-09-15T00:11:59.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026174549999999999 2024-09-15T00:11:59.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.566485+0000", 2024-09-15T00:11:59.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5318999999999999e-05 2024-09-15T00:11:59.314 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.314 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.314 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.315 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:106 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-09-15T00:11:59.315 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:11:59.315 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.599580089, 2024-09-15T00:11:59.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029384250000000002, 2024-09-15T00:11:59.315 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.315 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.315 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.315 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:11:59.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:11:59.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696631+0000", 2024-09-15T00:11:59.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4299999999999998e-07 2024-09-15T00:11:59.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696692+0000", 2024-09-15T00:11:59.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1229999999999995e-05 2024-09-15T00:11:59.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696699+0000", 2024-09-15T00:11:59.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5399999999999998e-06 2024-09-15T00:11:59.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696705+0000", 2024-09-15T00:11:59.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5770000000000001e-06 2024-09-15T00:11:59.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696732+0000", 2024-09-15T00:11:59.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7022e-05 2024-09-15T00:11:59.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696743+0000", 2024-09-15T00:11:59.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1426999999999999e-05 2024-09-15T00:11:59.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699542+0000", 2024-09-15T00:11:59.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002798521 2024-09-15T00:11:59.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699569+0000", 2024-09-15T00:11:59.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6665e-05 2024-09-15T00:11:59.319 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.319 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.319 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.319 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:107 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-09-15T00:11:59.319 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:59.319 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.598684659, 2024-09-15T00:11:59.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048949229999999998, 2024-09-15T00:11:59.320 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.320 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.320 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.320 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:59.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:59.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:11:59.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5799999999999999e-07 2024-09-15T00:11:59.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697531+0000", 2024-09-15T00:11:59.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3100000000000002e-06 2024-09-15T00:11:59.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697539+0000", 2024-09-15T00:11:59.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6410000000000008e-06 2024-09-15T00:11:59.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697543+0000", 2024-09-15T00:11:59.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1060000000000002e-06 2024-09-15T00:11:59.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697573+0000", 2024-09-15T00:11:59.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9978e-05 2024-09-15T00:11:59.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697585+0000", 2024-09-15T00:11:59.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1279e-05 2024-09-15T00:11:59.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702398+0000", 2024-09-15T00:11:59.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004812981 2024-09-15T00:11:59.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702421+0000", 2024-09-15T00:11:59.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3070000000000001e-05 2024-09-15T00:11:59.323 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.323 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.323 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.324 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:108 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'66)", 2024-09-15T00:11:59.324 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:11:59.324 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.597791168999997, 2024-09-15T00:11:59.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040179439999999999, 2024-09-15T00:11:59.324 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.324 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.324 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.324 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:11:59.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:11:59.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698420+0000", 2024-09-15T00:11:59.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0900000000000002e-07 2024-09-15T00:11:59.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698423+0000", 2024-09-15T00:11:59.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8319999999999999e-06 2024-09-15T00:11:59.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698431+0000", 2024-09-15T00:11:59.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8949999999999998e-06 2024-09-15T00:11:59.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698435+0000", 2024-09-15T00:11:59.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5939999999999998e-06 2024-09-15T00:11:59.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698469+0000", 2024-09-15T00:11:59.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4258000000000003e-05 2024-09-15T00:11:59.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698481+0000", 2024-09-15T00:11:59.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1763000000000001e-05 2024-09-15T00:11:59.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702422+0000", 2024-09-15T00:11:59.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003940603 2024-09-15T00:11:59.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702437+0000", 2024-09-15T00:11:59.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5489999999999999e-05 2024-09-15T00:11:59.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.328 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.328 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:109 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'66)", 2024-09-15T00:11:59.328 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:59.328 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.597260732999999, 2024-09-15T00:11:59.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038555479999999999, 2024-09-15T00:11:59.328 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.328 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.328 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.329 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:59.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:59.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:11:59.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.13e-07 2024-09-15T00:11:59.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699018+0000", 2024-09-15T00:11:59.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7540999999999996e-05 2024-09-15T00:11:59.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699024+0000", 2024-09-15T00:11:59.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4869999999999999e-06 2024-09-15T00:11:59.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699044+0000", 2024-09-15T00:11:59.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9400999999999999e-05 2024-09-15T00:11:59.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699104+0000", 2024-09-15T00:11:59.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0331999999999998e-05 2024-09-15T00:11:59.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699117+0000", 2024-09-15T00:11:59.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2789000000000001e-05 2024-09-15T00:11:59.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702777+0000", 2024-09-15T00:11:59.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0036604900000000002 2024-09-15T00:11:59.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702805+0000", 2024-09-15T00:11:59.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7994999999999999e-05 2024-09-15T00:11:59.332 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.332 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.332 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.332 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:110 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'72, mlcod=16'66)", 2024-09-15T00:11:59.332 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:11:59.333 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.596501199999999, 2024-09-15T00:11:59.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031108189999999999, 2024-09-15T00:11:59.333 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.333 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.333 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.333 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:11:59.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:11:59.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699710+0000", 2024-09-15T00:11:59.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:11:59.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699714+0000", 2024-09-15T00:11:59.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0010000000000003e-06 2024-09-15T00:11:59.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699722+0000", 2024-09-15T00:11:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4840000000000004e-06 2024-09-15T00:11:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699726+0000", 2024-09-15T00:11:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:11:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699755+0000", 2024-09-15T00:11:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8433999999999999e-05 2024-09-15T00:11:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699775+0000", 2024-09-15T00:11:59.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0017000000000002e-05 2024-09-15T00:11:59.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702807+0000", 2024-09-15T00:11:59.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030320239999999999 2024-09-15T00:11:59.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702820+0000", 2024-09-15T00:11:59.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3169999999999999e-05 2024-09-15T00:11:59.336 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.336 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.337 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.337 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:120 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'82, mlcod=16'78)", 2024-09-15T00:11:59.337 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:11:59.337 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.590597854000002, 2024-09-15T00:11:59.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028965829999999999, 2024-09-15T00:11:59.337 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.337 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.337 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.337 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:11:59.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:11:59.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705613+0000", 2024-09-15T00:11:59.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1800000000000001e-07 2024-09-15T00:11:59.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705684+0000", 2024-09-15T00:11:59.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0903999999999997e-05 2024-09-15T00:11:59.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705691+0000", 2024-09-15T00:11:59.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4290000000000004e-06 2024-09-15T00:11:59.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705695+0000", 2024-09-15T00:11:59.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4790000000000001e-06 2024-09-15T00:11:59.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705729+0000", 2024-09-15T00:11:59.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4415999999999997e-05 2024-09-15T00:11:59.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705741+0000", 2024-09-15T00:11:59.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1443999999999999e-05 2024-09-15T00:11:59.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.708476+0000", 2024-09-15T00:11:59.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027356630000000002 2024-09-15T00:11:59.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.708509+0000", 2024-09-15T00:11:59.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2830000000000002e-05 2024-09-15T00:11:59.341 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.341 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.341 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.341 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-15T00:11:59.341 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:59.341 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.589811388999998, 2024-09-15T00:11:59.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047824809999999999, 2024-09-15T00:11:59.341 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.341 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.341 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.342 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:59.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:59.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:11:59.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:11:59.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706471+0000", 2024-09-15T00:11:59.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1496999999999997e-05 2024-09-15T00:11:59.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706478+0000", 2024-09-15T00:11:59.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7460000000000003e-06 2024-09-15T00:11:59.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706481+0000", 2024-09-15T00:11:59.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.258e-06 2024-09-15T00:11:59.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706513+0000", 2024-09-15T00:11:59.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2567000000000002e-05 2024-09-15T00:11:59.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706524+0000", 2024-09-15T00:11:59.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1028e-05 2024-09-15T00:11:59.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.711150+0000", 2024-09-15T00:11:59.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046259450000000002 2024-09-15T00:11:59.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.711181+0000", 2024-09-15T00:11:59.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1041000000000003e-05 2024-09-15T00:11:59.345 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.345 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.345 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.345 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:136 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-15T00:11:59.346 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:59.346 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.236374455000004, 2024-09-15T00:11:59.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031270970000000001, 2024-09-15T00:11:59.346 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.346 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.346 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.346 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:59.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:59.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:11:59.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3000000000000002e-07 2024-09-15T00:11:59.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059839+0000", 2024-09-15T00:11:59.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3050000000000001e-06 2024-09-15T00:11:59.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059847+0000", 2024-09-15T00:11:59.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.622e-06 2024-09-15T00:11:59.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059851+0000", 2024-09-15T00:11:59.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8120000000000001e-06 2024-09-15T00:11:59.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059884+0000", 2024-09-15T00:11:59.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2620999999999998e-05 2024-09-15T00:11:59.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059894+0000", 2024-09-15T00:11:59.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0924000000000001e-05 2024-09-15T00:11:59.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062935+0000", 2024-09-15T00:11:59.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030406830000000002 2024-09-15T00:11:59.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062963+0000", 2024-09-15T00:11:59.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7800000000000001e-05 2024-09-15T00:11:59.350 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.350 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.350 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.350 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:139 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'92)", 2024-09-15T00:11:59.350 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:59.350 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.234587232000003, 2024-09-15T00:11:59.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040690240000000001, 2024-09-15T00:11:59.350 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.351 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.351 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.351 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:59.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:59.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:11:59.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:11:59.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061631+0000", 2024-09-15T00:11:59.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.17e-06 2024-09-15T00:11:59.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061636+0000", 2024-09-15T00:11:59.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7679999999999997e-06 2024-09-15T00:11:59.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061639+0000", 2024-09-15T00:11:59.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9069999999999999e-06 2024-09-15T00:11:59.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062733+0000", 2024-09-15T00:11:59.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010937119999999999 2024-09-15T00:11:59.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062744+0000", 2024-09-15T00:11:59.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1439e-05 2024-09-15T00:11:59.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065658+0000", 2024-09-15T00:11:59.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029134970000000001 2024-09-15T00:11:59.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065692+0000", 2024-09-15T00:11:59.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4147999999999999e-05 2024-09-15T00:11:59.354 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.354 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.354 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.355 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'92)", 2024-09-15T00:11:59.355 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:59.355 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.234198169999999, 2024-09-15T00:11:59.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003986608, 2024-09-15T00:11:59.355 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.355 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.355 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.355 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:59.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:59.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:11:59.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7000000000000001e-07 2024-09-15T00:11:59.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062075+0000", 2024-09-15T00:11:59.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2638000000000006e-05 2024-09-15T00:11:59.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062083+0000", 2024-09-15T00:11:59.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4330000000000004e-06 2024-09-15T00:11:59.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062087+0000", 2024-09-15T00:11:59.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3799999999999998e-06 2024-09-15T00:11:59.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.063001+0000", 2024-09-15T00:11:59.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091419500000000002 2024-09-15T00:11:59.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.063012+0000", 2024-09-15T00:11:59.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1117999999999999e-05 2024-09-15T00:11:59.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065977+0000", 2024-09-15T00:11:59.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029650840000000002 2024-09-15T00:11:59.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065999+0000", 2024-09-15T00:11:59.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1489999999999999e-05 2024-09-15T00:11:59.359 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.359 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.359 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.359 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:154 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'108, mlcod=16'105)", 2024-09-15T00:11:59.359 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:59.359 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 52.771950545000003, 2024-09-15T00:11:59.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053187770000000002, 2024-09-15T00:11:59.359 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.359 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.359 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.359 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:59.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:59.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:11:59.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:11:59.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524264+0000", 2024-09-15T00:11:59.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.738e-06 2024-09-15T00:11:59.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524270+0000", 2024-09-15T00:11:59.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9379999999999997e-06 2024-09-15T00:11:59.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524273+0000", 2024-09-15T00:11:59.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6370000000000001e-06 2024-09-15T00:11:59.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524310+0000", 2024-09-15T00:11:59.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6266999999999997e-05 2024-09-15T00:11:59.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524322+0000", 2024-09-15T00:11:59.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2741999999999999e-05 2024-09-15T00:11:59.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.529547+0000", 2024-09-15T00:11:59.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005225001 2024-09-15T00:11:59.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.529579+0000", 2024-09-15T00:11:59.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1065999999999997e-05 2024-09-15T00:11:59.363 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.363 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.363 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.363 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-15T00:11:59.363 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:11:59.363 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 52.771512907000002, 2024-09-15T00:11:59.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077021529999999998, 2024-09-15T00:11:59.364 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.364 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.364 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.364 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:11:59.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:11:59.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524698+0000", 2024-09-15T00:11:59.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6300000000000001e-07 2024-09-15T00:11:59.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524778+0000", 2024-09-15T00:11:59.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0598000000000003e-05 2024-09-15T00:11:59.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524785+0000", 2024-09-15T00:11:59.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6359999999999997e-06 2024-09-15T00:11:59.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524788+0000", 2024-09-15T00:11:59.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2540000000000001e-06 2024-09-15T00:11:59.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524817+0000", 2024-09-15T00:11:59.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8784000000000001e-05 2024-09-15T00:11:59.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524828+0000", 2024-09-15T00:11:59.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-15T00:11:59.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532360+0000", 2024-09-15T00:11:59.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007531757 2024-09-15T00:11:59.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532400+0000", 2024-09-15T00:11:59.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9835000000000001e-05 2024-09-15T00:11:59.367 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.367 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.367 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.367 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=16'106)", 2024-09-15T00:11:59.368 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:59.368 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 52.770570753999998, 2024-09-15T00:11:59.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067923050000000002, 2024-09-15T00:11:59.368 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.368 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.368 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.368 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:59.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:59.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:11:59.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-15T00:11:59.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525644+0000", 2024-09-15T00:11:59.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.709e-06 2024-09-15T00:11:59.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525651+0000", 2024-09-15T00:11:59.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7970000000000003e-06 2024-09-15T00:11:59.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525654+0000", 2024-09-15T00:11:59.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.433e-06 2024-09-15T00:11:59.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525685+0000", 2024-09-15T00:11:59.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0970000000000003e-05 2024-09-15T00:11:59.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525697+0000", 2024-09-15T00:11:59.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1759e-05 2024-09-15T00:11:59.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532401+0000", 2024-09-15T00:11:59.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006704425 2024-09-15T00:11:59.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532432+0000", 2024-09-15T00:11:59.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0709e-05 2024-09-15T00:11:59.372 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.372 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.372 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.372 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'106)", 2024-09-15T00:11:59.372 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:59.372 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 52.770108499999999, 2024-09-15T00:11:59.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063539190000000004, 2024-09-15T00:11:59.372 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.372 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.372 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.372 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:59.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:59.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:11:59.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:11:59.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526107+0000", 2024-09-15T00:11:59.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:11:59.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526117+0000", 2024-09-15T00:11:59.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7750000000000004e-06 2024-09-15T00:11:59.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526121+0000", 2024-09-15T00:11:59.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:11:59.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526163+0000", 2024-09-15T00:11:59.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1360999999999999e-05 2024-09-15T00:11:59.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526175+0000", 2024-09-15T00:11:59.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2729e-05 2024-09-15T00:11:59.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532433+0000", 2024-09-15T00:11:59.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062576999999999997 2024-09-15T00:11:59.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532456+0000", 2024-09-15T00:11:59.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2773e-05 2024-09-15T00:11:59.376 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.376 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.376 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.376 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'106)", 2024-09-15T00:11:59.376 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:11:59.376 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 52.769639097999999, 2024-09-15T00:11:59.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059082689999999998, 2024-09-15T00:11:59.377 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.377 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.377 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.377 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:11:59.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:11:59.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526572+0000", 2024-09-15T00:11:59.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9500000000000003e-07 2024-09-15T00:11:59.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526641+0000", 2024-09-15T00:11:59.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8858999999999999e-05 2024-09-15T00:11:59.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526648+0000", 2024-09-15T00:11:59.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:11:59.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526654+0000", 2024-09-15T00:11:59.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7509999999999997e-06 2024-09-15T00:11:59.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526689+0000", 2024-09-15T00:11:59.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5800000000000003e-05 2024-09-15T00:11:59.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526700+0000", 2024-09-15T00:11:59.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0467e-05 2024-09-15T00:11:59.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532457+0000", 2024-09-15T00:11:59.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057568180000000004 2024-09-15T00:11:59.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532479+0000", 2024-09-15T00:11:59.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2739e-05 2024-09-15T00:11:59.380 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.380 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.380 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.381 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'106)", 2024-09-15T00:11:59.381 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:11:59.381 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 52.768982833000003, 2024-09-15T00:11:59.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005274801, 2024-09-15T00:11:59.381 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.381 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.381 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.381 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:11:59.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:11:59.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527228+0000", 2024-09-15T00:11:59.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1600000000000001e-07 2024-09-15T00:11:59.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527232+0000", 2024-09-15T00:11:59.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8920000000000002e-06 2024-09-15T00:11:59.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527239+0000", 2024-09-15T00:11:59.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2880000000000004e-06 2024-09-15T00:11:59.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527243+0000", 2024-09-15T00:11:59.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.143e-06 2024-09-15T00:11:59.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527275+0000", 2024-09-15T00:11:59.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1866000000000003e-05 2024-09-15T00:11:59.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527288+0000", 2024-09-15T00:11:59.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469999999999999e-05 2024-09-15T00:11:59.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532480+0000", 2024-09-15T00:11:59.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051927129999999998 2024-09-15T00:11:59.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532502+0000", 2024-09-15T00:11:59.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1912999999999998e-05 2024-09-15T00:11:59.385 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.385 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.385 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.385 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:25 3.10 e26/23 3:0d9e4e9e:::rbd_header.11395e6383ec:head v 26'2, mlcod=0'0)", 2024-09-15T00:11:59.385 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:21.619783+0000", 2024-09-15T00:11:59.385 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.676427167, 2024-09-15T00:11:59.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031992600000000002, 2024-09-15T00:11:59.385 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.385 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.385 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619783+0000", 2024-09-15T00:11:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619783+0000", 2024-09-15T00:11:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619784+0000", 2024-09-15T00:11:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9900000000000009e-07 2024-09-15T00:11:59.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619795+0000", 2024-09-15T00:11:59.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0689e-05 2024-09-15T00:11:59.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619809+0000", 2024-09-15T00:11:59.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3757000000000001e-05 2024-09-15T00:11:59.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619818+0000", 2024-09-15T00:11:59.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4909999999999994e-06 2024-09-15T00:11:59.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619868+0000", 2024-09-15T00:11:59.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0046999999999997e-05 2024-09-15T00:11:59.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619887+0000", 2024-09-15T00:11:59.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780999999999999e-05 2024-09-15T00:11:59.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:11:59.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.622947+0000", 2024-09-15T00:11:59.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030601069999999998 2024-09-15T00:11:59.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:11:59.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.622982+0000", 2024-09-15T00:11:59.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5389000000000003e-05 2024-09-15T00:11:59.389 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.389 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.389 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.389 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.389 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.389 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:59.390 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6] out_osds: [0, 2, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T00:11:59.390 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:11:59.392 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-15T00:11:59.392 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T00:11:59.393 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:11:59.298+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:11:59.394 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:59.437 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:11:59.435+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:11:59.476 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-15T00:11:59.490 INFO:teuthology.orchestra.run.smithi171.stdout:2748 trunc from 0x718c8a3 to 0x68592fe 2024-09-15T00:11:59.502 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:59.500+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:59.512 INFO:teuthology.orchestra.run.smithi171.stdout:2749 write 0x583f148 thru 0x5841df0 (0x2ca9 bytes) 2024-09-15T00:11:59.513 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:59.513 INFO:teuthology.orchestra.run.smithi171.stdout:2751 write 0xcb993 thru 0xcc9d9 (0x1047 bytes) 2024-09-15T00:11:59.529 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:11:59.531 INFO:teuthology.orchestra.run.smithi171.stdout:2754 trunc from 0x68592fe to 0x4d91b4b 2024-09-15T00:11:59.538 INFO:teuthology.orchestra.run.smithi171.stdout:2756 write 0x18ae5bd thru 0x18b9dc0 (0xb804 bytes) 2024-09-15T00:11:59.557 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:11:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:84612 3.9 e59/55 3:95907793:::rbd_data.11396cb1f0e4.0000000000000007:head v 59'728, mlcod=58'726)", 2024-09-15T00:11:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:59.547277+0000", 2024-09-15T00:11:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0091426240000000002, 2024-09-15T00:11:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0091638959999999995, 2024-09-15T00:11:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:11:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:11:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:11:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:11:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:11:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.547277+0000", 2024-09-15T00:11:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:11:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.547277+0000", 2024-09-15T00:11:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:11:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:11:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.547279+0000", 2024-09-15T00:11:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-06 2024-09-15T00:11:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:11:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.548830+0000", 2024-09-15T00:11:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015507609999999999 2024-09-15T00:11:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:11:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.548855+0000", 2024-09-15T00:11:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5873000000000001e-05 2024-09-15T00:11:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:11:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.548868+0000", 2024-09-15T00:11:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2765999999999999e-05 2024-09-15T00:11:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:11:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.548943+0000", 2024-09-15T00:11:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4845999999999995e-05 2024-09-15T00:11:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:11:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.548967+0000", 2024-09-15T00:11:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3892000000000001e-05 2024-09-15T00:11:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:11:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 1 2024-09-15T00:11:59.562 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:59.562 INFO:teuthology.orchestra.run.smithi171.stdout:2757 write 0x8418b3 thru 0x84474c (0x2e9a bytes) 2024-09-15T00:11:59.563 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:11:59.568 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:59.578 INFO:teuthology.orchestra.run.smithi171.stdout:2758 read 0x39412cf thru 0x39435ab (0x22dd bytes) 2024-09-15T00:11:59.586 INFO:teuthology.orchestra.run.smithi171.stdout:2759 writesame 0x6b2b84c thru 0x6b3b7bc data_size 0x1(0xff71 bytes) 2024-09-15T00:11:59.592 INFO:teuthology.orchestra.run.smithi171.stdout:2762 trunc from 0x6b3b7bd to 0x30e7e4b 2024-09-15T00:11:59.605 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:11:59.603+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:11:59.640 INFO:teuthology.orchestra.run.smithi171.stdout:2763 read 0x24f78e thru 0x25a194 (0xaa07 bytes) 2024-09-15T00:11:59.641 INFO:teuthology.orchestra.run.smithi171.stdout:2764 clone 20 order 18 su 2048 sc 2 2024-09-15T00:11:59.697 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:59.697 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:11:59.697 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:11:59.697 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:11:59.697 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:59.704 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:59.702+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:59.706 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:59.707 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:11:59.708 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:59.725 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:11:59.725 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:59.742 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:11:59.742 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:59.759 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:11:59.759 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:59.767 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:11:59.767 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:11:59.768 INFO:teuthology.orchestra.run.smithi171.stdout: "epoch": 59, 2024-09-15T00:11:59.768 INFO:teuthology.orchestra.run.smithi171.stdout: "fsid": "bad2533c-9c09-4af4-ab93-147afbacb705", 2024-09-15T00:11:59.768 INFO:teuthology.orchestra.run.smithi171.stdout: "created": "2024-09-15T00:10:50.203030+0000", 2024-09-15T00:11:59.768 INFO:teuthology.orchestra.run.smithi171.stdout: "modified": "2024-09-15T00:11:59.164055+0000", 2024-09-15T00:11:59.768 INFO:teuthology.orchestra.run.smithi171.stdout: "last_up_change": "2024-09-15T00:11:03.441129+0000", 2024-09-15T00:11:59.769 INFO:teuthology.orchestra.run.smithi171.stdout: "last_in_change": "2024-09-15T00:11:49.124399+0000", 2024-09-15T00:11:59.769 INFO:teuthology.orchestra.run.smithi171.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T00:11:59.769 INFO:teuthology.orchestra.run.smithi171.stdout: "flags_num": 5806080, 2024-09-15T00:11:59.769 INFO:teuthology.orchestra.run.smithi171.stdout: "flags_set": [ 2024-09-15T00:11:59.769 INFO:teuthology.orchestra.run.smithi171.stdout: "nodeep-scrub", 2024-09-15T00:11:59.769 INFO:teuthology.orchestra.run.smithi171.stdout: "noscrub", 2024-09-15T00:11:59.769 INFO:teuthology.orchestra.run.smithi171.stdout: "pglog_hardlimit", 2024-09-15T00:11:59.770 INFO:teuthology.orchestra.run.smithi171.stdout: "purged_snapdirs", 2024-09-15T00:11:59.770 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_deletes", 2024-09-15T00:11:59.770 INFO:teuthology.orchestra.run.smithi171.stdout: "sortbitwise" 2024-09-15T00:11:59.770 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:11:59.770 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_version": 5, 2024-09-15T00:11:59.770 INFO:teuthology.orchestra.run.smithi171.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T00:11:59.770 INFO:teuthology.orchestra.run.smithi171.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T00:11:59.770 INFO:teuthology.orchestra.run.smithi171.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T00:11:59.770 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_snapshot": "", 2024-09-15T00:11:59.770 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_max": 3, 2024-09-15T00:11:59.771 INFO:teuthology.orchestra.run.smithi171.stdout: "max_osd": 8, 2024-09-15T00:11:59.771 INFO:teuthology.orchestra.run.smithi171.stdout: "require_min_compat_client": "luminous", 2024-09-15T00:11:59.771 INFO:teuthology.orchestra.run.smithi171.stdout: "min_compat_client": "jewel", 2024-09-15T00:11:59.771 INFO:teuthology.orchestra.run.smithi171.stdout: "require_osd_release": "squid", 2024-09-15T00:11:59.771 INFO:teuthology.orchestra.run.smithi171.stdout: "allow_crimson": false, 2024-09-15T00:11:59.771 INFO:teuthology.orchestra.run.smithi171.stdout: "pools": [ 2024-09-15T00:11:59.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.771 INFO:teuthology.orchestra.run.smithi171.stdout: "pool": 1, 2024-09-15T00:11:59.771 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_name": ".mgr", 2024-09-15T00:11:59.771 INFO:teuthology.orchestra.run.smithi171.stdout: "create_time": "2024-09-15T00:11:04.326356+0000", 2024-09-15T00:11:59.771 INFO:teuthology.orchestra.run.smithi171.stdout: "flags": 1, 2024-09-15T00:11:59.771 INFO:teuthology.orchestra.run.smithi171.stdout: "flags_names": "hashpspool", 2024-09-15T00:11:59.771 INFO:teuthology.orchestra.run.smithi171.stdout: "type": 1, 2024-09-15T00:11:59.772 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 2, 2024-09-15T00:11:59.772 INFO:teuthology.orchestra.run.smithi171.stdout: "min_size": 1, 2024-09-15T00:11:59.772 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_rule": 0, 2024-09-15T00:11:59.772 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:11:59.772 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:11:59.772 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:11:59.772 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:11:59.772 INFO:teuthology.orchestra.run.smithi171.stdout: "is_stretch_pool": false, 2024-09-15T00:11:59.773 INFO:teuthology.orchestra.run.smithi171.stdout: "object_hash": 2, 2024-09-15T00:11:59.773 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:11:59.773 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num": 1, 2024-09-15T00:11:59.773 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num": 1, 2024-09-15T00:11:59.773 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num_target": 1, 2024-09-15T00:11:59.773 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_target": 1, 2024-09-15T00:11:59.773 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_pending": 1, 2024-09-15T00:11:59.773 INFO:teuthology.orchestra.run.smithi171.stdout: "last_pg_merge_meta": { 2024-09-15T00:11:59.773 INFO:teuthology.orchestra.run.smithi171.stdout: "source_pgid": "0.0", 2024-09-15T00:11:59.773 INFO:teuthology.orchestra.run.smithi171.stdout: "ready_epoch": 0, 2024-09-15T00:11:59.773 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_started": 0, 2024-09-15T00:11:59.773 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_clean": 0, 2024-09-15T00:11:59.774 INFO:teuthology.orchestra.run.smithi171.stdout: "source_version": "0'0", 2024-09-15T00:11:59.774 INFO:teuthology.orchestra.run.smithi171.stdout: "target_version": "0'0" 2024-09-15T00:11:59.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.774 INFO:teuthology.orchestra.run.smithi171.stdout: "last_change": "17", 2024-09-15T00:11:59.774 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend": "0", 2024-09-15T00:11:59.774 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:11:59.774 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:11:59.774 INFO:teuthology.orchestra.run.smithi171.stdout: "auid": 0, 2024-09-15T00:11:59.774 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:11:59.774 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_seq": 0, 2024-09-15T00:11:59.774 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_epoch": 0, 2024-09-15T00:11:59.774 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_snaps": [], 2024-09-15T00:11:59.774 INFO:teuthology.orchestra.run.smithi171.stdout: "removed_snaps": "[]", 2024-09-15T00:11:59.774 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_bytes": 0, 2024-09-15T00:11:59.774 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_objects": 0, 2024-09-15T00:11:59.775 INFO:teuthology.orchestra.run.smithi171.stdout: "tiers": [], 2024-09-15T00:11:59.775 INFO:teuthology.orchestra.run.smithi171.stdout: "tier_of": -1, 2024-09-15T00:11:59.775 INFO:teuthology.orchestra.run.smithi171.stdout: "read_tier": -1, 2024-09-15T00:11:59.775 INFO:teuthology.orchestra.run.smithi171.stdout: "write_tier": -1, 2024-09-15T00:11:59.775 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_mode": "none", 2024-09-15T00:11:59.775 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_bytes": 0, 2024-09-15T00:11:59.775 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_objects": 0, 2024-09-15T00:11:59.775 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:11:59.775 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:11:59.775 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:11:59.775 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_flush_age": 0, 2024-09-15T00:11:59.776 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_evict_age": 0, 2024-09-15T00:11:59.776 INFO:teuthology.orchestra.run.smithi171.stdout: "erasure_code_profile": "", 2024-09-15T00:11:59.776 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_params": { 2024-09-15T00:11:59.776 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "none" 2024-09-15T00:11:59.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.776 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_period": 0, 2024-09-15T00:11:59.776 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_count": 0, 2024-09-15T00:11:59.776 INFO:teuthology.orchestra.run.smithi171.stdout: "use_gmt_hitset": true, 2024-09-15T00:11:59.776 INFO:teuthology.orchestra.run.smithi171.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:11:59.776 INFO:teuthology.orchestra.run.smithi171.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:11:59.776 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:11:59.776 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:11:59.777 INFO:teuthology.orchestra.run.smithi171.stdout: "grade_table": [], 2024-09-15T00:11:59.777 INFO:teuthology.orchestra.run.smithi171.stdout: "stripe_width": 0, 2024-09-15T00:11:59.777 INFO:teuthology.orchestra.run.smithi171.stdout: "expected_num_objects": 0, 2024-09-15T00:11:59.777 INFO:teuthology.orchestra.run.smithi171.stdout: "fast_read": false, 2024-09-15T00:11:59.777 INFO:teuthology.orchestra.run.smithi171.stdout: "options": { 2024-09-15T00:11:59.777 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_max": 32, 2024-09-15T00:11:59.778 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_min": 1 2024-09-15T00:11:59.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.778 INFO:teuthology.orchestra.run.smithi171.stdout: "application_metadata": { 2024-09-15T00:11:59.778 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr": {} 2024-09-15T00:11:59.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.778 INFO:teuthology.orchestra.run.smithi171.stdout: "read_balance": { 2024-09-15T00:11:59.779 INFO:teuthology.orchestra.run.smithi171.stdout: "score_type": "Fair distribution", 2024-09-15T00:11:59.779 INFO:teuthology.orchestra.run.smithi171.stdout: "score_acting": 6.9000000953674316, 2024-09-15T00:11:59.779 INFO:teuthology.orchestra.run.smithi171.stdout: "score_stable": 6.9000000953674316, 2024-09-15T00:11:59.779 INFO:teuthology.orchestra.run.smithi171.stdout: "optimal_score": 0.28999999165534973, 2024-09-15T00:11:59.779 INFO:teuthology.orchestra.run.smithi171.stdout: "raw_score_acting": 2, 2024-09-15T00:11:59.779 INFO:teuthology.orchestra.run.smithi171.stdout: "raw_score_stable": 2, 2024-09-15T00:11:59.779 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity_weighted": 0.875, 2024-09-15T00:11:59.779 INFO:teuthology.orchestra.run.smithi171.stdout: "average_primary_affinity": 1, 2024-09-15T00:11:59.779 INFO:teuthology.orchestra.run.smithi171.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-15T00:11:59.779 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.780 INFO:teuthology.orchestra.run.smithi171.stdout: "pool": 2, 2024-09-15T00:11:59.780 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_name": "rbd", 2024-09-15T00:11:59.780 INFO:teuthology.orchestra.run.smithi171.stdout: "create_time": "2024-09-15T00:11:08.005967+0000", 2024-09-15T00:11:59.780 INFO:teuthology.orchestra.run.smithi171.stdout: "flags": 8193, 2024-09-15T00:11:59.780 INFO:teuthology.orchestra.run.smithi171.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:11:59.780 INFO:teuthology.orchestra.run.smithi171.stdout: "type": 1, 2024-09-15T00:11:59.780 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 2, 2024-09-15T00:11:59.780 INFO:teuthology.orchestra.run.smithi171.stdout: "min_size": 1, 2024-09-15T00:11:59.780 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_rule": 0, 2024-09-15T00:11:59.780 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:11:59.780 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:11:59.780 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:11:59.780 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:11:59.781 INFO:teuthology.orchestra.run.smithi171.stdout: "is_stretch_pool": false, 2024-09-15T00:11:59.781 INFO:teuthology.orchestra.run.smithi171.stdout: "object_hash": 2, 2024-09-15T00:11:59.781 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:11:59.781 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num": 8, 2024-09-15T00:11:59.781 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num": 8, 2024-09-15T00:11:59.781 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num_target": 8, 2024-09-15T00:11:59.781 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_target": 8, 2024-09-15T00:11:59.781 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_pending": 8, 2024-09-15T00:11:59.781 INFO:teuthology.orchestra.run.smithi171.stdout: "last_pg_merge_meta": { 2024-09-15T00:11:59.781 INFO:teuthology.orchestra.run.smithi171.stdout: "source_pgid": "0.0", 2024-09-15T00:11:59.781 INFO:teuthology.orchestra.run.smithi171.stdout: "ready_epoch": 0, 2024-09-15T00:11:59.782 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_started": 0, 2024-09-15T00:11:59.782 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_clean": 0, 2024-09-15T00:11:59.782 INFO:teuthology.orchestra.run.smithi171.stdout: "source_version": "0'0", 2024-09-15T00:11:59.782 INFO:teuthology.orchestra.run.smithi171.stdout: "target_version": "0'0" 2024-09-15T00:11:59.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.782 INFO:teuthology.orchestra.run.smithi171.stdout: "last_change": "21", 2024-09-15T00:11:59.782 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend": "0", 2024-09-15T00:11:59.782 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:11:59.782 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:11:59.782 INFO:teuthology.orchestra.run.smithi171.stdout: "auid": 0, 2024-09-15T00:11:59.783 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:11:59.783 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_seq": 2, 2024-09-15T00:11:59.783 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_epoch": 21, 2024-09-15T00:11:59.783 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_snaps": [], 2024-09-15T00:11:59.783 INFO:teuthology.orchestra.run.smithi171.stdout: "removed_snaps": "[]", 2024-09-15T00:11:59.783 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_bytes": 0, 2024-09-15T00:11:59.783 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_objects": 0, 2024-09-15T00:11:59.783 INFO:teuthology.orchestra.run.smithi171.stdout: "tiers": [], 2024-09-15T00:11:59.783 INFO:teuthology.orchestra.run.smithi171.stdout: "tier_of": -1, 2024-09-15T00:11:59.783 INFO:teuthology.orchestra.run.smithi171.stdout: "read_tier": -1, 2024-09-15T00:11:59.783 INFO:teuthology.orchestra.run.smithi171.stdout: "write_tier": -1, 2024-09-15T00:11:59.784 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_mode": "none", 2024-09-15T00:11:59.784 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_bytes": 0, 2024-09-15T00:11:59.784 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_objects": 0, 2024-09-15T00:11:59.784 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:11:59.784 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:11:59.784 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:11:59.784 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_flush_age": 0, 2024-09-15T00:11:59.784 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_evict_age": 0, 2024-09-15T00:11:59.784 INFO:teuthology.orchestra.run.smithi171.stdout: "erasure_code_profile": "", 2024-09-15T00:11:59.784 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_params": { 2024-09-15T00:11:59.784 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "none" 2024-09-15T00:11:59.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.784 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_period": 0, 2024-09-15T00:11:59.784 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_count": 0, 2024-09-15T00:11:59.785 INFO:teuthology.orchestra.run.smithi171.stdout: "use_gmt_hitset": true, 2024-09-15T00:11:59.785 INFO:teuthology.orchestra.run.smithi171.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:11:59.785 INFO:teuthology.orchestra.run.smithi171.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:11:59.785 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:11:59.785 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:11:59.785 INFO:teuthology.orchestra.run.smithi171.stdout: "grade_table": [], 2024-09-15T00:11:59.785 INFO:teuthology.orchestra.run.smithi171.stdout: "stripe_width": 0, 2024-09-15T00:11:59.785 INFO:teuthology.orchestra.run.smithi171.stdout: "expected_num_objects": 0, 2024-09-15T00:11:59.785 INFO:teuthology.orchestra.run.smithi171.stdout: "fast_read": false, 2024-09-15T00:11:59.785 INFO:teuthology.orchestra.run.smithi171.stdout: "options": {}, 2024-09-15T00:11:59.785 INFO:teuthology.orchestra.run.smithi171.stdout: "application_metadata": { 2024-09-15T00:11:59.786 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd": {} 2024-09-15T00:11:59.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.786 INFO:teuthology.orchestra.run.smithi171.stdout: "read_balance": { 2024-09-15T00:11:59.786 INFO:teuthology.orchestra.run.smithi171.stdout: "score_type": "Fair distribution", 2024-09-15T00:11:59.786 INFO:teuthology.orchestra.run.smithi171.stdout: "score_acting": 2.6500000953674316, 2024-09-15T00:11:59.786 INFO:teuthology.orchestra.run.smithi171.stdout: "score_stable": 2.6500000953674316, 2024-09-15T00:11:59.786 INFO:teuthology.orchestra.run.smithi171.stdout: "optimal_score": 0.70999997854232788, 2024-09-15T00:11:59.786 INFO:teuthology.orchestra.run.smithi171.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-15T00:11:59.786 INFO:teuthology.orchestra.run.smithi171.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-15T00:11:59.786 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity_weighted": 0.875, 2024-09-15T00:11:59.786 INFO:teuthology.orchestra.run.smithi171.stdout: "average_primary_affinity": 1, 2024-09-15T00:11:59.786 INFO:teuthology.orchestra.run.smithi171.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-15T00:11:59.786 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.787 INFO:teuthology.orchestra.run.smithi171.stdout: "pool": 3, 2024-09-15T00:11:59.787 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_name": "pool_client.0", 2024-09-15T00:11:59.787 INFO:teuthology.orchestra.run.smithi171.stdout: "create_time": "2024-09-15T00:11:17.676102+0000", 2024-09-15T00:11:59.787 INFO:teuthology.orchestra.run.smithi171.stdout: "flags": 8193, 2024-09-15T00:11:59.787 INFO:teuthology.orchestra.run.smithi171.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:11:59.787 INFO:teuthology.orchestra.run.smithi171.stdout: "type": 1, 2024-09-15T00:11:59.787 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 2, 2024-09-15T00:11:59.787 INFO:teuthology.orchestra.run.smithi171.stdout: "min_size": 1, 2024-09-15T00:11:59.787 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_rule": 0, 2024-09-15T00:11:59.787 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:11:59.787 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:11:59.787 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:11:59.787 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:11:59.787 INFO:teuthology.orchestra.run.smithi171.stdout: "is_stretch_pool": false, 2024-09-15T00:11:59.787 INFO:teuthology.orchestra.run.smithi171.stdout: "object_hash": 2, 2024-09-15T00:11:59.788 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:11:59.788 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num": 32, 2024-09-15T00:11:59.788 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num": 32, 2024-09-15T00:11:59.788 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num_target": 32, 2024-09-15T00:11:59.788 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_target": 32, 2024-09-15T00:11:59.788 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_pending": 32, 2024-09-15T00:11:59.788 INFO:teuthology.orchestra.run.smithi171.stdout: "last_pg_merge_meta": { 2024-09-15T00:11:59.788 INFO:teuthology.orchestra.run.smithi171.stdout: "source_pgid": "0.0", 2024-09-15T00:11:59.788 INFO:teuthology.orchestra.run.smithi171.stdout: "ready_epoch": 0, 2024-09-15T00:11:59.788 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_started": 0, 2024-09-15T00:11:59.788 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_clean": 0, 2024-09-15T00:11:59.788 INFO:teuthology.orchestra.run.smithi171.stdout: "source_version": "0'0", 2024-09-15T00:11:59.788 INFO:teuthology.orchestra.run.smithi171.stdout: "target_version": "0'0" 2024-09-15T00:11:59.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.789 INFO:teuthology.orchestra.run.smithi171.stdout: "last_change": "59", 2024-09-15T00:11:59.789 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend": "0", 2024-09-15T00:11:59.789 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:11:59.789 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:11:59.789 INFO:teuthology.orchestra.run.smithi171.stdout: "auid": 0, 2024-09-15T00:11:59.789 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:11:59.790 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_seq": 21, 2024-09-15T00:11:59.790 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_epoch": 59, 2024-09-15T00:11:59.790 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_snaps": [], 2024-09-15T00:11:59.790 INFO:teuthology.orchestra.run.smithi171.stdout: "removed_snaps": "[]", 2024-09-15T00:11:59.790 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_bytes": 0, 2024-09-15T00:11:59.790 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_objects": 0, 2024-09-15T00:11:59.790 INFO:teuthology.orchestra.run.smithi171.stdout: "tiers": [], 2024-09-15T00:11:59.790 INFO:teuthology.orchestra.run.smithi171.stdout: "tier_of": -1, 2024-09-15T00:11:59.790 INFO:teuthology.orchestra.run.smithi171.stdout: "read_tier": -1, 2024-09-15T00:11:59.790 INFO:teuthology.orchestra.run.smithi171.stdout: "write_tier": -1, 2024-09-15T00:11:59.790 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_mode": "none", 2024-09-15T00:11:59.790 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_bytes": 0, 2024-09-15T00:11:59.790 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_objects": 0, 2024-09-15T00:11:59.790 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:11:59.790 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:11:59.791 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:11:59.791 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_flush_age": 0, 2024-09-15T00:11:59.791 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_evict_age": 0, 2024-09-15T00:11:59.791 INFO:teuthology.orchestra.run.smithi171.stdout: "erasure_code_profile": "", 2024-09-15T00:11:59.791 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_params": { 2024-09-15T00:11:59.791 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "none" 2024-09-15T00:11:59.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.791 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_period": 0, 2024-09-15T00:11:59.791 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_count": 0, 2024-09-15T00:11:59.791 INFO:teuthology.orchestra.run.smithi171.stdout: "use_gmt_hitset": true, 2024-09-15T00:11:59.791 INFO:teuthology.orchestra.run.smithi171.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:11:59.791 INFO:teuthology.orchestra.run.smithi171.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:11:59.791 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:11:59.791 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:11:59.792 INFO:teuthology.orchestra.run.smithi171.stdout: "grade_table": [], 2024-09-15T00:11:59.792 INFO:teuthology.orchestra.run.smithi171.stdout: "stripe_width": 0, 2024-09-15T00:11:59.792 INFO:teuthology.orchestra.run.smithi171.stdout: "expected_num_objects": 0, 2024-09-15T00:11:59.792 INFO:teuthology.orchestra.run.smithi171.stdout: "fast_read": false, 2024-09-15T00:11:59.792 INFO:teuthology.orchestra.run.smithi171.stdout: "options": {}, 2024-09-15T00:11:59.792 INFO:teuthology.orchestra.run.smithi171.stdout: "application_metadata": { 2024-09-15T00:11:59.792 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd": {} 2024-09-15T00:11:59.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.792 INFO:teuthology.orchestra.run.smithi171.stdout: "read_balance": { 2024-09-15T00:11:59.792 INFO:teuthology.orchestra.run.smithi171.stdout: "score_type": "Fair distribution", 2024-09-15T00:11:59.792 INFO:teuthology.orchestra.run.smithi171.stdout: "score_acting": 2.2000000476837158, 2024-09-15T00:11:59.792 INFO:teuthology.orchestra.run.smithi171.stdout: "score_stable": 2.2000000476837158, 2024-09-15T00:11:59.792 INFO:teuthology.orchestra.run.smithi171.stdout: "optimal_score": 0.70999997854232788, 2024-09-15T00:11:59.792 INFO:teuthology.orchestra.run.smithi171.stdout: "raw_score_acting": 1.559999942779541, 2024-09-15T00:11:59.793 INFO:teuthology.orchestra.run.smithi171.stdout: "raw_score_stable": 1.559999942779541, 2024-09-15T00:11:59.793 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity_weighted": 0.875, 2024-09-15T00:11:59.793 INFO:teuthology.orchestra.run.smithi171.stdout: "average_primary_affinity": 1, 2024-09-15T00:11:59.793 INFO:teuthology.orchestra.run.smithi171.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-15T00:11:59.793 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.793 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.793 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:11:59.793 INFO:teuthology.orchestra.run.smithi171.stdout: "osds": [ 2024-09-15T00:11:59.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.794 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 0, 2024-09-15T00:11:59.794 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "7337111f-0e02-490d-976b-3814a03371f0", 2024-09-15T00:11:59.794 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:11:59.794 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 0, 2024-09-15T00:11:59.794 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 0, 2024-09-15T00:11:59.794 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 0, 2024-09-15T00:11:59.794 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:11:59.794 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:11:59.794 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:11:59.795 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 0, 2024-09-15T00:11:59.795 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:11:59.795 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:11:59.795 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:11:59.795 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.795 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.795 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6824", 2024-09-15T00:11:59.795 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:11:59.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.796 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.796 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6825", 2024-09-15T00:11:59.796 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:11:59.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.796 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.796 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:11:59.796 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.796 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.796 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6826", 2024-09-15T00:11:59.796 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:11:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6827", 2024-09-15T00:11:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:11:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:11:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6830", 2024-09-15T00:11:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:11:59.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.798 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.798 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6831", 2024-09-15T00:11:59.798 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:11:59.798 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.798 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.798 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:11:59.798 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.798 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.798 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6828", 2024-09-15T00:11:59.799 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:11:59.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.799 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.799 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6829", 2024-09-15T00:11:59.799 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:11:59.799 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.799 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.799 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.171:6825/2906515165", 2024-09-15T00:11:59.799 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.171:6827/2906515165", 2024-09-15T00:11:59.799 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.171:6831/2906515165", 2024-09-15T00:11:59.799 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.171:6829/2906515165", 2024-09-15T00:11:59.799 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:11:59.800 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:11:59.800 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:11:59.800 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.800 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 1, 2024-09-15T00:11:59.800 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "64412633-4dd4-4b8d-b3f1-ed3c664ddf48", 2024-09-15T00:11:59.800 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:11:59.800 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:11:59.800 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:11:59.800 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:11:59.800 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:11:59.800 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:11:59.801 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:11:59.801 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 52, 2024-09-15T00:11:59.801 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:11:59.801 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:11:59.804 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:11:59.804 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.805 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.805 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6816", 2024-09-15T00:11:59.805 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:11:59.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.805 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.805 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6817", 2024-09-15T00:11:59.805 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:11:59.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.805 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.805 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:11:59.805 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.806 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.806 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6818", 2024-09-15T00:11:59.806 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:11:59.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.806 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.806 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6819", 2024-09-15T00:11:59.806 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:11:59.806 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.806 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.807 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:11:59.807 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.807 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.807 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6822", 2024-09-15T00:11:59.807 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:11:59.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.807 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.807 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6823", 2024-09-15T00:11:59.808 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:11:59.808 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.808 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.808 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:11:59.808 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.808 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.808 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6820", 2024-09-15T00:11:59.808 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:11:59.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.808 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.809 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6821", 2024-09-15T00:11:59.809 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:11:59.809 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.809 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.809 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.171:6817/3039831960", 2024-09-15T00:11:59.809 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.171:6819/3039831960", 2024-09-15T00:11:59.809 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.171:6823/3039831960", 2024-09-15T00:11:59.809 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.171:6821/3039831960", 2024-09-15T00:11:59.809 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:11:59.810 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:11:59.810 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:11:59.810 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.810 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 2, 2024-09-15T00:11:59.811 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "0300adee-5ce3-43ce-997b-2c0e970edbc7", 2024-09-15T00:11:59.811 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:11:59.811 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 0, 2024-09-15T00:11:59.811 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 0, 2024-09-15T00:11:59.811 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:11:59.811 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:11:59.811 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:11:59.812 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:11:59.812 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 18, 2024-09-15T00:11:59.812 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:11:59.812 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:11:59.812 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:11:59.812 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.812 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.812 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6808", 2024-09-15T00:11:59.813 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:11:59.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.813 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.813 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6809", 2024-09-15T00:11:59.813 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:11:59.813 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.813 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.814 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:11:59.814 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.814 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.814 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6810", 2024-09-15T00:11:59.814 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:11:59.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.815 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.815 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6811", 2024-09-15T00:11:59.815 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:11:59.815 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.815 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.815 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:11:59.816 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.816 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.816 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6814", 2024-09-15T00:11:59.816 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:11:59.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.817 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.817 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6815", 2024-09-15T00:11:59.817 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:11:59.817 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.817 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.818 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:11:59.818 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.818 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.819 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6812", 2024-09-15T00:11:59.819 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:11:59.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.819 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.819 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6813", 2024-09-15T00:11:59.820 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:11:59.820 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.820 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.820 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.171:6809/3950855930", 2024-09-15T00:11:59.821 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.171:6811/3950855930", 2024-09-15T00:11:59.821 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.171:6815/3950855930", 2024-09-15T00:11:59.821 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.171:6813/3950855930", 2024-09-15T00:11:59.821 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:11:59.821 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:11:59.822 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:11:59.822 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.822 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 3, 2024-09-15T00:11:59.823 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "ec668801-9b8f-401a-829c-0b1f8d8f08fb", 2024-09-15T00:11:59.823 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:11:59.823 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:11:59.823 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:11:59.823 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:11:59.823 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:11:59.824 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:11:59.824 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:11:59.824 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 55, 2024-09-15T00:11:59.824 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:11:59.824 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:11:59.825 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:11:59.825 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.825 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.825 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6800", 2024-09-15T00:11:59.826 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:11:59.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.826 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.826 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6801", 2024-09-15T00:11:59.827 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:11:59.827 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.827 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.827 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:11:59.828 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.828 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.828 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6802", 2024-09-15T00:11:59.828 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:11:59.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.829 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.829 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6803", 2024-09-15T00:11:59.829 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:11:59.830 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.830 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.830 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:11:59.830 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.831 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.831 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6806", 2024-09-15T00:11:59.831 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:11:59.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.832 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.832 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6807", 2024-09-15T00:11:59.832 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:11:59.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.832 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.833 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:11:59.833 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.833 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.833 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6804", 2024-09-15T00:11:59.834 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:11:59.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.834 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.834 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6805", 2024-09-15T00:11:59.834 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:11:59.835 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.835 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.835 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.171:6801/1819098683", 2024-09-15T00:11:59.835 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.171:6803/1819098683", 2024-09-15T00:11:59.836 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.171:6807/1819098683", 2024-09-15T00:11:59.836 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.171:6805/1819098683", 2024-09-15T00:11:59.836 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:11:59.836 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:11:59.836 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:11:59.837 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.837 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 4, 2024-09-15T00:11:59.837 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "9cc48936-5644-44c6-84bb-06510e933e8d", 2024-09-15T00:11:59.838 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:11:59.838 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:11:59.838 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:11:59.838 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:11:59.838 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:11:59.839 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:11:59.839 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:11:59.839 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 55, 2024-09-15T00:11:59.839 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:11:59.839 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:11:59.840 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:11:59.840 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.840 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.840 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6800", 2024-09-15T00:11:59.841 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:11:59.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.841 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.841 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6801", 2024-09-15T00:11:59.842 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:11:59.842 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.842 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.842 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:11:59.843 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.843 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.843 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6802", 2024-09-15T00:11:59.843 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:11:59.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.844 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.844 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6803", 2024-09-15T00:11:59.844 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:11:59.845 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.845 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.845 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:11:59.846 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.846 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.846 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6806", 2024-09-15T00:11:59.846 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:11:59.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.847 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.847 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6807", 2024-09-15T00:11:59.848 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:11:59.848 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.848 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.848 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:11:59.849 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.849 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.849 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6804", 2024-09-15T00:11:59.849 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:11:59.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.850 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.851 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6805", 2024-09-15T00:11:59.851 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:11:59.852 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.852 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.852 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.184:6801/2801626173", 2024-09-15T00:11:59.853 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.184:6803/2801626173", 2024-09-15T00:11:59.853 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.184:6807/2801626173", 2024-09-15T00:11:59.853 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.184:6805/2801626173", 2024-09-15T00:11:59.853 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:11:59.853 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:11:59.853 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:11:59.854 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.854 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 5, 2024-09-15T00:11:59.854 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "f6c53297-9b00-476f-9be8-e76ff762b044", 2024-09-15T00:11:59.855 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:11:59.855 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:11:59.855 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:11:59.855 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:11:59.855 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:11:59.856 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:11:59.856 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:11:59.856 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 55, 2024-09-15T00:11:59.856 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:11:59.856 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:11:59.857 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:11:59.857 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.857 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.857 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6808", 2024-09-15T00:11:59.858 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:11:59.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.858 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.858 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6809", 2024-09-15T00:11:59.859 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:11:59.859 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.859 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.859 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:11:59.859 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.860 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.860 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6810", 2024-09-15T00:11:59.860 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:11:59.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.861 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.861 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6811", 2024-09-15T00:11:59.861 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:11:59.861 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.862 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.862 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:11:59.862 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.863 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.863 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6814", 2024-09-15T00:11:59.863 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:11:59.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.864 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.864 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6815", 2024-09-15T00:11:59.864 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:11:59.864 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.865 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.865 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:11:59.865 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.866 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.866 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6812", 2024-09-15T00:11:59.866 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:11:59.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.866 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.867 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6813", 2024-09-15T00:11:59.867 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:11:59.867 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.867 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.868 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.184:6809/3271066302", 2024-09-15T00:11:59.868 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.184:6811/3271066302", 2024-09-15T00:11:59.868 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.184:6815/3271066302", 2024-09-15T00:11:59.868 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.184:6813/3271066302", 2024-09-15T00:11:59.868 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:11:59.869 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:11:59.869 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:11:59.869 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.870 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 6, 2024-09-15T00:11:59.870 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "737f08a6-2223-46b3-b9fe-6ae18b067ec3", 2024-09-15T00:11:59.870 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:11:59.870 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:11:59.870 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:11:59.870 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:11:59.870 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:11:59.871 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:11:59.871 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:11:59.871 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 55, 2024-09-15T00:11:59.871 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:11:59.871 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:11:59.871 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:11:59.871 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.871 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.871 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6816", 2024-09-15T00:11:59.871 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:11:59.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.872 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.872 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6817", 2024-09-15T00:11:59.872 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:11:59.872 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.872 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.872 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:11:59.872 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.872 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.872 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6818", 2024-09-15T00:11:59.873 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:11:59.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.873 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.873 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6819", 2024-09-15T00:11:59.873 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:11:59.873 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.873 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.873 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:11:59.873 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.873 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.873 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6822", 2024-09-15T00:11:59.874 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:11:59.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.874 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.874 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6823", 2024-09-15T00:11:59.874 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:11:59.874 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.874 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.874 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:11:59.874 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.874 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.874 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6820", 2024-09-15T00:11:59.875 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:11:59.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.875 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.875 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6821", 2024-09-15T00:11:59.875 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:11:59.875 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.875 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.875 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.184:6817/1903994256", 2024-09-15T00:11:59.875 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.184:6819/1903994256", 2024-09-15T00:11:59.875 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.184:6823/1903994256", 2024-09-15T00:11:59.875 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.184:6821/1903994256", 2024-09-15T00:11:59.876 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:11:59.876 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:11:59.876 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:11:59.876 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.876 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 7, 2024-09-15T00:11:59.876 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "5a5fba64-9249-426d-814b-fd251e8aa910", 2024-09-15T00:11:59.876 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:11:59.876 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 0, 2024-09-15T00:11:59.876 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 0, 2024-09-15T00:11:59.876 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:11:59.876 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:11:59.876 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:11:59.877 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:11:59.877 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 31, 2024-09-15T00:11:59.877 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:11:59.877 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:11:59.877 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:11:59.877 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.877 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.877 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6824", 2024-09-15T00:11:59.877 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:11:59.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.877 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.877 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6825", 2024-09-15T00:11:59.878 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:11:59.878 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.878 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.878 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:11:59.878 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.878 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.878 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6826", 2024-09-15T00:11:59.878 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:11:59.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.878 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.878 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6827", 2024-09-15T00:11:59.879 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:11:59.879 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.879 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.879 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:11:59.879 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.879 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.879 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6830", 2024-09-15T00:11:59.879 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:11:59.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.879 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.880 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6831", 2024-09-15T00:11:59.880 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:11:59.880 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.880 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.880 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:11:59.880 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:11:59.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.880 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:11:59.880 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6828", 2024-09-15T00:11:59.880 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:11:59.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.880 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:11:59.881 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6829", 2024-09-15T00:11:59.881 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:11:59.881 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.881 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.881 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.184:6825/978565004", 2024-09-15T00:11:59.881 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.184:6827/978565004", 2024-09-15T00:11:59.881 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.184:6831/978565004", 2024-09-15T00:11:59.881 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.184:6829/978565004", 2024-09-15T00:11:59.881 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:11:59.881 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:11:59.881 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:11:59.881 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:11:59.881 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.882 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:11:59.882 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_xinfo": [ 2024-09-15T00:11:59.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.882 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 0, 2024-09-15T00:11:59.882 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:11:59.882 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:11:59.882 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:11:59.882 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:11:59.882 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 65536, 2024-09-15T00:11:59.882 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:59.070337+0000", 2024-09-15T00:11:59.882 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:11:59.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.883 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 1, 2024-09-15T00:11:59.883 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:11:59.883 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:11:59.883 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:11:59.883 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:11:59.883 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:11:59.883 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.884683+0000", 2024-09-15T00:11:59.883 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:11:59.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.883 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 2, 2024-09-15T00:11:59.883 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:11:59.883 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:11:59.883 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:11:59.883 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:11:59.884 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 65536, 2024-09-15T00:11:59.884 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.998882+0000", 2024-09-15T00:11:59.884 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:11:59.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.884 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 3, 2024-09-15T00:11:59.884 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:11:59.884 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:11:59.884 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:11:59.884 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:11:59.884 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:11:59.884 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.656630+0000", 2024-09-15T00:11:59.884 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:11:59.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.885 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 4, 2024-09-15T00:11:59.885 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:11:59.885 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:11:59.885 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:11:59.885 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:11:59.885 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:11:59.885 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.802744+0000", 2024-09-15T00:11:59.885 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:11:59.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.885 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 5, 2024-09-15T00:11:59.885 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:11:59.886 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:11:59.886 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:11:59.886 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:11:59.886 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:11:59.886 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.567865+0000", 2024-09-15T00:11:59.886 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:11:59.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.886 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 6, 2024-09-15T00:11:59.886 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:11:59.886 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:11:59.886 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:11:59.886 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:11:59.886 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:11:59.886 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.739964+0000", 2024-09-15T00:11:59.887 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:11:59.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:11:59.887 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 7, 2024-09-15T00:11:59.887 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:11:59.887 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:11:59.887 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:11:59.887 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:11:59.887 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 65536, 2024-09-15T00:11:59.887 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:59.565310+0000", 2024-09-15T00:11:59.887 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:11:59.887 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.887 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:11:59.888 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_upmap": [], 2024-09-15T00:11:59.888 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_upmap_items": [], 2024-09-15T00:11:59.888 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_upmap_primaries": [], 2024-09-15T00:11:59.888 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_temp": [], 2024-09-15T00:11:59.888 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_temp": [], 2024-09-15T00:11:59.888 INFO:teuthology.orchestra.run.smithi171.stdout: "blocklist": {}, 2024-09-15T00:11:59.888 INFO:teuthology.orchestra.run.smithi171.stdout: "range_blocklist": {}, 2024-09-15T00:11:59.888 INFO:teuthology.orchestra.run.smithi171.stdout: "erasure_code_profiles": { 2024-09-15T00:11:59.888 INFO:teuthology.orchestra.run.smithi171.stdout: "default": { 2024-09-15T00:11:59.888 INFO:teuthology.orchestra.run.smithi171.stdout: "crush-failure-domain": "osd", 2024-09-15T00:11:59.888 INFO:teuthology.orchestra.run.smithi171.stdout: "k": "2", 2024-09-15T00:11:59.888 INFO:teuthology.orchestra.run.smithi171.stdout: "m": "1", 2024-09-15T00:11:59.888 INFO:teuthology.orchestra.run.smithi171.stdout: "plugin": "jerasure", 2024-09-15T00:11:59.888 INFO:teuthology.orchestra.run.smithi171.stdout: "technique": "reed_sol_van" 2024-09-15T00:11:59.888 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:11:59.889 INFO:teuthology.orchestra.run.smithi171.stdout: "removed_snaps_queue": [], 2024-09-15T00:11:59.889 INFO:teuthology.orchestra.run.smithi171.stdout: "new_removed_snaps": [], 2024-09-15T00:11:59.889 INFO:teuthology.orchestra.run.smithi171.stdout: "new_purged_snaps": [], 2024-09-15T00:11:59.889 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_node_flags": {}, 2024-09-15T00:11:59.889 INFO:teuthology.orchestra.run.smithi171.stdout: "device_class_flags": {}, 2024-09-15T00:11:59.889 INFO:teuthology.orchestra.run.smithi171.stdout: "stretch_mode": { 2024-09-15T00:11:59.889 INFO:teuthology.orchestra.run.smithi171.stdout: "stretch_mode_enabled": false, 2024-09-15T00:11:59.889 INFO:teuthology.orchestra.run.smithi171.stdout: "stretch_bucket_count": 0, 2024-09-15T00:11:59.889 INFO:teuthology.orchestra.run.smithi171.stdout: "degraded_stretch_mode": 0, 2024-09-15T00:11:59.889 INFO:teuthology.orchestra.run.smithi171.stdout: "recovering_stretch_mode": 0, 2024-09-15T00:11:59.889 INFO:teuthology.orchestra.run.smithi171.stdout: "stretch_mode_bucket": 0 2024-09-15T00:11:59.889 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:11:59.889 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:11:59.890 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:11:59.891 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:59.892 INFO:tasks.thrashosds.thrasher:j is {'epoch': 59, 'fsid': 'bad2533c-9c09-4af4-ab93-147afbacb705', 'created': '2024-09-15T00:10:50.203030+0000', 'modified': '2024-09-15T00:11:59.164055+0000', 'last_up_change': '2024-09-15T00:11:03.441129+0000', 'last_in_change': '2024-09-15T00:11:49.124399+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T00:11:04.326356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T00:11:08.005967+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.7099999785423279, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-15T00:11:17.676102+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 59, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.200000047683716, 'score_stable': 2.200000047683716, 'optimal_score': 0.7099999785423279, 'raw_score_acting': 1.559999942779541, 'raw_score_stable': 1.559999942779541, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '7337111f-0e02-490d-976b-3814a03371f0', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6824', 'nonce': 2906515165}, {'type': 'v1', 'addr': '172.21.15.171:6825', 'nonce': 2906515165}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6826', 'nonce': 2906515165}, {'type': 'v1', 'addr': '172.21.15.171:6827', 'nonce': 2906515165}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6830', 'nonce': 2906515165}, {'type': 'v1', 'addr': '172.21.15.171:6831', 'nonce': 2906515165}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6828', 'nonce': 2906515165}, {'type': 'v1', 'addr': '172.21.15.171:6829', 'nonce': 2906515165}]}, 'public_addr': '172.21.15.171:6825/2906515165', 'cluster_addr': '172.21.15.171:6827/2906515165', 'heartbeat_back_addr': '172.21.15.171:6831/2906515165', 'heartbeat_front_addr': '172.21.15.171:6829/2906515165', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '64412633-4dd4-4b8d-b3f1-ed3c664ddf48', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6816', 'nonce': 3039831960}, {'type': 'v1', 'addr': '172.21.15.171:6817', 'nonce': 3039831960}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6818', 'nonce': 3039831960}, {'type': 'v1', 'addr': '172.21.15.171:6819', 'nonce': 3039831960}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6822', 'nonce': 3039831960}, {'type': 'v1', 'addr': '172.21.15.171:6823', 'nonce': 3039831960}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6820', 'nonce': 3039831960}, {'type': 'v1', 'addr': '172.21.15.171:6821', 'nonce': 3039831960}]}, 'public_addr': '172.21.15.171:6817/3039831960', 'cluster_addr': '172.21.15.171:6819/3039831960', 'heartbeat_back_addr': '172.21.15.171:6823/3039831960', 'heartbeat_front_addr': '172.21.15.171:6821/3039831960', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0300adee-5ce3-43ce-997b-2c0e970edbc7', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6808', 'nonce': 3950855930}, {'type': 'v1', 'addr': '172.21.15.171:6809', 'nonce': 3950855930}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6810', 'nonce': 3950855930}, {'type': 'v1', 'addr': '172.21.15.171:6811', 'nonce': 3950855930}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6814', 'nonce': 3950855930}, {'type': 'v1', 'addr': '172.21.15.171:6815', 'nonce': 3950855930}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6812', 'nonce': 3950855930}, {'type': 'v1', 'addr': '172.21.15.171:6813', 'nonce': 3950855930}]}, 'public_addr': '172.21.15.171:6809/3950855930', 'cluster_addr': '172.21.15.171:6811/3950855930', 'heartbeat_back_addr': '172.21.15.171:6815/3950855930', 'heartbeat_front_addr': '172.21.15.171:6813/3950855930', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ec668801-9b8f-401a-829c-0b1f8d8f08fb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6800', 'nonce': 1819098683}, {'type': 'v1', 'addr': '172.21.15.171:6801', 'nonce': 1819098683}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6802', 'nonce': 1819098683}, {'type': 'v1', 'addr': '172.21.15.171:6803', 'nonce': 1819098683}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6806', 'nonce': 1819098683}, {'type': 'v1', 'addr': '172.21.15.171:6807', 'nonce': 1819098683}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6804', 'nonce': 1819098683}, {'type': 'v1', 'addr': '172.21.15.171:6805', 'nonce': 1819098683}]}, 'public_addr': '172.21.15.171:6801/1819098683', 'cluster_addr': '172.21.15.171:6803/1819098683', 'heartbeat_back_addr': '172.21.15.171:6807/1819098683', 'heartbeat_front_addr': '172.21.15.171:6805/1819098683', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '9cc48936-5644-44c6-84bb-06510e933e8d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6800', 'nonce': 2801626173}, {'type': 'v1', 'addr': '172.21.15.184:6801', 'nonce': 2801626173}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6802', 'nonce': 2801626173}, {'type': 'v1', 'addr': '172.21.15.184:6803', 'nonce': 2801626173}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6806', 'nonce': 2801626173}, {'type': 'v1', 'addr': '172.21.15.184:6807', 'nonce': 2801626173}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6804', 'nonce': 2801626173}, {'type': 'v1', 'addr': '172.21.15.184:6805', 'nonce': 2801626173}]}, 'public_addr': '172.21.15.184:6801/2801626173', 'cluster_addr': '172.21.15.184:6803/2801626173', 'heartbeat_back_addr': '172.21.15.184:6807/2801626173', 'heartbeat_front_addr': '172.21.15.184:6805/2801626173', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f6c53297-9b00-476f-9be8-e76ff762b044', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6808', 'nonce': 3271066302}, {'type': 'v1', 'addr': '172.21.15.184:6809', 'nonce': 3271066302}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6810', 'nonce': 3271066302}, {'type': 'v1', 'addr': '172.21.15.184:6811', 'nonce': 3271066302}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6814', 'nonce': 3271066302}, {'type': 'v1', 'addr': '172.21.15.184:6815', 'nonce': 3271066302}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6812', 'nonce': 3271066302}, {'type': 'v1', 'addr': '172.21.15.184:6813', 'nonce': 3271066302}]}, 'public_addr': '172.21.15.184:6809/3271066302', 'cluster_addr': '172.21.15.184:6811/3271066302', 'heartbeat_back_addr': '172.21.15.184:6815/3271066302', 'heartbeat_front_addr': '172.21.15.184:6813/3271066302', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '737f08a6-2223-46b3-b9fe-6ae18b067ec3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6816', 'nonce': 1903994256}, {'type': 'v1', 'addr': '172.21.15.184:6817', 'nonce': 1903994256}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6818', 'nonce': 1903994256}, {'type': 'v1', 'addr': '172.21.15.184:6819', 'nonce': 1903994256}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6822', 'nonce': 1903994256}, {'type': 'v1', 'addr': '172.21.15.184:6823', 'nonce': 1903994256}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6820', 'nonce': 1903994256}, {'type': 'v1', 'addr': '172.21.15.184:6821', 'nonce': 1903994256}]}, 'public_addr': '172.21.15.184:6817/1903994256', 'cluster_addr': '172.21.15.184:6819/1903994256', 'heartbeat_back_addr': '172.21.15.184:6823/1903994256', 'heartbeat_front_addr': '172.21.15.184:6821/1903994256', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '5a5fba64-9249-426d-814b-fd251e8aa910', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6824', 'nonce': 978565004}, {'type': 'v1', 'addr': '172.21.15.184:6825', 'nonce': 978565004}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6826', 'nonce': 978565004}, {'type': 'v1', 'addr': '172.21.15.184:6827', 'nonce': 978565004}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6830', 'nonce': 978565004}, {'type': 'v1', 'addr': '172.21.15.184:6831', 'nonce': 978565004}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6828', 'nonce': 978565004}, {'type': 'v1', 'addr': '172.21.15.184:6829', 'nonce': 978565004}]}, 'public_addr': '172.21.15.184:6825/978565004', 'cluster_addr': '172.21.15.184:6827/978565004', 'heartbeat_back_addr': '172.21.15.184:6831/978565004', 'heartbeat_front_addr': '172.21.15.184:6829/978565004', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-15T00:10:59.070337+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.884683+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.998882+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.656630+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.802744+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.567865+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.739964+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-15T00:10:59.565310+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T00:11:59.892 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-15T00:11:59.893 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:11:59.893 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:59.893 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:11:59.893 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:11:59.894 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:59.894 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:59.895 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:59.895 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:11:59.896 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:59.889+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:11:59.896 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:11:59.991 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:11:59.990+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:00.036 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:00.037 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:00.045 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:12:00.092 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:00.090+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:00.149 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:00.149 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:00.157 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:12:00.177 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is unset 2024-09-15T00:12:00.193 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:00.191+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:00.239 INFO:teuthology.orchestra.run.smithi171.stdout:flattening image image_client.0-clone19 2024-09-15T00:12:00.289 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:00.289 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:00.293 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:00.292+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:00.299 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:00.314 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-15T00:12:00.395 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:00.393+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:00.431 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:00.431 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:00.440 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:00.450 INFO:teuthology.orchestra.run.smithi171.stdout:2765 punch from 0x2209d9c to 0x220b0d6, (0x133a bytes) 2024-09-15T00:12:00.454 INFO:teuthology.orchestra.run.smithi171.stdout:2768 write 0x26bc70a thru 0x26c4de3 (0x86da bytes) 2024-09-15T00:12:00.473 INFO:teuthology.orchestra.run.smithi171.stdout:2769 trunc from 0x30e7e4b to 0x2135412 2024-09-15T00:12:00.495 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:00.493+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:00.516 INFO:teuthology.orchestra.run.smithi171.stdout:2771 writesame 0xc30ae61 thru 0xc31a04b data_size 0x1(0xf1eb bytes) 2024-09-15T00:12:00.516 INFO:teuthology.orchestra.run.smithi171.stdout:2772 punch from 0x7166d5d to 0x71705dc, (0x987f bytes) 2024-09-15T00:12:00.516 INFO:teuthology.orchestra.run.smithi171.stdout:2773 write 0xbb8cef7 thru 0xbb8ff76 (0x3080 bytes) 2024-09-15T00:12:00.519 INFO:teuthology.orchestra.run.smithi171.stdout:2774 punch from 0x4fbfd98 to 0x4fc9103, (0x936b bytes) 2024-09-15T00:12:00.520 INFO:teuthology.orchestra.run.smithi171.stdout:2775 read 0x4f04552 thru 0x4f13f36 (0xf9e5 bytes) 2024-09-15T00:12:00.521 INFO:teuthology.orchestra.run.smithi171.stdout:2776 trunc from 0xc31a04c to 0xdcb19e5 2024-09-15T00:12:00.527 INFO:teuthology.orchestra.run.smithi171.stdout:2777 write 0x3ed506a thru 0x3ee39fc (0xe993 bytes) 2024-09-15T00:12:00.532 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:00.532 INFO:teuthology.orchestra.run.smithi171.stdout:2779 read 0xcb2a7b0 thru 0xcb2c35d (0x1bae bytes) 2024-09-15T00:12:00.533 INFO:teuthology.orchestra.run.smithi171.stdout:2780 read 0x24c72f9 thru 0x24c9209 (0x1f11 bytes) 2024-09-15T00:12:00.533 INFO:teuthology.orchestra.run.smithi171.stdout:2781 write 0xc48fe54 thru 0xc49cc3e (0xcdeb bytes) 2024-09-15T00:12:00.538 INFO:teuthology.orchestra.run.smithi171.stdout:2782 writesame 0x8d026de thru 0x8d0e43c data_size 0x1(0xbd5f bytes) 2024-09-15T00:12:00.561 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:00.562 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:00.573 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:00.591 INFO:teuthology.orchestra.run.smithi171.stdout:2783 read 0x39f33ea thru 0x39f835e (0x4f75 bytes) 2024-09-15T00:12:00.591 INFO:teuthology.orchestra.run.smithi171.stdout:2784 read 0x6c8ef49 thru 0x6c94e0c (0x5ec4 bytes) 2024-09-15T00:12:00.591 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:00.591 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:00.591 INFO:teuthology.orchestra.run.smithi171.stdout:2789 write 0xb554e2f thru 0xb55bfda (0x71ac bytes) 2024-09-15T00:12:00.596 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:00.594+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:00.610 INFO:teuthology.orchestra.run.smithi171.stdout:2790 trunc from 0xdcb19e5 to 0x129fcd0 2024-09-15T00:12:00.657 INFO:teuthology.orchestra.run.smithi171.stdout:2791 writesame 0xee63130 thru 0xee6b27f data_size 0x8(0x8150 bytes) 2024-09-15T00:12:00.658 INFO:teuthology.orchestra.run.smithi171.stdout:2793 read 0x17c9ada thru 0x17cab81 (0x10a8 bytes) 2024-09-15T00:12:00.675 INFO:teuthology.orchestra.run.smithi171.stdout:2794 trunc from 0xee6b280 to 0xaba215 2024-09-15T00:12:00.697 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:00.695+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:00.701 INFO:teuthology.orchestra.run.smithi171.stdout:2797 trunc from 0xaba215 to 0x259f254 2024-09-15T00:12:00.711 INFO:teuthology.orchestra.run.smithi171.stdout:2798 write 0x3045c66 thru 0x304bebf (0x625a bytes) 2024-09-15T00:12:00.719 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:00.719 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:00.729 INFO:teuthology.orchestra.run.smithi171.stdout:2799 write 0x9633dda thru 0x963c700 (0x8927 bytes) 2024-09-15T00:12:00.732 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:00.744 INFO:teuthology.orchestra.run.smithi171.stdout:2800 writesame 0xb98f7d7 thru 0xb990234 data_size 0x1(0xa5e bytes) 2024-09-15T00:12:00.745 INFO:teuthology.orchestra.run.smithi171.stdout:2801 write 0x8cf4f93 thru 0x8d04f92 (0x10000 bytes) 2024-09-15T00:12:00.757 INFO:teuthology.orchestra.run.smithi171.stdout:2802 write 0x1280ab4 thru 0x128ca7c (0xbfc9 bytes) 2024-09-15T00:12:00.763 INFO:teuthology.orchestra.run.smithi171.stdout:2803 write 0x1f8a69d thru 0x1f8b007 (0x96b bytes) 2024-09-15T00:12:00.771 INFO:teuthology.orchestra.run.smithi171.stdout:2804 trunc from 0xb990235 to 0xcb945c0 2024-09-15T00:12:00.776 INFO:teuthology.orchestra.run.smithi171.stdout:2806 read 0x4a5eef6 thru 0x4a64821 (0x592c bytes) 2024-09-15T00:12:00.776 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:00.776 INFO:teuthology.orchestra.run.smithi171.stdout:2808 write 0x723d3ce thru 0x724bded (0xea20 bytes) 2024-09-15T00:12:00.782 INFO:teuthology.orchestra.run.smithi171.stdout:2809 writesame 0x859636a thru 0x8599f50 data_size 0x1(0x3be7 bytes) 2024-09-15T00:12:00.794 INFO:teuthology.orchestra.run.smithi171.stdout:2812 punch from 0xb71534f to 0xb71e5c4, (0x9275 bytes) 2024-09-15T00:12:00.794 INFO:teuthology.orchestra.run.smithi171.stdout:2813 read 0x7fcd56e thru 0x7fd986d (0xc300 bytes) 2024-09-15T00:12:00.794 INFO:teuthology.orchestra.run.smithi171.stdout:2814 punch from 0x2051c7d to 0x205d579, (0xb8fc bytes) 2024-09-15T00:12:00.795 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:00.795 INFO:teuthology.orchestra.run.smithi171.stdout:2816 trunc from 0xcb945c0 to 0xdb11ce4 2024-09-15T00:12:00.797 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:00.796+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:00.801 INFO:teuthology.orchestra.run.smithi171.stdout:2817 write 0x66c521d thru 0x66d1de9 (0xcbcd bytes) 2024-09-15T00:12:00.807 INFO:teuthology.orchestra.run.smithi171.stdout:2820 writesame 0x7943ea5 thru 0x794e96a data_size 0x1(0xaac6 bytes) 2024-09-15T00:12:00.842 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:00.843 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:00.851 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:00.854 INFO:teuthology.orchestra.run.smithi171.stdout:2822 trunc from 0xdb11ce4 to 0xbcafc51 2024-09-15T00:12:00.859 INFO:teuthology.orchestra.run.smithi171.stdout:2823 write 0x9335c4f thru 0x9344ace (0xee80 bytes) 2024-09-15T00:12:00.864 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:00.864 INFO:teuthology.orchestra.run.smithi171.stdout:2826 read 0x57ebdc5 thru 0x57f785a (0xba96 bytes) 2024-09-15T00:12:00.871 INFO:teuthology.orchestra.run.smithi171.stdout:2827 write 0xd0b2082 thru 0xd0b4725 (0x26a4 bytes) 2024-09-15T00:12:00.882 INFO:teuthology.orchestra.run.smithi171.stdout:2829 trunc from 0xd0b4726 to 0x90b363e 2024-09-15T00:12:00.891 INFO:teuthology.orchestra.run.smithi171.stdout:2830 read 0x8457acf thru 0x845d619 (0x5b4b bytes) 2024-09-15T00:12:00.898 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:00.897+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:00.901 INFO:teuthology.orchestra.run.smithi171.stdout:2831 write 0xbfb5ff9 thru 0xbfc2138 (0xc140 bytes) 2024-09-15T00:12:00.907 INFO:teuthology.orchestra.run.smithi171.stdout:2832 trunc from 0xbfc2139 to 0xd64bc71 2024-09-15T00:12:00.912 INFO:teuthology.orchestra.run.smithi171.stdout:2833 read 0x80a3c58 thru 0x80a913a (0x54e3 bytes) 2024-09-15T00:12:00.923 INFO:teuthology.orchestra.run.smithi171.stdout:2834 trunc from 0xd64bc71 to 0x4bb952a 2024-09-15T00:12:00.952 INFO:teuthology.orchestra.run.smithi171.stdout:2835 write 0xee35acc thru 0xee40f3c (0xb471 bytes) 2024-09-15T00:12:00.957 INFO:teuthology.orchestra.run.smithi171.stdout:2837 trunc from 0xee40f3d to 0xd587bcb 2024-09-15T00:12:00.962 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:00.962 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:00.967 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:00.967 INFO:teuthology.orchestra.run.smithi171.stdout:2840 write 0xb053a98 thru 0xb05cdf9 (0x9362 bytes) 2024-09-15T00:12:00.972 INFO:teuthology.orchestra.run.smithi171.stdout:2841 punch from 0x122db26 to 0x122ec09, (0x10e3 bytes) 2024-09-15T00:12:00.972 INFO:teuthology.orchestra.run.smithi171.stdout:2842 punch from 0x3876799 to 0x387dd31, (0x7598 bytes) 2024-09-15T00:12:00.972 INFO:teuthology.orchestra.run.smithi171.stdout:2843 read 0x663905a thru 0x664897f (0xf926 bytes) 2024-09-15T00:12:00.973 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:00.984 INFO:teuthology.orchestra.run.smithi171.stdout:2844 trunc from 0xd587bcb to 0x344b3df 2024-09-15T00:12:01.000 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:00.999+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:01.012 INFO:teuthology.orchestra.run.smithi171.stdout:2845 writesame 0xcf2dc2b thru 0xcf31984 data_size 0x1(0x3d5a bytes) 2024-09-15T00:12:01.012 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:01.012 INFO:teuthology.orchestra.run.smithi171.stdout:2847 punch from 0x5be4a22 to 0x5be72aa, (0x2888 bytes) 2024-09-15T00:12:01.012 INFO:teuthology.orchestra.run.smithi171.stdout:2849 read 0x1f57e45 thru 0x1f65aec (0xdca8 bytes) 2024-09-15T00:12:01.012 INFO:teuthology.orchestra.run.smithi171.stdout:2851 write 0x5c801bb thru 0x5c8bc48 (0xba8e bytes) 2024-09-15T00:12:01.030 INFO:teuthology.orchestra.run.smithi171.stdout:2852 trunc from 0xcf31985 to 0x37e75a7 2024-09-15T00:12:01.040 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:01.048 INFO:teuthology.orchestra.run.smithi171.stdout:2856 trunc from 0x37e75a7 to 0xa5277d4 2024-09-15T00:12:01.053 INFO:teuthology.orchestra.run.smithi171.stdout:2857 writesame 0x3e3fe90 thru 0x3e42788 data_size 0x1(0x28f9 bytes) 2024-09-15T00:12:01.060 INFO:teuthology.orchestra.run.smithi171.stdout:2859 writesame 0x25ca34a thru 0x25d8923 data_size 0x3f(0xe5da bytes) 2024-09-15T00:12:01.065 INFO:teuthology.orchestra.run.smithi171.stdout:2862 write 0x44e7e71 thru 0x44f072e (0x88be bytes) 2024-09-15T00:12:01.071 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:01.072 INFO:teuthology.orchestra.run.smithi171.stdout:2864 trunc from 0xa5277d4 to 0x92f2bc6 2024-09-15T00:12:01.078 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:01.078 INFO:teuthology.orchestra.run.smithi171.stdout:2866 read 0x5ea145d thru 0x5ea9822 (0x83c6 bytes) 2024-09-15T00:12:01.078 INFO:teuthology.orchestra.run.smithi171.stdout:2870 writesame 0x2ba7ac0 thru 0x2bb6cc3 data_size 0x12(0xf204 bytes) 2024-09-15T00:12:01.095 INFO:teuthology.orchestra.run.smithi171.stdout:2871 write 0xdcb5be1 thru 0xdcb63dc (0x7fc bytes) 2024-09-15T00:12:01.100 INFO:teuthology.orchestra.run.smithi171.stdout:2872 punch from 0x98db586 to 0x98dc521, (0xf9b bytes) 2024-09-15T00:12:01.101 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:01.099+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:01.103 INFO:teuthology.orchestra.run.smithi171.stdout:2874 trunc from 0xdcb63dd to 0xb55c01d 2024-09-15T00:12:01.120 INFO:teuthology.orchestra.run.smithi171.stdout:2876 write 0xcd48810 thru 0xcd55705 (0xcef6 bytes) 2024-09-15T00:12:01.121 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:01.121 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:01.128 INFO:teuthology.orchestra.run.smithi171.stdout:2877 trunc from 0xcd55706 to 0xaa35d83 2024-09-15T00:12:01.130 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:01.137 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:01.137 INFO:teuthology.orchestra.run.smithi171.stdout:2879 write 0x93872a thru 0x944209 (0xbae0 bytes) 2024-09-15T00:12:01.151 INFO:teuthology.orchestra.run.smithi171.stdout:2880 writesame 0x96de1f5 thru 0x96ebd31 data_size 0x19(0xdb3d bytes) 2024-09-15T00:12:01.158 INFO:teuthology.orchestra.run.smithi171.stdout:2881 punch from 0x740567d to 0x7406124, (0xaa7 bytes) 2024-09-15T00:12:01.168 INFO:teuthology.orchestra.run.smithi171.stdout:2884 write 0xe08b3da thru 0xe08ced4 (0x1afb bytes) 2024-09-15T00:12:01.173 INFO:teuthology.orchestra.run.smithi171.stdout:2887 punch from 0x1d971d5 to 0x1da586d, (0xe698 bytes) 2024-09-15T00:12:01.174 INFO:teuthology.orchestra.run.smithi171.stdout:2888 write 0xd45a9de thru 0xd46758f (0xcbb2 bytes) 2024-09-15T00:12:01.181 INFO:teuthology.orchestra.run.smithi171.stdout:2890 writesame 0xc59bb54 thru 0xc5a37ea data_size 0x1(0x7c97 bytes) 2024-09-15T00:12:01.211 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:01.211 INFO:teuthology.orchestra.run.smithi171.stdout:2892 read 0x3301c57 thru 0x330d407 (0xb7b1 bytes) 2024-09-15T00:12:01.211 INFO:teuthology.orchestra.run.smithi171.stdout:2893 write 0x4039ce8 thru 0x4041457 (0x7770 bytes) 2024-09-15T00:12:01.214 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:01.213+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:01.218 INFO:teuthology.orchestra.run.smithi171.stdout:2894 trunc from 0xe08ced5 to 0xb876d71 2024-09-15T00:12:01.237 INFO:teuthology.orchestra.run.smithi171.stdout:2895 write 0xd5850ae thru 0xd58c2a4 (0x71f7 bytes) 2024-09-15T00:12:01.240 INFO:teuthology.orchestra.run.smithi171.stdout:2896 punch from 0xb08b4f5 to 0xb0927e7, (0x72f2 bytes) 2024-09-15T00:12:01.241 INFO:teuthology.orchestra.run.smithi171.stdout:2897 punch from 0x23b9bd1 to 0x23c506c, (0xb49b bytes) 2024-09-15T00:12:01.241 INFO:teuthology.orchestra.run.smithi171.stdout:2899 read 0x9fbdf94 thru 0x9fc13d3 (0x3440 bytes) 2024-09-15T00:12:01.241 INFO:teuthology.orchestra.run.smithi171.stdout:2900 write 0x94cf21c thru 0x94dd9c4 (0xe7a9 bytes) 2024-09-15T00:12:01.255 INFO:teuthology.orchestra.run.smithi171.stdout:2901 trunc from 0xd58c2a5 to 0x4c77cc0 2024-09-15T00:12:01.274 INFO:teuthology.orchestra.run.smithi171.stdout:2903 writesame 0x829a8d8 thru 0x82a419d data_size 0x5(0x98c6 bytes) 2024-09-15T00:12:01.275 INFO:teuthology.orchestra.run.smithi171.stdout:2904 read 0x46042aa thru 0x4605b78 (0x18cf bytes) 2024-09-15T00:12:01.282 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:01.282 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:01.287 INFO:teuthology.orchestra.run.smithi171.stdout:2906 writesame 0xdfd9ad1 thru 0xdfe93c5 data_size 0x17(0xf8f5 bytes) 2024-09-15T00:12:01.291 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:01.297 INFO:teuthology.orchestra.run.smithi171.stdout:2907 trunc from 0xdfe93c6 to 0x5258f85 2024-09-15T00:12:01.303 INFO:teuthology.orchestra.run.smithi171.stdout:2910 read 0x5092e6b thru 0x509b2e4 (0x847a bytes) 2024-09-15T00:12:01.303 INFO:teuthology.orchestra.run.smithi171.stdout:2911 punch from 0x6610e9 to 0x662a7f, (0x1996 bytes) 2024-09-15T00:12:01.303 INFO:teuthology.orchestra.run.smithi171.stdout:2913 clone 21 order 24 su 131072 sc 14 2024-09-15T00:12:01.332 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:01.330+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:01.403 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:01.401+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:01.437 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:01.437 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:01.442 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:01.504 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:01.502+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:01.545 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:01.545 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:01.554 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:01.605 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:01.603+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:01.698 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:01.698 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:01.705 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:01.704+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:01.706 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:01.806 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:01.805+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:01.849 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:01.849 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:01.858 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:01.898 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:12:01.941 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:01.940+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:01.960 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:01.961 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:01.969 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:02.013 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:02.011+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:02.020 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:02.020 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:02.036 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:02.036 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:02.052 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:12:02.052 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:12:02.068 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:02.068 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:02.072 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:02.072 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:02.081 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:02.084 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:02.084 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:02.100 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:02.100 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:02.108 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:02.107+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:02.116 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:02.117 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:02.133 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:02.133 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:02.193 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:12:02.231 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:02.231 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:02.232 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:02.232 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:02.232 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:02.236 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:02.235+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:02.241 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:02.244 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone20 from 0x5258f85 (overlap 0xaba215) to 0x0 2024-09-15T00:12:02.312 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:02.310+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:02.412 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:02.410+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:02.423 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:02.423 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:02.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:02.424 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:02.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.424 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:02.424 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:02.424 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.341540600999998, 2024-09-15T00:12:02.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:12:02.424 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.424 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.424 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.424 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.424 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.424 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.425 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:12:02.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.425 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:02.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:02.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:12:02.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:12:02.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:12:02.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:12:02.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:12:02.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:12:02.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:12:02.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:12:02.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:12:02.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:12:02.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:12:02.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:12:02.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:02.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:12:02.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:12:02.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:02.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:12:02.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:12:02.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:02.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:12:02.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:12:02.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:02.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:12:02.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:12:02.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:12:02.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:12:02.430 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.430 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.430 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.430 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:12:02.430 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:02.430 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.328800059000002, 2024-09-15T00:12:02.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:12:02.430 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.430 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.430 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:02.431 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:02.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:02.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:12:02.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:12:02.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:12:02.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:12:02.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:12:02.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:12:02.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:12:02.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:12:02.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:12:02.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:12:02.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:12:02.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:12:02.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:12:02.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:12:02.434 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.434 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.434 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.434 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:02.434 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:02.434 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.196444300000003, 2024-09-15T00:12:02.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:12:02.434 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.434 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.434 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:12:02.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:12:02.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:12:02.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:12:02.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:12:02.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:12:02.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:12:02.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:12:02.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:12:02.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:12:02.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:12:02.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:12:02.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:02.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:12:02.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:12:02.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:02.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:12:02.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:12:02.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.440 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:12:02.440 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:02.440 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.182982655000004, 2024-09-15T00:12:02.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:12:02.440 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.440 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.440 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:02.440 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:02.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:02.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:12:02.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:12:02.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:12:02.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:12:02.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:12:02.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:12:02.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:12:02.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:12:02.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:12:02.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:12:02.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:12:02.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:12:02.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:12:02.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:12:02.443 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.443 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.443 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.444 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:02.444 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:02.444 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 26.997981171999999, 2024-09-15T00:12:02.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:12:02.444 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.444 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.444 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.444 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.444 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.444 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.444 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:12:02.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.444 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:02.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:02.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:12:02.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:12:02.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:12:02.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:12:02.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:12:02.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:12:02.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:12:02.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:12:02.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:12:02.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:12:02.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:12:02.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:12:02.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:02.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:12:02.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:12:02.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:02.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:12:02.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:12:02.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:02.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:12:02.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:12:02.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:02.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:12:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:12:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:12:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:12:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:12:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 26.985104905, 2024-09-15T00:12:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:12:02.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:12:02.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:12:02.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:12:02.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:12:02.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:12:02.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:12:02.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:12:02.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:12:02.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:12:02.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:12:02.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:12:02.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:12:02.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:12:02.453 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.453 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.453 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.453 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:12:02.453 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:02.453 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 21.526589761, 2024-09-15T00:12:02.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:12:02.453 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.453 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.454 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.454 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.454 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.454 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.454 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:12:02.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.454 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:02.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:02.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:12:02.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:02.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:12:02.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:12:02.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:12:02.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:12:02.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:12:02.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:12:02.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:12:02.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:12:02.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:12:02.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:12:02.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:02.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:12:02.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:12:02.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:02.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:12:02.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:12:02.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:02.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:12:02.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:12:02.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:02.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:12:02.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:12:02.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:12:02.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:12:02.459 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.459 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.459 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.459 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:02.459 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:02.459 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.809079200999999, 2024-09-15T00:12:02.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:12:02.459 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.459 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.459 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:02.460 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.460 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.460 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.460 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:12:02.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.460 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:02.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:02.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:12:02.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:12:02.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:12:02.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:12:02.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:12:02.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:12:02.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:12:02.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:12:02.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:12:02.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:12:02.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:12:02.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:12:02.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:12:02.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:12:02.463 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.463 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.463 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.463 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:02.464 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:02.464 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.809060592000002, 2024-09-15T00:12:02.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:12:02.464 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.464 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.464 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:02.464 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.464 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.464 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.464 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:12:02.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.464 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:02.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:02.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:02.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:02.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:12:02.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:12:02.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:12:02.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:12:02.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:12:02.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:12:02.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:12:02.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:12:02.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:12:02.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:12:02.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:12:02.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:12:02.468 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.468 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.468 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.468 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:02.468 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:02.468 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.809035267999999, 2024-09-15T00:12:02.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:12:02.468 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.468 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.468 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:02.468 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.469 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.469 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.469 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:12:02.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.469 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:02.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:02.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:02.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:12:02.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:12:02.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:12:02.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:12:02.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:12:02.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:12:02.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:12:02.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:12:02.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:12:02.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:12:02.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:12:02.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:12:02.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:12:02.472 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.472 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.472 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.472 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:02.473 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:02.473 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.809016944, 2024-09-15T00:12:02.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:12:02.473 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.473 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.473 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:02.473 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.473 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.473 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.473 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:12:02.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.473 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:02.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:02.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:12:02.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:12:02.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:12:02.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:12:02.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:12:02.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:02.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:12:02.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:12:02.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:12:02.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:12:02.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:12:02.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:12:02.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:12:02.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:12:02.477 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.477 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.477 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.477 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:02.477 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:02.477 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.808834371, 2024-09-15T00:12:02.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:12:02.477 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.477 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.477 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:02.477 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.478 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.478 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.478 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:12:02.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.478 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:02.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:02.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:12:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:12:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:12:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:12:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:12:02.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:12:02.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:02.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:12:02.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:12:02.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:12:02.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:12:02.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:12:02.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:12:02.481 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.481 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.481 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.481 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:02.481 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:02.482 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.808805530000001, 2024-09-15T00:12:02.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:12:02.482 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.482 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.482 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:02.482 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.482 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.482 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.482 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:12:02.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.482 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:02.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:02.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:02.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:12:02.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:12:02.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:12:02.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:12:02.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:12:02.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:12:02.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:12:02.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:12:02.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:12:02.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:12:02.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:12:02.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:12:02.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:12:02.486 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.486 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.486 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.486 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:02.486 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:02.486 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.808783319, 2024-09-15T00:12:02.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:12:02.486 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.486 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.486 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:02.486 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.487 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.487 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.487 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:12:02.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.487 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:02.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:02.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:02.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:12:02.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:12:02.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:12:02.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:12:02.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:12:02.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:12:02.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:02.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:12:02.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:12:02.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:12:02.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:12:02.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:12:02.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:12:02.490 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.490 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.490 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.490 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:02.490 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:02.491 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.808762160000001, 2024-09-15T00:12:02.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:12:02.491 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.491 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.491 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:02.491 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.491 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.491 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.491 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:12:02.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.491 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:02.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:02.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:02.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:02.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:12:02.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:12:02.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:12:02.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:12:02.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:12:02.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:12:02.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:12:02.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:12:02.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:12:02.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:12:02.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:12:02.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:12:02.495 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.495 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.495 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.495 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:02.495 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:02.495 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.808732211999999, 2024-09-15T00:12:02.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:12:02.495 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.495 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.495 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:02.495 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.496 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.496 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.496 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:12:02.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.496 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:02.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:02.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:02.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:12:02.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:12:02.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:12:02.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:12:02.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:12:02.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:12:02.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:12:02.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:12:02.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:12:02.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:12:02.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:12:02.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:12:02.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:12:02.499 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.499 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.499 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.500 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:02.500 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:02.500 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.808710248000001, 2024-09-15T00:12:02.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:12:02.500 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.500 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.500 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:02.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.500 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:12:02.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.501 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:02.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:02.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:02.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:12:02.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:12:02.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:12:02.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:12:02.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:12:02.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:12:02.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:12:02.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:12:02.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:12:02.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:12:02.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:12:02.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:12:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:12:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.808692821000001, 2024-09-15T00:12:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:12:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.505 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.505 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:12:02.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.505 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:02.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:02.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:12:02.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:12:02.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:12:02.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:12:02.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:12:02.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:02.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:12:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:12:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:12:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:12:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:12:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:12:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:12:02.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:12:02.508 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.508 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.508 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.508 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:02.508 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:02.508 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.808671021999999, 2024-09-15T00:12:02.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:12:02.508 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.508 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.509 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:02.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.509 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:12:02.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.509 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:02.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:02.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:12:02.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:12:02.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:12:02.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:12:02.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:12:02.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:12:02.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:12:02.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:12:02.512 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.512 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.512 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.512 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:02.512 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:02.512 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.759442663000002, 2024-09-15T00:12:02.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:12:02.512 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:02.513 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:02.513 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.513 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:02.513 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:02.513 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:02.513 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:12:02.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.513 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:02.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:02.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:02.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:02.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:02.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:02.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:02.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:12:02.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:12:02.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:02.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:12:02.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:12:02.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:02.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:12:02.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:12:02.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:12:02.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:12:02.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:02.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:12:02.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:12:02.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:02.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:12:02.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:12:02.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:02.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:12:02.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:12:02.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:02.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:12:02.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:12:02.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:12:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:12:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:12:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:12:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:12:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:12:02.517 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.518 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.518 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.518 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:02.518 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:02.518 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:02.519 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:02.517+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:02.519 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:02.594 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-15T00:12:02.619 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:02.618+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:02.635 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:02.636 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:02.645 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:02.680 INFO:teuthology.orchestra.run.smithi171.stdout:2914 write 0xc57c6ae thru 0xc586326 (0x9c79 bytes) 2024-09-15T00:12:02.700 INFO:teuthology.orchestra.run.smithi171.stdout:2915 write 0x569bea1 thru 0x56a96dc (0xd83c bytes) 2024-09-15T00:12:02.720 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:02.719+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:02.737 INFO:teuthology.orchestra.run.smithi171.stdout:2916 write 0xd163f1a thru 0xd167572 (0x3659 bytes) 2024-09-15T00:12:02.756 INFO:teuthology.orchestra.run.smithi171.stdout:2917 write 0xcfbe3c7 thru 0xcfc0239 (0x1e73 bytes) 2024-09-15T00:12:02.779 INFO:teuthology.orchestra.run.smithi171.stdout:2918 trunc from 0xd167573 to 0x67d36eb 2024-09-15T00:12:02.814 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:02.814 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:02.822 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:02.820+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:02.824 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:02.829 INFO:teuthology.orchestra.run.smithi171.stdout:2920 writesame 0x10d81a7 thru 0x10dcd26 data_size 0x40(0x4b80 bytes) 2024-09-15T00:12:02.845 INFO:teuthology.orchestra.run.smithi171.stdout:2923 write 0xea0e752 thru 0xea11803 (0x30b2 bytes) 2024-09-15T00:12:02.848 INFO:teuthology.orchestra.run.smithi171.stdout:2924 write 0xe66856a thru 0xe676625 (0xe0bc bytes) 2024-09-15T00:12:02.861 INFO:teuthology.orchestra.run.smithi171.stdout:2925 trunc from 0xea11804 to 0x8a5851b 2024-09-15T00:12:02.873 INFO:teuthology.orchestra.run.smithi171.stdout:2926 read 0x67bbd thru 0x74598 (0xc9dc bytes) 2024-09-15T00:12:02.885 INFO:teuthology.orchestra.run.smithi171.stdout:2928 writesame 0xc6e0b32 thru 0xc6ec37c data_size 0x1(0xb84b bytes) 2024-09-15T00:12:02.887 INFO:teuthology.orchestra.run.smithi171.stdout:2931 trunc from 0xc6ec37d to 0xd2cf1df 2024-09-15T00:12:02.893 INFO:teuthology.orchestra.run.smithi171.stdout:2934 read 0xce31def thru 0xce3f173 (0xd385 bytes) 2024-09-15T00:12:02.893 INFO:teuthology.orchestra.run.smithi171.stdout:2935 trunc from 0xd2cf1df to 0xd449a62 2024-09-15T00:12:02.899 INFO:teuthology.orchestra.run.smithi171.stdout:2938 clone 22 order 21 su 524288 sc 3 2024-09-15T00:12:02.923 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:02.921+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:02.945 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-15T00:12:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:12:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.407128125, 2024-09-15T00:12:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0035574980000000001, 2024-09-15T00:12:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:12:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535369+0000", 2024-09-15T00:12:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535370+0000", 2024-09-15T00:12:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0999999999999999e-07 2024-09-15T00:12:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535438+0000", 2024-09-15T00:12:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8199000000000002e-05 2024-09-15T00:12:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535446+0000", 2024-09-15T00:12:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1640000000000001e-06 2024-09-15T00:12:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535450+0000", 2024-09-15T00:12:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1520000000000002e-06 2024-09-15T00:12:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535491+0000", 2024-09-15T00:12:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0969e-05 2024-09-15T00:12:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.535510+0000", 2024-09-15T00:12:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9031000000000001e-05 2024-09-15T00:12:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.538893+0000", 2024-09-15T00:12:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033823870000000002 2024-09-15T00:12:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.538927+0000", 2024-09-15T00:12:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4085999999999999e-05 2024-09-15T00:12:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:70 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-09-15T00:12:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:12:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.406387905000003, 2024-09-15T00:12:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057871490000000001, 2024-09-15T00:12:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:12:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:12:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536110+0000", 2024-09-15T00:12:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-15T00:12:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536115+0000", 2024-09-15T00:12:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8960000000000002e-06 2024-09-15T00:12:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536123+0000", 2024-09-15T00:12:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3979999999999999e-06 2024-09-15T00:12:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536128+0000", 2024-09-15T00:12:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5569999999999997e-06 2024-09-15T00:12:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537011+0000", 2024-09-15T00:12:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088276299999999995 2024-09-15T00:12:02.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:02.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537034+0000", 2024-09-15T00:12:02.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3448999999999999e-05 2024-09-15T00:12:02.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:02.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541867+0000", 2024-09-15T00:12:02.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048332949999999996 2024-09-15T00:12:02.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:02.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541896+0000", 2024-09-15T00:12:02.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9179999999999998e-05 2024-09-15T00:12:02.954 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.954 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:02.954 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.954 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-15T00:12:02.954 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:12:02.954 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.378769452, 2024-09-15T00:12:02.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027567730000000001, 2024-09-15T00:12:02.954 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:02.954 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:02.954 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.955 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:02.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:02.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:12:02.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:02.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:12:02.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:02.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563728+0000", 2024-09-15T00:12:02.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5599999999999995e-07 2024-09-15T00:12:02.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:02.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563787+0000", 2024-09-15T00:12:02.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9092000000000003e-05 2024-09-15T00:12:02.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:02.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563796+0000", 2024-09-15T00:12:02.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0059999999999998e-06 2024-09-15T00:12:02.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563801+0000", 2024-09-15T00:12:02.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3290000000000004e-06 2024-09-15T00:12:02.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:02.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563830+0000", 2024-09-15T00:12:02.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9369999999999998e-05 2024-09-15T00:12:02.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:02.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.563842+0000", 2024-09-15T00:12:02.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1646e-05 2024-09-15T00:12:02.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:02.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.566459+0000", 2024-09-15T00:12:02.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026174549999999999 2024-09-15T00:12:02.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:02.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.566485+0000", 2024-09-15T00:12:02.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5318999999999999e-05 2024-09-15T00:12:02.958 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.958 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:02.958 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.959 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:106 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-09-15T00:12:02.959 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:12:02.959 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.245866989, 2024-09-15T00:12:02.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029384250000000002, 2024-09-15T00:12:02.959 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:02.959 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:02.959 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.959 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:02.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:02.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:12:02.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:02.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696630+0000", 2024-09-15T00:12:02.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:02.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696631+0000", 2024-09-15T00:12:02.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4299999999999998e-07 2024-09-15T00:12:02.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:02.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696692+0000", 2024-09-15T00:12:02.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1229999999999995e-05 2024-09-15T00:12:02.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:02.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696699+0000", 2024-09-15T00:12:02.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5399999999999998e-06 2024-09-15T00:12:02.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696705+0000", 2024-09-15T00:12:02.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5770000000000001e-06 2024-09-15T00:12:02.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:02.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696732+0000", 2024-09-15T00:12:02.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7022e-05 2024-09-15T00:12:02.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:02.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.696743+0000", 2024-09-15T00:12:02.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1426999999999999e-05 2024-09-15T00:12:02.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:02.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699542+0000", 2024-09-15T00:12:02.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002798521 2024-09-15T00:12:02.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:02.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699569+0000", 2024-09-15T00:12:02.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6665e-05 2024-09-15T00:12:02.963 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.963 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:02.963 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.963 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:107 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-09-15T00:12:02.963 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:12:02.963 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.244971559, 2024-09-15T00:12:02.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048949229999999998, 2024-09-15T00:12:02.963 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:02.963 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:02.963 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.963 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:02.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:02.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:12:02.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:02.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:12:02.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:02.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697526+0000", 2024-09-15T00:12:02.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5799999999999999e-07 2024-09-15T00:12:02.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:02.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697531+0000", 2024-09-15T00:12:02.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3100000000000002e-06 2024-09-15T00:12:02.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:02.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697539+0000", 2024-09-15T00:12:02.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6410000000000008e-06 2024-09-15T00:12:02.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697543+0000", 2024-09-15T00:12:02.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1060000000000002e-06 2024-09-15T00:12:02.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:02.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697573+0000", 2024-09-15T00:12:02.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9978e-05 2024-09-15T00:12:02.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:02.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.697585+0000", 2024-09-15T00:12:02.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1279e-05 2024-09-15T00:12:02.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:02.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702398+0000", 2024-09-15T00:12:02.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004812981 2024-09-15T00:12:02.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:02.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702421+0000", 2024-09-15T00:12:02.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3070000000000001e-05 2024-09-15T00:12:02.967 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.967 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:02.967 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.967 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:108 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'66)", 2024-09-15T00:12:02.968 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:12:02.968 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.244078068999997, 2024-09-15T00:12:02.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040179439999999999, 2024-09-15T00:12:02.968 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:02.968 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:02.968 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.968 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:02.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:02.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:12:02.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:02.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698419+0000", 2024-09-15T00:12:02.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:02.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698420+0000", 2024-09-15T00:12:02.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0900000000000002e-07 2024-09-15T00:12:02.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:02.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698423+0000", 2024-09-15T00:12:02.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8319999999999999e-06 2024-09-15T00:12:02.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:02.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698431+0000", 2024-09-15T00:12:02.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8949999999999998e-06 2024-09-15T00:12:02.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698435+0000", 2024-09-15T00:12:02.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5939999999999998e-06 2024-09-15T00:12:02.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:02.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698469+0000", 2024-09-15T00:12:02.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4258000000000003e-05 2024-09-15T00:12:02.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:02.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698481+0000", 2024-09-15T00:12:02.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1763000000000001e-05 2024-09-15T00:12:02.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:02.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702422+0000", 2024-09-15T00:12:02.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003940603 2024-09-15T00:12:02.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:02.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702437+0000", 2024-09-15T00:12:02.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5489999999999999e-05 2024-09-15T00:12:02.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.972 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:02.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.972 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:109 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'66)", 2024-09-15T00:12:02.972 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:12:02.972 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.243547632999999, 2024-09-15T00:12:02.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038555479999999999, 2024-09-15T00:12:02.972 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:02.972 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:02.972 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.973 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:02.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:02.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:12:02.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:02.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:12:02.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:02.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.698950+0000", 2024-09-15T00:12:02.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.13e-07 2024-09-15T00:12:02.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:02.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699018+0000", 2024-09-15T00:12:02.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7540999999999996e-05 2024-09-15T00:12:02.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:02.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699024+0000", 2024-09-15T00:12:02.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4869999999999999e-06 2024-09-15T00:12:02.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699044+0000", 2024-09-15T00:12:02.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9400999999999999e-05 2024-09-15T00:12:02.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:02.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699104+0000", 2024-09-15T00:12:02.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0331999999999998e-05 2024-09-15T00:12:02.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:02.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699117+0000", 2024-09-15T00:12:02.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2789000000000001e-05 2024-09-15T00:12:02.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:02.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702777+0000", 2024-09-15T00:12:02.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0036604900000000002 2024-09-15T00:12:02.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:02.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702805+0000", 2024-09-15T00:12:02.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7994999999999999e-05 2024-09-15T00:12:02.976 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.976 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:02.976 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.977 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:110 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'72, mlcod=16'66)", 2024-09-15T00:12:02.977 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:12:02.977 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.242788099999999, 2024-09-15T00:12:02.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031108189999999999, 2024-09-15T00:12:02.977 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:02.977 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:02.977 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.977 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:02.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:02.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:12:02.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:02.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699709+0000", 2024-09-15T00:12:02.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:02.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699710+0000", 2024-09-15T00:12:02.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:12:02.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:02.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699714+0000", 2024-09-15T00:12:02.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0010000000000003e-06 2024-09-15T00:12:02.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:02.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699722+0000", 2024-09-15T00:12:02.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4840000000000004e-06 2024-09-15T00:12:02.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699726+0000", 2024-09-15T00:12:02.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:12:02.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:02.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699755+0000", 2024-09-15T00:12:02.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8433999999999999e-05 2024-09-15T00:12:02.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:02.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.699775+0000", 2024-09-15T00:12:02.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0017000000000002e-05 2024-09-15T00:12:02.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:02.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702807+0000", 2024-09-15T00:12:02.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030320239999999999 2024-09-15T00:12:02.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:02.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.702820+0000", 2024-09-15T00:12:02.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3169999999999999e-05 2024-09-15T00:12:02.981 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.981 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:02.981 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.981 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:120 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'82, mlcod=16'78)", 2024-09-15T00:12:02.981 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:12:02.981 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.236884754000002, 2024-09-15T00:12:02.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028965829999999999, 2024-09-15T00:12:02.981 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:02.981 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:02.981 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.981 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:02.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:02.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:12:02.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:02.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705612+0000", 2024-09-15T00:12:02.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:02.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705613+0000", 2024-09-15T00:12:02.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1800000000000001e-07 2024-09-15T00:12:02.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:02.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705684+0000", 2024-09-15T00:12:02.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0903999999999997e-05 2024-09-15T00:12:02.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:02.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705691+0000", 2024-09-15T00:12:02.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4290000000000004e-06 2024-09-15T00:12:02.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705695+0000", 2024-09-15T00:12:02.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4790000000000001e-06 2024-09-15T00:12:02.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:02.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705729+0000", 2024-09-15T00:12:02.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4415999999999997e-05 2024-09-15T00:12:02.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:02.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.705741+0000", 2024-09-15T00:12:02.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1443999999999999e-05 2024-09-15T00:12:02.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:02.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.708476+0000", 2024-09-15T00:12:02.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027356630000000002 2024-09-15T00:12:02.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:02.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.708509+0000", 2024-09-15T00:12:02.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2830000000000002e-05 2024-09-15T00:12:02.985 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.985 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:02.985 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.985 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-15T00:12:02.985 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:12:02.986 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.236098288999997, 2024-09-15T00:12:02.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047824809999999999, 2024-09-15T00:12:02.986 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:02.986 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:02.986 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.986 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:02.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:02.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:12:02.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:02.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:12:02.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:02.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706399+0000", 2024-09-15T00:12:02.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:12:02.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:02.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706471+0000", 2024-09-15T00:12:02.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1496999999999997e-05 2024-09-15T00:12:02.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:02.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706478+0000", 2024-09-15T00:12:02.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7460000000000003e-06 2024-09-15T00:12:02.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706481+0000", 2024-09-15T00:12:02.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.258e-06 2024-09-15T00:12:02.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:02.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706513+0000", 2024-09-15T00:12:02.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2567000000000002e-05 2024-09-15T00:12:02.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:02.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.706524+0000", 2024-09-15T00:12:02.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1028e-05 2024-09-15T00:12:02.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:02.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.711150+0000", 2024-09-15T00:12:02.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046259450000000002 2024-09-15T00:12:02.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:02.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.711181+0000", 2024-09-15T00:12:02.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1041000000000003e-05 2024-09-15T00:12:02.989 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.989 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:02.989 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:136 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.882661355000003, 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031270970000000001, 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:12:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059836+0000", 2024-09-15T00:12:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3000000000000002e-07 2024-09-15T00:12:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059839+0000", 2024-09-15T00:12:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3050000000000001e-06 2024-09-15T00:12:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:02.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059847+0000", 2024-09-15T00:12:02.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.622e-06 2024-09-15T00:12:02.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059851+0000", 2024-09-15T00:12:02.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8120000000000001e-06 2024-09-15T00:12:02.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:02.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059884+0000", 2024-09-15T00:12:02.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2620999999999998e-05 2024-09-15T00:12:02.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:02.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.059894+0000", 2024-09-15T00:12:02.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0924000000000001e-05 2024-09-15T00:12:02.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:02.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062935+0000", 2024-09-15T00:12:02.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030406830000000002 2024-09-15T00:12:02.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:02.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062963+0000", 2024-09-15T00:12:02.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7800000000000001e-05 2024-09-15T00:12:02.994 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.994 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:02.994 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.994 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:139 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'92)", 2024-09-15T00:12:02.994 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:12:02.994 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.880874132000002, 2024-09-15T00:12:02.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040690240000000001, 2024-09-15T00:12:02.994 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:02.994 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:02.994 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.994 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:02.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061623+0000", 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:02.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061631+0000", 2024-09-15T00:12:02.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.17e-06 2024-09-15T00:12:02.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:02.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061636+0000", 2024-09-15T00:12:02.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7679999999999997e-06 2024-09-15T00:12:02.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:02.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.061639+0000", 2024-09-15T00:12:02.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9069999999999999e-06 2024-09-15T00:12:02.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:02.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062733+0000", 2024-09-15T00:12:02.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010937119999999999 2024-09-15T00:12:02.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:02.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062744+0000", 2024-09-15T00:12:02.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1439e-05 2024-09-15T00:12:02.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:02.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065658+0000", 2024-09-15T00:12:02.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029134970000000001 2024-09-15T00:12:02.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:02.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065692+0000", 2024-09-15T00:12:02.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4147999999999999e-05 2024-09-15T00:12:02.998 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.998 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:02.998 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:02.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.998 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'92)", 2024-09-15T00:12:02.998 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:12:02.998 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.880485069999999, 2024-09-15T00:12:02.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003986608, 2024-09-15T00:12:02.998 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:02.998 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:02.998 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:02.999 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:02.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:02.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:12:02.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:02.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:12:02.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:02.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:02.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:02.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:02.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062012+0000", 2024-09-15T00:12:02.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7000000000000001e-07 2024-09-15T00:12:03.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062075+0000", 2024-09-15T00:12:03.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2638000000000006e-05 2024-09-15T00:12:03.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062083+0000", 2024-09-15T00:12:03.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4330000000000004e-06 2024-09-15T00:12:03.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.062087+0000", 2024-09-15T00:12:03.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3799999999999998e-06 2024-09-15T00:12:03.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.063001+0000", 2024-09-15T00:12:03.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091419500000000002 2024-09-15T00:12:03.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.063012+0000", 2024-09-15T00:12:03.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1117999999999999e-05 2024-09-15T00:12:03.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065977+0000", 2024-09-15T00:12:03.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029650840000000002 2024-09-15T00:12:03.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.065999+0000", 2024-09-15T00:12:03.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1489999999999999e-05 2024-09-15T00:12:03.002 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.002 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.002 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.002 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:154 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'108, mlcod=16'105)", 2024-09-15T00:12:03.002 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:12:03.003 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.418237445000003, 2024-09-15T00:12:03.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053187770000000002, 2024-09-15T00:12:03.003 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.003 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.003 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.003 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:12:03.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:12:03.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:12:03.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:12:03.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524264+0000", 2024-09-15T00:12:03.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.738e-06 2024-09-15T00:12:03.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524270+0000", 2024-09-15T00:12:03.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9379999999999997e-06 2024-09-15T00:12:03.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524273+0000", 2024-09-15T00:12:03.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6370000000000001e-06 2024-09-15T00:12:03.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524310+0000", 2024-09-15T00:12:03.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6266999999999997e-05 2024-09-15T00:12:03.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524322+0000", 2024-09-15T00:12:03.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2741999999999999e-05 2024-09-15T00:12:03.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.529547+0000", 2024-09-15T00:12:03.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005225001 2024-09-15T00:12:03.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.529579+0000", 2024-09-15T00:12:03.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1065999999999997e-05 2024-09-15T00:12:03.006 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.007 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.007 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.007 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-15T00:12:03.007 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:03.007 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.417799807000002, 2024-09-15T00:12:03.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077021529999999998, 2024-09-15T00:12:03.007 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.007 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.007 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.007 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:03.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:03.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524698+0000", 2024-09-15T00:12:03.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6300000000000001e-07 2024-09-15T00:12:03.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524778+0000", 2024-09-15T00:12:03.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0598000000000003e-05 2024-09-15T00:12:03.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524785+0000", 2024-09-15T00:12:03.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6359999999999997e-06 2024-09-15T00:12:03.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524788+0000", 2024-09-15T00:12:03.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2540000000000001e-06 2024-09-15T00:12:03.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524817+0000", 2024-09-15T00:12:03.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8784000000000001e-05 2024-09-15T00:12:03.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524828+0000", 2024-09-15T00:12:03.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-15T00:12:03.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532360+0000", 2024-09-15T00:12:03.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007531757 2024-09-15T00:12:03.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532400+0000", 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9835000000000001e-05 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=16'106)", 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.416857653999998, 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067923050000000002, 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.011 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:03.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:03.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:03.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-15T00:12:03.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525644+0000", 2024-09-15T00:12:03.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.709e-06 2024-09-15T00:12:03.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525651+0000", 2024-09-15T00:12:03.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7970000000000003e-06 2024-09-15T00:12:03.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525654+0000", 2024-09-15T00:12:03.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.433e-06 2024-09-15T00:12:03.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525685+0000", 2024-09-15T00:12:03.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0970000000000003e-05 2024-09-15T00:12:03.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525697+0000", 2024-09-15T00:12:03.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1759e-05 2024-09-15T00:12:03.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532401+0000", 2024-09-15T00:12:03.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006704425 2024-09-15T00:12:03.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532432+0000", 2024-09-15T00:12:03.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0709e-05 2024-09-15T00:12:03.015 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.015 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.015 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.015 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'106)", 2024-09-15T00:12:03.015 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:03.016 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.416395399999999, 2024-09-15T00:12:03.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063539190000000004, 2024-09-15T00:12:03.016 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.016 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.016 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.016 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:03.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:03.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:03.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:12:03.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526107+0000", 2024-09-15T00:12:03.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:12:03.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526117+0000", 2024-09-15T00:12:03.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7750000000000004e-06 2024-09-15T00:12:03.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526121+0000", 2024-09-15T00:12:03.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:12:03.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526163+0000", 2024-09-15T00:12:03.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1360999999999999e-05 2024-09-15T00:12:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526175+0000", 2024-09-15T00:12:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2729e-05 2024-09-15T00:12:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532433+0000", 2024-09-15T00:12:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062576999999999997 2024-09-15T00:12:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532456+0000", 2024-09-15T00:12:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2773e-05 2024-09-15T00:12:03.020 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.020 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.020 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.020 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'106)", 2024-09-15T00:12:03.020 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:12:03.020 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.415925997999999, 2024-09-15T00:12:03.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059082689999999998, 2024-09-15T00:12:03.020 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.020 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.020 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.020 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:12:03.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:12:03.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526572+0000", 2024-09-15T00:12:03.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9500000000000003e-07 2024-09-15T00:12:03.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526641+0000", 2024-09-15T00:12:03.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8858999999999999e-05 2024-09-15T00:12:03.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526648+0000", 2024-09-15T00:12:03.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:12:03.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526654+0000", 2024-09-15T00:12:03.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7509999999999997e-06 2024-09-15T00:12:03.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526689+0000", 2024-09-15T00:12:03.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5800000000000003e-05 2024-09-15T00:12:03.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526700+0000", 2024-09-15T00:12:03.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0467e-05 2024-09-15T00:12:03.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532457+0000", 2024-09-15T00:12:03.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057568180000000004 2024-09-15T00:12:03.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532479+0000", 2024-09-15T00:12:03.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2739e-05 2024-09-15T00:12:03.024 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.024 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.024 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.025 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'106)", 2024-09-15T00:12:03.025 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:12:03.025 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.415269733000002, 2024-09-15T00:12:03.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005274801, 2024-09-15T00:12:03.025 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.025 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.025 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.025 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:12:03.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:12:03.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527228+0000", 2024-09-15T00:12:03.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1600000000000001e-07 2024-09-15T00:12:03.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527232+0000", 2024-09-15T00:12:03.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8920000000000002e-06 2024-09-15T00:12:03.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527239+0000", 2024-09-15T00:12:03.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2880000000000004e-06 2024-09-15T00:12:03.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527243+0000", 2024-09-15T00:12:03.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.143e-06 2024-09-15T00:12:03.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527275+0000", 2024-09-15T00:12:03.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1866000000000003e-05 2024-09-15T00:12:03.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527288+0000", 2024-09-15T00:12:03.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469999999999999e-05 2024-09-15T00:12:03.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532480+0000", 2024-09-15T00:12:03.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051927129999999998 2024-09-15T00:12:03.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532502+0000", 2024-09-15T00:12:03.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1912999999999998e-05 2024-09-15T00:12:03.028 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.029 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.029 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.029 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:25 3.10 e26/23 3:0d9e4e9e:::rbd_header.11395e6383ec:head v 26'2, mlcod=0'0)", 2024-09-15T00:12:03.029 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:21.619783+0000", 2024-09-15T00:12:03.029 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.322714067, 2024-09-15T00:12:03.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031992600000000002, 2024-09-15T00:12:03.029 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.029 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.029 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.029 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619783+0000", 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619783+0000", 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619784+0000", 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9900000000000009e-07 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619795+0000", 2024-09-15T00:12:03.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0689e-05 2024-09-15T00:12:03.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619809+0000", 2024-09-15T00:12:03.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3757000000000001e-05 2024-09-15T00:12:03.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619818+0000", 2024-09-15T00:12:03.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4909999999999994e-06 2024-09-15T00:12:03.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619868+0000", 2024-09-15T00:12:03.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0046999999999997e-05 2024-09-15T00:12:03.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.619887+0000", 2024-09-15T00:12:03.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8780999999999999e-05 2024-09-15T00:12:03.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.622947+0000", 2024-09-15T00:12:03.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030601069999999998 2024-09-15T00:12:03.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:21.622982+0000", 2024-09-15T00:12:03.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5389000000000003e-05 2024-09-15T00:12:03.033 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.033 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.033 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.033 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.033 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.033 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:03.035 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:03.035 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:03.033+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:03.135 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:03.134+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:03.179 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:03.179 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:03.179 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:03.179 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:03.188 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:03.197 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is unset 2024-09-15T00:12:03.253 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:03.252+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:03.265 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone21 from 0xd449a62 (overlap 0x5258f85) to 0x41e0c6a 2024-09-15T00:12:03.304 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:03.305 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:03.305 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:03.305 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:03.305 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:03.315 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:03.356 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:03.354+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:03.364 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-15T00:12:03.366 INFO:teuthology.orchestra.run.smithi171.stdout:2939 punch from 0x8062d82 to 0x8071926, (0xeba4 bytes) 2024-09-15T00:12:03.369 INFO:teuthology.orchestra.run.smithi171.stdout:2940 read 0xa1a1bf2 thru 0xa1a31b5 (0x15c4 bytes) 2024-09-15T00:12:03.369 INFO:teuthology.orchestra.run.smithi171.stdout:2941 read 0x601b01b thru 0x601b1ae (0x194 bytes) 2024-09-15T00:12:03.369 INFO:teuthology.orchestra.run.smithi171.stdout:2943 read 0xd420be1 thru 0xd42c627 (0xba47 bytes) 2024-09-15T00:12:03.377 INFO:teuthology.orchestra.run.smithi171.stdout:2944 write 0xe804eeb thru 0xe80b3c1 (0x64d7 bytes) 2024-09-15T00:12:03.381 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:03.381 INFO:teuthology.orchestra.run.smithi171.stdout:2946 write 0x63af892 thru 0x63b9a42 (0xa1b1 bytes) 2024-09-15T00:12:03.390 INFO:teuthology.orchestra.run.smithi171.stdout:2947 trunc from 0xe80b3c2 to 0xb4af37d 2024-09-15T00:12:03.403 INFO:teuthology.orchestra.run.smithi171.stdout:2949 writesame 0xa48413f thru 0xa487c0f data_size 0x15(0x3ad1 bytes) 2024-09-15T00:12:03.407 INFO:teuthology.orchestra.run.smithi171.stdout:2950 punch from 0x44240e7 to 0x442ea83, (0xa99c bytes) 2024-09-15T00:12:03.407 INFO:teuthology.orchestra.run.smithi171.stdout:2951 write 0x9ffa221 thru 0xa005384 (0xb164 bytes) 2024-09-15T00:12:03.421 INFO:teuthology.orchestra.run.smithi171.stdout:2952 write 0xd867b47 thru 0xd870472 (0x892c bytes) 2024-09-15T00:12:03.424 INFO:teuthology.orchestra.run.smithi171.stdout:2953 read 0x109bc9f thru 0x10a5ebc (0xa21e bytes) 2024-09-15T00:12:03.424 INFO:teuthology.orchestra.run.smithi171.stdout:2954 flatten 2024-09-15T00:12:03.439 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:03.437+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:03.492 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:03.492 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:03.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:03.492 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:03.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.493 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:03.493 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:03.493 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.308740151000002, 2024-09-15T00:12:03.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:12:03.493 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.493 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.493 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.493 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:12:03.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.493 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:03.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:03.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:12:03.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:12:03.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:12:03.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:12:03.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:12:03.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:12:03.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:12:03.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:12:03.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:12:03.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:12:03.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:12:03.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:12:03.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:03.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:12:03.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:12:03.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:03.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:12:03.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:12:03.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:03.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:12:03.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:12:03.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:12:03.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:12:03.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:12:03.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:12:03.498 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.498 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.498 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.498 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:03.498 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:12:03.498 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.305312241999999, 2024-09-15T00:12:03.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032661645000000003, 2024-09-15T00:12:03.498 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.498 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.499 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for ondisk", 2024-09-15T00:12:03.499 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.499 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.499 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.499 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:12:03.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.499 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:12:03.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:12:03.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:12:03.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5300000000000004e-07 2024-09-15T00:12:03.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183257+0000", 2024-09-15T00:12:03.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9380000000000002e-06 2024-09-15T00:12:03.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183265+0000", 2024-09-15T00:12:03.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6599999999999995e-06 2024-09-15T00:12:03.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183270+0000", 2024-09-15T00:12:03.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6489999999999996e-06 2024-09-15T00:12:03.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215767+0000", 2024-09-15T00:12:03.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032496490000000003 2024-09-15T00:12:03.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for ondisk", 2024-09-15T00:12:03.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215819+0000", 2024-09-15T00:12:03.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1647999999999997e-05 2024-09-15T00:12:03.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215914+0000", 2024-09-15T00:12:03.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5706999999999997e-05 2024-09-15T00:12:03.502 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.502 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.502 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.502 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:03.503 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:03.503 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.578898972000001, 2024-09-15T00:12:03.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:12:03.503 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.503 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.503 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.503 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.503 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.503 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.503 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:12:03.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.503 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:03.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:03.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:12:03.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:12:03.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:12:03.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:12:03.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:12:03.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:12:03.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:12:03.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:12:03.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:12:03.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:12:03.508 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.508 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.508 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.508 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:12:03.508 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:03.508 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.571940608999999, 2024-09-15T00:12:03.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:12:03.508 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.508 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.508 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:03.508 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:12:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:12:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.512 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49026 3.19 3:9c3e88bf:::rbd_data.113916cf513.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:03.512 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.660548+0000", 2024-09-15T00:12:03.512 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.828017009, 2024-09-15T00:12:03.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032745261999999997, 2024-09-15T00:12:03.512 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.512 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.512 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.512 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49026 2024-09-15T00:12:03.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.512 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.660548+0000", 2024-09-15T00:12:03.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.660548+0000", 2024-09-15T00:12:03.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.660550+0000", 2024-09-15T00:12:03.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3599999999999999e-06 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663739+0000", 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031890009999999999 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663773+0000", 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4048000000000003e-05 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663787+0000", 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.38e-05 2024-09-15T00:12:03.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663858+0000", 2024-09-15T00:12:03.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0883000000000006e-05 2024-09-15T00:12:03.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.664010+0000", 2024-09-15T00:12:03.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015168500000000001 2024-09-15T00:12:03.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:03.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.664511+0000", 2024-09-15T00:12:03.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00050103699999999997 2024-09-15T00:12:03.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:03.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693153+0000", 2024-09-15T00:12:03.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.028642203000000001 2024-09-15T00:12:03.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:03.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693197+0000", 2024-09-15T00:12:03.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4190000000000002e-05 2024-09-15T00:12:03.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693257+0000", 2024-09-15T00:12:03.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0299e-05 2024-09-15T00:12:03.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693293+0000", 2024-09-15T00:12:03.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5756000000000001e-05 2024-09-15T00:12:03.517 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.517 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.517 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.517 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:03.517 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:03.517 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.824751771999999, 2024-09-15T00:12:03.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:12:03.518 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.518 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.518 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.518 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.518 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.518 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.518 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:12:03.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.518 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:03.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:03.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:12:03.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:12:03.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:12:03.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:12:03.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:12:03.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:12:03.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:12:03.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:12:03.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:12:03.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:12:03.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:12:03.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:12:03.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:03.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:12:03.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:12:03.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:03.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:12:03.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:12:03.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:03.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:12:03.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:12:03.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:12:03.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:12:03.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:12:03.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:12:03.523 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.523 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.523 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.523 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65739 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1556480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:03.523 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892527+0000", 2024-09-15T00:12:03.523 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5960378140000007, 2024-09-15T00:12:03.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031693398999999997, 2024-09-15T00:12:03.523 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.523 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.523 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:03.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.524 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65739 2024-09-15T00:12:03.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.524 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892527+0000", 2024-09-15T00:12:03.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892527+0000", 2024-09-15T00:12:03.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892528+0000", 2024-09-15T00:12:03.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:12:03.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892530+0000", 2024-09-15T00:12:03.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1569999999999998e-06 2024-09-15T00:12:03.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892536+0000", 2024-09-15T00:12:03.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5420000000000002e-06 2024-09-15T00:12:03.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892541+0000", 2024-09-15T00:12:03.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6959999999999996e-06 2024-09-15T00:12:03.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924050+0000", 2024-09-15T00:12:03.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031508748000000003 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924126+0000", 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6508999999999995e-05 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924220+0000", 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4316999999999998e-05 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.527 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65740 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892547+0000", 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5960174309999999, 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031821012000000003, 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65740 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892547+0000", 2024-09-15T00:12:03.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892547+0000", 2024-09-15T00:12:03.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892548+0000", 2024-09-15T00:12:03.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-15T00:12:03.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892550+0000", 2024-09-15T00:12:03.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1409999999999999e-06 2024-09-15T00:12:03.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892556+0000", 2024-09-15T00:12:03.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:12:03.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892561+0000", 2024-09-15T00:12:03.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9969999999999998e-06 2024-09-15T00:12:03.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924251+0000", 2024-09-15T00:12:03.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031689175999999999 2024-09-15T00:12:03.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924319+0000", 2024-09-15T00:12:03.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7875999999999999e-05 2024-09-15T00:12:03.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924368+0000", 2024-09-15T00:12:03.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9942999999999999e-05 2024-09-15T00:12:03.531 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.531 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.532 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.532 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65741 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:03.532 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892567+0000", 2024-09-15T00:12:03.532 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5959973900000008, 2024-09-15T00:12:03.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032103363000000003, 2024-09-15T00:12:03.532 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.532 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.532 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:03.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.532 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65741 2024-09-15T00:12:03.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.533 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892567+0000", 2024-09-15T00:12:03.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892567+0000", 2024-09-15T00:12:03.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892568+0000", 2024-09-15T00:12:03.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:12:03.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892570+0000", 2024-09-15T00:12:03.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9290000000000001e-06 2024-09-15T00:12:03.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892576+0000", 2024-09-15T00:12:03.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9689999999999999e-06 2024-09-15T00:12:03.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892581+0000", 2024-09-15T00:12:03.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2310000000000004e-06 2024-09-15T00:12:03.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924509+0000", 2024-09-15T00:12:03.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031927668999999999 2024-09-15T00:12:03.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924586+0000", 2024-09-15T00:12:03.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6755000000000006e-05 2024-09-15T00:12:03.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924671+0000", 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5269e-05 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65742 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1789952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892587+0000", 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5959777370000001, 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032279749000000003, 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65742 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892587+0000", 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892587+0000", 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892588+0000", 2024-09-15T00:12:03.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2e-07 2024-09-15T00:12:03.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892590+0000", 2024-09-15T00:12:03.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1789999999999998e-06 2024-09-15T00:12:03.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892596+0000", 2024-09-15T00:12:03.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2430000000000002e-06 2024-09-15T00:12:03.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892601+0000", 2024-09-15T00:12:03.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0100000000000003e-06 2024-09-15T00:12:03.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924699+0000", 2024-09-15T00:12:03.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032098427999999998 2024-09-15T00:12:03.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924786+0000", 2024-09-15T00:12:03.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6960999999999996e-05 2024-09-15T00:12:03.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924867+0000", 2024-09-15T00:12:03.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0507999999999996e-05 2024-09-15T00:12:03.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.540 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.540 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65743 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1802240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:03.540 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892609+0000", 2024-09-15T00:12:03.540 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5959556490000004, 2024-09-15T00:12:03.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032672310000000003, 2024-09-15T00:12:03.540 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.540 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.541 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:03.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.541 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65743 2024-09-15T00:12:03.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.541 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892609+0000", 2024-09-15T00:12:03.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892609+0000", 2024-09-15T00:12:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892610+0000", 2024-09-15T00:12:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:12:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892612+0000", 2024-09-15T00:12:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.198e-06 2024-09-15T00:12:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892619+0000", 2024-09-15T00:12:03.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6490000000000002e-06 2024-09-15T00:12:03.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892623+0000", 2024-09-15T00:12:03.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8409999999999999e-06 2024-09-15T00:12:03.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924902+0000", 2024-09-15T00:12:03.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032278159000000001 2024-09-15T00:12:03.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924972+0000", 2024-09-15T00:12:03.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0411000000000001e-05 2024-09-15T00:12:03.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925282+0000", 2024-09-15T00:12:03.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00030952099999999999 2024-09-15T00:12:03.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.544 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.544 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65744 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1912832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:03.544 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892630+0000", 2024-09-15T00:12:03.545 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5959346520000004, 2024-09-15T00:12:03.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032874603000000002, 2024-09-15T00:12:03.545 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.545 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.545 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:03.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.545 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65744 2024-09-15T00:12:03.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.545 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892630+0000", 2024-09-15T00:12:03.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892630+0000", 2024-09-15T00:12:03.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892780+0000", 2024-09-15T00:12:03.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014963699999999999 2024-09-15T00:12:03.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892784+0000", 2024-09-15T00:12:03.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8310000000000003e-06 2024-09-15T00:12:03.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892793+0000", 2024-09-15T00:12:03.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7750000000000004e-06 2024-09-15T00:12:03.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892800+0000", 2024-09-15T00:12:03.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:12:03.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925314+0000", 2024-09-15T00:12:03.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032513895000000001 2024-09-15T00:12:03.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925387+0000", 2024-09-15T00:12:03.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2596999999999996e-05 2024-09-15T00:12:03.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925505+0000", 2024-09-15T00:12:03.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000117958 2024-09-15T00:12:03.548 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.548 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.549 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.549 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65745 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:03.549 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892811+0000", 2024-09-15T00:12:03.549 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5957539399999998, 2024-09-15T00:12:03.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032904337999999998, 2024-09-15T00:12:03.549 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.549 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.549 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:03.549 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.549 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.549 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.549 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65745 2024-09-15T00:12:03.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.550 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892811+0000", 2024-09-15T00:12:03.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892811+0000", 2024-09-15T00:12:03.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892811+0000", 2024-09-15T00:12:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4600000000000005e-07 2024-09-15T00:12:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892814+0000", 2024-09-15T00:12:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.734e-06 2024-09-15T00:12:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892822+0000", 2024-09-15T00:12:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.289e-06 2024-09-15T00:12:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892827+0000", 2024-09-15T00:12:03.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4559999999999999e-06 2024-09-15T00:12:03.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925548+0000", 2024-09-15T00:12:03.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032721223000000001 2024-09-15T00:12:03.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925623+0000", 2024-09-15T00:12:03.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5142999999999995e-05 2024-09-15T00:12:03.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925715+0000", 2024-09-15T00:12:03.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1947000000000006e-05 2024-09-15T00:12:03.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.553 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.553 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65746 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 2035712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:03.553 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892835+0000", 2024-09-15T00:12:03.553 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5957299749999994, 2024-09-15T00:12:03.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033055660000000001, 2024-09-15T00:12:03.553 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.553 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.554 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:03.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.554 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65746 2024-09-15T00:12:03.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.554 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892835+0000", 2024-09-15T00:12:03.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892835+0000", 2024-09-15T00:12:03.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893296+0000", 2024-09-15T00:12:03.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00046083000000000002 2024-09-15T00:12:03.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893301+0000", 2024-09-15T00:12:03.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0830000000000003e-06 2024-09-15T00:12:03.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893309+0000", 2024-09-15T00:12:03.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.551e-06 2024-09-15T00:12:03.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893326+0000", 2024-09-15T00:12:03.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6840000000000001e-05 2024-09-15T00:12:03.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925746+0000", 2024-09-15T00:12:03.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032419388 2024-09-15T00:12:03.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925824+0000", 2024-09-15T00:12:03.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8081e-05 2024-09-15T00:12:03.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925891+0000", 2024-09-15T00:12:03.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6886999999999998e-05 2024-09-15T00:12:03.557 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.557 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.557 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.557 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65747 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:03.558 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.893344+0000", 2024-09-15T00:12:03.558 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5952211700000003, 2024-09-15T00:12:03.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032673708000000003, 2024-09-15T00:12:03.558 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.558 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.558 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:03.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.558 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65747 2024-09-15T00:12:03.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.558 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893344+0000", 2024-09-15T00:12:03.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893344+0000", 2024-09-15T00:12:03.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893345+0000", 2024-09-15T00:12:03.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:12:03.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893349+0000", 2024-09-15T00:12:03.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000003e-06 2024-09-15T00:12:03.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893358+0000", 2024-09-15T00:12:03.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4050000000000006e-06 2024-09-15T00:12:03.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893365+0000", 2024-09-15T00:12:03.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1919999999999999e-06 2024-09-15T00:12:03.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925916+0000", 2024-09-15T00:12:03.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032550826999999997 2024-09-15T00:12:03.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925975+0000", 2024-09-15T00:12:03.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8378000000000002e-05 2024-09-15T00:12:03.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.926017+0000", 2024-09-15T00:12:03.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2815999999999998e-05 2024-09-15T00:12:03.561 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.562 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.562 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.562 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:66000 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1818624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:03.562 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.901160+0000", 2024-09-15T00:12:03.562 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5874052669999994, 2024-09-15T00:12:03.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031781113, 2024-09-15T00:12:03.562 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.562 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.562 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:03.562 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.562 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.562 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.563 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 66000 2024-09-15T00:12:03.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.563 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901160+0000", 2024-09-15T00:12:03.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901160+0000", 2024-09-15T00:12:03.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901160+0000", 2024-09-15T00:12:03.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:12:03.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901162+0000", 2024-09-15T00:12:03.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:12:03.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901168+0000", 2024-09-15T00:12:03.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.908e-06 2024-09-15T00:12:03.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901173+0000", 2024-09-15T00:12:03.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5299999999999998e-06 2024-09-15T00:12:03.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.932742+0000", 2024-09-15T00:12:03.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031569209000000001 2024-09-15T00:12:03.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.932848+0000", 2024-09-15T00:12:03.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000106505 2024-09-15T00:12:03.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.932941+0000", 2024-09-15T00:12:03.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2289999999999999e-05 2024-09-15T00:12:03.566 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.566 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.566 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.567 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:66001 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:03.567 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.901179+0000", 2024-09-15T00:12:03.567 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5873857660000006, 2024-09-15T00:12:03.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032023827999999997, 2024-09-15T00:12:03.567 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.567 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.567 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:03.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.567 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 66001 2024-09-15T00:12:03.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.567 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901179+0000", 2024-09-15T00:12:03.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901179+0000", 2024-09-15T00:12:03.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901490+0000", 2024-09-15T00:12:03.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00031079499999999999 2024-09-15T00:12:03.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901498+0000", 2024-09-15T00:12:03.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.841e-06 2024-09-15T00:12:03.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901509+0000", 2024-09-15T00:12:03.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0872e-05 2024-09-15T00:12:03.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901516+0000", 2024-09-15T00:12:03.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4830000000000002e-06 2024-09-15T00:12:03.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.932970+0000", 2024-09-15T00:12:03.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031453864999999998 2024-09-15T00:12:03.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933099+0000", 2024-09-15T00:12:03.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000128629 2024-09-15T00:12:03.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933203+0000", 2024-09-15T00:12:03.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104343 2024-09-15T00:12:03.570 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.571 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.571 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.571 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:66002 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1941504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:03.571 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.901529+0000", 2024-09-15T00:12:03.571 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5870358979999999, 2024-09-15T00:12:03.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031929382999999999, 2024-09-15T00:12:03.571 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.571 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.571 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:03.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.572 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 66002 2024-09-15T00:12:03.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.572 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901529+0000", 2024-09-15T00:12:03.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901529+0000", 2024-09-15T00:12:03.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901530+0000", 2024-09-15T00:12:03.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:12:03.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901532+0000", 2024-09-15T00:12:03.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-15T00:12:03.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901540+0000", 2024-09-15T00:12:03.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5610000000000001e-06 2024-09-15T00:12:03.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901545+0000", 2024-09-15T00:12:03.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.553e-06 2024-09-15T00:12:03.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933235+0000", 2024-09-15T00:12:03.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031689945999999997 2024-09-15T00:12:03.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933322+0000", 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6848999999999995e-05 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933458+0000", 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013624899999999999 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:66003 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.901553+0000", 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5870118190000007, 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032108112000000001, 2024-09-15T00:12:03.575 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 66003 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901553+0000", 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901553+0000", 2024-09-15T00:12:03.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901554+0000", 2024-09-15T00:12:03.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8999999999999996e-07 2024-09-15T00:12:03.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901556+0000", 2024-09-15T00:12:03.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2189999999999998e-06 2024-09-15T00:12:03.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901564+0000", 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4409999999999993e-06 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901570+0000", 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8239999999999996e-06 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933491+0000", 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031921110000000003 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933576+0000", 2024-09-15T00:12:03.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4814000000000005e-05 2024-09-15T00:12:03.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933661+0000", 2024-09-15T00:12:03.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5113999999999999e-05 2024-09-15T00:12:03.579 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.579 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.579 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.579 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:66004 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:03.579 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.901578+0000", 2024-09-15T00:12:03.579 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5869871740000008, 2024-09-15T00:12:03.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032286845000000002, 2024-09-15T00:12:03.580 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.580 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.580 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:03.580 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:03.580 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:03.580 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:03.580 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 66004 2024-09-15T00:12:03.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.580 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901578+0000", 2024-09-15T00:12:03.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901578+0000", 2024-09-15T00:12:03.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901578+0000", 2024-09-15T00:12:03.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:12:03.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901580+0000", 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0640000000000001e-06 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901587+0000", 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901592+0000", 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0139999999999998e-06 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933705+0000", 2024-09-15T00:12:03.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032112778000000002 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933792+0000", 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6849999999999997e-05 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933865+0000", 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2742000000000003e-05 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.583 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:03.584 INFO:teuthology.orchestra.run.smithi171.stdout:2955 read 0x4e2cab9 thru 0x4e3196a (0x4eb2 bytes) 2024-09-15T00:12:03.585 INFO:teuthology.orchestra.run.smithi171.stdout:2956 write 0x1f8bc51 thru 0x1f9342f (0x77df bytes) 2024-09-15T00:12:03.586 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:12:03.586 INFO:teuthology.orchestra.run.smithi171.stdout:2957 writesame 0xa6bfa47 thru 0xa6c275f data_size 0x1(0x2d19 bytes) 2024-09-15T00:12:03.586 INFO:teuthology.orchestra.run.smithi171.stdout:2958 trunc from 0xd870473 to 0x6adafb7 2024-09-15T00:12:03.586 INFO:teuthology.orchestra.run.smithi171.stdout:2959 trunc from 0x6adafb7 to 0xbc1375f 2024-09-15T00:12:03.586 INFO:teuthology.orchestra.run.smithi171.stdout:2960 writesame 0x36454e5 thru 0x364af93 data_size 0x1(0x5aaf bytes) 2024-09-15T00:12:03.587 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:03.583+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:03.593 INFO:teuthology.orchestra.run.smithi171.stdout:2962 read 0x42346aa thru 0x42409e6 (0xc33d bytes) 2024-09-15T00:12:03.593 INFO:teuthology.orchestra.run.smithi171.stdout:2964 write 0x61de79d thru 0x61ede87 (0xf6eb bytes) 2024-09-15T00:12:03.598 INFO:teuthology.orchestra.run.smithi171.stdout:2965 write 0x95d1cc9 thru 0x95db66d (0x99a5 bytes) 2024-09-15T00:12:03.602 INFO:teuthology.orchestra.run.smithi171.stdout:2966 write 0x44fc5c9 thru 0x45061ed (0x9c25 bytes) 2024-09-15T00:12:03.607 INFO:teuthology.orchestra.run.smithi171.stdout:2967 read 0x2af59fa thru 0x2b03d83 (0xe38a bytes) 2024-09-15T00:12:03.607 INFO:teuthology.orchestra.run.smithi171.stdout:2968 write 0x37e668b thru 0x37ed756 (0x70cc bytes) 2024-09-15T00:12:03.609 INFO:teuthology.orchestra.run.smithi171.stdout:2969 write 0x84355ee thru 0x844060e (0xb021 bytes) 2024-09-15T00:12:03.614 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:03.626 INFO:teuthology.orchestra.run.smithi171.stdout:2971 trunc from 0xbc1375f to 0x1821d7d 2024-09-15T00:12:03.638 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:03.638 INFO:teuthology.orchestra.run.smithi171.stdout:2974 read 0x964e8f thru 0x965bf7 (0xd69 bytes) 2024-09-15T00:12:03.640 INFO:teuthology.orchestra.run.smithi171.stdout:2975 trunc from 0x1821d7d to 0x356095c 2024-09-15T00:12:03.651 INFO:teuthology.orchestra.run.smithi171.stdout:2976 write 0x3a8e70b thru 0x3a9cd0a (0xe600 bytes) 2024-09-15T00:12:03.655 INFO:teuthology.orchestra.run.smithi171.stdout:2977 read 0x202dc26 thru 0x2031891 (0x3c6c bytes) 2024-09-15T00:12:03.655 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:03.655 INFO:teuthology.orchestra.run.smithi171.stdout:2979 read 0x1369faa thru 0x13748c7 (0xa91e bytes) 2024-09-15T00:12:03.655 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:03.655 INFO:teuthology.orchestra.run.smithi171.stdout:2983 punch from 0x38e3919 to 0x38ef694, (0xbd7b bytes) 2024-09-15T00:12:03.655 INFO:teuthology.orchestra.run.smithi171.stdout:2986 read 0x98c0c thru 0xa7efa (0xf2ef bytes) 2024-09-15T00:12:03.655 INFO:teuthology.orchestra.run.smithi171.stdout:2987 punch from 0x39f755c to 0x39fbe26, (0x48ca bytes) 2024-09-15T00:12:03.655 INFO:teuthology.orchestra.run.smithi171.stdout:2988 read 0x1b0f5d thru 0x1b7d1c (0x6dc0 bytes) 2024-09-15T00:12:03.656 INFO:teuthology.orchestra.run.smithi171.stdout:2989 write 0x21cd9f4 thru 0x21d4e1d (0x742a bytes) 2024-09-15T00:12:03.659 INFO:teuthology.orchestra.run.smithi171.stdout:2992 read 0x2c7a56 thru 0x2c94e0 (0x1a8b bytes) 2024-09-15T00:12:03.660 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:03.663 INFO:teuthology.orchestra.run.smithi171.stdout:2994 trunc from 0x3a9cd0b to 0x5f03e8e 2024-09-15T00:12:03.668 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:03.680 INFO:teuthology.orchestra.run.smithi171.stdout:2996 write 0x9db08b5 thru 0x9db4af2 (0x423e bytes) 2024-09-15T00:12:03.682 INFO:teuthology.orchestra.run.smithi171.stdout:2997 read 0xa8222f thru 0xa8fb57 (0xd929 bytes) 2024-09-15T00:12:03.683 INFO:teuthology.orchestra.run.smithi171.stdout:2998 write 0x1c312d9 thru 0x1c3673c (0x5464 bytes) 2024-09-15T00:12:03.686 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:03.684+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:03.686 INFO:teuthology.orchestra.run.smithi171.stdout:2999 write 0x2d9c17e thru 0x2da8eb7 (0xcd3a bytes) 2024-09-15T00:12:03.691 INFO:teuthology.orchestra.run.smithi171.stdout:3000 write 0xa2d1fb thru 0xa3a056 (0xce5c bytes) 2024-09-15T00:12:03.702 INFO:teuthology.orchestra.run.smithi171.stdout:3001 write 0xb6281b5 thru 0xb6291ca (0x1016 bytes) 2024-09-15T00:12:03.708 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:03.708 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:03.708 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:03.708 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:03.712 INFO:teuthology.orchestra.run.smithi171.stdout:3002 write 0xbfe62eb thru 0xbfe8de4 (0x2afa bytes) 2024-09-15T00:12:03.715 INFO:teuthology.orchestra.run.smithi171.stdout:3003 read 0x6726754 thru 0x67306dd (0x9f8a bytes) 2024-09-15T00:12:03.715 INFO:teuthology.orchestra.run.smithi171.stdout:3005 write 0x7a12b31 thru 0x7a1cfb4 (0xa484 bytes) 2024-09-15T00:12:03.718 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:12:03.719 INFO:teuthology.orchestra.run.smithi171.stdout:3007 read 0xa5ea3f5 thru 0xa5f7f50 (0xdb5c bytes) 2024-09-15T00:12:03.719 INFO:teuthology.orchestra.run.smithi171.stdout:3008 write 0x3c1b38 thru 0x3cced3 (0xb39c bytes) 2024-09-15T00:12:03.723 INFO:teuthology.orchestra.run.smithi171.stdout:3009 read 0x8f6d809 thru 0x8f729ed (0x51e5 bytes) 2024-09-15T00:12:03.723 INFO:teuthology.orchestra.run.smithi171.stdout:3010 clone 23 order 24 su 524288 sc 14 2024-09-15T00:12:03.787 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:03.786+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:03.821 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:03.821 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:03.821 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:03.821 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:03.821 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:03.829 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:12:03.888 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:03.886+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:03.976 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:03.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:462 3.0 e29/23 3:04479396:::rbd_id.image_client.0-clone2:head v 29'14, mlcod=27'12)", 2024-09-15T00:12:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.957432736999998, 2024-09-15T00:12:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030963029999999999, 2024-09-15T00:12:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:03.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:12:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016208+0000", 2024-09-15T00:12:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3669999999999997e-06 2024-09-15T00:12:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016250+0000", 2024-09-15T00:12:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1845999999999999e-05 2024-09-15T00:12:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016260+0000", 2024-09-15T00:12:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9750000000000002e-06 2024-09-15T00:12:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016342+0000", 2024-09-15T00:12:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1595999999999996e-05 2024-09-15T00:12:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016363+0000", 2024-09-15T00:12:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1073e-05 2024-09-15T00:12:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019246+0000", 2024-09-15T00:12:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028835240000000002 2024-09-15T00:12:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019296+0000", 2024-09-15T00:12:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9533999999999997e-05 2024-09-15T00:12:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.981 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:463 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'9, mlcod=27'7)", 2024-09-15T00:12:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.952819732999998, 2024-09-15T00:12:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040015709999999998, 2024-09-15T00:12:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7599999999999999e-07 2024-09-15T00:12:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020818+0000", 2024-09-15T00:12:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6609999999999999e-06 2024-09-15T00:12:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020826+0000", 2024-09-15T00:12:03.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4770000000000003e-06 2024-09-15T00:12:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020833+0000", 2024-09-15T00:12:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0809999999999996e-06 2024-09-15T00:12:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020878+0000", 2024-09-15T00:12:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4752000000000001e-05 2024-09-15T00:12:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020892+0000", 2024-09-15T00:12:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3899e-05 2024-09-15T00:12:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024767+0000", 2024-09-15T00:12:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038752259999999998 2024-09-15T00:12:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024814+0000", 2024-09-15T00:12:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7299000000000002e-05 2024-09-15T00:12:03.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.987 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.987 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:464 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'10, mlcod=27'8)", 2024-09-15T00:12:03.987 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:03.987 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.947899890999999, 2024-09-15T00:12:03.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048109290000000002, 2024-09-15T00:12:03.988 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.988 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.988 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.988 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:03.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:03.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:03.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-15T00:12:03.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025743+0000", 2024-09-15T00:12:03.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5789999999999993e-06 2024-09-15T00:12:03.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025752+0000", 2024-09-15T00:12:03.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8440000000000004e-06 2024-09-15T00:12:03.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025757+0000", 2024-09-15T00:12:03.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2610000000000001e-06 2024-09-15T00:12:03.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025796+0000", 2024-09-15T00:12:03.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9428999999999998e-05 2024-09-15T00:12:03.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025810+0000", 2024-09-15T00:12:03.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3129000000000001e-05 2024-09-15T00:12:03.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030514+0000", 2024-09-15T00:12:03.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047040099999999998 2024-09-15T00:12:03.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030544+0000", 2024-09-15T00:12:03.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9966000000000001e-05 2024-09-15T00:12:03.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.992 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.992 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:610 3.4 e29/23 3:255b0913:::rbd_data.11397bff333d.0000000000000039:head v 29'2, mlcod=0'0)", 2024-09-15T00:12:03.992 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:12:03.992 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.745156295999998, 2024-09-15T00:12:03.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003163014, 2024-09-15T00:12:03.993 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.993 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.993 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.993 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:12:03.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:12:03.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228477+0000", 2024-09-15T00:12:03.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1100000000000004e-07 2024-09-15T00:12:03.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228483+0000", 2024-09-15T00:12:03.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3720000000000001e-06 2024-09-15T00:12:03.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:03.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228497+0000", 2024-09-15T00:12:03.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3944000000000001e-05 2024-09-15T00:12:03.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:03.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228505+0000", 2024-09-15T00:12:03.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8140000000000001e-06 2024-09-15T00:12:03.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:03.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228549+0000", 2024-09-15T00:12:03.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4116000000000002e-05 2024-09-15T00:12:03.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:03.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228568+0000", 2024-09-15T00:12:03.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8473000000000001e-05 2024-09-15T00:12:03.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:03.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231599+0000", 2024-09-15T00:12:03.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030315590000000001 2024-09-15T00:12:03.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:03.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231639+0000", 2024-09-15T00:12:03.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9824999999999999e-05 2024-09-15T00:12:03.997 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.997 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:03.997 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:03.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.997 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:766 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'81, mlcod=29'78)", 2024-09-15T00:12:03.997 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:12:03.997 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.540607219000002, 2024-09-15T00:12:03.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029025880000000002, 2024-09-15T00:12:03.998 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:03.998 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:03.998 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:03.998 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:03.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:03.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:12:03.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:03.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:12:03.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:03.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:03.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433026+0000", 2024-09-15T00:12:03.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:12:03.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:03.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:03.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433030+0000", 2024-09-15T00:12:03.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6280000000000004e-06 2024-09-15T00:12:03.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:03.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433038+0000", 2024-09-15T00:12:04.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.072e-06 2024-09-15T00:12:04.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433045+0000", 2024-09-15T00:12:04.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1510000000000001e-06 2024-09-15T00:12:04.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433101+0000", 2024-09-15T00:12:04.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6336999999999999e-05 2024-09-15T00:12:04.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:04.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433113+0000", 2024-09-15T00:12:04.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1639e-05 2024-09-15T00:12:04.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:04.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435897+0000", 2024-09-15T00:12:04.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027845779999999998 2024-09-15T00:12:04.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435928+0000", 2024-09-15T00:12:04.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0689999999999999e-05 2024-09-15T00:12:04.002 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.002 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.002 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.002 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:832 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'100, mlcod=29'97)", 2024-09-15T00:12:04.002 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:04.002 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.448361718999998, 2024-09-15T00:12:04.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005498925, 2024-09-15T00:12:04.003 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:04.003 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:04.003 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:04.003 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:04.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:04.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:04.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:04.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:04.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:04.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:04.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7999999999999995e-07 2024-09-15T00:12:04.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:04.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525276+0000", 2024-09-15T00:12:04.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6739999999999996e-06 2024-09-15T00:12:04.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525287+0000", 2024-09-15T00:12:04.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0906e-05 2024-09-15T00:12:04.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525293+0000", 2024-09-15T00:12:04.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4409999999999998e-06 2024-09-15T00:12:04.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525329+0000", 2024-09-15T00:12:04.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5237000000000002e-05 2024-09-15T00:12:04.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:04.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525340+0000", 2024-09-15T00:12:04.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1782e-05 2024-09-15T00:12:04.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:04.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530710+0000", 2024-09-15T00:12:04.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053695460000000002 2024-09-15T00:12:04.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530770+0000", 2024-09-15T00:12:04.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9658999999999999e-05 2024-09-15T00:12:04.007 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.007 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.007 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.008 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:876 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'114, mlcod=29'111)", 2024-09-15T00:12:04.008 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:04.008 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.277322718999997, 2024-09-15T00:12:04.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028465830000000002, 2024-09-15T00:12:04.008 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:04.008 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:04.008 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:04.008 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:04.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:04.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:04.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:04.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:04.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:04.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:04.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2399999999999998e-07 2024-09-15T00:12:04.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:04.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696316+0000", 2024-09-15T00:12:04.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2800000000000003e-06 2024-09-15T00:12:04.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696326+0000", 2024-09-15T00:12:04.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0546e-05 2024-09-15T00:12:04.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696333+0000", 2024-09-15T00:12:04.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7660000000000001e-06 2024-09-15T00:12:04.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696369+0000", 2024-09-15T00:12:04.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5933000000000003e-05 2024-09-15T00:12:04.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:04.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696381+0000", 2024-09-15T00:12:04.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2417e-05 2024-09-15T00:12:04.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:04.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699124+0000", 2024-09-15T00:12:04.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027428000000000001 2024-09-15T00:12:04.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699156+0000", 2024-09-15T00:12:04.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2217e-05 2024-09-15T00:12:04.012 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.012 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.012 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1313 3.1f e30/28 3:fa9cde43:::rbd_data.11397bff333d.000000000000003a:head v 30'3, mlcod=26'1)", 2024-09-15T00:12:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:12:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.689483082000002, 2024-09-15T00:12:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044064359999999997, 2024-09-15T00:12:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:04.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:12:04.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:04.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:12:04.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:04.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284150+0000", 2024-09-15T00:12:04.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3799999999999999e-07 2024-09-15T00:12:04.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:04.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284228+0000", 2024-09-15T00:12:04.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:12:04.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284235+0000", 2024-09-15T00:12:04.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:12:04.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284239+0000", 2024-09-15T00:12:04.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3449999999999999e-06 2024-09-15T00:12:04.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284282+0000", 2024-09-15T00:12:04.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2506000000000003e-05 2024-09-15T00:12:04.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:04.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284297+0000", 2024-09-15T00:12:04.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5523999999999999e-05 2024-09-15T00:12:04.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:04.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288517+0000", 2024-09-15T00:12:04.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042194019999999997 2024-09-15T00:12:04.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288556+0000", 2024-09-15T00:12:04.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8952000000000002e-05 2024-09-15T00:12:04.018 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.018 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.018 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.018 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1321 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'7, mlcod=29'5)", 2024-09-15T00:12:04.018 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:04.018 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.665389822999998, 2024-09-15T00:12:04.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003131254, 2024-09-15T00:12:04.018 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:04.018 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:04.019 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:04.019 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:04.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:04.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:04.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:04.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:04.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:04.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:04.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:12:04.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:04.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308250+0000", 2024-09-15T00:12:04.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6109999999999999e-06 2024-09-15T00:12:04.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308264+0000", 2024-09-15T00:12:04.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4473e-05 2024-09-15T00:12:04.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308272+0000", 2024-09-15T00:12:04.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3540000000000002e-06 2024-09-15T00:12:04.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308340+0000", 2024-09-15T00:12:04.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8305000000000004e-05 2024-09-15T00:12:04.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:04.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308361+0000", 2024-09-15T00:12:04.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0409999999999999e-05 2024-09-15T00:12:04.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:04.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311332+0000", 2024-09-15T00:12:04.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029715950000000001 2024-09-15T00:12:04.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311374+0000", 2024-09-15T00:12:04.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1786000000000003e-05 2024-09-15T00:12:04.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.023 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.023 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.9 epoch 30/23 rep_tid 277 entries 30'217 (0'0) error 3:942069cf:::rbd_header.11397bff333d:head by client.4409.0:1373 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'214)", 2024-09-15T00:12:04.024 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:12:04.024 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.619837226000001, 2024-09-15T00:12:04.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034617960000000001, 2024-09-15T00:12:04.024 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:04.024 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:04.025 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:04.025 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:04.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:04.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:12:04.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:04.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:12:04.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:04.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353796+0000", 2024-09-15T00:12:04.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6699999999999999e-07 2024-09-15T00:12:04.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:04.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353803+0000", 2024-09-15T00:12:04.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.832e-06 2024-09-15T00:12:04.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353814+0000", 2024-09-15T00:12:04.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1815e-05 2024-09-15T00:12:04.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353822+0000", 2024-09-15T00:12:04.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7440000000000007e-06 2024-09-15T00:12:04.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353910+0000", 2024-09-15T00:12:04.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7491000000000006e-05 2024-09-15T00:12:04.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.357257+0000", 2024-09-15T00:12:04.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033474469999999999 2024-09-15T00:12:04.028 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.028 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.028 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.030 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1374 3.9 e30/23 3:942069cf:::rbd_header.11397bff333d:head v 30'218, mlcod=30'215)", 2024-09-15T00:12:04.030 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:12:04.030 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.618437583999999, 2024-09-15T00:12:04.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047299830000000001, 2024-09-15T00:12:04.030 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:04.030 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:04.030 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:04.030 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:04.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:04.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:12:04.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:04.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:12:04.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:04.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355196+0000", 2024-09-15T00:12:04.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3900000000000004e-07 2024-09-15T00:12:04.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:04.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355200+0000", 2024-09-15T00:12:04.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5700000000000003e-06 2024-09-15T00:12:04.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355210+0000", 2024-09-15T00:12:04.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6050000000000004e-06 2024-09-15T00:12:04.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355215+0000", 2024-09-15T00:12:04.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.933e-06 2024-09-15T00:12:04.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355249+0000", 2024-09-15T00:12:04.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4110999999999999e-05 2024-09-15T00:12:04.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:04.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355260+0000", 2024-09-15T00:12:04.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1235e-05 2024-09-15T00:12:04.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:04.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359895+0000", 2024-09-15T00:12:04.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046345550000000003 2024-09-15T00:12:04.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359925+0000", 2024-09-15T00:12:04.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0335e-05 2024-09-15T00:12:04.034 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.035 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.035 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.035 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1385 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'221, mlcod=30'218)", 2024-09-15T00:12:04.035 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:12:04.035 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.605981049, 2024-09-15T00:12:04.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030443950000000001, 2024-09-15T00:12:04.035 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:04.035 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:04.035 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:04.036 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:04.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:04.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:12:04.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:04.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:12:04.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:04.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367652+0000", 2024-09-15T00:12:04.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:12:04.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:04.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367657+0000", 2024-09-15T00:12:04.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5410000000000002e-06 2024-09-15T00:12:04.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367666+0000", 2024-09-15T00:12:04.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9280000000000005e-06 2024-09-15T00:12:04.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367671+0000", 2024-09-15T00:12:04.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7870000000000001e-06 2024-09-15T00:12:04.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367707+0000", 2024-09-15T00:12:04.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6102000000000001e-05 2024-09-15T00:12:04.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:04.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367719+0000", 2024-09-15T00:12:04.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1571e-05 2024-09-15T00:12:04.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:04.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370666+0000", 2024-09-15T00:12:04.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029474750000000002 2024-09-15T00:12:04.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370696+0000", 2024-09-15T00:12:04.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9402000000000001e-05 2024-09-15T00:12:04.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.041 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.041 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1662 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'300, mlcod=30'297)", 2024-09-15T00:12:04.042 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:12:04.042 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.185616412000002, 2024-09-15T00:12:04.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031296090000000002, 2024-09-15T00:12:04.042 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:04.042 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:04.042 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:04.042 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:04.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:04.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:12:04.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:04.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:12:04.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:04.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788017+0000", 2024-09-15T00:12:04.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8999999999999996e-07 2024-09-15T00:12:04.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:04.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788021+0000", 2024-09-15T00:12:04.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4970000000000003e-06 2024-09-15T00:12:04.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788027+0000", 2024-09-15T00:12:04.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1840000000000004e-06 2024-09-15T00:12:04.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788032+0000", 2024-09-15T00:12:04.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.135e-06 2024-09-15T00:12:04.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788070+0000", 2024-09-15T00:12:04.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7073999999999998e-05 2024-09-15T00:12:04.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:04.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788081+0000", 2024-09-15T00:12:04.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0987e-05 2024-09-15T00:12:04.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:04.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791116+0000", 2024-09-15T00:12:04.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030355209999999998 2024-09-15T00:12:04.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791146+0000", 2024-09-15T00:12:04.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9621e-05 2024-09-15T00:12:04.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.045 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.046 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1677 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'305, mlcod=30'302)", 2024-09-15T00:12:04.046 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:12:04.046 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.161888050000002, 2024-09-15T00:12:04.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031364320000000002, 2024-09-15T00:12:04.046 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:04.046 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:04.046 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:04.046 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:04.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:04.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:12:04.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:04.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:12:04.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:04.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811745+0000", 2024-09-15T00:12:04.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:12:04.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:04.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811749+0000", 2024-09-15T00:12:04.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9369999999999997e-06 2024-09-15T00:12:04.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811756+0000", 2024-09-15T00:12:04.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:12:04.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811765+0000", 2024-09-15T00:12:04.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7930000000000003e-06 2024-09-15T00:12:04.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811797+0000", 2024-09-15T00:12:04.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2203999999999999e-05 2024-09-15T00:12:04.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:04.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811810+0000", 2024-09-15T00:12:04.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2359e-05 2024-09-15T00:12:04.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:04.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814852+0000", 2024-09-15T00:12:04.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003042276 2024-09-15T00:12:04.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814881+0000", 2024-09-15T00:12:04.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9037999999999999e-05 2024-09-15T00:12:04.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.050 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.050 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1724 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'9, mlcod=30'7)", 2024-09-15T00:12:04.050 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:12:04.050 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.092962366000002, 2024-09-15T00:12:04.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003115769, 2024-09-15T00:12:04.050 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:04.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880671+0000", 2024-09-15T00:12:04.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4900000000000005e-07 2024-09-15T00:12:04.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:04.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880678+0000", 2024-09-15T00:12:04.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0979999999999996e-06 2024-09-15T00:12:04.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880691+0000", 2024-09-15T00:12:04.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3288999999999999e-05 2024-09-15T00:12:04.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880698+0000", 2024-09-15T00:12:04.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9240000000000001e-06 2024-09-15T00:12:04.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880757+0000", 2024-09-15T00:12:04.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8866000000000002e-05 2024-09-15T00:12:04.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:04.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880790+0000", 2024-09-15T00:12:04.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2926999999999998e-05 2024-09-15T00:12:04.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:04.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883738+0000", 2024-09-15T00:12:04.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002948418 2024-09-15T00:12:04.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883786+0000", 2024-09-15T00:12:04.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7398000000000003e-05 2024-09-15T00:12:04.054 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.054 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.054 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.054 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1790 3.9 e30/23 3:9676e01b:::rbd_data.11397bff333d.0000000000000030:head v 30'338, mlcod=30'335)", 2024-09-15T00:12:04.054 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:12:04.054 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.014079996, 2024-09-15T00:12:04.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044496980000000002, 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959553+0000", 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3600000000000004e-07 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959569+0000", 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6152999999999999e-05 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959577+0000", 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5179999999999998e-06 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959581+0000", 2024-09-15T00:12:04.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3669999999999999e-06 2024-09-15T00:12:04.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959619+0000", 2024-09-15T00:12:04.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8209e-05 2024-09-15T00:12:04.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:04.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959632+0000", 2024-09-15T00:12:04.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2405000000000001e-05 2024-09-15T00:12:04.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:04.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.963954+0000", 2024-09-15T00:12:04.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043226660000000002 2024-09-15T00:12:04.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.964002+0000", 2024-09-15T00:12:04.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7843999999999998e-05 2024-09-15T00:12:04.059 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.059 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.059 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.059 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:3177 3.4 e33/23 3:23818c00:::rbd_data.1139b531b717.0000000000000022:head v 33'58, mlcod=33'56)", 2024-09-15T00:12:04.059 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:12:04.059 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 34.315791429999997, 2024-09-15T00:12:04.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030564120000000001, 2024-09-15T00:12:04.060 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:04.060 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:04.060 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:04.060 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:04.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:04.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:12:04.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:04.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:12:04.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:04.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657842+0000", 2024-09-15T00:12:04.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.19e-06 2024-09-15T00:12:04.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:04.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657853+0000", 2024-09-15T00:12:04.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:12:04.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657871+0000", 2024-09-15T00:12:04.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8233e-05 2024-09-15T00:12:04.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657880+0000", 2024-09-15T00:12:04.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8319999999999995e-06 2024-09-15T00:12:04.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657949+0000", 2024-09-15T00:12:04.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8847e-05 2024-09-15T00:12:04.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:04.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657977+0000", 2024-09-15T00:12:04.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8308000000000001e-05 2024-09-15T00:12:04.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:04.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.660851+0000", 2024-09-15T00:12:04.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028743190000000002 2024-09-15T00:12:04.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.660897+0000", 2024-09-15T00:12:04.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6142e-05 2024-09-15T00:12:04.063 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.064 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.064 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.064 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:3602 3.1f e34/28 3:fd28af01:::rbd_data.1139b531b717.0000000000000032:head v 34'24, mlcod=33'22)", 2024-09-15T00:12:04.064 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:12:04.064 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.928394034999997, 2024-09-15T00:12:04.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034159709999999999, 2024-09-15T00:12:04.064 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:04.064 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:04.064 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:04.064 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:04.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:04.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:12:04.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:04.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:12:04.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:04.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045239+0000", 2024-09-15T00:12:04.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:12:04.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:04.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045252+0000", 2024-09-15T00:12:04.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2891000000000001e-05 2024-09-15T00:12:04.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045263+0000", 2024-09-15T00:12:04.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0875e-05 2024-09-15T00:12:04.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045271+0000", 2024-09-15T00:12:04.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3529999999999995e-06 2024-09-15T00:12:04.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045338+0000", 2024-09-15T00:12:04.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7377000000000002e-05 2024-09-15T00:12:04.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:04.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045359+0000", 2024-09-15T00:12:04.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1078e-05 2024-09-15T00:12:04.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:04.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.048621+0000", 2024-09-15T00:12:04.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003261543 2024-09-15T00:12:04.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.048654+0000", 2024-09-15T00:12:04.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3550999999999998e-05 2024-09-15T00:12:04.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.068 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.068 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:4253 3.4 e35/23 3:242f5431:::rbd_data.11396389f8f2.0000000000000000:head v 35'87, mlcod=35'85)", 2024-09-15T00:12:04.068 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:12:04.068 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.580905176999998, 2024-09-15T00:12:04.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002944997, 2024-09-15T00:12:04.068 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:04.068 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:04.068 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:12:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:12:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392728+0000", 2024-09-15T00:12:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.06e-07 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392782+0000", 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.448e-05 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392794+0000", 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1768e-05 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392801+0000", 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9589999999999998e-06 2024-09-15T00:12:04.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392845+0000", 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3884999999999997e-05 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392860+0000", 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5364999999999999e-05 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.395641+0000", 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027807629999999999 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.395672+0000", 2024-09-15T00:12:04.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1170999999999997e-05 2024-09-15T00:12:04.072 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.072 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.072 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.072 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:18791 3.0 e38/23 3:05ecfc97:::rbd_data.11398e1a091a.0000000000000030:head v 38'66, mlcod=38'64)", 2024-09-15T00:12:04.072 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:12:04.072 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.937989641000001, 2024-09-15T00:12:04.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029447940000000001, 2024-09-15T00:12:04.072 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:04.072 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:04.072 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:04.073 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:04.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:04.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:12:04.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:04.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:12:04.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:04.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035644+0000", 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4e-07 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035684+0000", 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9857999999999997e-05 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035694+0000", 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0259e-05 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:04.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035699+0000", 2024-09-15T00:12:04.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6389999999999997e-06 2024-09-15T00:12:04.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:04.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035745+0000", 2024-09-15T00:12:04.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5640999999999999e-05 2024-09-15T00:12:04.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:04.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035764+0000", 2024-09-15T00:12:04.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9368000000000002e-05 2024-09-15T00:12:04.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:04.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.038551+0000", 2024-09-15T00:12:04.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027860879999999999 2024-09-15T00:12:04.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:04.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:04.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:04.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.038588+0000", 2024-09-15T00:12:04.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7101000000000003e-05 2024-09-15T00:12:04.076 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.076 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.076 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.076 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:04.076 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:04.076 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:04.078 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:04.078 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:04.076+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:04.145 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:12:04.183 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:04.183 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:04.183 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:04.183 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:04.184 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:04.182+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:04.193 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:04.278 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:04.278 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:04.279 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:04.278+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:04.296 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:04.296 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:04.314 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:12:04.314 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:12:04.332 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:04.332 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:04.346 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:04.347 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:04.349 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:04.349 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:04.357 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:04.367 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:04.368 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:04.380 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:04.378+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:04.385 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:04.385 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:04.403 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:04.403 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:04.407 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-15T00:12:04.480 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:04.479+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:04.490 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:04.490 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:04.499 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:04.503 INFO:teuthology.orchestra.run.smithi171.stdout:3011 write 0x82074e0 thru 0x8217025 (0xfb46 bytes) 2024-09-15T00:12:04.518 INFO:teuthology.orchestra.run.smithi171.stdout:3012 trunc from 0xbfe8de5 to 0xb891062 2024-09-15T00:12:04.573 INFO:teuthology.orchestra.run.smithi171.stdout:3013 write 0x4b0b3d3 thru 0x4b0bdc0 (0x9ee bytes) 2024-09-15T00:12:04.575 INFO:teuthology.orchestra.run.smithi171.stdout:3015 punch from 0x499b9af to 0x49a9b16, (0xe167 bytes) 2024-09-15T00:12:04.575 INFO:teuthology.orchestra.run.smithi171.stdout:3016 read 0xb43a6d6 thru 0xb448de9 (0xe714 bytes) 2024-09-15T00:12:04.575 INFO:teuthology.orchestra.run.smithi171.stdout:3017 read 0x328436b thru 0x3286b16 (0x27ac bytes) 2024-09-15T00:12:04.575 INFO:teuthology.orchestra.run.smithi171.stdout:3018 write 0x20470ec thru 0x2054d7b (0xdc90 bytes) 2024-09-15T00:12:04.578 INFO:teuthology.orchestra.run.smithi171.stdout:3019 write 0x696e28a thru 0x6973aca (0x5841 bytes) 2024-09-15T00:12:04.582 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:04.580+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:04.589 INFO:teuthology.orchestra.run.smithi171.stdout:3020 writesame 0xdb68f77 thru 0xdb70f94 data_size 0x2e(0x801e bytes) 2024-09-15T00:12:04.596 INFO:teuthology.orchestra.run.smithi171.stdout:3021 write 0xe1b9a82 thru 0xe1bc647 (0x2bc6 bytes) 2024-09-15T00:12:04.600 INFO:teuthology.orchestra.run.smithi171.stdout:3022 write 0x8687ca6 thru 0x868d1e4 (0x553f bytes) 2024-09-15T00:12:04.603 INFO:teuthology.orchestra.run.smithi171.stdout:3024 punch from 0x59729 to 0x59b5f, (0x436 bytes) 2024-09-15T00:12:04.603 INFO:teuthology.orchestra.run.smithi171.stdout:3026 write 0x8eafaf5 thru 0x8eafe20 (0x32c bytes) 2024-09-15T00:12:04.606 INFO:teuthology.orchestra.run.smithi171.stdout:3027 trunc from 0xe1bc648 to 0xd265c49 2024-09-15T00:12:04.620 INFO:teuthology.orchestra.run.smithi171.stdout:3028 trunc from 0xd265c49 to 0xc9f9db4 2024-09-15T00:12:04.628 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:04.628 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:04.630 INFO:teuthology.orchestra.run.smithi171.stdout:3030 read 0x89a2b8c thru 0x89aa753 (0x7bc8 bytes) 2024-09-15T00:12:04.630 INFO:teuthology.orchestra.run.smithi171.stdout:3031 write 0xa51e0e1 thru 0xa5234ce (0x53ee bytes) 2024-09-15T00:12:04.637 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:04.647 INFO:teuthology.orchestra.run.smithi171.stdout:3032 trunc from 0xc9f9db4 to 0x1b8c82 2024-09-15T00:12:04.673 INFO:teuthology.orchestra.run.smithi171.stdout:3033 read 0x160949 thru 0x1697af (0x8e67 bytes) 2024-09-15T00:12:04.673 INFO:teuthology.orchestra.run.smithi171.stdout:3034 punch from 0x1b061e to 0x1b23bd, (0x1d9f bytes) 2024-09-15T00:12:04.675 INFO:teuthology.orchestra.run.smithi171.stdout:3035 trunc from 0x1b8c82 to 0x18b1738 2024-09-15T00:12:04.679 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:04.680 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:04.680 INFO:teuthology.orchestra.run.smithi171.stdout:3039 writesame 0xe67943 thru 0xe7047a data_size 0x87(0x8b38 bytes) 2024-09-15T00:12:04.681 INFO:teuthology.orchestra.run.smithi171.stdout:3040 punch from 0xd3e1bd to 0xd421ff, (0x4042 bytes) 2024-09-15T00:12:04.682 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:04.680+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:04.687 INFO:teuthology.orchestra.run.smithi171.stdout:3041 trunc from 0x18b1738 to 0x6b3c2a7 2024-09-15T00:12:04.693 INFO:teuthology.orchestra.run.smithi171.stdout:3042 read 0x9e23af thru 0x9ead70 (0x89c2 bytes) 2024-09-15T00:12:04.693 INFO:teuthology.orchestra.run.smithi171.stdout:3043 read 0x5f641a0 thru 0x5f679ed (0x384e bytes) 2024-09-15T00:12:04.693 INFO:teuthology.orchestra.run.smithi171.stdout:3044 read 0x13d2b9 thru 0x1413a5 (0x40ed bytes) 2024-09-15T00:12:04.694 INFO:teuthology.orchestra.run.smithi171.stdout:3045 punch from 0x56e6828 to 0x56ee136, (0x790e bytes) 2024-09-15T00:12:04.694 INFO:teuthology.orchestra.run.smithi171.stdout:3047 read 0x22a7c91 thru 0x22b35f3 (0xb963 bytes) 2024-09-15T00:12:04.694 INFO:teuthology.orchestra.run.smithi171.stdout:3049 punch from 0x5655398 to 0x5660169, (0xadd1 bytes) 2024-09-15T00:12:04.694 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:04.694 INFO:teuthology.orchestra.run.smithi171.stdout:3051 read 0x358d80a thru 0x359b1dc (0xd9d3 bytes) 2024-09-15T00:12:04.694 INFO:teuthology.orchestra.run.smithi171.stdout:3052 read 0x428b08e thru 0x4292818 (0x778b bytes) 2024-09-15T00:12:04.694 INFO:teuthology.orchestra.run.smithi171.stdout:3054 read 0x4da8926 thru 0x4db7bf8 (0xf2d3 bytes) 2024-09-15T00:12:04.701 INFO:teuthology.orchestra.run.smithi171.stdout:3056 trunc from 0x6b3c2a7 to 0xb176ce9 2024-09-15T00:12:04.704 INFO:teuthology.orchestra.run.smithi171.stdout:3057 write 0xb72d69 thru 0xb7c18b (0x9423 bytes) 2024-09-15T00:12:04.709 INFO:teuthology.orchestra.run.smithi171.stdout:3058 read 0x46e153d thru 0x46e5803 (0x42c7 bytes) 2024-09-15T00:12:04.719 INFO:teuthology.orchestra.run.smithi171.stdout:3060 write 0xe6f20c0 thru 0xe6f53d8 (0x3319 bytes) 2024-09-15T00:12:04.722 INFO:teuthology.orchestra.run.smithi171.stdout:3061 writesame 0x516512 thru 0x5227d8 data_size 0xb(0xc2c7 bytes) 2024-09-15T00:12:04.742 INFO:teuthology.orchestra.run.smithi171.stdout:3062 trunc from 0xe6f53d9 to 0x6b4f9e1 2024-09-15T00:12:04.753 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:04.753 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:04.755 INFO:teuthology.orchestra.run.smithi171.stdout:3063 punch from 0x5e7dc0f to 0x5e8c255, (0xe646 bytes) 2024-09-15T00:12:04.755 INFO:teuthology.orchestra.run.smithi171.stdout:3064 write 0x18ef090 thru 0x18f76f4 (0x8665 bytes) 2024-09-15T00:12:04.759 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:04.759 INFO:teuthology.orchestra.run.smithi171.stdout:3066 read 0x4393f63 thru 0x43a3145 (0xf1e3 bytes) 2024-09-15T00:12:04.763 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:04.768 INFO:teuthology.orchestra.run.smithi171.stdout:3067 writesame 0xa1a0716 thru 0xa1a30fd data_size 0x12(0x29e8 bytes) 2024-09-15T00:12:04.779 INFO:teuthology.orchestra.run.smithi171.stdout:3068 write 0xd58ae42 thru 0xd58d4d5 (0x2694 bytes) 2024-09-15T00:12:04.781 INFO:teuthology.orchestra.run.smithi171.stdout:3070 writesame 0xa2df2b6 thru 0xa2e514d data_size 0xc(0x5e98 bytes) 2024-09-15T00:12:04.783 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:04.781+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:04.787 INFO:teuthology.orchestra.run.smithi171.stdout:3072 writesame 0x25cc64e thru 0x25dc122 data_size 0x1(0xfad5 bytes) 2024-09-15T00:12:04.787 INFO:teuthology.orchestra.run.smithi171.stdout:3073 write 0xa357803 thru 0xa35f936 (0x8134 bytes) 2024-09-15T00:12:04.791 INFO:teuthology.orchestra.run.smithi171.stdout:3075 writesame 0x91c5d6d thru 0x91c6b7f data_size 0x1(0xe13 bytes) 2024-09-15T00:12:04.795 INFO:teuthology.orchestra.run.smithi171.stdout:3077 punch from 0x6f855fd to 0x6f9144d, (0xbe50 bytes) 2024-09-15T00:12:04.795 INFO:teuthology.orchestra.run.smithi171.stdout:3078 punch from 0x825a167 to 0x825d012, (0x2eab bytes) 2024-09-15T00:12:04.795 INFO:teuthology.orchestra.run.smithi171.stdout:3081 write 0x6cdbb50 thru 0x6ce10dd (0x558e bytes) 2024-09-15T00:12:04.797 INFO:teuthology.orchestra.run.smithi171.stdout:3082 writesame 0x68f799d thru 0x6903af8 data_size 0xc6(0xc15c bytes) 2024-09-15T00:12:04.801 INFO:teuthology.orchestra.run.smithi171.stdout:3083 trunc from 0xd58d4d6 to 0xca84e82 2024-09-15T00:12:04.808 INFO:teuthology.orchestra.run.smithi171.stdout:3084 read 0x92447cd thru 0x924c9ce (0x8202 bytes) 2024-09-15T00:12:04.809 INFO:teuthology.orchestra.run.smithi171.stdout:3086 write 0x196c4f6 thru 0x1971a9a (0x55a5 bytes) 2024-09-15T00:12:04.810 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:04.810 INFO:teuthology.orchestra.run.smithi171.stdout:3089 punch from 0xa9bafc6 to 0xa9c4563, (0x959d bytes) 2024-09-15T00:12:04.814 INFO:teuthology.orchestra.run.smithi171.stdout:3090 trunc from 0xca84e82 to 0x997bbff 2024-09-15T00:12:04.821 INFO:teuthology.orchestra.run.smithi171.stdout:3091 read 0x5bdc66f thru 0x5be1723 (0x50b5 bytes) 2024-09-15T00:12:04.822 INFO:teuthology.orchestra.run.smithi171.stdout:3092 punch from 0x7a4fff6 to 0x7a5bdc9, (0xbdd3 bytes) 2024-09-15T00:12:04.822 INFO:teuthology.orchestra.run.smithi171.stdout:3093 read 0x91c6a2c thru 0x91cfb4d (0x9122 bytes) 2024-09-15T00:12:04.822 INFO:teuthology.orchestra.run.smithi171.stdout:3094 writesame 0x903f7f2 thru 0x904b0ca data_size 0x1(0xb8d9 bytes) 2024-09-15T00:12:04.872 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:04.872 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:04.880 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:04.883 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:04.882+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:04.893 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6] out_osds: [0, 2, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T00:12:04.893 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:12:04.893 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-15T00:12:04.893 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-15T00:12:04.920 INFO:teuthology.orchestra.run.smithi171.stdout:3095 write 0xdf395ac thru 0xdf3e56d (0x4fc2 bytes) 2024-09-15T00:12:04.923 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:04.923 INFO:teuthology.orchestra.run.smithi171.stdout:3097 punch from 0x6443c2 to 0x64ceee, (0x8b2c bytes) 2024-09-15T00:12:04.924 INFO:teuthology.orchestra.run.smithi171.stdout:3098 writesame 0x1696d97 thru 0x169e25d data_size 0x5(0x74c7 bytes) 2024-09-15T00:12:04.936 INFO:teuthology.orchestra.run.smithi171.stdout:3100 read 0x2d0d6a6 thru 0x2d11433 (0x3d8e bytes) 2024-09-15T00:12:04.937 INFO:teuthology.orchestra.run.smithi171.stdout:3101 punch from 0x8324a88 to 0x83321bb, (0xd733 bytes) 2024-09-15T00:12:04.937 INFO:teuthology.orchestra.run.smithi171.stdout:3102 punch from 0x111ce94 to 0x112c6a2, (0xf80e bytes) 2024-09-15T00:12:04.937 INFO:teuthology.orchestra.run.smithi171.stdout:3103 read 0x539f19e thru 0x53adeba (0xed1d bytes) 2024-09-15T00:12:04.937 INFO:teuthology.orchestra.run.smithi171.stdout:3104 write 0x2df0255 thru 0x2dfd4e1 (0xd28d bytes) 2024-09-15T00:12:04.940 INFO:teuthology.orchestra.run.smithi171.stdout:3105 write 0xa970a4b thru 0xa975c4d (0x5203 bytes) 2024-09-15T00:12:04.943 INFO:teuthology.orchestra.run.smithi171.stdout:3107 write 0xa129f42 thru 0xa1385c2 (0xe681 bytes) 2024-09-15T00:12:04.947 INFO:teuthology.orchestra.run.smithi171.stdout:3108 write 0x5f8e5ec thru 0x5f91d7f (0x3794 bytes) 2024-09-15T00:12:04.950 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:04.950 INFO:teuthology.orchestra.run.smithi171.stdout:3110 write 0x3d5b125 thru 0x3d63986 (0x8862 bytes) 2024-09-15T00:12:04.953 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:04.953 INFO:teuthology.orchestra.run.smithi171.stdout:3112 read 0x5ecde91 thru 0x5ed9a8e (0xbbfe bytes) 2024-09-15T00:12:04.953 INFO:teuthology.orchestra.run.smithi171.stdout:3113 write 0xb89e62b thru 0xb8aa0cd (0xbaa3 bytes) 2024-09-15T00:12:04.958 INFO:teuthology.orchestra.run.smithi171.stdout:3114 write 0x316c9dd thru 0x3171910 (0x4f34 bytes) 2024-09-15T00:12:04.969 INFO:teuthology.orchestra.run.smithi171.stdout:3115 trunc from 0xdf3e56e to 0x641be21 2024-09-15T00:12:04.985 INFO:teuthology.orchestra.run.smithi171.stdout:3116 write 0x76f0263 thru 0x76ff9dd (0xf77b bytes) 2024-09-15T00:12:04.985 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:04.983+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:04.986 INFO:teuthology.orchestra.run.smithi171.stdout:3118 write 0x4ec5f88 thru 0x4ec8a02 (0x2a7b bytes) 2024-09-15T00:12:04.989 INFO:teuthology.orchestra.run.smithi171.stdout:3120 trunc from 0x76ff9de to 0x803a93e 2024-09-15T00:12:04.991 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:04.991 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:04.995 INFO:teuthology.orchestra.run.smithi171.stdout:3121 read 0x50b5e30 thru 0x50bb979 (0x5b4a bytes) 2024-09-15T00:12:04.995 INFO:teuthology.orchestra.run.smithi171.stdout:3122 write 0x75b3325 thru 0x75bd20f (0x9eeb bytes) 2024-09-15T00:12:05.001 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:05.004 INFO:teuthology.orchestra.run.smithi171.stdout:3123 write 0x89d5376 thru 0x89de717 (0x93a2 bytes) 2024-09-15T00:12:05.006 INFO:teuthology.orchestra.run.smithi171.stdout:3124 write 0x8890b59 thru 0x88a079f (0xfc47 bytes) 2024-09-15T00:12:05.010 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:05.010 INFO:teuthology.orchestra.run.smithi171.stdout:3126 writesame 0x717cf26 thru 0x718a430 data_size 0x1(0xd50b bytes) 2024-09-15T00:12:05.085 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:05.083+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:05.097 INFO:teuthology.orchestra.run.smithi171.stdout:3127 trunc from 0x89de718 to 0xe0ddf3e 2024-09-15T00:12:05.102 INFO:teuthology.orchestra.run.smithi171.stdout:3128 read 0xc54e6f4 thru 0xc55ccfa (0xe607 bytes) 2024-09-15T00:12:05.110 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:05.110 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:05.111 INFO:teuthology.orchestra.run.smithi171.stdout:3129 trunc from 0xe0ddf3e to 0x725d8b6 2024-09-15T00:12:05.121 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:05.122 INFO:teuthology.orchestra.run.smithi171.stdout:3131 writesame 0xec4990 thru 0xecf0f9 data_size 0x9(0xa76a bytes) 2024-09-15T00:12:05.131 INFO:teuthology.orchestra.run.smithi171.stdout:3132 punch from 0x38b74e3 to 0x38bd581, (0x609e bytes) 2024-09-15T00:12:05.131 INFO:teuthology.orchestra.run.smithi171.stdout:3134 read 0x54055cf thru 0x540b3b2 (0x5de4 bytes) 2024-09-15T00:12:05.144 INFO:teuthology.orchestra.run.smithi171.stdout:3135 write 0xd6690b3 thru 0xd670f21 (0x7e6f bytes) 2024-09-15T00:12:05.146 INFO:teuthology.orchestra.run.smithi171.stdout:3136 read 0x4462487 thru 0x4469e6e (0x79e8 bytes) 2024-09-15T00:12:05.147 INFO:teuthology.orchestra.run.smithi171.stdout:3137 read 0xc8459cf thru 0xc851448 (0xba7a bytes) 2024-09-15T00:12:05.147 INFO:teuthology.orchestra.run.smithi171.stdout:3138 punch from 0x886235b to 0x886ad47, (0x89ec bytes) 2024-09-15T00:12:05.147 INFO:teuthology.orchestra.run.smithi171.stdout:3139 punch from 0x723a19 to 0x731754, (0xdd3b bytes) 2024-09-15T00:12:05.147 INFO:teuthology.orchestra.run.smithi171.stdout:3140 read 0x2738ab2 thru 0x2741784 (0x8cd3 bytes) 2024-09-15T00:12:05.148 INFO:teuthology.orchestra.run.smithi171.stdout:3141 writesame 0xa4c037f thru 0xa4cc067 data_size 0x1(0xbce9 bytes) 2024-09-15T00:12:05.204 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:12:05.203+0000 7f65f1aa6640 0 --2- 172.21.15.171:0/3579905269 >> v2:172.21.15.171:3300/0 conn(0x7f65ec070ea0 0x7f65ec071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T00:12:05.205 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:05.204+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:05.213 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:12:05.239 INFO:teuthology.orchestra.run.smithi171.stdout:3142 read 0x8794131 thru 0x87a2b1b (0xe9eb bytes) 2024-09-15T00:12:05.240 INFO:teuthology.orchestra.run.smithi171.stdout:3145 read 0xb47c833 thru 0xb48bdf0 (0xf5be bytes) 2024-09-15T00:12:05.240 INFO:teuthology.orchestra.run.smithi171.stdout:3146 read 0x3fdc360 thru 0x3feaf56 (0xebf7 bytes) 2024-09-15T00:12:05.241 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:05.241 INFO:teuthology.orchestra.run.smithi171.stdout:3152 write 0x8bc58f0 thru 0x8bcf30b (0x9a1c bytes) 2024-09-15T00:12:05.244 INFO:teuthology.orchestra.run.smithi171.stdout:3153 read 0xb425c7b thru 0xb42f858 (0x9bde bytes) 2024-09-15T00:12:05.245 INFO:teuthology.orchestra.run.smithi171.stdout:3155 writesame 0x4297de1 thru 0x429ed6a data_size 0x3(0x6f8a bytes) 2024-09-15T00:12:05.253 INFO:teuthology.orchestra.run.smithi171.stdout:3156 writesame 0x3f3959e thru 0x3f43373 data_size 0x2(0x9dd6 bytes) 2024-09-15T00:12:05.267 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:05.268 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:05.268 INFO:teuthology.orchestra.run.smithi171.stdout:3157 punch from 0x9818252 to 0x982778e, (0xf53c bytes) 2024-09-15T00:12:05.268 INFO:teuthology.orchestra.run.smithi171.stdout:3159 writesame 0x526ed88 thru 0x527e077 data_size 0x34(0xf2f0 bytes) 2024-09-15T00:12:05.271 INFO:teuthology.orchestra.run.smithi171.stdout:3161 read 0x560a3ab thru 0x561489a (0xa4f0 bytes) 2024-09-15T00:12:05.277 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:05.290 INFO:teuthology.orchestra.run.smithi171.stdout:3163 trunc from 0xd670f22 to 0x36cb9 2024-09-15T00:12:05.319 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:05.317+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:05.328 INFO:teuthology.orchestra.run.smithi171.stdout:3164 trunc from 0x36cb9 to 0x72e190a 2024-09-15T00:12:05.342 INFO:teuthology.orchestra.run.smithi171.stdout:3166 write 0x96fa54d thru 0x970418b (0x9c3f bytes) 2024-09-15T00:12:05.348 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:05.348 INFO:teuthology.orchestra.run.smithi171.stdout:3170 write 0x6b791b thru 0x6c5502 (0xdbe8 bytes) 2024-09-15T00:12:05.356 INFO:teuthology.orchestra.run.smithi171.stdout:3173 trunc from 0x970418c to 0xb7c41bd 2024-09-15T00:12:05.361 INFO:teuthology.orchestra.run.smithi171.stdout:3174 write 0x4479abf thru 0x447d0cd (0x360f bytes) 2024-09-15T00:12:05.364 INFO:teuthology.orchestra.run.smithi171.stdout:3175 read 0x4f1b4f5 thru 0x4f1dd35 (0x2841 bytes) 2024-09-15T00:12:05.365 INFO:teuthology.orchestra.run.smithi171.stdout:3178 read 0x70e1af4 thru 0x70e4417 (0x2924 bytes) 2024-09-15T00:12:05.365 INFO:teuthology.orchestra.run.smithi171.stdout:3180 read 0x3260974 thru 0x326f8e7 (0xef74 bytes) 2024-09-15T00:12:05.365 INFO:teuthology.orchestra.run.smithi171.stdout:3181 punch from 0x9263177 to 0x92657b0, (0x2639 bytes) 2024-09-15T00:12:05.373 INFO:teuthology.orchestra.run.smithi171.stdout:3182 trunc from 0xb7c41bd to 0x41dad94 2024-09-15T00:12:05.381 INFO:teuthology.orchestra.run.smithi171.stdout:3183 punch from 0x81bf76 to 0x81efca, (0x3054 bytes) 2024-09-15T00:12:05.381 INFO:teuthology.orchestra.run.smithi171.stdout:3184 read 0x97a8e0 thru 0x97bfa4 (0x16c5 bytes) 2024-09-15T00:12:05.381 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:05.387 INFO:teuthology.orchestra.run.smithi171.stdout:3186 writesame 0x4b32652 thru 0x4b3f4b8 data_size 0x39(0xce67 bytes) 2024-09-15T00:12:05.388 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:05.386+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:05.389 INFO:teuthology.orchestra.run.smithi171.stdout:3187 read 0x693a5f thru 0x698fc1 (0x5563 bytes) 2024-09-15T00:12:05.405 INFO:teuthology.orchestra.run.smithi171.stdout:3188 write 0xc94e858 thru 0xc950747 (0x1ef0 bytes) 2024-09-15T00:12:05.407 INFO:teuthology.orchestra.run.smithi171.stdout:3189 punch from 0x2b20f49 to 0x2b2b3b5, (0xa46c bytes) 2024-09-15T00:12:05.407 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:05.416 INFO:teuthology.orchestra.run.smithi171.stdout:3191 trunc from 0xc950748 to 0x440b00e 2024-09-15T00:12:05.418 INFO:tasks.ceph.mon.a.smithi171.stderr:2024-09-15T00:12:05.416+0000 7f0425d35640 -1 mon.a@0(leader).osd e63 definitely_dead 0 2024-09-15T00:12:05.427 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:05.427 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:05.433 INFO:teuthology.orchestra.run.smithi171.stdout:3192 write 0x771751c thru 0x771eee3 (0x79c8 bytes) 2024-09-15T00:12:05.434 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:05.434 INFO:teuthology.orchestra.run.smithi171.stdout:3195 write 0x40468f4 thru 0x4055642 (0xed4f bytes) 2024-09-15T00:12:05.436 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:05.440 INFO:teuthology.orchestra.run.smithi171.stdout:3196 read 0x652c721 thru 0x6531609 (0x4ee9 bytes) 2024-09-15T00:12:05.440 INFO:teuthology.orchestra.run.smithi171.stdout:3199 trunc from 0x771eee4 to 0x78c7587 2024-09-15T00:12:05.449 INFO:teuthology.orchestra.run.smithi171.stdout:3200 trunc from 0x78c7587 to 0x382b2af 2024-09-15T00:12:05.463 INFO:teuthology.orchestra.run.smithi171.stdout:3201 write 0x5741739 thru 0x5746f6e (0x5836 bytes) 2024-09-15T00:12:05.467 INFO:teuthology.orchestra.run.smithi171.stdout:3202 trunc from 0x5746f6f to 0x44709e0 2024-09-15T00:12:05.482 INFO:teuthology.orchestra.run.smithi171.stdout:3203 writesame 0x5be9a88 thru 0x5bf0e94 data_size 0x3(0x740d bytes) 2024-09-15T00:12:05.482 INFO:teuthology.orchestra.run.smithi171.stdout:3204 punch from 0x41e17fe to 0x41e2a3a, (0x123c bytes) 2024-09-15T00:12:05.482 INFO:teuthology.orchestra.run.smithi171.stdout:3206 punch from 0x402f9b4 to 0x40351de, (0x582a bytes) 2024-09-15T00:12:05.482 INFO:teuthology.orchestra.run.smithi171.stdout:3208 read 0x3995974 thru 0x39a2c7a (0xd307 bytes) 2024-09-15T00:12:05.482 INFO:teuthology.orchestra.run.smithi171.stdout:3212 read 0x4c1b7b2 thru 0x4c2364e (0x7e9d bytes) 2024-09-15T00:12:05.482 INFO:teuthology.orchestra.run.smithi171.stdout:3213 trunc from 0x5bf0e95 to 0x56b14bc 2024-09-15T00:12:05.489 INFO:teuthology.orchestra.run.smithi171.stdout:3214 trunc from 0x56b14bc to 0x5f79c37 2024-09-15T00:12:05.494 INFO:teuthology.orchestra.run.smithi171.stdout:3215 read 0x416e1c1 thru 0x416e98e (0x7ce bytes) 2024-09-15T00:12:05.508 INFO:teuthology.orchestra.run.smithi171.stdout:3216 write 0xc1a9368 thru 0xc1af5b6 (0x624f bytes) 2024-09-15T00:12:05.511 INFO:teuthology.orchestra.run.smithi171.stdout:3217 read 0xb5e5e2a thru 0xb5e8313 (0x24ea bytes) 2024-09-15T00:12:05.512 INFO:teuthology.orchestra.run.smithi171.stdout:3218 punch from 0xabe9b2a to 0xabf0795, (0x6c6b bytes) 2024-09-15T00:12:05.512 INFO:teuthology.orchestra.run.smithi171.stdout:3219 write 0x61f08cc thru 0x61f5c18 (0x534d bytes) 2024-09-15T00:12:05.516 INFO:teuthology.orchestra.run.smithi171.stdout:3220 trunc from 0xc1af5b7 to 0xd9f4049 2024-09-15T00:12:05.520 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:05.518+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:05.523 INFO:teuthology.orchestra.run.smithi171.stdout:3221 write 0x93abe5d thru 0x93af4ef (0x3693 bytes) 2024-09-15T00:12:05.526 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:05.532 INFO:teuthology.orchestra.run.smithi171.stdout:3225 write 0xe39be8c thru 0xe3a8150 (0xc2c5 bytes) 2024-09-15T00:12:05.535 INFO:teuthology.orchestra.run.smithi171.stdout:3226 write 0x9ad8685 thru 0x9ae7cde (0xf65a bytes) 2024-09-15T00:12:05.539 INFO:teuthology.orchestra.run.smithi171.stdout:3227 read 0xb353081 thru 0xb35bd69 (0x8ce9 bytes) 2024-09-15T00:12:05.539 INFO:teuthology.orchestra.run.smithi171.stdout:3228 writesame 0xa9c907f thru 0xa9c9e58 data_size 0x9(0xdda bytes) 2024-09-15T00:12:05.542 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:05.542 INFO:teuthology.orchestra.run.smithi171.stdout:3232 write 0x3a4727c thru 0x3a4bce1 (0x4a66 bytes) 2024-09-15T00:12:05.545 INFO:teuthology.orchestra.run.smithi171.stdout:3234 writesame 0xe26d7b7 thru 0xe271a4a data_size 0x4(0x4294 bytes) 2024-09-15T00:12:05.550 INFO:teuthology.orchestra.run.smithi171.stdout:3235 write 0x50b8040 thru 0x50c4a7f (0xca40 bytes) 2024-09-15T00:12:05.555 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:05.555 INFO:teuthology.orchestra.run.smithi171.stdout:3237 punch from 0x990b547 to 0x9917924, (0xc3dd bytes) 2024-09-15T00:12:05.555 INFO:teuthology.orchestra.run.smithi171.stdout:3239 read 0xad5a639 thru 0xad66eb0 (0xc878 bytes) 2024-09-15T00:12:05.555 INFO:teuthology.orchestra.run.smithi171.stdout:3240 writesame 0xbe72f4a thru 0xbe81732 data_size 0x1(0xe7e9 bytes) 2024-09-15T00:12:05.585 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:05.586 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:05.589 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:05.587+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:05.595 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:05.662 INFO:teuthology.orchestra.run.smithi171.stdout:3241 writesame 0x83c8036 thru 0x83d6974 data_size 0x1(0xe93f bytes) 2024-09-15T00:12:05.690 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:05.688+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:05.746 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:05.747 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:05.756 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:05.774 INFO:teuthology.orchestra.run.smithi171.stdout:3242 punch from 0xe1e4988 to 0xe1e6a0d, (0x2085 bytes) 2024-09-15T00:12:05.774 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:05.774 INFO:teuthology.orchestra.run.smithi171.stdout:3244 write 0x222d6ab thru 0x223c643 (0xef99 bytes) 2024-09-15T00:12:05.780 INFO:teuthology.orchestra.run.smithi171.stdout:3246 write 0x32005ab thru 0x32007f3 (0x249 bytes) 2024-09-15T00:12:05.781 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:05.782 INFO:teuthology.orchestra.run.smithi171.stdout:3248 read 0x3976fd0 thru 0x397eb09 (0x7b3a bytes) 2024-09-15T00:12:05.782 INFO:teuthology.orchestra.run.smithi171.stdout:3249 read 0xae6777 thru 0xaf1b3b (0xb3c5 bytes) 2024-09-15T00:12:05.782 INFO:teuthology.orchestra.run.smithi171.stdout:3250 writesame 0xd77c693 thru 0xd789e23 data_size 0xd(0xd791 bytes) 2024-09-15T00:12:05.785 INFO:teuthology.orchestra.run.smithi171.stdout:3253 read 0x9e155e0 thru 0x9e1f022 (0x9a43 bytes) 2024-09-15T00:12:05.786 INFO:teuthology.orchestra.run.smithi171.stdout:3254 trunc from 0xe3a8151 to 0xe90e91c 2024-09-15T00:12:05.792 INFO:teuthology.orchestra.run.smithi171.stdout:3256 punch from 0x6217d7a to 0x6226945, (0xebcb bytes) 2024-09-15T00:12:05.798 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:05.796+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:05.806 INFO:teuthology.orchestra.run.smithi171.stdout:3257 trunc from 0xe90e91c to 0x3113864 2024-09-15T00:12:05.821 INFO:teuthology.orchestra.run.smithi171.stdout:3258 punch from 0x1c463ef to 0x1c4e750, (0x8361 bytes) 2024-09-15T00:12:05.821 INFO:teuthology.orchestra.run.smithi171.stdout:3259 read 0x118fb57 thru 0x11955e6 (0x5a90 bytes) 2024-09-15T00:12:05.827 INFO:teuthology.orchestra.run.smithi171.stdout:3261 write 0x3d11be5 thru 0x3d1f565 (0xd981 bytes) 2024-09-15T00:12:05.829 INFO:teuthology.orchestra.run.smithi171.stdout:3262 write 0x2ba2c65 thru 0x2ba395a (0xcf6 bytes) 2024-09-15T00:12:05.830 INFO:teuthology.orchestra.run.smithi171.stdout:3263 punch from 0x1e001c8 to 0x1e02d5b, (0x2b93 bytes) 2024-09-15T00:12:05.830 INFO:teuthology.orchestra.run.smithi171.stdout:3264 clone 24 order 20 su 4096 sc 14 2024-09-15T00:12:05.893 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:05.891+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:05.902 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:05.903 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:05.912 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:05.994 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:05.992+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:06.056 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:06.056 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:06.065 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:06.095 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:06.094+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:06.168 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:06.168 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:06.177 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:06.195 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:06.194+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:06.207 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is set 2024-09-15T00:12:06.219 INFO:teuthology.orchestra.run.smithi171.stderr:marked in osd.0. 2024-09-15T00:12:06.233 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-15T00:12:06.289 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:06.290 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:06.296 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:06.295+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:06.298 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:06.397 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:06.396+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:06.402 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:06.402 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:06.411 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:06.417 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:12:06.498 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:06.496+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:06.558 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:06.559 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:06.568 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:06.574 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:06.575 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:06.591 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:06.592 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:06.599 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:06.597+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:06.609 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:12:06.610 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:12:06.626 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:06.626 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:06.643 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:06.643 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:06.660 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:06.660 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:06.676 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:06.676 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:06.676 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:06.676 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:06.676 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:06.677 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:06.685 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:06.693 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:06.693 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:06.704 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:06.703+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:06.796 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:06.796 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:06.796 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:06.796 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:06.796 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:06.801 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:06.800+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:06.804 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:06.902 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:06.901+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:06.949 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:06.950 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:06.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:06.950 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:06.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.950 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:06.950 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:06.950 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.767530391000001, 2024-09-15T00:12:06.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:12:06.950 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:06.950 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:06.950 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:06.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:06.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:06.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:06.951 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:12:06.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.951 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:06.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:06.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:06.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:06.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:06.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:06.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:12:06.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:12:06.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:06.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:12:06.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:12:06.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:06.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:12:06.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:12:06.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:06.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:12:06.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:12:06.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:06.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:12:06.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:12:06.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:06.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:12:06.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:12:06.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:06.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:12:06.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:12:06.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:06.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:12:06.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:12:06.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:06.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:12:06.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:12:06.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:06.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:12:06.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:12:06.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:06.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:12:06.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:12:06.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.955 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:06.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.956 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:06.956 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:12:06.956 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.764102481999998, 2024-09-15T00:12:06.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032661645000000003, 2024-09-15T00:12:06.956 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:06.956 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:06.956 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for ondisk", 2024-09-15T00:12:06.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:06.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:06.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:06.956 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:12:06.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.957 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:06.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:06.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:12:06.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:06.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:12:06.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:06.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183253+0000", 2024-09-15T00:12:06.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5300000000000004e-07 2024-09-15T00:12:06.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:06.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183257+0000", 2024-09-15T00:12:06.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9380000000000002e-06 2024-09-15T00:12:06.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:06.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183265+0000", 2024-09-15T00:12:06.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6599999999999995e-06 2024-09-15T00:12:06.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:06.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.183270+0000", 2024-09-15T00:12:06.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6489999999999996e-06 2024-09-15T00:12:06.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:06.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215767+0000", 2024-09-15T00:12:06.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032496490000000003 2024-09-15T00:12:06.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for ondisk", 2024-09-15T00:12:06.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215819+0000", 2024-09-15T00:12:06.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1647999999999997e-05 2024-09-15T00:12:06.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:06.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215914+0000", 2024-09-15T00:12:06.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5706999999999997e-05 2024-09-15T00:12:06.960 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.960 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:06.960 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.960 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:06.961 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:06.961 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.037689212, 2024-09-15T00:12:06.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:12:06.961 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:06.961 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:06.961 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:06.961 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:06.961 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:06.961 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:06.961 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:12:06.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.961 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:06.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:06.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:06.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:06.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:06.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:06.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:12:06.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:12:06.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:06.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:12:06.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:12:06.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:06.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:12:06.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:12:06.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:06.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:12:06.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:12:06.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:06.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:12:06.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:12:06.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:06.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:12:06.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:12:06.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:06.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:12:06.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:12:06.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:06.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:12:06.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:12:06.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:06.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:12:06.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:12:06.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:06.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:12:06.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:12:06.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:06.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:12:06.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:12:06.966 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.966 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:06.967 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.967 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:12:06.967 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:06.967 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.030730849000001, 2024-09-15T00:12:06.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:12:06.967 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:06.967 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:06.967 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:06.967 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:06.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:06.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:06.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:06.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:06.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:06.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:12:06.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:12:06.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:06.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:12:06.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:12:06.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:06.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:12:06.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:12:06.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:06.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:12:06.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:12:06.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:06.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:12:06.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:12:06.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:06.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:12:06.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:12:06.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:06.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:12:06.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:12:06.971 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.971 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:06.971 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.971 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49026 3.19 3:9c3e88bf:::rbd_data.113916cf513.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:06.971 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.660548+0000", 2024-09-15T00:12:06.971 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.286807248999999, 2024-09-15T00:12:06.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032745261999999997, 2024-09-15T00:12:06.971 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:06.971 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:06.972 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:06.972 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:06.972 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:06.972 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:06.972 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49026 2024-09-15T00:12:06.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.972 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:06.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:06.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.660548+0000", 2024-09-15T00:12:06.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:06.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.660548+0000", 2024-09-15T00:12:06.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:06.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.660550+0000", 2024-09-15T00:12:06.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3599999999999999e-06 2024-09-15T00:12:06.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:06.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663739+0000", 2024-09-15T00:12:06.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031890009999999999 2024-09-15T00:12:06.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:06.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663773+0000", 2024-09-15T00:12:06.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4048000000000003e-05 2024-09-15T00:12:06.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:06.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663787+0000", 2024-09-15T00:12:06.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.38e-05 2024-09-15T00:12:06.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:06.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663858+0000", 2024-09-15T00:12:06.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0883000000000006e-05 2024-09-15T00:12:06.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:06.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.664010+0000", 2024-09-15T00:12:06.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015168500000000001 2024-09-15T00:12:06.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:06.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.664511+0000", 2024-09-15T00:12:06.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00050103699999999997 2024-09-15T00:12:06.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:06.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693153+0000", 2024-09-15T00:12:06.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.028642203000000001 2024-09-15T00:12:06.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:06.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693197+0000", 2024-09-15T00:12:06.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4190000000000002e-05 2024-09-15T00:12:06.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:06.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693257+0000", 2024-09-15T00:12:06.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0299e-05 2024-09-15T00:12:06.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:06.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.693293+0000", 2024-09-15T00:12:06.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5756000000000001e-05 2024-09-15T00:12:06.977 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.977 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:06.977 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.977 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:06.977 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:06.977 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.283542012000002, 2024-09-15T00:12:06.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:12:06.978 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:06.978 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:06.978 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:06.978 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:06.978 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:06.978 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:06.978 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:12:06.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.978 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:06.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:06.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:06.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:06.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:12:06.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:12:06.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:06.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:12:06.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:12:06.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:06.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:12:06.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:12:06.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:06.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:12:06.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:12:06.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:06.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:12:06.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:12:06.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:06.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:12:06.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:12:06.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:06.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:12:06.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:12:06.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:06.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:12:06.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:12:06.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:06.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:12:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:12:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65739 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1556480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892527+0000", 2024-09-15T00:12:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.054828054, 2024-09-15T00:12:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031693398999999997, 2024-09-15T00:12:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65739 2024-09-15T00:12:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892527+0000", 2024-09-15T00:12:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892527+0000", 2024-09-15T00:12:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892528+0000", 2024-09-15T00:12:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:12:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892530+0000", 2024-09-15T00:12:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1569999999999998e-06 2024-09-15T00:12:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892536+0000", 2024-09-15T00:12:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5420000000000002e-06 2024-09-15T00:12:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892541+0000", 2024-09-15T00:12:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6959999999999996e-06 2024-09-15T00:12:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924050+0000", 2024-09-15T00:12:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031508748000000003 2024-09-15T00:12:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924126+0000", 2024-09-15T00:12:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6508999999999995e-05 2024-09-15T00:12:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924220+0000", 2024-09-15T00:12:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4316999999999998e-05 2024-09-15T00:12:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65740 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892547+0000", 2024-09-15T00:12:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.054807671000001, 2024-09-15T00:12:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031821012000000003, 2024-09-15T00:12:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65740 2024-09-15T00:12:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892547+0000", 2024-09-15T00:12:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892547+0000", 2024-09-15T00:12:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892548+0000", 2024-09-15T00:12:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000001e-07 2024-09-15T00:12:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892550+0000", 2024-09-15T00:12:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1409999999999999e-06 2024-09-15T00:12:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892556+0000", 2024-09-15T00:12:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:12:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892561+0000", 2024-09-15T00:12:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9969999999999998e-06 2024-09-15T00:12:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924251+0000", 2024-09-15T00:12:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031689175999999999 2024-09-15T00:12:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924319+0000", 2024-09-15T00:12:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7875999999999999e-05 2024-09-15T00:12:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924368+0000", 2024-09-15T00:12:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9942999999999999e-05 2024-09-15T00:12:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65741 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892567+0000", 2024-09-15T00:12:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.05478763, 2024-09-15T00:12:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032103363000000003, 2024-09-15T00:12:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65741 2024-09-15T00:12:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892567+0000", 2024-09-15T00:12:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892567+0000", 2024-09-15T00:12:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892568+0000", 2024-09-15T00:12:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:12:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892570+0000", 2024-09-15T00:12:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9290000000000001e-06 2024-09-15T00:12:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892576+0000", 2024-09-15T00:12:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9689999999999999e-06 2024-09-15T00:12:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892581+0000", 2024-09-15T00:12:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2310000000000004e-06 2024-09-15T00:12:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924509+0000", 2024-09-15T00:12:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031927668999999999 2024-09-15T00:12:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924586+0000", 2024-09-15T00:12:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6755000000000006e-05 2024-09-15T00:12:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924671+0000", 2024-09-15T00:12:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5269e-05 2024-09-15T00:12:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65742 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1789952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892587+0000", 2024-09-15T00:12:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.054767976999999, 2024-09-15T00:12:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032279749000000003, 2024-09-15T00:12:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65742 2024-09-15T00:12:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892587+0000", 2024-09-15T00:12:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892587+0000", 2024-09-15T00:12:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892588+0000", 2024-09-15T00:12:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2e-07 2024-09-15T00:12:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892590+0000", 2024-09-15T00:12:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1789999999999998e-06 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892596+0000", 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2430000000000002e-06 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892601+0000", 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0100000000000003e-06 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924699+0000", 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032098427999999998 2024-09-15T00:12:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924786+0000", 2024-09-15T00:12:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6960999999999996e-05 2024-09-15T00:12:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924867+0000", 2024-09-15T00:12:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0507999999999996e-05 2024-09-15T00:12:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65743 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1802240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892609+0000", 2024-09-15T00:12:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.054745888999999, 2024-09-15T00:12:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032672310000000003, 2024-09-15T00:12:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65743 2024-09-15T00:12:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892609+0000", 2024-09-15T00:12:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892609+0000", 2024-09-15T00:12:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892610+0000", 2024-09-15T00:12:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:12:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892612+0000", 2024-09-15T00:12:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.198e-06 2024-09-15T00:12:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892619+0000", 2024-09-15T00:12:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6490000000000002e-06 2024-09-15T00:12:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892623+0000", 2024-09-15T00:12:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8409999999999999e-06 2024-09-15T00:12:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924902+0000", 2024-09-15T00:12:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032278159000000001 2024-09-15T00:12:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.924972+0000", 2024-09-15T00:12:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0411000000000001e-05 2024-09-15T00:12:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925282+0000", 2024-09-15T00:12:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00030952099999999999 2024-09-15T00:12:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65744 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1912832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892630+0000", 2024-09-15T00:12:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.054724891999999, 2024-09-15T00:12:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032874603000000002, 2024-09-15T00:12:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65744 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892630+0000", 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892630+0000", 2024-09-15T00:12:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892780+0000", 2024-09-15T00:12:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014963699999999999 2024-09-15T00:12:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892784+0000", 2024-09-15T00:12:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8310000000000003e-06 2024-09-15T00:12:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892793+0000", 2024-09-15T00:12:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7750000000000004e-06 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892800+0000", 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925314+0000", 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032513895000000001 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925387+0000", 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2596999999999996e-05 2024-09-15T00:12:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925505+0000", 2024-09-15T00:12:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000117958 2024-09-15T00:12:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65745 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892811+0000", 2024-09-15T00:12:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.054544180000001, 2024-09-15T00:12:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032904337999999998, 2024-09-15T00:12:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65745 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892811+0000", 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892811+0000", 2024-09-15T00:12:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892811+0000", 2024-09-15T00:12:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4600000000000005e-07 2024-09-15T00:12:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892814+0000", 2024-09-15T00:12:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.734e-06 2024-09-15T00:12:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892822+0000", 2024-09-15T00:12:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.289e-06 2024-09-15T00:12:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892827+0000", 2024-09-15T00:12:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4559999999999999e-06 2024-09-15T00:12:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925548+0000", 2024-09-15T00:12:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032721223000000001 2024-09-15T00:12:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925623+0000", 2024-09-15T00:12:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5142999999999995e-05 2024-09-15T00:12:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925715+0000", 2024-09-15T00:12:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1947000000000006e-05 2024-09-15T00:12:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65746 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 2035712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.892835+0000", 2024-09-15T00:12:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.054520215, 2024-09-15T00:12:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033055660000000001, 2024-09-15T00:12:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65746 2024-09-15T00:12:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892835+0000", 2024-09-15T00:12:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.892835+0000", 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893296+0000", 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00046083000000000002 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893301+0000", 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0830000000000003e-06 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893309+0000", 2024-09-15T00:12:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.551e-06 2024-09-15T00:12:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893326+0000", 2024-09-15T00:12:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6840000000000001e-05 2024-09-15T00:12:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925746+0000", 2024-09-15T00:12:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032419388 2024-09-15T00:12:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925824+0000", 2024-09-15T00:12:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8081e-05 2024-09-15T00:12:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925891+0000", 2024-09-15T00:12:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6886999999999998e-05 2024-09-15T00:12:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:65747 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.893344+0000", 2024-09-15T00:12:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.054011409999999, 2024-09-15T00:12:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032673708000000003, 2024-09-15T00:12:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 65747 2024-09-15T00:12:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893344+0000", 2024-09-15T00:12:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893344+0000", 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893345+0000", 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893349+0000", 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000003e-06 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893358+0000", 2024-09-15T00:12:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4050000000000006e-06 2024-09-15T00:12:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.893365+0000", 2024-09-15T00:12:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1919999999999999e-06 2024-09-15T00:12:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925916+0000", 2024-09-15T00:12:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032550826999999997 2024-09-15T00:12:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.925975+0000", 2024-09-15T00:12:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8378000000000002e-05 2024-09-15T00:12:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.926017+0000", 2024-09-15T00:12:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2815999999999998e-05 2024-09-15T00:12:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:66000 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1818624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.901160+0000", 2024-09-15T00:12:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.046195507, 2024-09-15T00:12:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031781113, 2024-09-15T00:12:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 66000 2024-09-15T00:12:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901160+0000", 2024-09-15T00:12:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901160+0000", 2024-09-15T00:12:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901160+0000", 2024-09-15T00:12:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:12:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901162+0000", 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901168+0000", 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.908e-06 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901173+0000", 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5299999999999998e-06 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.932742+0000", 2024-09-15T00:12:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031569209000000001 2024-09-15T00:12:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.932848+0000", 2024-09-15T00:12:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000106505 2024-09-15T00:12:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.932941+0000", 2024-09-15T00:12:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2289999999999999e-05 2024-09-15T00:12:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:66001 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.901179+0000", 2024-09-15T00:12:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.046176006, 2024-09-15T00:12:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032023827999999997, 2024-09-15T00:12:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 66001 2024-09-15T00:12:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901179+0000", 2024-09-15T00:12:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901179+0000", 2024-09-15T00:12:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901490+0000", 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00031079499999999999 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901498+0000", 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.841e-06 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901509+0000", 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0872e-05 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901516+0000", 2024-09-15T00:12:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4830000000000002e-06 2024-09-15T00:12:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.932970+0000", 2024-09-15T00:12:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031453864999999998 2024-09-15T00:12:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933099+0000", 2024-09-15T00:12:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000128629 2024-09-15T00:12:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933203+0000", 2024-09-15T00:12:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104343 2024-09-15T00:12:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:66002 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 1941504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.901529+0000", 2024-09-15T00:12:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.045826138000001, 2024-09-15T00:12:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031929382999999999, 2024-09-15T00:12:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 66002 2024-09-15T00:12:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901529+0000", 2024-09-15T00:12:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901529+0000", 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901530+0000", 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901532+0000", 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6869999999999999e-06 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901540+0000", 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5610000000000001e-06 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901545+0000", 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.553e-06 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933235+0000", 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031689945999999997 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933322+0000", 2024-09-15T00:12:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6848999999999995e-05 2024-09-15T00:12:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933458+0000", 2024-09-15T00:12:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013624899999999999 2024-09-15T00:12:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:66003 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.901553+0000", 2024-09-15T00:12:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.045802059, 2024-09-15T00:12:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032108112000000001, 2024-09-15T00:12:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 66003 2024-09-15T00:12:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901553+0000", 2024-09-15T00:12:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901553+0000", 2024-09-15T00:12:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901554+0000", 2024-09-15T00:12:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8999999999999996e-07 2024-09-15T00:12:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901556+0000", 2024-09-15T00:12:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2189999999999998e-06 2024-09-15T00:12:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901564+0000", 2024-09-15T00:12:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4409999999999993e-06 2024-09-15T00:12:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901570+0000", 2024-09-15T00:12:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8239999999999996e-06 2024-09-15T00:12:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933491+0000", 2024-09-15T00:12:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031921110000000003 2024-09-15T00:12:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933576+0000", 2024-09-15T00:12:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4814000000000005e-05 2024-09-15T00:12:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933661+0000", 2024-09-15T00:12:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5113999999999999e-05 2024-09-15T00:12:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:66004 3.4 3:2233e75d:::rbd_data.1139766160da.0000000000000008:13 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.901578+0000", 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.045777414, 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032286845000000002, 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 66004 2024-09-15T00:12:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901578+0000", 2024-09-15T00:12:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901578+0000", 2024-09-15T00:12:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901578+0000", 2024-09-15T00:12:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:12:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901580+0000", 2024-09-15T00:12:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0640000000000001e-06 2024-09-15T00:12:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901587+0000", 2024-09-15T00:12:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-15T00:12:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.901592+0000", 2024-09-15T00:12:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0139999999999998e-06 2024-09-15T00:12:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933705+0000", 2024-09-15T00:12:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032112778000000002 2024-09-15T00:12:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933792+0000", 2024-09-15T00:12:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6849999999999997e-05 2024-09-15T00:12:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.933865+0000", 2024-09-15T00:12:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2742000000000003e-05 2024-09-15T00:12:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.045 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:07.046 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:12:07.047 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:07.045+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:07.148 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:07.146+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:07.150 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:07.150 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:07.150 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:07.150 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:07.159 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:12:07.249 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:07.247+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:07.270 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:07.270 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:07.271 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:07.271 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:07.271 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:07.280 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:12:07.365 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:07.364+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:07.427 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:07.427 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:07.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:07.427 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:07.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.427 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:462 3.0 e29/23 3:04479396:::rbd_id.image_client.0-clone2:head v 29'14, mlcod=27'12)", 2024-09-15T00:12:07.427 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:07.427 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.408227746000001, 2024-09-15T00:12:07.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030963029999999999, 2024-09-15T00:12:07.427 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.427 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.427 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.428 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:07.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:07.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:07.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:12:07.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016208+0000", 2024-09-15T00:12:07.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3669999999999997e-06 2024-09-15T00:12:07.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016250+0000", 2024-09-15T00:12:07.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1845999999999999e-05 2024-09-15T00:12:07.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016260+0000", 2024-09-15T00:12:07.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9750000000000002e-06 2024-09-15T00:12:07.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016342+0000", 2024-09-15T00:12:07.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1595999999999996e-05 2024-09-15T00:12:07.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016363+0000", 2024-09-15T00:12:07.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1073e-05 2024-09-15T00:12:07.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019246+0000", 2024-09-15T00:12:07.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028835240000000002 2024-09-15T00:12:07.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019296+0000", 2024-09-15T00:12:07.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9533999999999997e-05 2024-09-15T00:12:07.431 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.431 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.431 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.431 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:463 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'9, mlcod=27'7)", 2024-09-15T00:12:07.432 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:07.432 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.403614742000002, 2024-09-15T00:12:07.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040015709999999998, 2024-09-15T00:12:07.432 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.432 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.432 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.432 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:07.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:07.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:07.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7599999999999999e-07 2024-09-15T00:12:07.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020818+0000", 2024-09-15T00:12:07.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6609999999999999e-06 2024-09-15T00:12:07.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020826+0000", 2024-09-15T00:12:07.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4770000000000003e-06 2024-09-15T00:12:07.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020833+0000", 2024-09-15T00:12:07.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0809999999999996e-06 2024-09-15T00:12:07.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020878+0000", 2024-09-15T00:12:07.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4752000000000001e-05 2024-09-15T00:12:07.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020892+0000", 2024-09-15T00:12:07.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3899e-05 2024-09-15T00:12:07.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024767+0000", 2024-09-15T00:12:07.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038752259999999998 2024-09-15T00:12:07.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024814+0000", 2024-09-15T00:12:07.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7299000000000002e-05 2024-09-15T00:12:07.436 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.436 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.436 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.436 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:464 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'10, mlcod=27'8)", 2024-09-15T00:12:07.436 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:07.437 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.398694900000002, 2024-09-15T00:12:07.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048109290000000002, 2024-09-15T00:12:07.437 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.437 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.437 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.437 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:07.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:07.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:07.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-15T00:12:07.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025743+0000", 2024-09-15T00:12:07.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5789999999999993e-06 2024-09-15T00:12:07.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025752+0000", 2024-09-15T00:12:07.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8440000000000004e-06 2024-09-15T00:12:07.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025757+0000", 2024-09-15T00:12:07.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2610000000000001e-06 2024-09-15T00:12:07.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025796+0000", 2024-09-15T00:12:07.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9428999999999998e-05 2024-09-15T00:12:07.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025810+0000", 2024-09-15T00:12:07.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3129000000000001e-05 2024-09-15T00:12:07.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030514+0000", 2024-09-15T00:12:07.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047040099999999998 2024-09-15T00:12:07.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030544+0000", 2024-09-15T00:12:07.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9966000000000001e-05 2024-09-15T00:12:07.440 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.441 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.441 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.441 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:610 3.4 e29/23 3:255b0913:::rbd_data.11397bff333d.0000000000000039:head v 29'2, mlcod=0'0)", 2024-09-15T00:12:07.441 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:12:07.441 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.195951305000001, 2024-09-15T00:12:07.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003163014, 2024-09-15T00:12:07.441 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.441 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.441 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.441 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:12:07.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:12:07.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228477+0000", 2024-09-15T00:12:07.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1100000000000004e-07 2024-09-15T00:12:07.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228483+0000", 2024-09-15T00:12:07.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3720000000000001e-06 2024-09-15T00:12:07.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228497+0000", 2024-09-15T00:12:07.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3944000000000001e-05 2024-09-15T00:12:07.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228505+0000", 2024-09-15T00:12:07.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8140000000000001e-06 2024-09-15T00:12:07.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228549+0000", 2024-09-15T00:12:07.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4116000000000002e-05 2024-09-15T00:12:07.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228568+0000", 2024-09-15T00:12:07.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8473000000000001e-05 2024-09-15T00:12:07.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231599+0000", 2024-09-15T00:12:07.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030315590000000001 2024-09-15T00:12:07.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231639+0000", 2024-09-15T00:12:07.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9824999999999999e-05 2024-09-15T00:12:07.445 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.445 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.445 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:766 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'81, mlcod=29'78)", 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.991402227999998, 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029025880000000002, 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:12:07.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433026+0000", 2024-09-15T00:12:07.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:12:07.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433030+0000", 2024-09-15T00:12:07.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6280000000000004e-06 2024-09-15T00:12:07.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433038+0000", 2024-09-15T00:12:07.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.072e-06 2024-09-15T00:12:07.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433045+0000", 2024-09-15T00:12:07.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1510000000000001e-06 2024-09-15T00:12:07.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433101+0000", 2024-09-15T00:12:07.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6336999999999999e-05 2024-09-15T00:12:07.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433113+0000", 2024-09-15T00:12:07.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1639e-05 2024-09-15T00:12:07.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435897+0000", 2024-09-15T00:12:07.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027845779999999998 2024-09-15T00:12:07.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435928+0000", 2024-09-15T00:12:07.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0689999999999999e-05 2024-09-15T00:12:07.449 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.449 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.450 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.450 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:832 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'100, mlcod=29'97)", 2024-09-15T00:12:07.450 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:07.450 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.899156728000001, 2024-09-15T00:12:07.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005498925, 2024-09-15T00:12:07.450 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.450 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.450 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.450 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:07.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7999999999999995e-07 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525276+0000", 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6739999999999996e-06 2024-09-15T00:12:07.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525287+0000", 2024-09-15T00:12:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0906e-05 2024-09-15T00:12:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525293+0000", 2024-09-15T00:12:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4409999999999998e-06 2024-09-15T00:12:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525329+0000", 2024-09-15T00:12:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5237000000000002e-05 2024-09-15T00:12:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525340+0000", 2024-09-15T00:12:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1782e-05 2024-09-15T00:12:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530710+0000", 2024-09-15T00:12:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053695460000000002 2024-09-15T00:12:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530770+0000", 2024-09-15T00:12:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9658999999999999e-05 2024-09-15T00:12:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:876 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'114, mlcod=29'111)", 2024-09-15T00:12:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.728117728000001, 2024-09-15T00:12:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028465830000000002, 2024-09-15T00:12:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2399999999999998e-07 2024-09-15T00:12:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696316+0000", 2024-09-15T00:12:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2800000000000003e-06 2024-09-15T00:12:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696326+0000", 2024-09-15T00:12:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0546e-05 2024-09-15T00:12:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696333+0000", 2024-09-15T00:12:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7660000000000001e-06 2024-09-15T00:12:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696369+0000", 2024-09-15T00:12:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5933000000000003e-05 2024-09-15T00:12:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696381+0000", 2024-09-15T00:12:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2417e-05 2024-09-15T00:12:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699124+0000", 2024-09-15T00:12:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027428000000000001 2024-09-15T00:12:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699156+0000", 2024-09-15T00:12:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2217e-05 2024-09-15T00:12:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1313 3.1f e30/28 3:fa9cde43:::rbd_data.11397bff333d.000000000000003a:head v 30'3, mlcod=26'1)", 2024-09-15T00:12:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:12:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.140278090999999, 2024-09-15T00:12:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044064359999999997, 2024-09-15T00:12:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:12:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:12:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284150+0000", 2024-09-15T00:12:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3799999999999999e-07 2024-09-15T00:12:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284228+0000", 2024-09-15T00:12:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:12:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284235+0000", 2024-09-15T00:12:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:12:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284239+0000", 2024-09-15T00:12:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3449999999999999e-06 2024-09-15T00:12:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284282+0000", 2024-09-15T00:12:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2506000000000003e-05 2024-09-15T00:12:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284297+0000", 2024-09-15T00:12:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5523999999999999e-05 2024-09-15T00:12:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288517+0000", 2024-09-15T00:12:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042194019999999997 2024-09-15T00:12:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288556+0000", 2024-09-15T00:12:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8952000000000002e-05 2024-09-15T00:12:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1321 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'7, mlcod=29'5)", 2024-09-15T00:12:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.116184832000002, 2024-09-15T00:12:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003131254, 2024-09-15T00:12:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:12:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308250+0000", 2024-09-15T00:12:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6109999999999999e-06 2024-09-15T00:12:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308264+0000", 2024-09-15T00:12:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4473e-05 2024-09-15T00:12:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308272+0000", 2024-09-15T00:12:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3540000000000002e-06 2024-09-15T00:12:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308340+0000", 2024-09-15T00:12:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8305000000000004e-05 2024-09-15T00:12:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308361+0000", 2024-09-15T00:12:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0409999999999999e-05 2024-09-15T00:12:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311332+0000", 2024-09-15T00:12:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029715950000000001 2024-09-15T00:12:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311374+0000", 2024-09-15T00:12:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1786000000000003e-05 2024-09-15T00:12:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.9 epoch 30/23 rep_tid 277 entries 30'217 (0'0) error 3:942069cf:::rbd_header.11397bff333d:head by client.4409.0:1373 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'214)", 2024-09-15T00:12:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:12:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.070632234999998, 2024-09-15T00:12:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034617960000000001, 2024-09-15T00:12:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:12:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:12:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353796+0000", 2024-09-15T00:12:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6699999999999999e-07 2024-09-15T00:12:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353803+0000", 2024-09-15T00:12:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.832e-06 2024-09-15T00:12:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353814+0000", 2024-09-15T00:12:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1815e-05 2024-09-15T00:12:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353822+0000", 2024-09-15T00:12:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7440000000000007e-06 2024-09-15T00:12:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353910+0000", 2024-09-15T00:12:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7491000000000006e-05 2024-09-15T00:12:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.357257+0000", 2024-09-15T00:12:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033474469999999999 2024-09-15T00:12:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1374 3.9 e30/23 3:942069cf:::rbd_header.11397bff333d:head v 30'218, mlcod=30'215)", 2024-09-15T00:12:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:12:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.069232593000002, 2024-09-15T00:12:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047299830000000001, 2024-09-15T00:12:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:12:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:12:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355196+0000", 2024-09-15T00:12:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3900000000000004e-07 2024-09-15T00:12:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355200+0000", 2024-09-15T00:12:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5700000000000003e-06 2024-09-15T00:12:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355210+0000", 2024-09-15T00:12:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6050000000000004e-06 2024-09-15T00:12:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355215+0000", 2024-09-15T00:12:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.933e-06 2024-09-15T00:12:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355249+0000", 2024-09-15T00:12:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4110999999999999e-05 2024-09-15T00:12:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355260+0000", 2024-09-15T00:12:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1235e-05 2024-09-15T00:12:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359895+0000", 2024-09-15T00:12:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046345550000000003 2024-09-15T00:12:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359925+0000", 2024-09-15T00:12:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0335e-05 2024-09-15T00:12:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1385 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'221, mlcod=30'218)", 2024-09-15T00:12:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:12:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.056776057999997, 2024-09-15T00:12:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030443950000000001, 2024-09-15T00:12:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:12:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:12:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367652+0000", 2024-09-15T00:12:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:12:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367657+0000", 2024-09-15T00:12:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5410000000000002e-06 2024-09-15T00:12:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367666+0000", 2024-09-15T00:12:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9280000000000005e-06 2024-09-15T00:12:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367671+0000", 2024-09-15T00:12:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7870000000000001e-06 2024-09-15T00:12:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367707+0000", 2024-09-15T00:12:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6102000000000001e-05 2024-09-15T00:12:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367719+0000", 2024-09-15T00:12:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1571e-05 2024-09-15T00:12:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370666+0000", 2024-09-15T00:12:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029474750000000002 2024-09-15T00:12:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370696+0000", 2024-09-15T00:12:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9402000000000001e-05 2024-09-15T00:12:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1662 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'300, mlcod=30'297)", 2024-09-15T00:12:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:12:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.636411420999998, 2024-09-15T00:12:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031296090000000002, 2024-09-15T00:12:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:12:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:12:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788017+0000", 2024-09-15T00:12:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8999999999999996e-07 2024-09-15T00:12:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788021+0000", 2024-09-15T00:12:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4970000000000003e-06 2024-09-15T00:12:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788027+0000", 2024-09-15T00:12:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1840000000000004e-06 2024-09-15T00:12:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788032+0000", 2024-09-15T00:12:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.135e-06 2024-09-15T00:12:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788070+0000", 2024-09-15T00:12:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7073999999999998e-05 2024-09-15T00:12:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788081+0000", 2024-09-15T00:12:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0987e-05 2024-09-15T00:12:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791116+0000", 2024-09-15T00:12:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030355209999999998 2024-09-15T00:12:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791146+0000", 2024-09-15T00:12:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9621e-05 2024-09-15T00:12:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1677 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'305, mlcod=30'302)", 2024-09-15T00:12:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:12:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.612683058999998, 2024-09-15T00:12:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031364320000000002, 2024-09-15T00:12:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:12:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:12:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811745+0000", 2024-09-15T00:12:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:12:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811749+0000", 2024-09-15T00:12:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9369999999999997e-06 2024-09-15T00:12:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811756+0000", 2024-09-15T00:12:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:12:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811765+0000", 2024-09-15T00:12:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7930000000000003e-06 2024-09-15T00:12:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811797+0000", 2024-09-15T00:12:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2203999999999999e-05 2024-09-15T00:12:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811810+0000", 2024-09-15T00:12:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2359e-05 2024-09-15T00:12:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814852+0000", 2024-09-15T00:12:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003042276 2024-09-15T00:12:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814881+0000", 2024-09-15T00:12:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9037999999999999e-05 2024-09-15T00:12:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1724 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'9, mlcod=30'7)", 2024-09-15T00:12:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:12:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.543757374999998, 2024-09-15T00:12:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003115769, 2024-09-15T00:12:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:12:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:12:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880671+0000", 2024-09-15T00:12:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4900000000000005e-07 2024-09-15T00:12:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880678+0000", 2024-09-15T00:12:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0979999999999996e-06 2024-09-15T00:12:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880691+0000", 2024-09-15T00:12:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3288999999999999e-05 2024-09-15T00:12:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880698+0000", 2024-09-15T00:12:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9240000000000001e-06 2024-09-15T00:12:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880757+0000", 2024-09-15T00:12:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8866000000000002e-05 2024-09-15T00:12:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880790+0000", 2024-09-15T00:12:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2926999999999998e-05 2024-09-15T00:12:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883738+0000", 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002948418 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883786+0000", 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7398000000000003e-05 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1790 3.9 e30/23 3:9676e01b:::rbd_data.11397bff333d.0000000000000030:head v 30'338, mlcod=30'335)", 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:12:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.464875005000003, 2024-09-15T00:12:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044496980000000002, 2024-09-15T00:12:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:12:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:12:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959553+0000", 2024-09-15T00:12:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3600000000000004e-07 2024-09-15T00:12:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959569+0000", 2024-09-15T00:12:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6152999999999999e-05 2024-09-15T00:12:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959577+0000", 2024-09-15T00:12:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5179999999999998e-06 2024-09-15T00:12:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959581+0000", 2024-09-15T00:12:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3669999999999999e-06 2024-09-15T00:12:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959619+0000", 2024-09-15T00:12:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8209e-05 2024-09-15T00:12:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959632+0000", 2024-09-15T00:12:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2405000000000001e-05 2024-09-15T00:12:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.963954+0000", 2024-09-15T00:12:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043226660000000002 2024-09-15T00:12:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.964002+0000", 2024-09-15T00:12:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7843999999999998e-05 2024-09-15T00:12:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.499 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:3177 3.4 e33/23 3:23818c00:::rbd_data.1139b531b717.0000000000000022:head v 33'58, mlcod=33'56)", 2024-09-15T00:12:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:12:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.766586439000001, 2024-09-15T00:12:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030564120000000001, 2024-09-15T00:12:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:12:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:12:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657842+0000", 2024-09-15T00:12:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.19e-06 2024-09-15T00:12:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657853+0000", 2024-09-15T00:12:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:12:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657871+0000", 2024-09-15T00:12:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8233e-05 2024-09-15T00:12:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657880+0000", 2024-09-15T00:12:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8319999999999995e-06 2024-09-15T00:12:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657949+0000", 2024-09-15T00:12:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8847e-05 2024-09-15T00:12:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657977+0000", 2024-09-15T00:12:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8308000000000001e-05 2024-09-15T00:12:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.660851+0000", 2024-09-15T00:12:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028743190000000002 2024-09-15T00:12:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.660897+0000", 2024-09-15T00:12:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6142e-05 2024-09-15T00:12:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:3602 3.1f e34/28 3:fd28af01:::rbd_data.1139b531b717.0000000000000032:head v 34'24, mlcod=33'22)", 2024-09-15T00:12:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:12:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.379189044, 2024-09-15T00:12:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034159709999999999, 2024-09-15T00:12:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:12:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:12:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045239+0000", 2024-09-15T00:12:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:12:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045252+0000", 2024-09-15T00:12:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2891000000000001e-05 2024-09-15T00:12:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045263+0000", 2024-09-15T00:12:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0875e-05 2024-09-15T00:12:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045271+0000", 2024-09-15T00:12:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3529999999999995e-06 2024-09-15T00:12:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045338+0000", 2024-09-15T00:12:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7377000000000002e-05 2024-09-15T00:12:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045359+0000", 2024-09-15T00:12:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1078e-05 2024-09-15T00:12:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.048621+0000", 2024-09-15T00:12:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003261543 2024-09-15T00:12:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.048654+0000", 2024-09-15T00:12:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3550999999999998e-05 2024-09-15T00:12:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:4253 3.4 e35/23 3:242f5431:::rbd_data.11396389f8f2.0000000000000000:head v 35'87, mlcod=35'85)", 2024-09-15T00:12:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:12:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.031700186000002, 2024-09-15T00:12:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002944997, 2024-09-15T00:12:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:12:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:12:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392728+0000", 2024-09-15T00:12:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.06e-07 2024-09-15T00:12:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392782+0000", 2024-09-15T00:12:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.448e-05 2024-09-15T00:12:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392794+0000", 2024-09-15T00:12:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1768e-05 2024-09-15T00:12:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392801+0000", 2024-09-15T00:12:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9589999999999998e-06 2024-09-15T00:12:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392845+0000", 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3884999999999997e-05 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392860+0000", 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5364999999999999e-05 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.395641+0000", 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027807629999999999 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.395672+0000", 2024-09-15T00:12:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1170999999999997e-05 2024-09-15T00:12:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:18791 3.0 e38/23 3:05ecfc97:::rbd_data.11398e1a091a.0000000000000030:head v 38'66, mlcod=38'64)", 2024-09-15T00:12:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:12:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.388784650000002, 2024-09-15T00:12:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029447940000000001, 2024-09-15T00:12:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:12:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:12:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035644+0000", 2024-09-15T00:12:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4e-07 2024-09-15T00:12:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035684+0000", 2024-09-15T00:12:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9857999999999997e-05 2024-09-15T00:12:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035694+0000", 2024-09-15T00:12:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0259e-05 2024-09-15T00:12:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035699+0000", 2024-09-15T00:12:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6389999999999997e-06 2024-09-15T00:12:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035745+0000", 2024-09-15T00:12:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5640999999999999e-05 2024-09-15T00:12:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035764+0000", 2024-09-15T00:12:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9368000000000002e-05 2024-09-15T00:12:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.038551+0000", 2024-09-15T00:12:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027860879999999999 2024-09-15T00:12:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.038588+0000", 2024-09-15T00:12:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7101000000000003e-05 2024-09-15T00:12:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.517 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:07.519 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:07.519 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:07.517+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:07.620 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:07.618+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:07.664 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:07.664 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:07.664 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:07.664 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:07.673 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:07.721 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:07.719+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:07.776 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:07.776 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:07.776 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:07.776 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:07.776 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:07.785 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:07.825 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:07.824+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:07.922 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:07.920+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:07.930 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:47798 3.7 3:e6004524:::rbd_data.113916cf513.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1789239~44328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.387407+0000", 2024-09-15T00:12:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.540050531999999, 2024-09-15T00:12:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033697973999999999, 2024-09-15T00:12:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 47798 2024-09-15T00:12:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.387407+0000", 2024-09-15T00:12:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.387407+0000", 2024-09-15T00:12:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.387408+0000", 2024-09-15T00:12:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2580000000000001e-06 2024-09-15T00:12:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389094+0000", 2024-09-15T00:12:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001685572 2024-09-15T00:12:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389116+0000", 2024-09-15T00:12:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2379999999999999e-05 2024-09-15T00:12:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389126+0000", 2024-09-15T00:12:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0044000000000001e-05 2024-09-15T00:12:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389203+0000", 2024-09-15T00:12:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6945000000000003e-05 2024-09-15T00:12:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389381+0000", 2024-09-15T00:12:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000177908 2024-09-15T00:12:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389890+0000", 2024-09-15T00:12:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00050844599999999996 2024-09-15T00:12:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.420647+0000", 2024-09-15T00:12:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030757201000000001 2024-09-15T00:12:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.420970+0000", 2024-09-15T00:12:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032290200000000002 2024-09-15T00:12:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.421037+0000", 2024-09-15T00:12:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6883000000000003e-05 2024-09-15T00:12:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.421105+0000", 2024-09-15T00:12:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8435000000000005e-05 2024-09-15T00:12:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56198 3.2 3:44ae5a28:::rbd_data.1139167e758e.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:12:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.238999+0000", 2024-09-15T00:12:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.688458962999999, 2024-09-15T00:12:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037097333000000003, 2024-09-15T00:12:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56198 2024-09-15T00:12:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.238999+0000", 2024-09-15T00:12:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.238999+0000", 2024-09-15T00:12:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.239000+0000", 2024-09-15T00:12:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.359e-06 2024-09-15T00:12:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242065+0000", 2024-09-15T00:12:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030647230000000001 2024-09-15T00:12:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242079+0000", 2024-09-15T00:12:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3801999999999999e-05 2024-09-15T00:12:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242108+0000", 2024-09-15T00:12:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9889000000000001e-05 2024-09-15T00:12:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242177+0000", 2024-09-15T00:12:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9072000000000005e-05 2024-09-15T00:12:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242332+0000", 2024-09-15T00:12:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000154816 2024-09-15T00:12:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.243189+0000", 2024-09-15T00:12:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00085691200000000004 2024-09-15T00:12:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.267650+0000", 2024-09-15T00:12:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024461146999999999 2024-09-15T00:12:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.275977+0000", 2024-09-15T00:12:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0083269729999999997 2024-09-15T00:12:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276040+0000", 2024-09-15T00:12:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2497999999999997e-05 2024-09-15T00:12:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276096+0000", 2024-09-15T00:12:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6141999999999999e-05 2024-09-15T00:12:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56199 3.2 3:44ae5a28:::rbd_data.1139167e758e.0000000000000000:head [write 0~0] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:12:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242772+0000", 2024-09-15T00:12:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.684685559999998, 2024-09-15T00:12:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033898029000000003, 2024-09-15T00:12:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56199 2024-09-15T00:12:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242772+0000", 2024-09-15T00:12:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242772+0000", 2024-09-15T00:12:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242774+0000", 2024-09-15T00:12:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.483e-06 2024-09-15T00:12:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242782+0000", 2024-09-15T00:12:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9740000000000002e-06 2024-09-15T00:12:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242806+0000", 2024-09-15T00:12:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4689e-05 2024-09-15T00:12:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242815+0000", 2024-09-15T00:12:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3410000000000003e-06 2024-09-15T00:12:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.265144+0000", 2024-09-15T00:12:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022329244000000002 2024-09-15T00:12:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.265237+0000", 2024-09-15T00:12:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3154999999999998e-05 2024-09-15T00:12:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.265426+0000", 2024-09-15T00:12:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018955600000000001 2024-09-15T00:12:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.267912+0000", 2024-09-15T00:12:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024857880000000001 2024-09-15T00:12:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276583+0000", 2024-09-15T00:12:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0086705689999999995 2024-09-15T00:12:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276634+0000", 2024-09-15T00:12:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1465000000000003e-05 2024-09-15T00:12:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276670+0000", 2024-09-15T00:12:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5765000000000001e-05 2024-09-15T00:12:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:12:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.684613513999999, 2024-09-15T00:12:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:12:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:12:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:12:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:12:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:12:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:12:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:12:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:12:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:12:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:12:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:12:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:12:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:12:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:12:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:12:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:12:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:12:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:12:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:12:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:12:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:12:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:12:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:12:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:12:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56207 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [write 0~0] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:12:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.245178+0000", 2024-09-15T00:12:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.682279814000001, 2024-09-15T00:12:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038624626000000002, 2024-09-15T00:12:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56207 2024-09-15T00:12:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245178+0000", 2024-09-15T00:12:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245178+0000", 2024-09-15T00:12:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245180+0000", 2024-09-15T00:12:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-15T00:12:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245185+0000", 2024-09-15T00:12:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1780000000000002e-06 2024-09-15T00:12:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245192+0000", 2024-09-15T00:12:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6420000000000007e-06 2024-09-15T00:12:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245198+0000", 2024-09-15T00:12:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5729999999999998e-06 2024-09-15T00:12:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.271149+0000", 2024-09-15T00:12:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025950534000000001 2024-09-15T00:12:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.271238+0000", 2024-09-15T00:12:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9364999999999996e-05 2024-09-15T00:12:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.271398+0000", 2024-09-15T00:12:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015962600000000001 2024-09-15T00:12:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.277582+0000", 2024-09-15T00:12:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061847669999999999 2024-09-15T00:12:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283748+0000", 2024-09-15T00:12:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061654170000000003 2024-09-15T00:12:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283782+0000", 2024-09-15T00:12:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4245000000000002e-05 2024-09-15T00:12:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283802+0000", 2024-09-15T00:12:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-05 2024-09-15T00:12:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58878 3.8 3:12456482:::rbd_data.1139766160da.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.63830793, 2024-09-15T00:12:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061956199000000003, 2024-09-15T00:12:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58878 2024-09-15T00:12:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289151+0000", 2024-09-15T00:12:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:12:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289473+0000", 2024-09-15T00:12:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032183800000000002 2024-09-15T00:12:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289487+0000", 2024-09-15T00:12:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-05 2024-09-15T00:12:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289493+0000", 2024-09-15T00:12:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4489999999999996e-06 2024-09-15T00:12:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289542+0000", 2024-09-15T00:12:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8844999999999997e-05 2024-09-15T00:12:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289642+0000", 2024-09-15T00:12:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9607999999999999e-05 2024-09-15T00:12:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.293908+0000", 2024-09-15T00:12:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659350000000002 2024-09-15T00:12:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351022+0000", 2024-09-15T00:12:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057114535000000001 2024-09-15T00:12:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351033+0000", 2024-09-15T00:12:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0263e-05 2024-09-15T00:12:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351070+0000", 2024-09-15T00:12:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7666000000000002e-05 2024-09-15T00:12:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351106+0000", 2024-09-15T00:12:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999997e-05 2024-09-15T00:12:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58879 3.7 3:e169894d:::rbd_data.1139766160da.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.635956441999999, 2024-09-15T00:12:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043716294000000003, 2024-09-15T00:12:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58879 2024-09-15T00:12:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291502+0000", 2024-09-15T00:12:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-15T00:12:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291664+0000", 2024-09-15T00:12:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016216599999999999 2024-09-15T00:12:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291674+0000", 2024-09-15T00:12:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0276e-05 2024-09-15T00:12:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291680+0000", 2024-09-15T00:12:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2580000000000003e-06 2024-09-15T00:12:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291732+0000", 2024-09-15T00:12:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2114999999999998e-05 2024-09-15T00:12:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291866+0000", 2024-09-15T00:12:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013416900000000001 2024-09-15T00:12:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.296426+0000", 2024-09-15T00:12:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045595779999999999 2024-09-15T00:12:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335075+0000", 2024-09-15T00:12:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038649347000000001 2024-09-15T00:12:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335097+0000", 2024-09-15T00:12:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2565000000000002e-05 2024-09-15T00:12:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335161+0000", 2024-09-15T00:12:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3928000000000002e-05 2024-09-15T00:12:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335217+0000", 2024-09-15T00:12:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6042000000000003e-05 2024-09-15T00:12:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58880 3.8 3:13d91cfd:::rbd_data.1139766160da.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.632564335, 2024-09-15T00:12:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056331463999999998, 2024-09-15T00:12:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58880 2024-09-15T00:12:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294896+0000", 2024-09-15T00:12:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.356e-06 2024-09-15T00:12:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295288+0000", 2024-09-15T00:12:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039223200000000001 2024-09-15T00:12:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295304+0000", 2024-09-15T00:12:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5937e-05 2024-09-15T00:12:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295312+0000", 2024-09-15T00:12:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1489999999999994e-06 2024-09-15T00:12:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333162+0000", 2024-09-15T00:12:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037849840000000003 2024-09-15T00:12:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333269+0000", 2024-09-15T00:12:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010749800000000001 2024-09-15T00:12:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.336198+0000", 2024-09-15T00:12:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029284739999999999 2024-09-15T00:12:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351137+0000", 2024-09-15T00:12:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014939103 2024-09-15T00:12:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351150+0000", 2024-09-15T00:12:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3542e-05 2024-09-15T00:12:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351191+0000", 2024-09-15T00:12:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0791000000000003e-05 2024-09-15T00:12:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351225+0000", 2024-09-15T00:12:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3541999999999998e-05 2024-09-15T00:12:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58878 3.8 e56/23 ondisk, result = 0)", 2024-09-15T00:12:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.626034552, 2024-09-15T00:12:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049689096000000002, 2024-09-15T00:12:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301425+0000", 2024-09-15T00:12:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.782e-06 2024-09-15T00:12:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301433+0000", 2024-09-15T00:12:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689999999999994e-06 2024-09-15T00:12:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301449+0000", 2024-09-15T00:12:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5767e-05 2024-09-15T00:12:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301458+0000", 2024-09-15T00:12:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:12:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351009+0000", 2024-09-15T00:12:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049551523 2024-09-15T00:12:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351017+0000", 2024-09-15T00:12:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8359999999999992e-06 2024-09-15T00:12:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351112+0000", 2024-09-15T00:12:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5173999999999994e-05 2024-09-15T00:12:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58879 3.7 e56/55 ondisk, result = 0)", 2024-09-15T00:12:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.303373+0000", 2024-09-15T00:12:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.62408443, 2024-09-15T00:12:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031714352000000001, 2024-09-15T00:12:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.303373+0000", 2024-09-15T00:12:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.303373+0000", 2024-09-15T00:12:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.303375+0000", 2024-09-15T00:12:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3510000000000001e-06 2024-09-15T00:12:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.303380+0000", 2024-09-15T00:12:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6840000000000001e-06 2024-09-15T00:12:07.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:07.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.303392+0000", 2024-09-15T00:12:07.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1847e-05 2024-09-15T00:12:07.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:07.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:07.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:07.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.303401+0000", 2024-09-15T00:12:07.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0219999999999993e-06 2024-09-15T00:12:07.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:08.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.334994+0000", 2024-09-15T00:12:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031592758999999998 2024-09-15T00:12:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335005+0000", 2024-09-15T00:12:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:12:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335088+0000", 2024-09-15T00:12:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2808999999999999e-05 2024-09-15T00:12:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:08.001 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:62811 3.1e 3:7e43c43a:::rbd_data.1139766160da.000000000000001b:13 [read 1937408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.656753+0000", 2024-09-15T00:12:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.270704485, 2024-09-15T00:12:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030335925999999999, 2024-09-15T00:12:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 62811 2024-09-15T00:12:08.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656753+0000", 2024-09-15T00:12:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656753+0000", 2024-09-15T00:12:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:08.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656754+0000", 2024-09-15T00:12:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2600000000000002e-07 2024-09-15T00:12:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656767+0000", 2024-09-15T00:12:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3088000000000001e-05 2024-09-15T00:12:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:08.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656773+0000", 2024-09-15T00:12:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3790000000000003e-06 2024-09-15T00:12:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656778+0000", 2024-09-15T00:12:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-15T00:12:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.686998+0000", 2024-09-15T00:12:08.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030219629000000001 2024-09-15T00:12:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687046+0000", 2024-09-15T00:12:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8547000000000001e-05 2024-09-15T00:12:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687089+0000", 2024-09-15T00:12:08.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2926999999999997e-05 2024-09-15T00:12:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:62812 3.1e 3:7e43c43a:::rbd_data.1139766160da.000000000000001b:13 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.656789+0000", 2024-09-15T00:12:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.270668549, 2024-09-15T00:12:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030406757999999999, 2024-09-15T00:12:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:08.007 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 62812 2024-09-15T00:12:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656789+0000", 2024-09-15T00:12:08.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656789+0000", 2024-09-15T00:12:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656790+0000", 2024-09-15T00:12:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.75e-07 2024-09-15T00:12:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656791+0000", 2024-09-15T00:12:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9400000000000001e-06 2024-09-15T00:12:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656801+0000", 2024-09-15T00:12:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5589999999999994e-06 2024-09-15T00:12:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656808+0000", 2024-09-15T00:12:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1489999999999997e-06 2024-09-15T00:12:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687116+0000", 2024-09-15T00:12:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030308162999999999 2024-09-15T00:12:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687161+0000", 2024-09-15T00:12:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4764e-05 2024-09-15T00:12:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687196+0000", 2024-09-15T00:12:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4708e-05 2024-09-15T00:12:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:62813 3.1e 3:7e43c43a:::rbd_data.1139766160da.000000000000001b:13 [read 2060288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.656815+0000", 2024-09-15T00:12:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.270642691000001, 2024-09-15T00:12:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030516998, 2024-09-15T00:12:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:08.013 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 62813 2024-09-15T00:12:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656815+0000", 2024-09-15T00:12:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656815+0000", 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656815+0000", 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656817+0000", 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.08e-06 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656824+0000", 2024-09-15T00:12:08.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1249999999999998e-06 2024-09-15T00:12:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656828+0000", 2024-09-15T00:12:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9010000000000002e-06 2024-09-15T00:12:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687224+0000", 2024-09-15T00:12:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030395884000000001 2024-09-15T00:12:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687274+0000", 2024-09-15T00:12:08.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9455e-05 2024-09-15T00:12:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687332+0000", 2024-09-15T00:12:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8143000000000002e-05 2024-09-15T00:12:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:62814 3.1e 3:7e43c43a:::rbd_data.1139766160da.000000000000001b:13 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:08.017 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.656834+0000", 2024-09-15T00:12:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.270623583999999, 2024-09-15T00:12:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030596983000000001, 2024-09-15T00:12:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 62814 2024-09-15T00:12:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656834+0000", 2024-09-15T00:12:08.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656834+0000", 2024-09-15T00:12:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656834+0000", 2024-09-15T00:12:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:12:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:08.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656842+0000", 2024-09-15T00:12:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.525e-06 2024-09-15T00:12:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656848+0000", 2024-09-15T00:12:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4169999999999997e-06 2024-09-15T00:12:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656864+0000", 2024-09-15T00:12:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5591999999999999e-05 2024-09-15T00:12:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687353+0000", 2024-09-15T00:12:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030488718000000001 2024-09-15T00:12:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687394+0000", 2024-09-15T00:12:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0973999999999998e-05 2024-09-15T00:12:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687431+0000", 2024-09-15T00:12:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7397999999999997e-05 2024-09-15T00:12:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:79533 3.1e 3:7f55ca9b:::rbd_data.1139741c8001.0000000000000015:14 [read 888832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:12:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:57.830591+0000", 2024-09-15T00:12:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.096866466, 2024-09-15T00:12:08.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030265637000000001, 2024-09-15T00:12:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79533 2024-09-15T00:12:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830591+0000", 2024-09-15T00:12:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830591+0000", 2024-09-15T00:12:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830592+0000", 2024-09-15T00:12:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7599999999999998e-07 2024-09-15T00:12:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830593+0000", 2024-09-15T00:12:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.885e-06 2024-09-15T00:12:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830599+0000", 2024-09-15T00:12:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-15T00:12:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830603+0000", 2024-09-15T00:12:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.33e-06 2024-09-15T00:12:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.860758+0000", 2024-09-15T00:12:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030154389 2024-09-15T00:12:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.860810+0000", 2024-09-15T00:12:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2568000000000002e-05 2024-09-15T00:12:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.860857+0000", 2024-09-15T00:12:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6360000000000003e-05 2024-09-15T00:12:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:79534 3.1e 3:7f55ca9b:::rbd_data.1139741c8001.0000000000000015:14 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:12:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:57.830609+0000", 2024-09-15T00:12:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.096848544, 2024-09-15T00:12:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030373054, 2024-09-15T00:12:08.027 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79534 2024-09-15T00:12:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830609+0000", 2024-09-15T00:12:08.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830609+0000", 2024-09-15T00:12:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830609+0000", 2024-09-15T00:12:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9700000000000003e-07 2024-09-15T00:12:08.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830611+0000", 2024-09-15T00:12:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4619999999999999e-06 2024-09-15T00:12:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830621+0000", 2024-09-15T00:12:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9580000000000002e-06 2024-09-15T00:12:08.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830625+0000", 2024-09-15T00:12:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-15T00:12:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.860890+0000", 2024-09-15T00:12:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030265071000000001 2024-09-15T00:12:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.860942+0000", 2024-09-15T00:12:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1938999999999999e-05 2024-09-15T00:12:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.860982+0000", 2024-09-15T00:12:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0407999999999997e-05 2024-09-15T00:12:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:08.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:79535 3.1e 3:7f55ca9b:::rbd_data.1139741c8001.0000000000000015:14 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:12:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:57.830630+0000", 2024-09-15T00:12:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.096827349, 2024-09-15T00:12:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030460928000000002, 2024-09-15T00:12:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79535 2024-09-15T00:12:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830630+0000", 2024-09-15T00:12:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830630+0000", 2024-09-15T00:12:08.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830631+0000", 2024-09-15T00:12:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1099999999999996e-07 2024-09-15T00:12:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830632+0000", 2024-09-15T00:12:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.672e-06 2024-09-15T00:12:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830636+0000", 2024-09-15T00:12:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-06 2024-09-15T00:12:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830640+0000", 2024-09-15T00:12:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2590000000000002e-06 2024-09-15T00:12:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:08.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.861010+0000", 2024-09-15T00:12:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030369213999999999 2024-09-15T00:12:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.861052+0000", 2024-09-15T00:12:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2401999999999998e-05 2024-09-15T00:12:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.861091+0000", 2024-09-15T00:12:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9279000000000001e-05 2024-09-15T00:12:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.037 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:84253 3.18 3:1a889b9d:::rbd_data.113947ee3a89.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-15T00:12:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:59.323633+0000", 2024-09-15T00:12:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.6038242300000007, 2024-09-15T00:12:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034950439999999999, 2024-09-15T00:12:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:08.038 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 84253 2024-09-15T00:12:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.323633+0000", 2024-09-15T00:12:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.323633+0000", 2024-09-15T00:12:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.323634+0000", 2024-09-15T00:12:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.34e-07 2024-09-15T00:12:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324115+0000", 2024-09-15T00:12:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00048104099999999998 2024-09-15T00:12:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324123+0000", 2024-09-15T00:12:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5459999999999992e-06 2024-09-15T00:12:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324130+0000", 2024-09-15T00:12:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3740000000000003e-06 2024-09-15T00:12:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.346309+0000", 2024-09-15T00:12:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022179194999999999 2024-09-15T00:12:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.346420+0000", 2024-09-15T00:12:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000111093 2024-09-15T00:12:08.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.346612+0000", 2024-09-15T00:12:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019197600000000001 2024-09-15T00:12:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.356985+0000", 2024-09-15T00:12:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01037306 2024-09-15T00:12:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:08.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.358458+0000", 2024-09-15T00:12:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001473001 2024-09-15T00:12:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.358532+0000", 2024-09-15T00:12:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3541000000000006e-05 2024-09-15T00:12:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.358584+0000", 2024-09-15T00:12:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2179000000000003e-05 2024-09-15T00:12:08.044 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:84254 3.18 3:1a889b9d:::rbd_data.113947ee3a89.0000000000000005:head [write 0~0] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-15T00:12:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:59.324141+0000", 2024-09-15T00:12:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.6033161939999996, 2024-09-15T00:12:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034565371999999997, 2024-09-15T00:12:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 84254 2024-09-15T00:12:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324141+0000", 2024-09-15T00:12:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324141+0000", 2024-09-15T00:12:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324142+0000", 2024-09-15T00:12:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7300000000000005e-07 2024-09-15T00:12:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:08.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324145+0000", 2024-09-15T00:12:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.661e-06 2024-09-15T00:12:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324151+0000", 2024-09-15T00:12:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5139999999999998e-06 2024-09-15T00:12:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324157+0000", 2024-09-15T00:12:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2569999999999998e-06 2024-09-15T00:12:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.351607+0000", 2024-09-15T00:12:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.027450862999999999 2024-09-15T00:12:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.351699+0000", 2024-09-15T00:12:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1247000000000002e-05 2024-09-15T00:12:08.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.351895+0000", 2024-09-15T00:12:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019662600000000001 2024-09-15T00:12:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.356991+0000", 2024-09-15T00:12:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005095774 2024-09-15T00:12:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.358642+0000", 2024-09-15T00:12:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001651306 2024-09-15T00:12:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.358682+0000", 2024-09-15T00:12:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000003e-05 2024-09-15T00:12:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.358707+0000", 2024-09-15T00:12:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5160999999999998e-05 2024-09-15T00:12:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:95167 3.1e 3:7941b88e:::rbd_data.1139e6d65aff.000000000000001b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T00:12:08.052 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:03.555273+0000", 2024-09-15T00:12:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3721850599999996, 2024-09-15T00:12:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036694817999999997, 2024-09-15T00:12:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 95167 2024-09-15T00:12:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.053 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555273+0000", 2024-09-15T00:12:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555273+0000", 2024-09-15T00:12:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555295+0000", 2024-09-15T00:12:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2189999999999999e-05 2024-09-15T00:12:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555303+0000", 2024-09-15T00:12:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8869999999999992e-06 2024-09-15T00:12:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555321+0000", 2024-09-15T00:12:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8202999999999998e-05 2024-09-15T00:12:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555331+0000", 2024-09-15T00:12:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.842e-06 2024-09-15T00:12:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555390+0000", 2024-09-15T00:12:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9422999999999997e-05 2024-09-15T00:12:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555507+0000", 2024-09-15T00:12:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116762 2024-09-15T00:12:08.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:12:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.558432+0000", 2024-09-15T00:12:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029248239999999999 2024-09-15T00:12:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.591882+0000", 2024-09-15T00:12:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033450739 2024-09-15T00:12:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.591896+0000", 2024-09-15T00:12:08.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4053000000000001e-05 2024-09-15T00:12:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.591934+0000", 2024-09-15T00:12:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7515000000000003e-05 2024-09-15T00:12:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.591967+0000", 2024-09-15T00:12:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000002e-05 2024-09-15T00:12:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:08.058 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:08.059 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:08.060 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:08.060 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:08.058+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:08.161 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:08.160+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:08.174 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:08.174 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:08.174 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:08.174 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:08.184 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:08.229 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:12:08.272 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:08.271+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:08.292 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:08.292 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:08.293 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:08.293 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:08.293 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:08.301 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:08.363 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:08.361+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:08.407 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:08.407 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:08.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:08.407 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:08.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.407 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:08.407 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:08.407 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.832422188000002, 2024-09-15T00:12:08.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:12:08.407 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.408 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.408 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:08.408 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.408 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.408 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.408 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:12:08.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.408 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:08.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:08.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:12:08.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:12:08.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:12:08.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:12:08.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:12:08.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:12:08.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:12:08.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:12:08.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:12:08.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:12:08.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:12:08.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:12:08.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:08.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:12:08.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:12:08.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:08.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:12:08.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:12:08.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:08.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:12:08.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:12:08.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:08.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:12:08.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:12:08.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:12:08.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:12:08.413 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.413 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.413 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.413 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:08.413 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:08.413 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.832387496000003, 2024-09-15T00:12:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:12:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:12:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:08.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:08.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:12:08.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:12:08.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:12:08.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:12:08.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:12:08.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:12:08.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:12:08.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:12:08.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:12:08.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:12:08.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:12:08.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:08.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:12:08.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:12:08.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:08.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:12:08.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:12:08.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:08.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:12:08.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:12:08.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:08.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:12:08.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:12:08.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:12:08.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:12:08.419 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.419 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.419 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.419 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48937 3.1a 3:5e7e71d0:::rbd_data.11395b9829f3.0000000000000000:e [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.419 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:12:08.419 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.792294541, 2024-09-15T00:12:08.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072322336000000001, 2024-09-15T00:12:08.420 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.420 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.420 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.420 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.420 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.420 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.420 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48937 2024-09-15T00:12:08.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.420 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:12:08.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:12:08.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612043+0000", 2024-09-15T00:12:08.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:12:08.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612045+0000", 2024-09-15T00:12:08.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2749999999999998e-06 2024-09-15T00:12:08.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612053+0000", 2024-09-15T00:12:08.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0439999999999996e-06 2024-09-15T00:12:08.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612062+0000", 2024-09-15T00:12:08.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6829999999999997e-06 2024-09-15T00:12:08.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684078+0000", 2024-09-15T00:12:08.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072016017000000002 2024-09-15T00:12:08.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684274+0000", 2024-09-15T00:12:08.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019636600000000001 2024-09-15T00:12:08.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684364+0000", 2024-09-15T00:12:08.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0420000000000005e-05 2024-09-15T00:12:08.424 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.424 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.424 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.424 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.424 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:08.424 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.791713808000001, 2024-09-15T00:12:08.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:12:08.424 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.425 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.425 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.425 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.425 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.425 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.425 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:12:08.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.425 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:08.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:08.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:08.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:12:08.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:12:08.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:12:08.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:12:08.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:12:08.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:12:08.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:12:08.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:12:08.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:12:08.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:12:08.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:12:08.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:12:08.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:12:08.428 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.429 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.429 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.429 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.429 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:08.429 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.791697277000001, 2024-09-15T00:12:08.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:12:08.429 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.429 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.429 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.429 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.430 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:12:08.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.430 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:08.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:08.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:12:08.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:12:08.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:12:08.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:12:08.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:12:08.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:12:08.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:12:08.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:12:08.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:12:08.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:12:08.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:12:08.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:12:08.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:12:08.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:12:08.433 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.433 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.433 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.434 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.434 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:08.434 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.791680498000002, 2024-09-15T00:12:08.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:12:08.434 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.434 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.434 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.434 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:12:08.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.434 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:08.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:08.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:08.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:12:08.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:12:08.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:12:08.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:12:08.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:12:08.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:12:08.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:12:08.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:12:08.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:12:08.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:12:08.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:12:08.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:12:08.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:12:08.438 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.438 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.438 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.438 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.438 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:08.439 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.791662767999998, 2024-09-15T00:12:08.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:12:08.439 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.439 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.439 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.439 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.439 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.439 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.439 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:12:08.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.439 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:08.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:08.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:08.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:12:08.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:12:08.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:12:08.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:12:08.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:12:08.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:12:08.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:12:08.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:12:08.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:12:08.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:12:08.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:12:08.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:12:08.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:12:08.443 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.443 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.443 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.443 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.443 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:08.443 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.791645603999999, 2024-09-15T00:12:08.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:12:08.443 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.443 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.443 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.443 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.444 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.444 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.444 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:12:08.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.444 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:08.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:08.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:08.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:12:08.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:12:08.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:12:08.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:12:08.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:12:08.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:12:08.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:12:08.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:12:08.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:12:08.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:12:08.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:12:08.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:12:08.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:12:08.447 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.447 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.447 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.448 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.448 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:08.448 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.791623928, 2024-09-15T00:12:08.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:12:08.448 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.448 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.448 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.448 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:12:08.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.448 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:08.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:08.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:08.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:12:08.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:12:08.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:12:08.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:12:08.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:12:08.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:12:08.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:12:08.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:12:08.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:12:08.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:12:08.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:12:08.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:12:08.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:12:08.452 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.452 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.452 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.452 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.452 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:08.452 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.791577803999999, 2024-09-15T00:12:08.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:12:08.453 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.453 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.453 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.453 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:12:08.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.453 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:08.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:08.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:08.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:12:08.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:12:08.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:12:08.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:12:08.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:12:08.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:12:08.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:12:08.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:12:08.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:12:08.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:12:08.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:12:08.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:12:08.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:12:08.456 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.456 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.457 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.457 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.457 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:08.457 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.791548046999999, 2024-09-15T00:12:08.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:12:08.457 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.457 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.457 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.457 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:12:08.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.458 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:08.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:08.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:12:08.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:12:08.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:12:08.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:12:08.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:12:08.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:12:08.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:12:08.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:12:08.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:12:08.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:12:08.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:12:08.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:12:08.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:12:08.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:12:08.461 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.461 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.461 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.461 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.461 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:08.461 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.791521727999999, 2024-09-15T00:12:08.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:12:08.462 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.462 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.462 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.462 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.462 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.462 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.462 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:12:08.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.462 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:08.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:08.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:08.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:12:08.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:12:08.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:12:08.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:12:08.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:12:08.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:12:08.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:12:08.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:12:08.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:12:08.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:12:08.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:12:08.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:12:08.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:12:08.466 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.466 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.466 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.466 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.466 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:08.466 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.791485865999999, 2024-09-15T00:12:08.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:12:08.466 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.466 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.466 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.467 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.467 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:12:08.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.467 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:08.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:08.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:08.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:12:08.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:12:08.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:12:08.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:12:08.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:12:08.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:12:08.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:12:08.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:12:08.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:12:08.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:12:08.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:12:08.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:12:08.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:12:08.470 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.470 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.470 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.471 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.471 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:08.471 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.791464147999999, 2024-09-15T00:12:08.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:12:08.471 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.471 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.471 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.471 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.471 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.471 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.471 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:12:08.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.472 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:08.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:08.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:12:08.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:12:08.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:12:08.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:12:08.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:12:08.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:12:08.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:12:08.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:12:08.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:12:08.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:12:08.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:12:08.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:12:08.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:12:08.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:12:08.475 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.475 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.475 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.476 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.476 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:08.476 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.791267681000001, 2024-09-15T00:12:08.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:12:08.476 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.476 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.476 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.476 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.476 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.476 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.476 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:12:08.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.476 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:08.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:08.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:08.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:12:08.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:12:08.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:12:08.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:12:08.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:12:08.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:12:08.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:12:08.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:12:08.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:12:08.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:12:08.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:12:08.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:12:08.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:12:08.480 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.480 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.480 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.480 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.480 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:08.480 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.791241842000002, 2024-09-15T00:12:08.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:12:08.480 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.480 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.481 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.481 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.481 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.481 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.481 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:12:08.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.481 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:08.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:08.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:08.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:12:08.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:12:08.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:12:08.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:12:08.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:12:08.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:12:08.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:12:08.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:12:08.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:12:08.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:12:08.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:12:08.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:12:08.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:12:08.485 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.485 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.485 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.485 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.485 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:08.485 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.791180195999999, 2024-09-15T00:12:08.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:12:08.485 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.485 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.486 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.486 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.486 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.486 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.486 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:12:08.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.486 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:08.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:08.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:12:08.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:12:08.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:12:08.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:12:08.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:12:08.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:12:08.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:12:08.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:12:08.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:12:08.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:12:08.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:12:08.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:12:08.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:12:08.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:12:08.490 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.490 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.490 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.490 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.490 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:08.490 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.791140867999999, 2024-09-15T00:12:08.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:12:08.490 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.490 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.490 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.491 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.491 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.491 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.491 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:12:08.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.491 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:08.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:08.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:12:08.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:12:08.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:12:08.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:12:08.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:12:08.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:12:08.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:12:08.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:12:08.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:12:08.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:12:08.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:12:08.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:12:08.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:12:08.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:12:08.494 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.494 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.494 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.495 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.495 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:08.495 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.789415468000001, 2024-09-15T00:12:08.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:12:08.495 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.495 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.495 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.495 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.495 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.495 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.495 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:12:08.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.496 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:08.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:08.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:12:08.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:12:08.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:12:08.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:12:08.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:12:08.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:12:08.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:12:08.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:12:08.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:12:08.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:12:08.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:12:08.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:12:08.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:12:08.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:12:08.499 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.499 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.499 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.499 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:08.499 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:08.499 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.789393307000001, 2024-09-15T00:12:08.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:12:08.500 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:08.500 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:08.500 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:08.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:08.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:08.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:08.500 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:12:08.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.500 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:08.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:08.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:08.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:08.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:08.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:08.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:08.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:08.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:12:08.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:08.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:12:08.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:12:08.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:08.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:12:08.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:12:08.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:08.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:12:08.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:12:08.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:08.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:12:08.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:12:08.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:08.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:12:08.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:12:08.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:08.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:08.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:08.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:12:08.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:12:08.503 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.503 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.503 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.503 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:08.504 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:08.504 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:08.504 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone23 from 0x3d1f566 (overlap 0x36cb9) to 0x1b65c 2024-09-15T00:12:08.505 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:08.506 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:08.503+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:08.551 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-15T00:12:08.606 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:08.604+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:08.652 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:08.652 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:08.661 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:08.706 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:12:08.707 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:08.705+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:08.805 INFO:teuthology.orchestra.run.smithi171.stdout:3265 writesame 0xd95bd93 thru 0xd968c9e data_size 0x15(0xcf0c bytes) 2024-09-15T00:12:08.805 INFO:teuthology.orchestra.run.smithi171.stdout:3266 read 0x3f3a698 thru 0x3f49094 (0xe9fd bytes) 2024-09-15T00:12:08.805 INFO:teuthology.orchestra.run.smithi171.stdout:3267 write 0x4857a1b thru 0x485978c (0x1d72 bytes) 2024-09-15T00:12:08.807 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:08.806+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:08.811 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:08.812 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:08.815 INFO:teuthology.orchestra.run.smithi171.stdout:3268 punch from 0x3659fd6 to 0x3666cab, (0xccd5 bytes) 2024-09-15T00:12:08.815 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:08.815 INFO:teuthology.orchestra.run.smithi171.stdout:3270 write 0x4ff7285 thru 0x5000647 (0x93c3 bytes) 2024-09-15T00:12:08.823 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:08.828 INFO:teuthology.orchestra.run.smithi171.stdout:3271 read 0x70c4ceb thru 0x70cded8 (0x91ee bytes) 2024-09-15T00:12:08.828 INFO:teuthology.orchestra.run.smithi171.stdout:3272 write 0xa67009d thru 0xa677dc3 (0x7d27 bytes) 2024-09-15T00:12:08.838 INFO:teuthology.orchestra.run.smithi171.stdout:3273 read 0x3ba7f8c thru 0x3bb4008 (0xc07d bytes) 2024-09-15T00:12:08.841 INFO:teuthology.orchestra.run.smithi171.stdout:3274 punch from 0x1b8a623 to 0x1b8bc26, (0x1603 bytes) 2024-09-15T00:12:08.841 INFO:teuthology.orchestra.run.smithi171.stdout:3275 punch from 0x3129de8 to 0x31372b6, (0xd4ce bytes) 2024-09-15T00:12:08.841 INFO:teuthology.orchestra.run.smithi171.stdout:3277 writesame 0x30699d9 thru 0x3076816 data_size 0x1(0xce3e bytes) 2024-09-15T00:12:08.856 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:08.856 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:08.890 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:08.890 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:08.909 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:08.907+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:08.918 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:12:08.918 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:12:08.944 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:08.944 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:08.994 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:08.994 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:09.009 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:09.008+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:09.019 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:09.019 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:09.035 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:09.035 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:09.041 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:09.041 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:09.047 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:09.062 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:09.062 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:09.074 INFO:teuthology.orchestra.run.smithi171.stdout:3278 trunc from 0xd968c9f to 0x2a2b993 2024-09-15T00:12:09.110 INFO:teuthology.orchestra.run.smithi171.stdout:3279 trunc from 0x2a2b993 to 0xab6d468 2024-09-15T00:12:09.111 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:09.109+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:09.116 INFO:teuthology.orchestra.run.smithi171.stdout:3280 writesame 0x48d1e58 thru 0x48d981e data_size 0x5(0x79c7 bytes) 2024-09-15T00:12:09.129 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:09.141 INFO:teuthology.orchestra.run.smithi171.stdout:3282 write 0xe4fbaa6 thru 0xe4fcbe0 (0x113b bytes) 2024-09-15T00:12:09.149 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:09.149 INFO:teuthology.orchestra.run.smithi171.stdout:3284 writesame 0xdeba6c9 thru 0xdebcd53 data_size 0x27(0x268b bytes) 2024-09-15T00:12:09.174 INFO:teuthology.orchestra.run.smithi171.stdout:3285 trunc from 0xe4fcbe1 to 0x60e131 2024-09-15T00:12:09.211 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:09.209+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:09.219 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:09.219 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:09.232 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is set 2024-09-15T00:12:09.236 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:09.312 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:09.310+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:09.393 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:09.393 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:09.403 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:09.412 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:09.411+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:09.513 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:09.511+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:09.514 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:09.514 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:09.524 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:09.613 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:09.612+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:09.675 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:09.675 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:09.684 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:09.714 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:09.713+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:09.816 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:09.814+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:09.830 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:09.830 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:09.839 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:09.917 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:09.915+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:09.984 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:09.985 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:09.993 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:10.038 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:10.036+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:10.118 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:10.117+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:10.132 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:10.132 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:10.144 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:10.227 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:10.226+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:10.301 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:10.301 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:10.310 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:10.320 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:10.318+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:10.421 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:10.420+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:10.455 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:10.455 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:10.464 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:10.522 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:10.521+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:10.608 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:10.608 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:10.616 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:10.623 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:10.622+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:10.719 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:10.719 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:10.724 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:10.722+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:10.728 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:10.824 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:10.823+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:10.831 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:10.831 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:10.840 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:12:10.925 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:10.924+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:10.983 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:10.983 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:10.992 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:12:11.026 INFO:teuthology.orchestra.run.smithi171.stdout:3287 read 0x422331 thru 0x428294 (0x5f64 bytes) 2024-09-15T00:12:11.027 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:11.024+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:11.082 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:12:11.127 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:11.126+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:11.131 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:11.131 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:11.140 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:12:11.205 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:11.205 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:11.223 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:11.223 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:11.229 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:11.227+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:11.234 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 0] out_osds: [2, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T00:12:11.234 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:12:11.234 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-15T00:12:11.234 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 1.000000 2024-09-15T00:12:11.235 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-15T00:12:11.241 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:12:11.241 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:12:11.251 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:12:11.260 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:11.261 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:11.278 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:11.278 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:11.301 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:462 3.0 e29/23 3:04479396:::rbd_id.image_client.0-clone2:head v 29'14, mlcod=27'12)", 2024-09-15T00:12:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.281353539000001, 2024-09-15T00:12:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030963029999999999, 2024-09-15T00:12:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.301 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:12:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016208+0000", 2024-09-15T00:12:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3669999999999997e-06 2024-09-15T00:12:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016250+0000", 2024-09-15T00:12:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1845999999999999e-05 2024-09-15T00:12:11.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016260+0000", 2024-09-15T00:12:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9750000000000002e-06 2024-09-15T00:12:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016342+0000", 2024-09-15T00:12:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1595999999999996e-05 2024-09-15T00:12:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016363+0000", 2024-09-15T00:12:11.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1073e-05 2024-09-15T00:12:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019246+0000", 2024-09-15T00:12:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028835240000000002 2024-09-15T00:12:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019296+0000", 2024-09-15T00:12:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9533999999999997e-05 2024-09-15T00:12:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.305 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:463 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'9, mlcod=27'7)", 2024-09-15T00:12:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.276740535000002, 2024-09-15T00:12:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040015709999999998, 2024-09-15T00:12:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:11.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7599999999999999e-07 2024-09-15T00:12:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020818+0000", 2024-09-15T00:12:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6609999999999999e-06 2024-09-15T00:12:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020826+0000", 2024-09-15T00:12:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4770000000000003e-06 2024-09-15T00:12:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020833+0000", 2024-09-15T00:12:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0809999999999996e-06 2024-09-15T00:12:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020878+0000", 2024-09-15T00:12:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4752000000000001e-05 2024-09-15T00:12:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020892+0000", 2024-09-15T00:12:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3899e-05 2024-09-15T00:12:11.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024767+0000", 2024-09-15T00:12:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038752259999999998 2024-09-15T00:12:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024814+0000", 2024-09-15T00:12:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7299000000000002e-05 2024-09-15T00:12:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:464 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'10, mlcod=27'8)", 2024-09-15T00:12:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.271820693000002, 2024-09-15T00:12:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048109290000000002, 2024-09-15T00:12:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-15T00:12:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025743+0000", 2024-09-15T00:12:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5789999999999993e-06 2024-09-15T00:12:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025752+0000", 2024-09-15T00:12:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8440000000000004e-06 2024-09-15T00:12:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025757+0000", 2024-09-15T00:12:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2610000000000001e-06 2024-09-15T00:12:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025796+0000", 2024-09-15T00:12:11.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9428999999999998e-05 2024-09-15T00:12:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025810+0000", 2024-09-15T00:12:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3129000000000001e-05 2024-09-15T00:12:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030514+0000", 2024-09-15T00:12:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047040099999999998 2024-09-15T00:12:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030544+0000", 2024-09-15T00:12:11.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9966000000000001e-05 2024-09-15T00:12:11.315 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.315 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.315 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.315 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:610 3.4 e29/23 3:255b0913:::rbd_data.11397bff333d.0000000000000039:head v 29'2, mlcod=0'0)", 2024-09-15T00:12:11.315 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:12:11.315 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.069077098000001, 2024-09-15T00:12:11.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003163014, 2024-09-15T00:12:11.315 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.315 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.315 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.315 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:12:11.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:12:11.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228477+0000", 2024-09-15T00:12:11.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1100000000000004e-07 2024-09-15T00:12:11.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228483+0000", 2024-09-15T00:12:11.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3720000000000001e-06 2024-09-15T00:12:11.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228497+0000", 2024-09-15T00:12:11.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3944000000000001e-05 2024-09-15T00:12:11.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228505+0000", 2024-09-15T00:12:11.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8140000000000001e-06 2024-09-15T00:12:11.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228549+0000", 2024-09-15T00:12:11.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4116000000000002e-05 2024-09-15T00:12:11.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228568+0000", 2024-09-15T00:12:11.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8473000000000001e-05 2024-09-15T00:12:11.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231599+0000", 2024-09-15T00:12:11.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030315590000000001 2024-09-15T00:12:11.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231639+0000", 2024-09-15T00:12:11.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9824999999999999e-05 2024-09-15T00:12:11.319 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.319 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.319 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.319 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:766 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'81, mlcod=29'78)", 2024-09-15T00:12:11.319 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:12:11.319 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.864528020999998, 2024-09-15T00:12:11.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029025880000000002, 2024-09-15T00:12:11.320 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.320 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.320 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.320 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:12:11.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:12:11.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433026+0000", 2024-09-15T00:12:11.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:12:11.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433030+0000", 2024-09-15T00:12:11.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6280000000000004e-06 2024-09-15T00:12:11.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433038+0000", 2024-09-15T00:12:11.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.072e-06 2024-09-15T00:12:11.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433045+0000", 2024-09-15T00:12:11.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1510000000000001e-06 2024-09-15T00:12:11.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433101+0000", 2024-09-15T00:12:11.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6336999999999999e-05 2024-09-15T00:12:11.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433113+0000", 2024-09-15T00:12:11.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1639e-05 2024-09-15T00:12:11.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435897+0000", 2024-09-15T00:12:11.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027845779999999998 2024-09-15T00:12:11.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435928+0000", 2024-09-15T00:12:11.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0689999999999999e-05 2024-09-15T00:12:11.323 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.324 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.324 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.324 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:832 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'100, mlcod=29'97)", 2024-09-15T00:12:11.324 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:11.324 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.772282521000001, 2024-09-15T00:12:11.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005498925, 2024-09-15T00:12:11.324 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.324 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.324 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.324 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:11.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:11.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:11.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7999999999999995e-07 2024-09-15T00:12:11.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525276+0000", 2024-09-15T00:12:11.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6739999999999996e-06 2024-09-15T00:12:11.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525287+0000", 2024-09-15T00:12:11.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0906e-05 2024-09-15T00:12:11.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525293+0000", 2024-09-15T00:12:11.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4409999999999998e-06 2024-09-15T00:12:11.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525329+0000", 2024-09-15T00:12:11.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5237000000000002e-05 2024-09-15T00:12:11.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525340+0000", 2024-09-15T00:12:11.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1782e-05 2024-09-15T00:12:11.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530710+0000", 2024-09-15T00:12:11.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053695460000000002 2024-09-15T00:12:11.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530770+0000", 2024-09-15T00:12:11.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9658999999999999e-05 2024-09-15T00:12:11.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.328 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.328 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:876 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'114, mlcod=29'111)", 2024-09-15T00:12:11.328 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:11.328 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.601243521000001, 2024-09-15T00:12:11.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028465830000000002, 2024-09-15T00:12:11.328 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.328 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.328 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.329 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:11.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:11.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:11.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2399999999999998e-07 2024-09-15T00:12:11.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696316+0000", 2024-09-15T00:12:11.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2800000000000003e-06 2024-09-15T00:12:11.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696326+0000", 2024-09-15T00:12:11.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0546e-05 2024-09-15T00:12:11.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696333+0000", 2024-09-15T00:12:11.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7660000000000001e-06 2024-09-15T00:12:11.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696369+0000", 2024-09-15T00:12:11.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5933000000000003e-05 2024-09-15T00:12:11.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696381+0000", 2024-09-15T00:12:11.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2417e-05 2024-09-15T00:12:11.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699124+0000", 2024-09-15T00:12:11.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027428000000000001 2024-09-15T00:12:11.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699156+0000", 2024-09-15T00:12:11.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2217e-05 2024-09-15T00:12:11.333 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.333 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.333 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.333 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1313 3.1f e30/28 3:fa9cde43:::rbd_data.11397bff333d.000000000000003a:head v 30'3, mlcod=26'1)", 2024-09-15T00:12:11.333 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:12:11.333 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.013403883999999, 2024-09-15T00:12:11.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044064359999999997, 2024-09-15T00:12:11.333 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.333 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.333 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.333 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:12:11.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:12:11.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284150+0000", 2024-09-15T00:12:11.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3799999999999999e-07 2024-09-15T00:12:11.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284228+0000", 2024-09-15T00:12:11.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:12:11.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284235+0000", 2024-09-15T00:12:11.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:12:11.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284239+0000", 2024-09-15T00:12:11.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3449999999999999e-06 2024-09-15T00:12:11.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284282+0000", 2024-09-15T00:12:11.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2506000000000003e-05 2024-09-15T00:12:11.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284297+0000", 2024-09-15T00:12:11.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5523999999999999e-05 2024-09-15T00:12:11.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288517+0000", 2024-09-15T00:12:11.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042194019999999997 2024-09-15T00:12:11.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288556+0000", 2024-09-15T00:12:11.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8952000000000002e-05 2024-09-15T00:12:11.337 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.337 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.338 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1321 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'7, mlcod=29'5)", 2024-09-15T00:12:11.338 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:11.338 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 44.989310625000002, 2024-09-15T00:12:11.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003131254, 2024-09-15T00:12:11.338 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.338 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.338 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.338 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:11.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:11.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:11.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:12:11.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308250+0000", 2024-09-15T00:12:11.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6109999999999999e-06 2024-09-15T00:12:11.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308264+0000", 2024-09-15T00:12:11.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4473e-05 2024-09-15T00:12:11.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308272+0000", 2024-09-15T00:12:11.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3540000000000002e-06 2024-09-15T00:12:11.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308340+0000", 2024-09-15T00:12:11.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8305000000000004e-05 2024-09-15T00:12:11.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308361+0000", 2024-09-15T00:12:11.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0409999999999999e-05 2024-09-15T00:12:11.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311332+0000", 2024-09-15T00:12:11.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029715950000000001 2024-09-15T00:12:11.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311374+0000", 2024-09-15T00:12:11.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1786000000000003e-05 2024-09-15T00:12:11.342 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.342 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.342 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.342 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.9 epoch 30/23 rep_tid 277 entries 30'217 (0'0) error 3:942069cf:::rbd_header.11397bff333d:head by client.4409.0:1373 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'214)", 2024-09-15T00:12:11.342 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:12:11.342 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 44.943758027999998, 2024-09-15T00:12:11.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034617960000000001, 2024-09-15T00:12:11.343 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.343 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.343 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:11.343 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:12:11.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:12:11.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353796+0000", 2024-09-15T00:12:11.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6699999999999999e-07 2024-09-15T00:12:11.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353803+0000", 2024-09-15T00:12:11.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.832e-06 2024-09-15T00:12:11.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353814+0000", 2024-09-15T00:12:11.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1815e-05 2024-09-15T00:12:11.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353822+0000", 2024-09-15T00:12:11.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7440000000000007e-06 2024-09-15T00:12:11.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353910+0000", 2024-09-15T00:12:11.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7491000000000006e-05 2024-09-15T00:12:11.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.357257+0000", 2024-09-15T00:12:11.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033474469999999999 2024-09-15T00:12:11.346 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.346 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.346 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.346 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1374 3.9 e30/23 3:942069cf:::rbd_header.11397bff333d:head v 30'218, mlcod=30'215)", 2024-09-15T00:12:11.346 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:12:11.346 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 44.942358386000002, 2024-09-15T00:12:11.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047299830000000001, 2024-09-15T00:12:11.346 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.346 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.347 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.347 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:12:11.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:12:11.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355196+0000", 2024-09-15T00:12:11.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3900000000000004e-07 2024-09-15T00:12:11.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355200+0000", 2024-09-15T00:12:11.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5700000000000003e-06 2024-09-15T00:12:11.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355210+0000", 2024-09-15T00:12:11.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6050000000000004e-06 2024-09-15T00:12:11.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355215+0000", 2024-09-15T00:12:11.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.933e-06 2024-09-15T00:12:11.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355249+0000", 2024-09-15T00:12:11.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4110999999999999e-05 2024-09-15T00:12:11.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355260+0000", 2024-09-15T00:12:11.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1235e-05 2024-09-15T00:12:11.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359895+0000", 2024-09-15T00:12:11.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046345550000000003 2024-09-15T00:12:11.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359925+0000", 2024-09-15T00:12:11.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0335e-05 2024-09-15T00:12:11.350 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.350 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.350 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.351 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1385 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'221, mlcod=30'218)", 2024-09-15T00:12:11.351 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:12:11.351 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 44.929901850999997, 2024-09-15T00:12:11.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030443950000000001, 2024-09-15T00:12:11.351 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.351 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.351 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.351 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:12:11.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:12:11.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367652+0000", 2024-09-15T00:12:11.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:12:11.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367657+0000", 2024-09-15T00:12:11.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5410000000000002e-06 2024-09-15T00:12:11.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367666+0000", 2024-09-15T00:12:11.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9280000000000005e-06 2024-09-15T00:12:11.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367671+0000", 2024-09-15T00:12:11.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7870000000000001e-06 2024-09-15T00:12:11.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367707+0000", 2024-09-15T00:12:11.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6102000000000001e-05 2024-09-15T00:12:11.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367719+0000", 2024-09-15T00:12:11.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1571e-05 2024-09-15T00:12:11.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370666+0000", 2024-09-15T00:12:11.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029474750000000002 2024-09-15T00:12:11.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370696+0000", 2024-09-15T00:12:11.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9402000000000001e-05 2024-09-15T00:12:11.355 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.355 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.355 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.355 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1662 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'300, mlcod=30'297)", 2024-09-15T00:12:11.355 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:12:11.355 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 44.509537213999998, 2024-09-15T00:12:11.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031296090000000002, 2024-09-15T00:12:11.355 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.355 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.355 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.355 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:12:11.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:12:11.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788017+0000", 2024-09-15T00:12:11.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8999999999999996e-07 2024-09-15T00:12:11.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788021+0000", 2024-09-15T00:12:11.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4970000000000003e-06 2024-09-15T00:12:11.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788027+0000", 2024-09-15T00:12:11.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1840000000000004e-06 2024-09-15T00:12:11.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788032+0000", 2024-09-15T00:12:11.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.135e-06 2024-09-15T00:12:11.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788070+0000", 2024-09-15T00:12:11.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7073999999999998e-05 2024-09-15T00:12:11.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788081+0000", 2024-09-15T00:12:11.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0987e-05 2024-09-15T00:12:11.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791116+0000", 2024-09-15T00:12:11.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030355209999999998 2024-09-15T00:12:11.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791146+0000", 2024-09-15T00:12:11.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9621e-05 2024-09-15T00:12:11.359 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.359 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.359 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.360 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1677 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'305, mlcod=30'302)", 2024-09-15T00:12:11.360 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:12:11.360 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 44.485808851999998, 2024-09-15T00:12:11.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031364320000000002, 2024-09-15T00:12:11.360 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.360 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.360 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.360 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:12:11.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:12:11.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811745+0000", 2024-09-15T00:12:11.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:12:11.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811749+0000", 2024-09-15T00:12:11.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9369999999999997e-06 2024-09-15T00:12:11.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811756+0000", 2024-09-15T00:12:11.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:12:11.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811765+0000", 2024-09-15T00:12:11.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7930000000000003e-06 2024-09-15T00:12:11.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811797+0000", 2024-09-15T00:12:11.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2203999999999999e-05 2024-09-15T00:12:11.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811810+0000", 2024-09-15T00:12:11.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2359e-05 2024-09-15T00:12:11.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814852+0000", 2024-09-15T00:12:11.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003042276 2024-09-15T00:12:11.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814881+0000", 2024-09-15T00:12:11.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9037999999999999e-05 2024-09-15T00:12:11.364 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.364 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.364 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.364 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1724 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'9, mlcod=30'7)", 2024-09-15T00:12:11.364 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:12:11.364 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 44.416883167999998, 2024-09-15T00:12:11.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003115769, 2024-09-15T00:12:11.364 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.364 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.364 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.365 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:12:11.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:12:11.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880671+0000", 2024-09-15T00:12:11.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4900000000000005e-07 2024-09-15T00:12:11.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880678+0000", 2024-09-15T00:12:11.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0979999999999996e-06 2024-09-15T00:12:11.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880691+0000", 2024-09-15T00:12:11.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3288999999999999e-05 2024-09-15T00:12:11.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880698+0000", 2024-09-15T00:12:11.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9240000000000001e-06 2024-09-15T00:12:11.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880757+0000", 2024-09-15T00:12:11.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8866000000000002e-05 2024-09-15T00:12:11.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880790+0000", 2024-09-15T00:12:11.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2926999999999998e-05 2024-09-15T00:12:11.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883738+0000", 2024-09-15T00:12:11.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002948418 2024-09-15T00:12:11.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883786+0000", 2024-09-15T00:12:11.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7398000000000003e-05 2024-09-15T00:12:11.368 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.368 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.368 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.369 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1790 3.9 e30/23 3:9676e01b:::rbd_data.11397bff333d.0000000000000030:head v 30'338, mlcod=30'335)", 2024-09-15T00:12:11.369 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:12:11.369 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 44.338000798000003, 2024-09-15T00:12:11.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044496980000000002, 2024-09-15T00:12:11.369 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.369 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.369 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.369 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:12:11.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:12:11.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959553+0000", 2024-09-15T00:12:11.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3600000000000004e-07 2024-09-15T00:12:11.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959569+0000", 2024-09-15T00:12:11.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6152999999999999e-05 2024-09-15T00:12:11.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959577+0000", 2024-09-15T00:12:11.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5179999999999998e-06 2024-09-15T00:12:11.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959581+0000", 2024-09-15T00:12:11.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3669999999999999e-06 2024-09-15T00:12:11.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959619+0000", 2024-09-15T00:12:11.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8209e-05 2024-09-15T00:12:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959632+0000", 2024-09-15T00:12:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2405000000000001e-05 2024-09-15T00:12:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.963954+0000", 2024-09-15T00:12:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043226660000000002 2024-09-15T00:12:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.964002+0000", 2024-09-15T00:12:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7843999999999998e-05 2024-09-15T00:12:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:3177 3.4 e33/23 3:23818c00:::rbd_data.1139b531b717.0000000000000022:head v 33'58, mlcod=33'56)", 2024-09-15T00:12:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:12:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.639712232000001, 2024-09-15T00:12:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030564120000000001, 2024-09-15T00:12:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:12:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:12:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657842+0000", 2024-09-15T00:12:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.19e-06 2024-09-15T00:12:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657853+0000", 2024-09-15T00:12:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:12:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657871+0000", 2024-09-15T00:12:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8233e-05 2024-09-15T00:12:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657880+0000", 2024-09-15T00:12:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8319999999999995e-06 2024-09-15T00:12:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657949+0000", 2024-09-15T00:12:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8847e-05 2024-09-15T00:12:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657977+0000", 2024-09-15T00:12:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8308000000000001e-05 2024-09-15T00:12:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.660851+0000", 2024-09-15T00:12:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028743190000000002 2024-09-15T00:12:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.660897+0000", 2024-09-15T00:12:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6142e-05 2024-09-15T00:12:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:3602 3.1f e34/28 3:fd28af01:::rbd_data.1139b531b717.0000000000000032:head v 34'24, mlcod=33'22)", 2024-09-15T00:12:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:12:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.252314837, 2024-09-15T00:12:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034159709999999999, 2024-09-15T00:12:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:12:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:12:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045239+0000", 2024-09-15T00:12:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:12:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045252+0000", 2024-09-15T00:12:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2891000000000001e-05 2024-09-15T00:12:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045263+0000", 2024-09-15T00:12:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0875e-05 2024-09-15T00:12:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045271+0000", 2024-09-15T00:12:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3529999999999995e-06 2024-09-15T00:12:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045338+0000", 2024-09-15T00:12:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7377000000000002e-05 2024-09-15T00:12:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045359+0000", 2024-09-15T00:12:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1078e-05 2024-09-15T00:12:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.048621+0000", 2024-09-15T00:12:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003261543 2024-09-15T00:12:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.048654+0000", 2024-09-15T00:12:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3550999999999998e-05 2024-09-15T00:12:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:4253 3.4 e35/23 3:242f5431:::rbd_data.11396389f8f2.0000000000000000:head v 35'87, mlcod=35'85)", 2024-09-15T00:12:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:12:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.904825979000002, 2024-09-15T00:12:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002944997, 2024-09-15T00:12:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:12:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:12:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392728+0000", 2024-09-15T00:12:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.06e-07 2024-09-15T00:12:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392782+0000", 2024-09-15T00:12:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.448e-05 2024-09-15T00:12:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392794+0000", 2024-09-15T00:12:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1768e-05 2024-09-15T00:12:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392801+0000", 2024-09-15T00:12:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9589999999999998e-06 2024-09-15T00:12:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392845+0000", 2024-09-15T00:12:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3884999999999997e-05 2024-09-15T00:12:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392860+0000", 2024-09-15T00:12:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5364999999999999e-05 2024-09-15T00:12:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.395641+0000", 2024-09-15T00:12:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027807629999999999 2024-09-15T00:12:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.395672+0000", 2024-09-15T00:12:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1170999999999997e-05 2024-09-15T00:12:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:18791 3.0 e38/23 3:05ecfc97:::rbd_data.11398e1a091a.0000000000000030:head v 38'66, mlcod=38'64)", 2024-09-15T00:12:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:12:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 35.261910442999998, 2024-09-15T00:12:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029447940000000001, 2024-09-15T00:12:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:12:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:12:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035644+0000", 2024-09-15T00:12:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4e-07 2024-09-15T00:12:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035684+0000", 2024-09-15T00:12:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9857999999999997e-05 2024-09-15T00:12:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035694+0000", 2024-09-15T00:12:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0259e-05 2024-09-15T00:12:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035699+0000", 2024-09-15T00:12:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6389999999999997e-06 2024-09-15T00:12:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035745+0000", 2024-09-15T00:12:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5640999999999999e-05 2024-09-15T00:12:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035764+0000", 2024-09-15T00:12:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9368000000000002e-05 2024-09-15T00:12:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.038551+0000", 2024-09-15T00:12:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027860879999999999 2024-09-15T00:12:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.038588+0000", 2024-09-15T00:12:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7101000000000003e-05 2024-09-15T00:12:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.392 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.392 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:11.393 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:11.393 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:11.394 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:11.394 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:11.395 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:11.395 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:11.396 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:11.396 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:11.392+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:11.493 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:11.492+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:11.539 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:11.539 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:11.539 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:11.539 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:11.548 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:11.594 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:11.592+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:11.652 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:11.652 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:11.652 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:11.660 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:11.703 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:11.702+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:11.765 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:47798 3.7 3:e6004524:::rbd_data.113916cf513.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 1789239~44328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.387407+0000", 2024-09-15T00:12:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.374932119, 2024-09-15T00:12:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033697973999999999, 2024-09-15T00:12:11.765 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 47798 2024-09-15T00:12:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.387407+0000", 2024-09-15T00:12:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.387407+0000", 2024-09-15T00:12:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.387408+0000", 2024-09-15T00:12:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2580000000000001e-06 2024-09-15T00:12:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389094+0000", 2024-09-15T00:12:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001685572 2024-09-15T00:12:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389116+0000", 2024-09-15T00:12:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2379999999999999e-05 2024-09-15T00:12:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389126+0000", 2024-09-15T00:12:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0044000000000001e-05 2024-09-15T00:12:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389203+0000", 2024-09-15T00:12:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6945000000000003e-05 2024-09-15T00:12:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389381+0000", 2024-09-15T00:12:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000177908 2024-09-15T00:12:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.389890+0000", 2024-09-15T00:12:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00050844599999999996 2024-09-15T00:12:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.420647+0000", 2024-09-15T00:12:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030757201000000001 2024-09-15T00:12:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.420970+0000", 2024-09-15T00:12:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032290200000000002 2024-09-15T00:12:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.421037+0000", 2024-09-15T00:12:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6883000000000003e-05 2024-09-15T00:12:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.421105+0000", 2024-09-15T00:12:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8435000000000005e-05 2024-09-15T00:12:11.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56198 3.2 3:44ae5a28:::rbd_data.1139167e758e.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:12:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.238999+0000", 2024-09-15T00:12:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.52334055, 2024-09-15T00:12:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037097333000000003, 2024-09-15T00:12:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.771 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56198 2024-09-15T00:12:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.238999+0000", 2024-09-15T00:12:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.238999+0000", 2024-09-15T00:12:11.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.239000+0000", 2024-09-15T00:12:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.359e-06 2024-09-15T00:12:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242065+0000", 2024-09-15T00:12:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030647230000000001 2024-09-15T00:12:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242079+0000", 2024-09-15T00:12:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3801999999999999e-05 2024-09-15T00:12:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242108+0000", 2024-09-15T00:12:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9889000000000001e-05 2024-09-15T00:12:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242177+0000", 2024-09-15T00:12:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9072000000000005e-05 2024-09-15T00:12:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242332+0000", 2024-09-15T00:12:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000154816 2024-09-15T00:12:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.243189+0000", 2024-09-15T00:12:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00085691200000000004 2024-09-15T00:12:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.267650+0000", 2024-09-15T00:12:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024461146999999999 2024-09-15T00:12:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.275977+0000", 2024-09-15T00:12:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0083269729999999997 2024-09-15T00:12:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276040+0000", 2024-09-15T00:12:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2497999999999997e-05 2024-09-15T00:12:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276096+0000", 2024-09-15T00:12:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6141999999999999e-05 2024-09-15T00:12:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.776 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56199 3.2 3:44ae5a28:::rbd_data.1139167e758e.0000000000000000:head [write 0~0] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:12:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242772+0000", 2024-09-15T00:12:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.519567147, 2024-09-15T00:12:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033898029000000003, 2024-09-15T00:12:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.777 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56199 2024-09-15T00:12:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242772+0000", 2024-09-15T00:12:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242772+0000", 2024-09-15T00:12:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242774+0000", 2024-09-15T00:12:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.483e-06 2024-09-15T00:12:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242782+0000", 2024-09-15T00:12:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9740000000000002e-06 2024-09-15T00:12:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242806+0000", 2024-09-15T00:12:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4689e-05 2024-09-15T00:12:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242815+0000", 2024-09-15T00:12:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3410000000000003e-06 2024-09-15T00:12:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.265144+0000", 2024-09-15T00:12:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022329244000000002 2024-09-15T00:12:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.265237+0000", 2024-09-15T00:12:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3154999999999998e-05 2024-09-15T00:12:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.265426+0000", 2024-09-15T00:12:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018955600000000001 2024-09-15T00:12:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.267912+0000", 2024-09-15T00:12:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024857880000000001 2024-09-15T00:12:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276583+0000", 2024-09-15T00:12:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0086705689999999995 2024-09-15T00:12:11.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276634+0000", 2024-09-15T00:12:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1465000000000003e-05 2024-09-15T00:12:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.276670+0000", 2024-09-15T00:12:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5765000000000001e-05 2024-09-15T00:12:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:12:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.519495101, 2024-09-15T00:12:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:12:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:12:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:12:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:12:11.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:12:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:12:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:12:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:12:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:12:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:12:11.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:12:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:12:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:12:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:12:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:12:11.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:12:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:12:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:12:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:12:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:12:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:12:11.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:12:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:12:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:12:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56207 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [write 0~0] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:12:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.245178+0000", 2024-09-15T00:12:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.517161400999999, 2024-09-15T00:12:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038624626000000002, 2024-09-15T00:12:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56207 2024-09-15T00:12:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245178+0000", 2024-09-15T00:12:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245178+0000", 2024-09-15T00:12:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245180+0000", 2024-09-15T00:12:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8589999999999999e-06 2024-09-15T00:12:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245185+0000", 2024-09-15T00:12:11.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1780000000000002e-06 2024-09-15T00:12:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245192+0000", 2024-09-15T00:12:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6420000000000007e-06 2024-09-15T00:12:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245198+0000", 2024-09-15T00:12:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5729999999999998e-06 2024-09-15T00:12:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.271149+0000", 2024-09-15T00:12:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025950534000000001 2024-09-15T00:12:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.271238+0000", 2024-09-15T00:12:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9364999999999996e-05 2024-09-15T00:12:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.271398+0000", 2024-09-15T00:12:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015962600000000001 2024-09-15T00:12:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.277582+0000", 2024-09-15T00:12:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061847669999999999 2024-09-15T00:12:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283748+0000", 2024-09-15T00:12:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061654170000000003 2024-09-15T00:12:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283782+0000", 2024-09-15T00:12:11.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4245000000000002e-05 2024-09-15T00:12:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283802+0000", 2024-09-15T00:12:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-05 2024-09-15T00:12:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58878 3.8 3:12456482:::rbd_data.1139766160da.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.473189517000002, 2024-09-15T00:12:11.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061956199000000003, 2024-09-15T00:12:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58878 2024-09-15T00:12:11.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289151+0000", 2024-09-15T00:12:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:12:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289473+0000", 2024-09-15T00:12:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032183800000000002 2024-09-15T00:12:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289487+0000", 2024-09-15T00:12:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-05 2024-09-15T00:12:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289493+0000", 2024-09-15T00:12:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4489999999999996e-06 2024-09-15T00:12:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289542+0000", 2024-09-15T00:12:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8844999999999997e-05 2024-09-15T00:12:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289642+0000", 2024-09-15T00:12:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9607999999999999e-05 2024-09-15T00:12:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.293908+0000", 2024-09-15T00:12:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659350000000002 2024-09-15T00:12:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351022+0000", 2024-09-15T00:12:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057114535000000001 2024-09-15T00:12:11.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351033+0000", 2024-09-15T00:12:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0263e-05 2024-09-15T00:12:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351070+0000", 2024-09-15T00:12:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7666000000000002e-05 2024-09-15T00:12:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351106+0000", 2024-09-15T00:12:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999997e-05 2024-09-15T00:12:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58879 3.7 3:e169894d:::rbd_data.1139766160da.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.470838028999999, 2024-09-15T00:12:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043716294000000003, 2024-09-15T00:12:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58879 2024-09-15T00:12:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:11.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291502+0000", 2024-09-15T00:12:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-15T00:12:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291664+0000", 2024-09-15T00:12:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016216599999999999 2024-09-15T00:12:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291674+0000", 2024-09-15T00:12:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0276e-05 2024-09-15T00:12:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291680+0000", 2024-09-15T00:12:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2580000000000003e-06 2024-09-15T00:12:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291732+0000", 2024-09-15T00:12:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2114999999999998e-05 2024-09-15T00:12:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291866+0000", 2024-09-15T00:12:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013416900000000001 2024-09-15T00:12:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.296426+0000", 2024-09-15T00:12:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045595779999999999 2024-09-15T00:12:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335075+0000", 2024-09-15T00:12:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038649347000000001 2024-09-15T00:12:11.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335097+0000", 2024-09-15T00:12:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2565000000000002e-05 2024-09-15T00:12:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335161+0000", 2024-09-15T00:12:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3928000000000002e-05 2024-09-15T00:12:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335217+0000", 2024-09-15T00:12:11.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6042000000000003e-05 2024-09-15T00:12:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58880 3.8 3:13d91cfd:::rbd_data.1139766160da.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.467445922, 2024-09-15T00:12:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056331463999999998, 2024-09-15T00:12:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58880 2024-09-15T00:12:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294896+0000", 2024-09-15T00:12:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.356e-06 2024-09-15T00:12:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295288+0000", 2024-09-15T00:12:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039223200000000001 2024-09-15T00:12:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295304+0000", 2024-09-15T00:12:11.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5937e-05 2024-09-15T00:12:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295312+0000", 2024-09-15T00:12:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1489999999999994e-06 2024-09-15T00:12:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333162+0000", 2024-09-15T00:12:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037849840000000003 2024-09-15T00:12:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333269+0000", 2024-09-15T00:12:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010749800000000001 2024-09-15T00:12:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.336198+0000", 2024-09-15T00:12:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029284739999999999 2024-09-15T00:12:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351137+0000", 2024-09-15T00:12:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014939103 2024-09-15T00:12:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351150+0000", 2024-09-15T00:12:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3542e-05 2024-09-15T00:12:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351191+0000", 2024-09-15T00:12:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0791000000000003e-05 2024-09-15T00:12:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351225+0000", 2024-09-15T00:12:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3541999999999998e-05 2024-09-15T00:12:11.812 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58878 3.8 e56/23 ondisk, result = 0)", 2024-09-15T00:12:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.460916138999998, 2024-09-15T00:12:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049689096000000002, 2024-09-15T00:12:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301425+0000", 2024-09-15T00:12:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.782e-06 2024-09-15T00:12:11.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301433+0000", 2024-09-15T00:12:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689999999999994e-06 2024-09-15T00:12:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301449+0000", 2024-09-15T00:12:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5767e-05 2024-09-15T00:12:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301458+0000", 2024-09-15T00:12:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:12:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351009+0000", 2024-09-15T00:12:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049551523 2024-09-15T00:12:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351017+0000", 2024-09-15T00:12:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8359999999999992e-06 2024-09-15T00:12:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351112+0000", 2024-09-15T00:12:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5173999999999994e-05 2024-09-15T00:12:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58879 3.7 e56/55 ondisk, result = 0)", 2024-09-15T00:12:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.303373+0000", 2024-09-15T00:12:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.458966017000002, 2024-09-15T00:12:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031714352000000001, 2024-09-15T00:12:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.817 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.303373+0000", 2024-09-15T00:12:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.303373+0000", 2024-09-15T00:12:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.303375+0000", 2024-09-15T00:12:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3510000000000001e-06 2024-09-15T00:12:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.303380+0000", 2024-09-15T00:12:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6840000000000001e-06 2024-09-15T00:12:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.303392+0000", 2024-09-15T00:12:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1847e-05 2024-09-15T00:12:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.303401+0000", 2024-09-15T00:12:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0219999999999993e-06 2024-09-15T00:12:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.334994+0000", 2024-09-15T00:12:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.031592758999999998 2024-09-15T00:12:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335005+0000", 2024-09-15T00:12:11.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:12:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335088+0000", 2024-09-15T00:12:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2808999999999999e-05 2024-09-15T00:12:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:62811 3.1e 3:7e43c43a:::rbd_data.1139766160da.000000000000001b:13 [read 1937408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:11.821 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.656753+0000", 2024-09-15T00:12:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 17.105586072000001, 2024-09-15T00:12:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030335925999999999, 2024-09-15T00:12:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 62811 2024-09-15T00:12:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656753+0000", 2024-09-15T00:12:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656753+0000", 2024-09-15T00:12:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656754+0000", 2024-09-15T00:12:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2600000000000002e-07 2024-09-15T00:12:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656767+0000", 2024-09-15T00:12:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3088000000000001e-05 2024-09-15T00:12:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656773+0000", 2024-09-15T00:12:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3790000000000003e-06 2024-09-15T00:12:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656778+0000", 2024-09-15T00:12:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-15T00:12:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.686998+0000", 2024-09-15T00:12:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030219629000000001 2024-09-15T00:12:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687046+0000", 2024-09-15T00:12:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8547000000000001e-05 2024-09-15T00:12:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687089+0000", 2024-09-15T00:12:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2926999999999997e-05 2024-09-15T00:12:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:62812 3.1e 3:7e43c43a:::rbd_data.1139766160da.000000000000001b:13 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.656789+0000", 2024-09-15T00:12:11.826 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 17.105550136000002, 2024-09-15T00:12:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030406757999999999, 2024-09-15T00:12:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 62812 2024-09-15T00:12:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656789+0000", 2024-09-15T00:12:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656789+0000", 2024-09-15T00:12:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656790+0000", 2024-09-15T00:12:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.75e-07 2024-09-15T00:12:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656791+0000", 2024-09-15T00:12:11.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9400000000000001e-06 2024-09-15T00:12:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656801+0000", 2024-09-15T00:12:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5589999999999994e-06 2024-09-15T00:12:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656808+0000", 2024-09-15T00:12:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1489999999999997e-06 2024-09-15T00:12:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687116+0000", 2024-09-15T00:12:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030308162999999999 2024-09-15T00:12:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687161+0000", 2024-09-15T00:12:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4764e-05 2024-09-15T00:12:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687196+0000", 2024-09-15T00:12:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4708e-05 2024-09-15T00:12:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.830 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:62813 3.1e 3:7e43c43a:::rbd_data.1139766160da.000000000000001b:13 [read 2060288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.656815+0000", 2024-09-15T00:12:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 17.105524278000001, 2024-09-15T00:12:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030516998, 2024-09-15T00:12:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 62813 2024-09-15T00:12:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.831 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656815+0000", 2024-09-15T00:12:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656815+0000", 2024-09-15T00:12:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656815+0000", 2024-09-15T00:12:11.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:12:11.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656817+0000", 2024-09-15T00:12:11.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.08e-06 2024-09-15T00:12:11.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656824+0000", 2024-09-15T00:12:11.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1249999999999998e-06 2024-09-15T00:12:11.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656828+0000", 2024-09-15T00:12:11.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9010000000000002e-06 2024-09-15T00:12:11.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687224+0000", 2024-09-15T00:12:11.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030395884000000001 2024-09-15T00:12:11.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687274+0000", 2024-09-15T00:12:11.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9455e-05 2024-09-15T00:12:11.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687332+0000", 2024-09-15T00:12:11.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8143000000000002e-05 2024-09-15T00:12:11.835 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.835 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.835 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.835 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:62814 3.1e 3:7e43c43a:::rbd_data.1139766160da.000000000000001b:13 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:11.835 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:54.656834+0000", 2024-09-15T00:12:11.835 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 17.105505171000001, 2024-09-15T00:12:11.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030596983000000001, 2024-09-15T00:12:11.835 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.835 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.835 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:11.835 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.836 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 62814 2024-09-15T00:12:11.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.836 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656834+0000", 2024-09-15T00:12:11.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656834+0000", 2024-09-15T00:12:11.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656834+0000", 2024-09-15T00:12:11.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:12:11.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656842+0000", 2024-09-15T00:12:11.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.525e-06 2024-09-15T00:12:11.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656848+0000", 2024-09-15T00:12:11.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4169999999999997e-06 2024-09-15T00:12:11.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.656864+0000", 2024-09-15T00:12:11.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5591999999999999e-05 2024-09-15T00:12:11.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687353+0000", 2024-09-15T00:12:11.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030488718000000001 2024-09-15T00:12:11.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687394+0000", 2024-09-15T00:12:11.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0973999999999998e-05 2024-09-15T00:12:11.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:54.687431+0000", 2024-09-15T00:12:11.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7397999999999997e-05 2024-09-15T00:12:11.839 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.839 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.839 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.839 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:79533 3.1e 3:7f55ca9b:::rbd_data.1139741c8001.0000000000000015:14 [read 888832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:12:11.839 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:57.830591+0000", 2024-09-15T00:12:11.840 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.931748053, 2024-09-15T00:12:11.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030265637000000001, 2024-09-15T00:12:11.840 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.840 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.840 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:11.840 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.840 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.840 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.840 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79533 2024-09-15T00:12:11.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.841 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830591+0000", 2024-09-15T00:12:11.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830591+0000", 2024-09-15T00:12:11.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830592+0000", 2024-09-15T00:12:11.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7599999999999998e-07 2024-09-15T00:12:11.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830593+0000", 2024-09-15T00:12:11.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.885e-06 2024-09-15T00:12:11.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830599+0000", 2024-09-15T00:12:11.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-15T00:12:11.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830603+0000", 2024-09-15T00:12:11.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.33e-06 2024-09-15T00:12:11.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.860758+0000", 2024-09-15T00:12:11.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030154389 2024-09-15T00:12:11.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.860810+0000", 2024-09-15T00:12:11.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2568000000000002e-05 2024-09-15T00:12:11.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.860857+0000", 2024-09-15T00:12:11.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6360000000000003e-05 2024-09-15T00:12:11.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.844 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.844 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:79534 3.1e 3:7f55ca9b:::rbd_data.1139741c8001.0000000000000015:14 [read 954368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:12:11.844 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:57.830609+0000", 2024-09-15T00:12:11.844 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.931730131, 2024-09-15T00:12:11.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030373054, 2024-09-15T00:12:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79534 2024-09-15T00:12:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830609+0000", 2024-09-15T00:12:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830609+0000", 2024-09-15T00:12:11.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830609+0000", 2024-09-15T00:12:11.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9700000000000003e-07 2024-09-15T00:12:11.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830611+0000", 2024-09-15T00:12:11.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4619999999999999e-06 2024-09-15T00:12:11.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830621+0000", 2024-09-15T00:12:11.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9580000000000002e-06 2024-09-15T00:12:11.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830625+0000", 2024-09-15T00:12:11.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000001e-06 2024-09-15T00:12:11.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.860890+0000", 2024-09-15T00:12:11.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030265071000000001 2024-09-15T00:12:11.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.860942+0000", 2024-09-15T00:12:11.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1938999999999999e-05 2024-09-15T00:12:11.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.860982+0000", 2024-09-15T00:12:11.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0407999999999997e-05 2024-09-15T00:12:11.848 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.848 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.848 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.849 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:79535 3.1e 3:7f55ca9b:::rbd_data.1139741c8001.0000000000000015:14 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:12:11.849 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:57.830630+0000", 2024-09-15T00:12:11.849 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.931708936, 2024-09-15T00:12:11.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030460928000000002, 2024-09-15T00:12:11.849 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.849 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.849 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:11.849 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.849 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.849 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.849 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79535 2024-09-15T00:12:11.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.849 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830630+0000", 2024-09-15T00:12:11.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830630+0000", 2024-09-15T00:12:11.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830631+0000", 2024-09-15T00:12:11.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1099999999999996e-07 2024-09-15T00:12:11.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830632+0000", 2024-09-15T00:12:11.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.672e-06 2024-09-15T00:12:11.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830636+0000", 2024-09-15T00:12:11.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-06 2024-09-15T00:12:11.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.830640+0000", 2024-09-15T00:12:11.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2590000000000002e-06 2024-09-15T00:12:11.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.861010+0000", 2024-09-15T00:12:11.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.030369213999999999 2024-09-15T00:12:11.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.861052+0000", 2024-09-15T00:12:11.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2401999999999998e-05 2024-09-15T00:12:11.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:57.861091+0000", 2024-09-15T00:12:11.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9279000000000001e-05 2024-09-15T00:12:11.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.853 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.853 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:84253 3.18 3:1a889b9d:::rbd_data.113947ee3a89.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-15T00:12:11.853 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:59.323633+0000", 2024-09-15T00:12:11.853 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.438705817000001, 2024-09-15T00:12:11.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034950439999999999, 2024-09-15T00:12:11.853 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.853 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.853 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.854 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 84253 2024-09-15T00:12:11.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.854 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.323633+0000", 2024-09-15T00:12:11.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.323633+0000", 2024-09-15T00:12:11.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.323634+0000", 2024-09-15T00:12:11.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.34e-07 2024-09-15T00:12:11.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324115+0000", 2024-09-15T00:12:11.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00048104099999999998 2024-09-15T00:12:11.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324123+0000", 2024-09-15T00:12:11.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5459999999999992e-06 2024-09-15T00:12:11.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324130+0000", 2024-09-15T00:12:11.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3740000000000003e-06 2024-09-15T00:12:11.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.346309+0000", 2024-09-15T00:12:11.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022179194999999999 2024-09-15T00:12:11.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.346420+0000", 2024-09-15T00:12:11.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000111093 2024-09-15T00:12:11.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:11.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.346612+0000", 2024-09-15T00:12:11.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019197600000000001 2024-09-15T00:12:11.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:11.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.356985+0000", 2024-09-15T00:12:11.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01037306 2024-09-15T00:12:11.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:11.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.358458+0000", 2024-09-15T00:12:11.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001473001 2024-09-15T00:12:11.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.358532+0000", 2024-09-15T00:12:11.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3541000000000006e-05 2024-09-15T00:12:11.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.358584+0000", 2024-09-15T00:12:11.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2179000000000003e-05 2024-09-15T00:12:11.859 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.859 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.859 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.859 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:84254 3.18 3:1a889b9d:::rbd_data.113947ee3a89.0000000000000005:head [write 0~0] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-15T00:12:11.859 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:59.324141+0000", 2024-09-15T00:12:11.859 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.438197781, 2024-09-15T00:12:11.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034565371999999997, 2024-09-15T00:12:11.859 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.859 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.859 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 84254 2024-09-15T00:12:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324141+0000", 2024-09-15T00:12:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324141+0000", 2024-09-15T00:12:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324142+0000", 2024-09-15T00:12:11.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7300000000000005e-07 2024-09-15T00:12:11.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324145+0000", 2024-09-15T00:12:11.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.661e-06 2024-09-15T00:12:11.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324151+0000", 2024-09-15T00:12:11.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5139999999999998e-06 2024-09-15T00:12:11.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.324157+0000", 2024-09-15T00:12:11.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2569999999999998e-06 2024-09-15T00:12:11.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.351607+0000", 2024-09-15T00:12:11.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.027450862999999999 2024-09-15T00:12:11.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.351699+0000", 2024-09-15T00:12:11.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1247000000000002e-05 2024-09-15T00:12:11.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:11.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.351895+0000", 2024-09-15T00:12:11.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019662600000000001 2024-09-15T00:12:11.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:11.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.356991+0000", 2024-09-15T00:12:11.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005095774 2024-09-15T00:12:11.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:11.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.358642+0000", 2024-09-15T00:12:11.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001651306 2024-09-15T00:12:11.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.358682+0000", 2024-09-15T00:12:11.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9190000000000003e-05 2024-09-15T00:12:11.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:59.358707+0000", 2024-09-15T00:12:11.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5160999999999998e-05 2024-09-15T00:12:11.864 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.864 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.864 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.864 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:95167 3.1e 3:7941b88e:::rbd_data.1139e6d65aff.000000000000001b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T00:12:11.865 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:03.555273+0000", 2024-09-15T00:12:11.865 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2070666469999995, 2024-09-15T00:12:11.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036694817999999997, 2024-09-15T00:12:11.865 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:11.865 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:11.865 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:11.865 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:11.865 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:11.865 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:11.865 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 95167 2024-09-15T00:12:11.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.865 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:11.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555273+0000", 2024-09-15T00:12:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555273+0000", 2024-09-15T00:12:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555295+0000", 2024-09-15T00:12:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2189999999999999e-05 2024-09-15T00:12:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:11.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555303+0000", 2024-09-15T00:12:11.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8869999999999992e-06 2024-09-15T00:12:11.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:11.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555321+0000", 2024-09-15T00:12:11.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8202999999999998e-05 2024-09-15T00:12:11.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:11.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555331+0000", 2024-09-15T00:12:11.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.842e-06 2024-09-15T00:12:11.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:11.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555390+0000", 2024-09-15T00:12:11.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9422999999999997e-05 2024-09-15T00:12:11.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:11.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.555507+0000", 2024-09-15T00:12:11.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116762 2024-09-15T00:12:11.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 6", 2024-09-15T00:12:11.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.558432+0000", 2024-09-15T00:12:11.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029248239999999999 2024-09-15T00:12:11.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:11.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.591882+0000", 2024-09-15T00:12:11.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033450739 2024-09-15T00:12:11.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:11.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.591896+0000", 2024-09-15T00:12:11.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4053000000000001e-05 2024-09-15T00:12:11.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:11.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.591934+0000", 2024-09-15T00:12:11.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7515000000000003e-05 2024-09-15T00:12:11.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:11.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:11.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:11.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:03.591967+0000", 2024-09-15T00:12:11.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3380000000000002e-05 2024-09-15T00:12:11.870 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.870 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.870 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.870 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:11.870 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:11.870 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:11.872 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:11.872 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:11.870+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:11.972 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:11.970+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:11.985 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:11.985 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:11.985 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:11.986 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:11.994 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:12.017 INFO:teuthology.orchestra.run.smithi171.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-15T00:12:12.020 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is unset 2024-09-15T00:12:12.023 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:12.048 INFO:teuthology.orchestra.run.smithi171.stdout:3290 write 0xec7dbe9 thru 0xec825ff (0x4a17 bytes) 2024-09-15T00:12:12.061 INFO:teuthology.orchestra.run.smithi171.stdout:3291 punch from 0xa1de785 to 0xa1dec26, (0x4a1 bytes) 2024-09-15T00:12:12.061 INFO:teuthology.orchestra.run.smithi171.stdout:3293 writesame 0x586edc7 thru 0x587a074 data_size 0x1(0xb2ae bytes) 2024-09-15T00:12:12.079 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:12.077+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:12.081 INFO:teuthology.orchestra.run.smithi171.stdout:3294 write 0xc349860 thru 0xc3587b9 (0xef5a bytes) 2024-09-15T00:12:12.100 INFO:teuthology.orchestra.run.smithi171.stdout:3298 write 0xeae41e7 thru 0xeae7e8d (0x3ca7 bytes) 2024-09-15T00:12:12.112 INFO:teuthology.orchestra.run.smithi171.stdout:3299 write 0xda8be96 thru 0xda8bfaa (0x115 bytes) 2024-09-15T00:12:12.116 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:12.116 INFO:teuthology.orchestra.run.smithi171.stdout:3301 read 0xc185f53 thru 0xc1895a8 (0x3656 bytes) 2024-09-15T00:12:12.117 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:12.122 INFO:teuthology.orchestra.run.smithi171.stdout:3306 write 0xecc7d46 thru 0xeccbdd7 (0x4092 bytes) 2024-09-15T00:12:12.127 INFO:teuthology.orchestra.run.smithi171.stdout:3307 read 0x9782dc2 thru 0x978677d (0x39bc bytes) 2024-09-15T00:12:12.127 INFO:teuthology.orchestra.run.smithi171.stdout:3308 write 0x67ffe10 thru 0x680f3fa (0xf5eb bytes) 2024-09-15T00:12:12.143 INFO:teuthology.orchestra.run.smithi171.stdout:3309 writesame 0xea0019e thru 0xea04d20 data_size 0x1(0x4b83 bytes) 2024-09-15T00:12:12.149 INFO:teuthology.orchestra.run.smithi171.stdout:3310 punch from 0xa4fc510 to 0xa4ffa71, (0x3561 bytes) 2024-09-15T00:12:12.155 INFO:teuthology.orchestra.run.smithi171.stdout:3311 trunc from 0xeccbdd8 to 0x97bb04b 2024-09-15T00:12:12.164 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:12.164 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:12.164 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:12.164 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:12.164 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:12.171 INFO:teuthology.orchestra.run.smithi171.stdout:3312 read 0x56811aa thru 0x56815b0 (0x407 bytes) 2024-09-15T00:12:12.171 INFO:teuthology.orchestra.run.smithi171.stdout:3314 read 0x226732e thru 0x2267346 (0x19 bytes) 2024-09-15T00:12:12.173 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:12.171+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:12.174 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:12.182 INFO:teuthology.orchestra.run.smithi171.stdout:3315 writesame 0xe542461 thru 0xe544bc5 data_size 0x1(0x2765 bytes) 2024-09-15T00:12:12.182 INFO:teuthology.orchestra.run.smithi171.stdout:3316 write 0x676784 thru 0x6799a3 (0x3220 bytes) 2024-09-15T00:12:12.184 INFO:teuthology.orchestra.run.smithi171.stdout:3317 read 0x8ba61d1 thru 0x8bb1959 (0xb789 bytes) 2024-09-15T00:12:12.184 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:12.184 INFO:teuthology.orchestra.run.smithi171.stdout:3320 read 0x52a7e90 thru 0x52b2228 (0xa399 bytes) 2024-09-15T00:12:12.184 INFO:teuthology.orchestra.run.smithi171.stdout:3322 read 0xcf90061 thru 0xcf9c3f7 (0xc397 bytes) 2024-09-15T00:12:12.200 INFO:teuthology.orchestra.run.smithi171.stdout:3323 trunc from 0xe544bc6 to 0xeb29ca 2024-09-15T00:12:12.234 INFO:teuthology.orchestra.run.smithi171.stdout:3324 writesame 0xb5009b3 thru 0xb503264 data_size 0x1(0x28b2 bytes) 2024-09-15T00:12:12.241 INFO:teuthology.orchestra.run.smithi171.stdout:3326 write 0xcd43995 thru 0xcd4a2e9 (0x6955 bytes) 2024-09-15T00:12:12.256 INFO:teuthology.orchestra.run.smithi171.stdout:3328 read 0x85289a thru 0x859e2e (0x7595 bytes) 2024-09-15T00:12:12.256 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:12.257 INFO:teuthology.orchestra.run.smithi171.stdout:3330 punch from 0x470388e to 0x470f5e4, (0xbd56 bytes) 2024-09-15T00:12:12.257 INFO:teuthology.orchestra.run.smithi171.stdout:3331 writesame 0xa2862e7 thru 0xa28c535 data_size 0x1(0x624f bytes) 2024-09-15T00:12:12.274 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:12.272+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:12.274 INFO:teuthology.orchestra.run.smithi171.stdout:3332 writesame 0xd0b6b56 thru 0xd0c4774 data_size 0x1(0xdc1f bytes) 2024-09-15T00:12:12.281 INFO:teuthology.orchestra.run.smithi171.stdout:3333 write 0x9783c73 thru 0x9789c54 (0x5fe2 bytes) 2024-09-15T00:12:12.293 INFO:teuthology.orchestra.run.smithi171.stdout:3334 read 0x2562131 thru 0x256d8a6 (0xb776 bytes) 2024-09-15T00:12:12.293 INFO:teuthology.orchestra.run.smithi171.stdout:3335 punch from 0x463091b to 0x46397cd, (0x8eb2 bytes) 2024-09-15T00:12:12.293 INFO:teuthology.orchestra.run.smithi171.stdout:3336 read 0x34b37c0 thru 0x34b395d (0x19e bytes) 2024-09-15T00:12:12.306 INFO:teuthology.orchestra.run.smithi171.stdout:3337 trunc from 0xd0c4775 to 0x1be6a7a 2024-09-15T00:12:12.325 INFO:teuthology.orchestra.run.smithi171.stdout:3338 trunc from 0x1be6a7a to 0xb15c53 2024-09-15T00:12:12.334 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:12.334 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:12.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:12.334 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:12.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.334 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:12.334 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:12.334 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.759804041000002, 2024-09-15T00:12:12.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:12:12.335 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.335 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.335 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:12.335 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.335 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.335 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.335 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:12:12.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.335 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:12.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:12.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:12:12.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:12:12.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:12:12.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:12:12.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:12:12.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:12:12.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:12:12.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:12:12.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:12:12.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:12:12.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:12:12.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:12:12.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:12.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:12:12.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:12:12.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:12.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:12:12.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:12:12.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:12.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:12:12.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:12:12.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:12.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:12:12.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:12:12.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:12:12.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:12:12.340 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.340 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.340 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.341 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:12.341 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:12.341 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.759769349000003, 2024-09-15T00:12:12.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:12:12.341 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.341 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.341 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:12.341 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.341 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.341 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.341 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:12:12.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.341 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:12.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:12.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:12.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:12:12.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:12:12.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:12:12.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:12:12.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:12:12.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:12:12.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:12:12.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:12:12.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:12:12.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:12:12.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:12:12.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:12.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:12:12.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:12:12.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:12.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:12:12.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:12:12.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:12.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:12:12.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:12:12.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:12.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:12:12.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:12:12.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:12:12.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:12:12.346 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.346 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.346 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.347 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48937 3.1a 3:5e7e71d0:::rbd_data.11395b9829f3.0000000000000000:e [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.347 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:12:12.347 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.719676394, 2024-09-15T00:12:12.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072322336000000001, 2024-09-15T00:12:12.347 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.347 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.347 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.347 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48937 2024-09-15T00:12:12.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.348 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:12:12.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:12:12.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612043+0000", 2024-09-15T00:12:12.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:12:12.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612045+0000", 2024-09-15T00:12:12.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2749999999999998e-06 2024-09-15T00:12:12.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612053+0000", 2024-09-15T00:12:12.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0439999999999996e-06 2024-09-15T00:12:12.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612062+0000", 2024-09-15T00:12:12.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6829999999999997e-06 2024-09-15T00:12:12.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684078+0000", 2024-09-15T00:12:12.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072016017000000002 2024-09-15T00:12:12.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684274+0000", 2024-09-15T00:12:12.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019636600000000001 2024-09-15T00:12:12.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684364+0000", 2024-09-15T00:12:12.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0420000000000005e-05 2024-09-15T00:12:12.351 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.351 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.351 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.351 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.351 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:12.351 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.719095661000001, 2024-09-15T00:12:12.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:12:12.351 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.352 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.352 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.352 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:12:12.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.352 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:12.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:12.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:12.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:12:12.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:12:12.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:12:12.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:12:12.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:12:12.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:12:12.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:12:12.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:12:12.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:12:12.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:12:12.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:12:12.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:12:12.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:12:12.355 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.355 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.355 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.356 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.356 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:12.356 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.719079130000001, 2024-09-15T00:12:12.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:12:12.356 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.356 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.356 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.356 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.356 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.356 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.356 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:12:12.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.356 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:12.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:12.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:12:12.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:12:12.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:12:12.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:12:12.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:12:12.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:12:12.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:12:12.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:12:12.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:12:12.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:12:12.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:12:12.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:12:12.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:12:12.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:12:12.360 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.360 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.360 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.360 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.360 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:12.360 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.719062351000002, 2024-09-15T00:12:12.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:12:12.360 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.360 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.360 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.361 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:12:12.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.361 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:12.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:12.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:12.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:12:12.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:12:12.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:12:12.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:12:12.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:12:12.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:12:12.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:12:12.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:12:12.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:12:12.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:12:12.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:12:12.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:12:12.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:12:12.364 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.364 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.364 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.364 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.364 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:12.365 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.719044620999998, 2024-09-15T00:12:12.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:12:12.365 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.365 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.365 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.365 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.365 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.365 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.365 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:12:12.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.365 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:12.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:12.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:12.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:12:12.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:12:12.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:12:12.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:12:12.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:12:12.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:12:12.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:12:12.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:12:12.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:12:12.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:12:12.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:12:12.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:12:12.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:12:12.369 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.369 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.369 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.369 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.369 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:12.369 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.719027456999999, 2024-09-15T00:12:12.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:12:12.369 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.369 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.369 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.369 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.370 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:12:12.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.370 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:12.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:12.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:12.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:12:12.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:12:12.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:12:12.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:12:12.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:12:12.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:12:12.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:12:12.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:12:12.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:12:12.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:12:12.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:12:12.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:12:12.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:12:12.373 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.373 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.373 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.373 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.373 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:12.374 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.719005781, 2024-09-15T00:12:12.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:12:12.374 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.374 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.374 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.374 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:12:12.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.374 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:12.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:12.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:12.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:12:12.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:12:12.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:12:12.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:12:12.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:12:12.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:12:12.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:12:12.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:12:12.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:12:12.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:12:12.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:12:12.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:12:12.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:12:12.378 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.378 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.378 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.378 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.378 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:12.378 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.718959656999999, 2024-09-15T00:12:12.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:12:12.378 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.378 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.378 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.379 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:12:12.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.379 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:12.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:12.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:12.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:12:12.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:12:12.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:12:12.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:12:12.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:12:12.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:12:12.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:12:12.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:12:12.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:12:12.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:12:12.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:12:12.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:12:12.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:12:12.382 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.382 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.382 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.383 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.383 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:12.383 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.718929899999999, 2024-09-15T00:12:12.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:12:12.383 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.383 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.383 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.384 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.384 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.384 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:12:12.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.384 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:12.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:12.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:12:12.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:12:12.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:12:12.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:12:12.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:12:12.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:12:12.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:12:12.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:12:12.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:12:12.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:12:12.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:12:12.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:12:12.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:12:12.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:12:12.388 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.388 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.388 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.388 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.388 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:12.388 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.718903580999999, 2024-09-15T00:12:12.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:12:12.389 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.389 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.389 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.389 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.389 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.389 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.389 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:12:12.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.389 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:12.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:12.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:12.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:12:12.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:12:12.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:12:12.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:12:12.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:12:12.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:12:12.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:12:12.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:12:12.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:12:12.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:12:12.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:12:12.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:12:12.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:12:12.393 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.393 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.393 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.393 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.393 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:12.393 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.718867718999999, 2024-09-15T00:12:12.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:12:12.393 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.393 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.393 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.394 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.394 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:12:12.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.394 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:12.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:12.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:12.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:12:12.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:12:12.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:12:12.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:12:12.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:12:12.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:12:12.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:12:12.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:12:12.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:12:12.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:12:12.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:12:12.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:12:12.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:12:12.397 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.397 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.397 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.398 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.398 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:12.398 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.718846000999999, 2024-09-15T00:12:12.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:12:12.398 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.398 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.398 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.398 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.399 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.399 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.399 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:12:12.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.399 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:12.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:12.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:12:12.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:12:12.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:12:12.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:12:12.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:12:12.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:12:12.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:12:12.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:12:12.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:12:12.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:12:12.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:12:12.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:12:12.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:12:12.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:12:12.402 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.402 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.402 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.403 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.403 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:12.403 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.718649534000001, 2024-09-15T00:12:12.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:12:12.403 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.403 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.403 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.403 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.403 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.403 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.403 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:12:12.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.403 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:12.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:12.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:12.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:12:12.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:12:12.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:12:12.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:12:12.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:12:12.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:12:12.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:12:12.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:12:12.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:12:12.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:12:12.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:12:12.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:12:12.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:12:12.407 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.407 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.407 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.407 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.407 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:12.407 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.718623695000002, 2024-09-15T00:12:12.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:12:12.407 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.407 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.408 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.408 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.408 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.408 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.408 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:12:12.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.408 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:12.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:12.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:12.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:12:12.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:12:12.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:12:12.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:12:12.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:12:12.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:12:12.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:12:12.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:12:12.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:12:12.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:12:12.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:12:12.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:12:12.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:12:12.411 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.411 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.411 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.412 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.412 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:12.412 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.718562048999999, 2024-09-15T00:12:12.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:12:12.412 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.412 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.412 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.412 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.412 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.413 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.413 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:12:12.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.413 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:12.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:12.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:12:12.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:12:12.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:12:12.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:12:12.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:12:12.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:12:12.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:12:12.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:12:12.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:12:12.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:12:12.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:12:12.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:12:12.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:12:12.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:12:12.416 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.416 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.416 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.417 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.417 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:12.417 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.718522720999999, 2024-09-15T00:12:12.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:12:12.417 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.417 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.417 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.417 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.417 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.417 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.417 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:12:12.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.418 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:12.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:12.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:12:12.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:12:12.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:12:12.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:12:12.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:12:12.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:12:12.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:12:12.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:12:12.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:12:12.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:12:12.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:12:12.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:12:12.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:12:12.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:12:12.421 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.421 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.421 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.421 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.421 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:12.421 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.716797321000001, 2024-09-15T00:12:12.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:12:12.422 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.422 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.422 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.422 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.422 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.422 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.422 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:12:12.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.422 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:12.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:12.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:12:12.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:12:12.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:12:12.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:12:12.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:12:12.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:12:12.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:12:12.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:12:12.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:12:12.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:12:12.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:12:12.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:12:12.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:12:12.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:12:12.425 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.426 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.426 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.426 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.426 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:12.426 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.716775160000001, 2024-09-15T00:12:12.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:12:12.426 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.426 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.426 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.426 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.426 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.426 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.426 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:12:12.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.427 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:12.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:12.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:12.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:12:12.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:12:12.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:12:12.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:12:12.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:12:12.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:12:12.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:12:12.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:12:12.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:12:12.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:12:12.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:12:12.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:12:12.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:12:12.430 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.430 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.430 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.430 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.430 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.430 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:12.432 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:12.433 INFO:teuthology.orchestra.run.smithi171.stdout:3339 write 0xb2be157 thru 0xb2bec13 (0xabd bytes) 2024-09-15T00:12:12.433 INFO:teuthology.orchestra.run.smithi171.stdout:3341 read 0x4842484 thru 0x4850956 (0xe4d3 bytes) 2024-09-15T00:12:12.433 INFO:teuthology.orchestra.run.smithi171.stdout:3342 writesame 0x8247d6b thru 0x82577ea data_size 0xa7(0xfa80 bytes) 2024-09-15T00:12:12.433 INFO:teuthology.orchestra.run.smithi171.stdout:3343 write 0x9609305 thru 0x960c412 (0x310e bytes) 2024-09-15T00:12:12.434 INFO:teuthology.orchestra.run.smithi171.stdout:3345 write 0xcf0ea57 thru 0xcf177fe (0x8da8 bytes) 2024-09-15T00:12:12.434 INFO:teuthology.orchestra.run.smithi171.stdout:3346 read 0x438e976 thru 0x439018d (0x1818 bytes) 2024-09-15T00:12:12.434 INFO:teuthology.orchestra.run.smithi171.stdout:3347 write 0x3d4ff87 thru 0x3d509af (0xa29 bytes) 2024-09-15T00:12:12.434 INFO:teuthology.orchestra.run.smithi171.stdout:3348 punch from 0x5334e81 to 0x533a00d, (0x518c bytes) 2024-09-15T00:12:12.434 INFO:teuthology.orchestra.run.smithi171.stdout:3349 punch from 0x2235292 to 0x2239df5, (0x4b63 bytes) 2024-09-15T00:12:12.434 INFO:teuthology.orchestra.run.smithi171.stdout:3350 write 0xe847f2f thru 0xe84efc0 (0x7092 bytes) 2024-09-15T00:12:12.434 INFO:teuthology.orchestra.run.smithi171.stdout:3351 write 0x82e64a0 thru 0x82f5ca9 (0xf80a bytes) 2024-09-15T00:12:12.434 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:12.430+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:12.443 INFO:teuthology.orchestra.run.smithi171.stdout:3354 trunc from 0xe84efc1 to 0x8939714 2024-09-15T00:12:12.467 INFO:teuthology.orchestra.run.smithi171.stdout:3355 punch from 0x12277e8 to 0x1229c35, (0x244d bytes) 2024-09-15T00:12:12.467 INFO:teuthology.orchestra.run.smithi171.stdout:3357 read 0x1dbbabc thru 0x1dc2abd (0x7002 bytes) 2024-09-15T00:12:12.467 INFO:teuthology.orchestra.run.smithi171.stdout:3359 read 0x4613647 thru 0x46179cf (0x4389 bytes) 2024-09-15T00:12:12.473 INFO:teuthology.orchestra.run.smithi171.stdout:3361 writesame 0xa164f3f thru 0xa166832 data_size 0x2(0x18f4 bytes) 2024-09-15T00:12:12.473 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:12.473 INFO:teuthology.orchestra.run.smithi171.stdout:3364 read 0x2b64959 thru 0x2b69229 (0x48d1 bytes) 2024-09-15T00:12:12.474 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:12.474 INFO:teuthology.orchestra.run.smithi171.stdout:3367 punch from 0x583a1e7 to 0x5843b07, (0x9920 bytes) 2024-09-15T00:12:12.479 INFO:teuthology.orchestra.run.smithi171.stdout:3370 writesame 0xafad8fe thru 0xafb17d5 data_size 0x4(0x3ed8 bytes) 2024-09-15T00:12:12.479 INFO:teuthology.orchestra.run.smithi171.stdout:3371 writesame 0x3724b8a thru 0x3727eab data_size 0x55(0x3322 bytes) 2024-09-15T00:12:12.492 INFO:teuthology.orchestra.run.smithi171.stdout:3372 write 0x9e819a6 thru 0x9e82141 (0x79c bytes) 2024-09-15T00:12:12.499 INFO:teuthology.orchestra.run.smithi171.stdout:3374 read 0x9ddddfb thru 0x9decab0 (0xecb6 bytes) 2024-09-15T00:12:12.499 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:12.499 INFO:teuthology.orchestra.run.smithi171.stdout:3378 writesame 0x508bcb2 thru 0x5093bec data_size 0x21(0x7f3b bytes) 2024-09-15T00:12:12.518 INFO:teuthology.orchestra.run.smithi171.stdout:3379 writesame 0x5542530 thru 0x554c161 data_size 0x1(0x9c32 bytes) 2024-09-15T00:12:12.532 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:12.531+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:12.534 INFO:teuthology.orchestra.run.smithi171.stdout:3380 read 0x7afa3d0 thru 0x7afb836 (0x1467 bytes) 2024-09-15T00:12:12.534 INFO:teuthology.orchestra.run.smithi171.stdout:3381 read 0x370a571 thru 0x370b068 (0xaf8 bytes) 2024-09-15T00:12:12.535 INFO:teuthology.orchestra.run.smithi171.stdout:3383 write 0x5db7a72 thru 0x5dbc471 (0x4a00 bytes) 2024-09-15T00:12:12.540 INFO:teuthology.orchestra.run.smithi171.stdout:3384 write 0x3a58877 thru 0x3a5ee70 (0x65fa bytes) 2024-09-15T00:12:12.546 INFO:teuthology.orchestra.run.smithi171.stdout:3385 read 0x81e8f4e thru 0x81ef53f (0x65f2 bytes) 2024-09-15T00:12:12.554 INFO:teuthology.orchestra.run.smithi171.stdout:3387 write 0xcac53ce thru 0xcaca2b2 (0x4ee5 bytes) 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:12.561 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:12.562 INFO:teuthology.orchestra.run.smithi171.stdout:3389 read 0x598a844 thru 0x598d479 (0x2c36 bytes) 2024-09-15T00:12:12.562 INFO:teuthology.orchestra.run.smithi171.stdout:3390 writesame 0xa4f101e thru 0xa4fed54 data_size 0x2b(0xdd37 bytes) 2024-09-15T00:12:12.572 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:12.579 INFO:teuthology.orchestra.run.smithi171.stdout:3392 read 0x2f4032b thru 0x2f410b7 (0xd8d bytes) 2024-09-15T00:12:12.579 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:12.585 INFO:teuthology.orchestra.run.smithi171.stdout:3394 trunc from 0xcaca2b3 to 0x7fb6b49 2024-09-15T00:12:12.608 INFO:teuthology.orchestra.run.smithi171.stdout:3396 trunc from 0x7fb6b49 to 0x9618323 2024-09-15T00:12:12.612 INFO:teuthology.orchestra.run.smithi171.stdout:3397 read 0x82bb21 thru 0x836e7f (0xb35f bytes) 2024-09-15T00:12:12.613 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:12.613 INFO:teuthology.orchestra.run.smithi171.stdout:3399 read 0x2296a2d thru 0x22978e7 (0xebb bytes) 2024-09-15T00:12:12.613 INFO:teuthology.orchestra.run.smithi171.stdout:3400 writesame 0x2e25b78 thru 0x2e27208 data_size 0x1(0x1691 bytes) 2024-09-15T00:12:12.621 INFO:teuthology.orchestra.run.smithi171.stdout:3402 read 0x8e7ecb0 thru 0x8e82867 (0x3bb8 bytes) 2024-09-15T00:12:12.621 INFO:teuthology.orchestra.run.smithi171.stdout:3405 write 0x4aebb22 thru 0x4af46e0 (0x8bbf bytes) 2024-09-15T00:12:12.631 INFO:teuthology.orchestra.run.smithi171.stdout:3406 write 0x1ccb436 thru 0x1cd641f (0xafea bytes) 2024-09-15T00:12:12.633 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:12.632+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:12.647 INFO:teuthology.orchestra.run.smithi171.stdout:3407 write 0x82d3586 thru 0x82deaf1 (0xb56c bytes) 2024-09-15T00:12:12.663 INFO:teuthology.orchestra.run.smithi171.stdout:3409 trunc from 0x9618323 to 0x2468f0d 2024-09-15T00:12:12.697 INFO:teuthology.orchestra.run.smithi171.stdout:3413 read 0xc90c0c thru 0xc9242b (0x1820 bytes) 2024-09-15T00:12:12.698 INFO:teuthology.orchestra.run.smithi171.stdout:3415 write 0x23906da thru 0x239e5a1 (0xdec8 bytes) 2024-09-15T00:12:12.722 INFO:teuthology.orchestra.run.smithi171.stdout:3416 write 0xcb52b41 thru 0xcb5d187 (0xa647 bytes) 2024-09-15T00:12:12.734 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:12.732+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:12.736 INFO:teuthology.orchestra.run.smithi171.stdout:3417 read 0x5a57748 thru 0x5a5a139 (0x29f2 bytes) 2024-09-15T00:12:12.736 INFO:teuthology.orchestra.run.smithi171.stdout:3419 writesame 0x73938d2 thru 0x739657f data_size 0x2b(0x2cae bytes) 2024-09-15T00:12:12.737 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:12.737 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:12.737 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:12.737 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:12.737 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:12.748 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:12.749 INFO:teuthology.orchestra.run.smithi171.stdout:3420 write 0xd9c35e8 thru 0xd9d2414 (0xee2d bytes) 2024-09-15T00:12:12.767 INFO:teuthology.orchestra.run.smithi171.stdout:3421 write 0x50b2935 thru 0x50c00d1 (0xd79d bytes) 2024-09-15T00:12:12.785 INFO:teuthology.orchestra.run.smithi171.stdout:3422 writesame 0x32ad3ee thru 0x32b1d00 data_size 0x1(0x4913 bytes) 2024-09-15T00:12:12.796 INFO:teuthology.orchestra.run.smithi171.stdout:3423 read 0x9309459 thru 0x9317001 (0xdba9 bytes) 2024-09-15T00:12:12.796 INFO:teuthology.orchestra.run.smithi171.stdout:3428 punch from 0xad934af to 0xad97015, (0x3b66 bytes) 2024-09-15T00:12:12.796 INFO:teuthology.orchestra.run.smithi171.stdout:3430 write 0x102d3e5 thru 0x102de93 (0xaaf bytes) 2024-09-15T00:12:12.801 INFO:teuthology.orchestra.run.smithi171.stdout:3431 punch from 0xcfc06c0 to 0xcfc3b5a, (0x349a bytes) 2024-09-15T00:12:12.801 INFO:teuthology.orchestra.run.smithi171.stdout:3433 read 0x4e5a29b thru 0x4e69356 (0xf0bc bytes) 2024-09-15T00:12:12.816 INFO:teuthology.orchestra.run.smithi171.stdout:3434 trunc from 0xd9d2415 to 0x1ab4de7 2024-09-15T00:12:12.835 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:12.833+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:12.872 INFO:teuthology.orchestra.run.smithi171.stdout:3436 write 0xd3a3a30 thru 0xd3b1eda (0xe4ab bytes) 2024-09-15T00:12:12.894 INFO:teuthology.orchestra.run.smithi171.stdout:3437 trunc from 0xd3b1edb to 0xeb455bc 2024-09-15T00:12:12.899 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:12.899 INFO:teuthology.orchestra.run.smithi171.stdout:3441 read 0x819272d thru 0x819e6ff (0xbfd3 bytes) 2024-09-15T00:12:12.905 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:12.905 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:12.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:12.906 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:12.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.906 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:47986 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.906 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:12.906 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.431465367000001, 2024-09-15T00:12:12.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045287635, 2024-09-15T00:12:12.906 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.906 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.906 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:12.906 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.907 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.907 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.907 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 47986 2024-09-15T00:12:12.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.907 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:12.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:12.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:12.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8400000000000004e-07 2024-09-15T00:12:12.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474928+0000", 2024-09-15T00:12:12.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033009430000000002 2024-09-15T00:12:12.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474943+0000", 2024-09-15T00:12:12.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5172e-05 2024-09-15T00:12:12.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474951+0000", 2024-09-15T00:12:12.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1779999999999997e-06 2024-09-15T00:12:12.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475058+0000", 2024-09-15T00:12:12.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000107595 2024-09-15T00:12:12.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475200+0000", 2024-09-15T00:12:12.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014180400000000001 2024-09-15T00:12:12.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:12.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.478046+0000", 2024-09-15T00:12:12.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028464150000000001 2024-09-15T00:12:12.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:12.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516790+0000", 2024-09-15T00:12:12.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038743278999999999 2024-09-15T00:12:12.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:12.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516832+0000", 2024-09-15T00:12:12.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1974000000000002e-05 2024-09-15T00:12:12.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:12.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516872+0000", 2024-09-15T00:12:12.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9928999999999997e-05 2024-09-15T00:12:12.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516914+0000", 2024-09-15T00:12:12.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2762000000000002e-05 2024-09-15T00:12:12.912 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.912 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.912 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.912 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48957 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 4456448~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.912 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:12:12.913 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.294026854999998, 2024-09-15T00:12:12.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044512426000000001, 2024-09-15T00:12:12.913 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.913 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.913 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.913 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48957 2024-09-15T00:12:12.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.913 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:12:12.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:12:12.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609066+0000", 2024-09-15T00:12:12.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:12:12.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609068+0000", 2024-09-15T00:12:12.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9700000000000002e-06 2024-09-15T00:12:12.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609073+0000", 2024-09-15T00:12:12.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7939999999999999e-06 2024-09-15T00:12:12.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609078+0000", 2024-09-15T00:12:12.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3209999999999998e-06 2024-09-15T00:12:12.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653082+0000", 2024-09-15T00:12:12.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044003852000000003 2024-09-15T00:12:12.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653136+0000", 2024-09-15T00:12:12.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4024000000000003e-05 2024-09-15T00:12:12.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653578+0000", 2024-09-15T00:12:12.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044211599999999999 2024-09-15T00:12:12.917 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.917 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.917 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.917 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48958 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.917 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:12.918 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.294009215999999, 2024-09-15T00:12:12.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044724939999999998, 2024-09-15T00:12:12.918 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.918 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.918 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.918 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.918 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.918 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.918 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48958 2024-09-15T00:12:12.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.918 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609085+0000", 2024-09-15T00:12:12.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6610000000000001e-06 2024-09-15T00:12:12.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609090+0000", 2024-09-15T00:12:12.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2719999999999997e-06 2024-09-15T00:12:12.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609094+0000", 2024-09-15T00:12:12.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.907e-06 2024-09-15T00:12:12.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653618+0000", 2024-09-15T00:12:12.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044523759000000003 2024-09-15T00:12:12.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653703+0000", 2024-09-15T00:12:12.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5550000000000006e-05 2024-09-15T00:12:12.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653808+0000", 2024-09-15T00:12:12.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010445700000000001 2024-09-15T00:12:12.921 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.921 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.921 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.922 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48959 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.922 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:12.922 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.293993276999998, 2024-09-15T00:12:12.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044884741999999998, 2024-09-15T00:12:12.922 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.922 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.922 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.922 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48959 2024-09-15T00:12:12.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.922 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:12.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:12.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:12.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2899999999999999e-07 2024-09-15T00:12:12.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609101+0000", 2024-09-15T00:12:12.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.826e-06 2024-09-15T00:12:12.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609108+0000", 2024-09-15T00:12:12.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.165e-06 2024-09-15T00:12:12.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609113+0000", 2024-09-15T00:12:12.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7849999999999999e-06 2024-09-15T00:12:12.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653842+0000", 2024-09-15T00:12:12.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044728954000000001 2024-09-15T00:12:12.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653896+0000", 2024-09-15T00:12:12.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4593999999999999e-05 2024-09-15T00:12:12.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653984+0000", 2024-09-15T00:12:12.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7089000000000006e-05 2024-09-15T00:12:12.926 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.926 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.926 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.926 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48960 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 9961472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.926 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:12.926 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.293972757999999, 2024-09-15T00:12:12.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045084194000000001, 2024-09-15T00:12:12.926 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.927 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.927 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.927 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48960 2024-09-15T00:12:12.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.927 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:12.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:12.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609120+0000", 2024-09-15T00:12:12.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7099999999999997e-07 2024-09-15T00:12:12.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609122+0000", 2024-09-15T00:12:12.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:12:12.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609129+0000", 2024-09-15T00:12:12.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4159999999999998e-06 2024-09-15T00:12:12.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609134+0000", 2024-09-15T00:12:12.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7210000000000002e-06 2024-09-15T00:12:12.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654023+0000", 2024-09-15T00:12:12.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044888979000000002 2024-09-15T00:12:12.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654104+0000", 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1108999999999999e-05 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654204+0000", 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9630000000000007e-05 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48961 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:12.931 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.293951936999999, 2024-09-15T00:12:12.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045197821999999999, 2024-09-15T00:12:12.931 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.931 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.931 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.931 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.931 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.931 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.931 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48961 2024-09-15T00:12:12.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.931 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:12.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:12.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609141+0000", 2024-09-15T00:12:12.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:12:12.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609143+0000", 2024-09-15T00:12:12.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-15T00:12:12.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609150+0000", 2024-09-15T00:12:12.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2420000000000003e-06 2024-09-15T00:12:12.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609155+0000", 2024-09-15T00:12:12.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.028e-06 2024-09-15T00:12:12.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654232+0000", 2024-09-15T00:12:12.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045076702000000003 2024-09-15T00:12:12.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654281+0000", 2024-09-15T00:12:12.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9798e-05 2024-09-15T00:12:12.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654338+0000", 2024-09-15T00:12:12.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6498e-05 2024-09-15T00:12:12.934 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.935 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.935 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.935 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48962 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 13631488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.935 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:12.935 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.293930886999998, 2024-09-15T00:12:12.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045312458, 2024-09-15T00:12:12.935 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.935 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.935 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.935 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.935 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.935 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.935 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48962 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609162+0000", 2024-09-15T00:12:12.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:12.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609164+0000", 2024-09-15T00:12:12.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0099999999999998e-06 2024-09-15T00:12:12.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609171+0000", 2024-09-15T00:12:12.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7530000000000001e-06 2024-09-15T00:12:12.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609178+0000", 2024-09-15T00:12:12.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1709999999999999e-06 2024-09-15T00:12:12.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654365+0000", 2024-09-15T00:12:12.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045187867999999999 2024-09-15T00:12:12.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654420+0000", 2024-09-15T00:12:12.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4376000000000003e-05 2024-09-15T00:12:12.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654474+0000", 2024-09-15T00:12:12.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3879999999999999e-05 2024-09-15T00:12:12.939 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.939 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.939 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.939 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48971 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.939 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:12.939 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.293899892999999, 2024-09-15T00:12:12.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045896151000000003, 2024-09-15T00:12:12.939 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.940 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.940 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.940 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.940 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.940 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.940 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48971 2024-09-15T00:12:12.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.940 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:12.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609193+0000", 2024-09-15T00:12:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7599999999999997e-07 2024-09-15T00:12:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609195+0000", 2024-09-15T00:12:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:12:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609210+0000", 2024-09-15T00:12:12.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4956999999999999e-05 2024-09-15T00:12:12.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609505+0000", 2024-09-15T00:12:12.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000295078 2024-09-15T00:12:12.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654498+0000", 2024-09-15T00:12:12.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044992277999999997 2024-09-15T00:12:12.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654541+0000", 2024-09-15T00:12:12.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3525000000000001e-05 2024-09-15T00:12:12.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655088+0000", 2024-09-15T00:12:12.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054726799999999995 2024-09-15T00:12:12.943 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.943 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.943 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.944 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48972 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 2097152~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.944 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:12.944 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.293573319, 2024-09-15T00:12:12.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046150885000000003, 2024-09-15T00:12:12.944 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.944 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.944 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.944 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.944 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.944 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.944 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48972 2024-09-15T00:12:12.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.944 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6400000000000002e-07 2024-09-15T00:12:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609523+0000", 2024-09-15T00:12:12.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5300000000000001e-06 2024-09-15T00:12:12.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609533+0000", 2024-09-15T00:12:12.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0023999999999999e-05 2024-09-15T00:12:12.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609539+0000", 2024-09-15T00:12:12.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3119999999999999e-06 2024-09-15T00:12:12.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655115+0000", 2024-09-15T00:12:12.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045576094999999997 2024-09-15T00:12:12.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655197+0000", 2024-09-15T00:12:12.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1129000000000002e-05 2024-09-15T00:12:12.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655670+0000", 2024-09-15T00:12:12.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00047313100000000001 2024-09-15T00:12:12.947 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.947 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.948 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.948 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.948 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:12.948 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.293545056999999, 2024-09-15T00:12:12.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:12:12.948 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.948 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.948 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.948 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.948 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.948 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.948 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:12:12.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.949 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:12.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:12.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:12.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:12:12.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:12:12.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:12:12.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:12:12.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:12:12.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:12:12.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:12:12.952 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.952 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.952 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.952 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.952 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:12.952 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.293523828000001, 2024-09-15T00:12:12.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:12:12.952 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.952 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.952 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.952 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.953 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.953 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.953 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:12:12.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.953 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:12.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:12.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:12:12.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:12:12.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:12:12.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:12:12.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:12:12.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:12:12.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:12:12.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:12:12.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:12:12.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:12:12.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:12:12.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:12:12.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:12:12.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:12:12.956 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.956 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.956 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.956 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.956 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:12.956 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.293498286999998, 2024-09-15T00:12:12.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:12:12.957 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.957 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.957 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.957 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.957 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.957 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.957 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:12:12.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.957 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:12.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:12:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:12:12.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:12:12.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:12:12.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:12:12.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:12:12.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:12:12.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:12:12.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:12:12.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:12:12.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:12:12.961 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.961 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.961 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.961 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.961 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:12.961 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.293475812000001, 2024-09-15T00:12:12.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:12:12.961 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.961 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.962 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.962 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.962 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.962 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.962 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:12:12.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.962 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:12.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:12.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:12:12.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:12:12.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:12:12.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:12:12.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:12:12.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:12:12.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:12:12.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:12:12.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:12:12.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:12:12.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:12:12.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:12:12.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:12:12.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:12:12.965 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.965 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.965 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.965 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.966 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:12.966 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.293455302000002, 2024-09-15T00:12:12.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:12:12.966 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.966 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.966 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.966 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:12:12.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.966 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:12:12.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:12:12.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:12:12.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:12:12.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:12:12.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:12:12.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:12:12.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:12:12.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:12:12.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:12:12.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:12:12.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:12:12.969 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.969 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.970 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.970 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.970 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:12.970 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.293434987000001, 2024-09-15T00:12:12.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:12:12.970 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.970 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.970 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.970 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.970 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.970 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.971 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:12:12.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.971 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:12.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:12.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:12:12.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:12:12.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:12:12.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:12:12.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:12:12.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:12:12.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:12:12.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:12:12.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:12:12.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:12:12.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:12:12.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:12:12.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:12:12.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:12:12.974 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.974 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.974 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.974 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.975 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:12.975 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.293410211000001, 2024-09-15T00:12:12.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:12:12.975 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.975 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.975 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.975 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.975 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.975 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.975 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:12:12.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.975 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:12.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:12.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:12.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:12:12.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:12:12.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:12:12.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:12:12.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:12:12.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:12:12.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:12:12.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:12:12.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:12:12.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:12:12.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:12:12.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:12:12.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:12:12.979 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.979 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.979 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.980 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:12.980 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:12.980 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.114068824, 2024-09-15T00:12:12.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:12:12.980 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.980 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.980 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:12.980 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.980 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.980 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.980 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:12:12.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.981 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:12.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:12.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:12:12.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:12:12.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:12:12.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:12:12.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:12:12.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:12:12.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:12:12.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:12:12.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:12:12.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:12:12.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:12:12.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:12:12.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:12.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:12:12.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:12:12.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:12.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:12:12.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:12:12.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:12.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:12:12.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:12:12.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:12.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:12:12.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:12:12.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:12:12.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:12:12.985 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.985 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.985 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.985 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:12:12.985 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:12.986 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.097115372000001, 2024-09-15T00:12:12.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:12:12.986 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.986 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.986 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.986 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:12.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:12:12.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:12:12.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:12:12.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:12:12.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:12:12.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:12:12.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:12:12.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:12:12.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:12:12.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:12:12.989 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.989 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.989 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.990 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:12:12.990 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:12.990 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.802684465, 2024-09-15T00:12:12.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:12:12.990 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.990 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.990 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:12.990 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:12.990 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:12.990 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:12.990 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:12:12.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.990 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:12.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:12.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:12:12.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:12:12.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:12:12.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:12:12.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:12:12.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:12:12.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:12:12.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:12:12.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:12:12.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:12:12.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:12:12.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:12:12.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:12.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:12:12.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:12:12.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:12.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:12:12.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:12:12.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:12.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:12:12.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:12:12.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:12.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:12:12.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:12:12.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:12:12.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:12:12.995 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.995 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.995 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.995 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:12:12.995 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:12.995 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.790228651, 2024-09-15T00:12:12.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:12:12.996 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:12.996 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:12.996 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:12.996 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:12.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:12.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:12.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:12.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:12.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:12.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:12.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:12:12.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:12:12.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:12.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:12:12.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:12:12.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:12.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:12:12.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:12:12.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:12.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:12:12.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:12:12.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:12.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:12:12.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:12:12.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:12.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:12:12.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:12:12.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:12.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:12.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:12.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:12:12.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:12:12.999 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.999 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.999 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.999 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:12.999 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:12.999 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:13.001 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:13.002 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:12.999+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:13.003 INFO:teuthology.orchestra.run.smithi171.stdout:3443 trunc from 0xeb455bc to 0x8bc19f5 2024-09-15T00:12:13.003 INFO:teuthology.orchestra.run.smithi171.stdout:3444 read 0x26af02d thru 0x26b823e (0x9212 bytes) 2024-09-15T00:12:13.003 INFO:teuthology.orchestra.run.smithi171.stdout:3445 punch from 0x520614d to 0x5206c20, (0xad3 bytes) 2024-09-15T00:12:13.003 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:13.003 INFO:teuthology.orchestra.run.smithi171.stdout:3447 writesame 0x7c6100c thru 0x7c626fd data_size 0x21(0x16f2 bytes) 2024-09-15T00:12:13.003 INFO:teuthology.orchestra.run.smithi171.stdout:3448 write 0xa06559a thru 0xa06d58b (0x7ff2 bytes) 2024-09-15T00:12:13.003 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:13.003 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:13.003 INFO:teuthology.orchestra.run.smithi171.stdout:3454 write 0x8ae9aec thru 0x8af89a4 (0xeeb9 bytes) 2024-09-15T00:12:13.003 INFO:teuthology.orchestra.run.smithi171.stdout:3455 punch from 0x4b64ca6 to 0x4b74ada, (0xfe34 bytes) 2024-09-15T00:12:13.004 INFO:teuthology.orchestra.run.smithi171.stdout:3456 write 0x9ab3617 thru 0x9abc54b (0x8f35 bytes) 2024-09-15T00:12:13.004 INFO:teuthology.orchestra.run.smithi171.stdout:3457 write 0x9df37c9 thru 0x9df62a0 (0x2ad8 bytes) 2024-09-15T00:12:13.004 INFO:teuthology.orchestra.run.smithi171.stdout:3459 write 0xc2392d7 thru 0xc24027c (0x6fa6 bytes) 2024-09-15T00:12:13.004 INFO:teuthology.orchestra.run.smithi171.stdout:3461 write 0x35971c6 thru 0x3599ff2 (0x2e2d bytes) 2024-09-15T00:12:13.004 INFO:teuthology.orchestra.run.smithi171.stdout:3462 punch from 0x4403bb2 to 0x440db22, (0x9f70 bytes) 2024-09-15T00:12:13.004 INFO:teuthology.orchestra.run.smithi171.stdout:3463 write 0xc09bdde thru 0xc0a6a75 (0xac98 bytes) 2024-09-15T00:12:13.004 INFO:teuthology.orchestra.run.smithi171.stdout:3464 read 0xaca12d3 thru 0xaca3b72 (0x28a0 bytes) 2024-09-15T00:12:13.005 INFO:teuthology.orchestra.run.smithi171.stdout:3465 read 0x1133243 thru 0x1134039 (0xdf7 bytes) 2024-09-15T00:12:13.005 INFO:teuthology.orchestra.run.smithi171.stdout:3466 trunc from 0xc24027d to 0xb0d7e45 2024-09-15T00:12:13.020 INFO:teuthology.orchestra.run.smithi171.stdout:3467 write 0x2ba96c0 thru 0x2bb679d (0xd0de bytes) 2024-09-15T00:12:13.033 INFO:teuthology.orchestra.run.smithi171.stdout:3468 write 0x65b9abb thru 0x65c6710 (0xcc56 bytes) 2024-09-15T00:12:13.056 INFO:teuthology.orchestra.run.smithi171.stdout:3469 writesame 0xd495b97 thru 0xd49b54e data_size 0x74(0x59b8 bytes) 2024-09-15T00:12:13.056 INFO:teuthology.orchestra.run.smithi171.stdout:3472 read 0xa8a14b8 thru 0xa8abad7 (0xa620 bytes) 2024-09-15T00:12:13.056 INFO:teuthology.orchestra.run.smithi171.stdout:3473 trunc from 0xd49b54f to 0xdab4ca5 2024-09-15T00:12:13.062 INFO:teuthology.orchestra.run.smithi171.stdout:3474 read 0x9e7514d thru 0x9e7978b (0x463f bytes) 2024-09-15T00:12:13.062 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:13.062 INFO:teuthology.orchestra.run.smithi171.stdout:3477 punch from 0x6a318d5 to 0x6a39b6e, (0x8299 bytes) 2024-09-15T00:12:13.068 INFO:teuthology.orchestra.run.smithi171.stdout:3478 write 0xe4d7989 thru 0xe4db954 (0x3fcc bytes) 2024-09-15T00:12:13.076 INFO:teuthology.orchestra.run.smithi171.stdout:3479 trunc from 0xe4db955 to 0xd8c556b 2024-09-15T00:12:13.085 INFO:teuthology.orchestra.run.smithi171.stdout:3480 read 0xa59fb1 thru 0xa69e14 (0xfe64 bytes) 2024-09-15T00:12:13.086 INFO:teuthology.orchestra.run.smithi171.stdout:3482 read 0x964ad90 thru 0x965a627 (0xf898 bytes) 2024-09-15T00:12:13.098 INFO:teuthology.orchestra.run.smithi171.stdout:3484 trunc from 0xd8c556b to 0x3554c6a 2024-09-15T00:12:13.101 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:13.100+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:13.121 INFO:teuthology.orchestra.run.smithi171.stdout:3485 read 0x14147ab thru 0x141e35d (0x9bb3 bytes) 2024-09-15T00:12:13.122 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:13.122 INFO:teuthology.orchestra.run.smithi171.stdout:3487 read 0x2ddb73c thru 0x2de9c12 (0xe4d7 bytes) 2024-09-15T00:12:13.122 INFO:teuthology.orchestra.run.smithi171.stdout:3488 punch from 0xdb388b to 0xdb790c, (0x4081 bytes) 2024-09-15T00:12:13.140 INFO:teuthology.orchestra.run.smithi171.stdout:3489 write 0xc8294ae thru 0xc82d617 (0x416a bytes) 2024-09-15T00:12:13.151 INFO:teuthology.orchestra.run.smithi171.stdout:3490 read 0xa7d50c2 thru 0xa7e1f94 (0xced3 bytes) 2024-09-15T00:12:13.151 INFO:teuthology.orchestra.run.smithi171.stdout:3491 read 0x29f6e08 thru 0x29fab4b (0x3d44 bytes) 2024-09-15T00:12:13.151 INFO:teuthology.orchestra.run.smithi171.stdout:3492 write 0x3950f03 thru 0x395ad41 (0x9e3f bytes) 2024-09-15T00:12:13.163 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:13.163 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:13.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.163 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:105060 3.14 e70/55 3:29659f17:::rbd_object_map.113978285a61:head v 70'720, mlcod=70'712)", 2024-09-15T00:12:13.163 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:13.160463+0000", 2024-09-15T00:12:13.163 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0014433110000000001, 2024-09-15T00:12:13.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00145849, 2024-09-15T00:12:13.163 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.164 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.164 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.164 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.160463+0000", 2024-09-15T00:12:13.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.160463+0000", 2024-09-15T00:12:13.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.160464+0000", 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4899999999999995e-07 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.160468+0000", 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8070000000000001e-06 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.160475+0000", 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6310000000000003e-06 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.160481+0000", 2024-09-15T00:12:13.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.071e-06 2024-09-15T00:12:13.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.160525+0000", 2024-09-15T00:12:13.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4181000000000003e-05 2024-09-15T00:12:13.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.160545+0000", 2024-09-15T00:12:13.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9967e-05 2024-09-15T00:12:13.166 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.166 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.166 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:105056 3.14 e70/55 3:29659f17:::rbd_object_map.113978285a61:head v 70'716, mlcod=70'710)", 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:13.156848+0000", 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.005058552, 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051097169999999997, 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.156848+0000", 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.156848+0000", 2024-09-15T00:12:13.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.156848+0000", 2024-09-15T00:12:13.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:12:13.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.156853+0000", 2024-09-15T00:12:13.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9280000000000001e-06 2024-09-15T00:12:13.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.156863+0000", 2024-09-15T00:12:13.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3689999999999996e-06 2024-09-15T00:12:13.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.156868+0000", 2024-09-15T00:12:13.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2279999999999998e-06 2024-09-15T00:12:13.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.156915+0000", 2024-09-15T00:12:13.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6742000000000001e-05 2024-09-15T00:12:13.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.156934+0000", 2024-09-15T00:12:13.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9247000000000001e-05 2024-09-15T00:12:13.170 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.170 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.170 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.170 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:105057 3.14 e70/55 3:29659f17:::rbd_object_map.113978285a61:head v 70'717, mlcod=70'710)", 2024-09-15T00:12:13.170 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:13.157908+0000", 2024-09-15T00:12:13.170 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0039982059999999998, 2024-09-15T00:12:13.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040793360000000002, 2024-09-15T00:12:13.170 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.157908+0000", 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.157908+0000", 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.157909+0000", 2024-09-15T00:12:13.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:12:13.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.157913+0000", 2024-09-15T00:12:13.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.929e-06 2024-09-15T00:12:13.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.157922+0000", 2024-09-15T00:12:13.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9549999999999998e-06 2024-09-15T00:12:13.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.157928+0000", 2024-09-15T00:12:13.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9959999999999999e-06 2024-09-15T00:12:13.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.157968+0000", 2024-09-15T00:12:13.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0391e-05 2024-09-15T00:12:13.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.157981+0000", 2024-09-15T00:12:13.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2835e-05 2024-09-15T00:12:13.173 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.173 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.173 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.174 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:105058 3.14 e70/55 3:29659f17:::rbd_object_map.113978285a61:head v 70'718, mlcod=70'710)", 2024-09-15T00:12:13.174 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:13.158606+0000", 2024-09-15T00:12:13.174 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0033001850000000002, 2024-09-15T00:12:13.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034098330000000001, 2024-09-15T00:12:13.174 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.174 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.174 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.174 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.158606+0000", 2024-09-15T00:12:13.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.158606+0000", 2024-09-15T00:12:13.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.158607+0000", 2024-09-15T00:12:13.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6000000000000003e-07 2024-09-15T00:12:13.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.158613+0000", 2024-09-15T00:12:13.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.587e-06 2024-09-15T00:12:13.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.158624+0000", 2024-09-15T00:12:13.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1080999999999999e-05 2024-09-15T00:12:13.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.158631+0000", 2024-09-15T00:12:13.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.827e-06 2024-09-15T00:12:13.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.158673+0000", 2024-09-15T00:12:13.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2253999999999999e-05 2024-09-15T00:12:13.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.158691+0000", 2024-09-15T00:12:13.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8598999999999999e-05 2024-09-15T00:12:13.177 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.177 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.177 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.177 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4409.0:105059 3.14 e70/55 3:29659f17:::rbd_object_map.113978285a61:head v 70'719, mlcod=70'712)", 2024-09-15T00:12:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:13.159664+0000", 2024-09-15T00:12:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0022422560000000002, 2024-09-15T00:12:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0023815310000000001, 2024-09-15T00:12:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.159664+0000", 2024-09-15T00:12:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.159664+0000", 2024-09-15T00:12:13.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.159665+0000", 2024-09-15T00:12:13.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9999999999999997e-07 2024-09-15T00:12:13.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.159671+0000", 2024-09-15T00:12:13.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8089999999999998e-06 2024-09-15T00:12:13.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.159701+0000", 2024-09-15T00:12:13.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0678e-05 2024-09-15T00:12:13.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.159709+0000", 2024-09-15T00:12:13.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5140000000000004e-06 2024-09-15T00:12:13.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.159757+0000", 2024-09-15T00:12:13.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8318000000000001e-05 2024-09-15T00:12:13.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:13.159777+0000", 2024-09-15T00:12:13.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9406e-05 2024-09-15T00:12:13.181 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.181 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.181 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.181 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.181 INFO:teuthology.orchestra.run.smithi184.stdout: ], 2024-09-15T00:12:13.181 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 5 2024-09-15T00:12:13.181 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:13.182 INFO:teuthology.orchestra.run.smithi171.stdout:3493 trunc from 0xc82d618 to 0xc11e904 2024-09-15T00:12:13.182 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:13.184 INFO:teuthology.orchestra.run.smithi171.stdout:3494 write 0xacc9a57 thru 0xacd201b (0x85c5 bytes) 2024-09-15T00:12:13.198 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:13.199 INFO:teuthology.orchestra.run.smithi171.stdout:3496 writesame 0x828a638 thru 0x829572d data_size 0x1(0xb0f6 bytes) 2024-09-15T00:12:13.203 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:13.201+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:13.218 INFO:teuthology.orchestra.run.smithi171.stdout:3498 write 0x504135b thru 0x5044a73 (0x3719 bytes) 2024-09-15T00:12:13.231 INFO:teuthology.orchestra.run.smithi171.stdout:3500 write 0xc58cb6e thru 0xc58debc (0x134f bytes) 2024-09-15T00:12:13.237 INFO:teuthology.orchestra.run.smithi171.stdout:3501 writesame 0x8663dcd thru 0x866672b data_size 0x11(0x295f bytes) 2024-09-15T00:12:13.239 INFO:teuthology.orchestra.run.smithi171.stdout:3502 punch from 0x2057aca to 0x205b915, (0x3e4b bytes) 2024-09-15T00:12:13.240 INFO:teuthology.orchestra.run.smithi171.stdout:3503 write 0xaf44337 thru 0xaf52f78 (0xec42 bytes) 2024-09-15T00:12:13.250 INFO:teuthology.orchestra.run.smithi171.stdout:3504 writesame 0x8605dd6 thru 0x86110f7 data_size 0x1(0xb322 bytes) 2024-09-15T00:12:13.263 INFO:teuthology.orchestra.run.smithi171.stdout:3506 clone 25 order 21 su 131072 sc 5 2024-09-15T00:12:13.300 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:13.301 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:13.301 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:13.301 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:13.301 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:13.304 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:13.302+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:13.309 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:13.397 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:12:13.414 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:19308 3.10 3:0ee1524d:::rbd_data.11398e1a091a.0000000000000017:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-15T00:12:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 34.426888734999999, 2024-09-15T00:12:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02547147, 2024-09-15T00:12:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 19308 2024-09-15T00:12:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985182+0000", 2024-09-15T00:12:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6500000000000001e-06 2024-09-15T00:12:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985189+0000", 2024-09-15T00:12:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9949999999999999e-06 2024-09-15T00:12:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985205+0000", 2024-09-15T00:12:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6055000000000001e-05 2024-09-15T00:12:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985212+0000", 2024-09-15T00:12:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:12:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985261+0000", 2024-09-15T00:12:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8380000000000001e-05 2024-09-15T00:12:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985311+0000", 2024-09-15T00:12:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0763999999999997e-05 2024-09-15T00:12:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.986489+0000", 2024-09-15T00:12:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0011773420000000001 2024-09-15T00:12:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010514+0000", 2024-09-15T00:12:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024025392999999999 2024-09-15T00:12:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010535+0000", 2024-09-15T00:12:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0652000000000001e-05 2024-09-15T00:12:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010597+0000", 2024-09-15T00:12:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2411999999999998e-05 2024-09-15T00:12:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010652+0000", 2024-09-15T00:12:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5074999999999997e-05 2024-09-15T00:12:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.744711766999998, 2024-09-15T00:12:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:12:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:12:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:12:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:12:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:12:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:12:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:12:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:12:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:12:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:12:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:12:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:12:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:12:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:12:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:12:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:12:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:12:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:12:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:12:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:12:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:12:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:12:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:12:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:12:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65520 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.529656712000001, 2024-09-15T00:12:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024513282000000001, 2024-09-15T00:12:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65520 2024-09-15T00:12:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.39e-07 2024-09-15T00:12:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882415+0000", 2024-09-15T00:12:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8020000000000001e-06 2024-09-15T00:12:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882421+0000", 2024-09-15T00:12:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7119999999999997e-06 2024-09-15T00:12:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882425+0000", 2024-09-15T00:12:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2660000000000003e-06 2024-09-15T00:12:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906919+0000", 2024-09-15T00:12:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024494438 2024-09-15T00:12:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906926+0000", 2024-09-15T00:12:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.725e-06 2024-09-15T00:12:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65522 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882465+0000", 2024-09-15T00:12:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.529604286000001, 2024-09-15T00:12:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024507998, 2024-09-15T00:12:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65522 2024-09-15T00:12:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882465+0000", 2024-09-15T00:12:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882465+0000", 2024-09-15T00:12:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882466+0000", 2024-09-15T00:12:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2399999999999999e-07 2024-09-15T00:12:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882472+0000", 2024-09-15T00:12:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7819999999999996e-06 2024-09-15T00:12:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882479+0000", 2024-09-15T00:12:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2859999999999996e-06 2024-09-15T00:12:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882484+0000", 2024-09-15T00:12:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0329999999999999e-06 2024-09-15T00:12:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906966+0000", 2024-09-15T00:12:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024481975 2024-09-15T00:12:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906973+0000", 2024-09-15T00:12:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.498e-06 2024-09-15T00:12:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65523 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882490+0000", 2024-09-15T00:12:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.529579049999999, 2024-09-15T00:12:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024506971999999998, 2024-09-15T00:12:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65523 2024-09-15T00:12:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882490+0000", 2024-09-15T00:12:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882490+0000", 2024-09-15T00:12:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882491+0000", 2024-09-15T00:12:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:12:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882493+0000", 2024-09-15T00:12:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9439999999999999e-06 2024-09-15T00:12:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882499+0000", 2024-09-15T00:12:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1489999999999999e-06 2024-09-15T00:12:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882503+0000", 2024-09-15T00:12:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.617e-06 2024-09-15T00:12:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906991+0000", 2024-09-15T00:12:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024487143999999999 2024-09-15T00:12:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906997+0000", 2024-09-15T00:12:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:12:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65524 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882509+0000", 2024-09-15T00:12:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.529559985999999, 2024-09-15T00:12:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024511689999999999, 2024-09-15T00:12:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65524 2024-09-15T00:12:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882509+0000", 2024-09-15T00:12:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882509+0000", 2024-09-15T00:12:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882510+0000", 2024-09-15T00:12:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:12:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882512+0000", 2024-09-15T00:12:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.818e-06 2024-09-15T00:12:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882518+0000", 2024-09-15T00:12:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8919999999999997e-06 2024-09-15T00:12:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882522+0000", 2024-09-15T00:12:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3939999999999998e-06 2024-09-15T00:12:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907015+0000", 2024-09-15T00:12:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024492726999999999 2024-09-15T00:12:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907021+0000", 2024-09-15T00:12:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5250000000000002e-06 2024-09-15T00:12:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65525 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.529541844000001, 2024-09-15T00:12:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024517753, 2024-09-15T00:12:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65525 2024-09-15T00:12:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:12:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882530+0000", 2024-09-15T00:12:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.804e-06 2024-09-15T00:12:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882535+0000", 2024-09-15T00:12:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5500000000000002e-06 2024-09-15T00:12:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882540+0000", 2024-09-15T00:12:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6909999999999996e-06 2024-09-15T00:12:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907039+0000", 2024-09-15T00:12:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024498809 2024-09-15T00:12:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907045+0000", 2024-09-15T00:12:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5459999999999997e-06 2024-09-15T00:12:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66541 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1449984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:12:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.497934255000001, 2024-09-15T00:12:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024574262999999999, 2024-09-15T00:12:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66541 2024-09-15T00:12:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:12:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:12:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914136+0000", 2024-09-15T00:12:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3599999999999999e-07 2024-09-15T00:12:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914137+0000", 2024-09-15T00:12:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9309999999999998e-06 2024-09-15T00:12:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914143+0000", 2024-09-15T00:12:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0120000000000002e-06 2024-09-15T00:12:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914148+0000", 2024-09-15T00:12:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.707e-06 2024-09-15T00:12:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938536+0000", 2024-09-15T00:12:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024387671999999999 2024-09-15T00:12:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938618+0000", 2024-09-15T00:12:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1811999999999995e-05 2024-09-15T00:12:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938709+0000", 2024-09-15T00:12:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1792999999999993e-05 2024-09-15T00:12:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.452 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66542 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1560576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.452 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:13.452 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.497915114000001, 2024-09-15T00:12:13.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024740410000000001, 2024-09-15T00:12:13.452 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.452 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.452 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.452 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.452 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.452 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.453 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66542 2024-09-15T00:12:13.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.453 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:13.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914155+0000", 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914157+0000", 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914162+0000", 2024-09-15T00:12:13.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:13.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914167+0000", 2024-09-15T00:12:13.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3789999999999999e-06 2024-09-15T00:12:13.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938730+0000", 2024-09-15T00:12:13.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024562694 2024-09-15T00:12:13.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938803+0000", 2024-09-15T00:12:13.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3325999999999996e-05 2024-09-15T00:12:13.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938895+0000", 2024-09-15T00:12:13.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1884000000000003e-05 2024-09-15T00:12:13.456 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.456 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.456 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.456 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66543 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1572864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.457 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:13.457 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.497884854999999, 2024-09-15T00:12:13.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024893976000000002, 2024-09-15T00:12:13.457 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.457 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.457 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.457 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66543 2024-09-15T00:12:13.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.457 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:13.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:13.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:13.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:13.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914188+0000", 2024-09-15T00:12:13.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:12:13.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914193+0000", 2024-09-15T00:12:13.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4129999999999997e-06 2024-09-15T00:12:13.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914198+0000", 2024-09-15T00:12:13.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9309999999999999e-06 2024-09-15T00:12:13.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938914+0000", 2024-09-15T00:12:13.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024716380999999999 2024-09-15T00:12:13.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938993+0000", 2024-09-15T00:12:13.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8763000000000005e-05 2024-09-15T00:12:13.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939079+0000", 2024-09-15T00:12:13.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5372999999999997e-05 2024-09-15T00:12:13.461 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.461 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.461 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.461 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66544 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1683456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.461 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:13.461 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.497862452, 2024-09-15T00:12:13.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025051225, 2024-09-15T00:12:13.461 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.461 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.461 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.461 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.461 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.462 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.462 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66544 2024-09-15T00:12:13.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.462 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:13.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:13.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:13.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:12:13.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914210+0000", 2024-09-15T00:12:13.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2450000000000001e-06 2024-09-15T00:12:13.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914215+0000", 2024-09-15T00:12:13.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7869999999999998e-06 2024-09-15T00:12:13.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914220+0000", 2024-09-15T00:12:13.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7899999999999999e-06 2024-09-15T00:12:13.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939105+0000", 2024-09-15T00:12:13.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024884381000000001 2024-09-15T00:12:13.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939177+0000", 2024-09-15T00:12:13.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2138999999999995e-05 2024-09-15T00:12:13.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939258+0000", 2024-09-15T00:12:13.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1462e-05 2024-09-15T00:12:13.465 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.465 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.465 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.465 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66545 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1695744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.466 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:13.466 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.497840178000001, 2024-09-15T00:12:13.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025209788, 2024-09-15T00:12:13.466 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.466 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.466 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.466 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66545 2024-09-15T00:12:13.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.466 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:13.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:13.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914230+0000", 2024-09-15T00:12:13.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1099999999999996e-07 2024-09-15T00:12:13.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914232+0000", 2024-09-15T00:12:13.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3080000000000002e-06 2024-09-15T00:12:13.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914238+0000", 2024-09-15T00:12:13.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.02e-06 2024-09-15T00:12:13.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914243+0000", 2024-09-15T00:12:13.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1200000000000001e-06 2024-09-15T00:12:13.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939285+0000", 2024-09-15T00:12:13.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025042212000000001 2024-09-15T00:12:13.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939361+0000", 2024-09-15T00:12:13.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5558999999999999e-05 2024-09-15T00:12:13.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939439+0000", 2024-09-15T00:12:13.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8058000000000004e-05 2024-09-15T00:12:13.470 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.470 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.470 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.470 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66546 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.470 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:13.470 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.497820203, 2024-09-15T00:12:13.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025375960999999999, 2024-09-15T00:12:13.470 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.470 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.471 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.471 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.471 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.471 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.471 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66546 2024-09-15T00:12:13.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.471 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:13.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:13.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914250+0000", 2024-09-15T00:12:13.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:12:13.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914252+0000", 2024-09-15T00:12:13.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.052e-06 2024-09-15T00:12:13.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914258+0000", 2024-09-15T00:12:13.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9039999999999997e-06 2024-09-15T00:12:13.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914262+0000", 2024-09-15T00:12:13.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4719999999999997e-06 2024-09-15T00:12:13.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939466+0000", 2024-09-15T00:12:13.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025203550000000002 2024-09-15T00:12:13.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939537+0000", 2024-09-15T00:12:13.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1780999999999996e-05 2024-09-15T00:12:13.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939625+0000", 2024-09-15T00:12:13.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7828e-05 2024-09-15T00:12:13.474 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.474 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.474 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.475 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66547 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1818624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.475 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:13.475 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.497800529999999, 2024-09-15T00:12:13.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025552638999999999, 2024-09-15T00:12:13.475 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.475 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.475 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.475 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.475 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.475 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.475 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66547 2024-09-15T00:12:13.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.476 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:13.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:13.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:13.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:13.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914271+0000", 2024-09-15T00:12:13.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8920000000000001e-06 2024-09-15T00:12:13.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914277+0000", 2024-09-15T00:12:13.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:13.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914282+0000", 2024-09-15T00:12:13.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7190000000000001e-06 2024-09-15T00:12:13.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939654+0000", 2024-09-15T00:12:13.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025372372000000001 2024-09-15T00:12:13.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939734+0000", 2024-09-15T00:12:13.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9946999999999999e-05 2024-09-15T00:12:13.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939822+0000", 2024-09-15T00:12:13.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7680000000000001e-05 2024-09-15T00:12:13.479 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.479 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.479 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.479 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66548 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.479 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:13.479 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.497770143, 2024-09-15T00:12:13.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025712727000000001, 2024-09-15T00:12:13.479 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.480 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.480 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.480 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.480 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.480 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.480 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66548 2024-09-15T00:12:13.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.480 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:13.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:13.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914300+0000", 2024-09-15T00:12:13.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3599999999999999e-07 2024-09-15T00:12:13.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914302+0000", 2024-09-15T00:12:13.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3369999999999998e-06 2024-09-15T00:12:13.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914313+0000", 2024-09-15T00:12:13.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1399999999999999e-05 2024-09-15T00:12:13.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914319+0000", 2024-09-15T00:12:13.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9100000000000002e-06 2024-09-15T00:12:13.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939850+0000", 2024-09-15T00:12:13.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025531087000000001 2024-09-15T00:12:13.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939932+0000", 2024-09-15T00:12:13.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1534999999999998e-05 2024-09-15T00:12:13.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940012+0000", 2024-09-15T00:12:13.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0022000000000001e-05 2024-09-15T00:12:13.483 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.483 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.483 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.484 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66549 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1941504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.484 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:13.484 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.497705052000001, 2024-09-15T00:12:13.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025813836999999999, 2024-09-15T00:12:13.484 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.484 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.484 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.484 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.484 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.484 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.484 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66549 2024-09-15T00:12:13.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.484 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:13.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:13.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914365+0000", 2024-09-15T00:12:13.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2799999999999995e-07 2024-09-15T00:12:13.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914368+0000", 2024-09-15T00:12:13.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.92e-06 2024-09-15T00:12:13.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914376+0000", 2024-09-15T00:12:13.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8110000000000003e-06 2024-09-15T00:12:13.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914381+0000", 2024-09-15T00:12:13.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5119999999999999e-06 2024-09-15T00:12:13.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940040+0000", 2024-09-15T00:12:13.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025658271999999999 2024-09-15T00:12:13.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940103+0000", 2024-09-15T00:12:13.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2891000000000005e-05 2024-09-15T00:12:13.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940178+0000", 2024-09-15T00:12:13.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5703000000000003e-05 2024-09-15T00:12:13.488 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.488 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.488 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.488 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66550 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.488 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:13.488 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.497561472000001, 2024-09-15T00:12:13.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025820701000000001, 2024-09-15T00:12:13.488 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.488 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.488 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.488 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.488 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.489 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.489 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66550 2024-09-15T00:12:13.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.489 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:13.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:13.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914509+0000", 2024-09-15T00:12:13.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1999999999999999e-07 2024-09-15T00:12:13.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914513+0000", 2024-09-15T00:12:13.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1640000000000003e-06 2024-09-15T00:12:13.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914522+0000", 2024-09-15T00:12:13.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7639999999999994e-06 2024-09-15T00:12:13.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914527+0000", 2024-09-15T00:12:13.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9170000000000003e-06 2024-09-15T00:12:13.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940199+0000", 2024-09-15T00:12:13.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025672037000000002 2024-09-15T00:12:13.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940269+0000", 2024-09-15T00:12:13.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9208999999999994e-05 2024-09-15T00:12:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940329+0000", 2024-09-15T00:12:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.999e-05 2024-09-15T00:12:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66551 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.497504724999999, 2024-09-15T00:12:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025917345000000001, 2024-09-15T00:12:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.493 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.493 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.493 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.493 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.493 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.493 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66551 2024-09-15T00:12:13.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.493 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:13.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:13.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:13.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1699999999999997e-07 2024-09-15T00:12:13.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914569+0000", 2024-09-15T00:12:13.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.225e-06 2024-09-15T00:12:13.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914582+0000", 2024-09-15T00:12:13.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3648999999999999e-05 2024-09-15T00:12:13.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914588+0000", 2024-09-15T00:12:13.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8449999999999999e-06 2024-09-15T00:12:13.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940355+0000", 2024-09-15T00:12:13.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025767110999999999 2024-09-15T00:12:13.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940423+0000", 2024-09-15T00:12:13.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7574e-05 2024-09-15T00:12:13.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940482+0000", 2024-09-15T00:12:13.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9224000000000001e-05 2024-09-15T00:12:13.497 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.497 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.497 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.497 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:88237 3.a 3:52924f23:::rbd_data.11396cb1f0e4.0000000000000001:16 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-15T00:12:13.497 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:13.497 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.134339762, 2024-09-15T00:12:13.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024612426999999999, 2024-09-15T00:12:13.497 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.497 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.497 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.497 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.497 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.498 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.498 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88237 2024-09-15T00:12:13.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.498 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:13.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:13.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:13.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7899999999999999e-07 2024-09-15T00:12:13.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277732+0000", 2024-09-15T00:12:13.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1679999999999998e-06 2024-09-15T00:12:13.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277738+0000", 2024-09-15T00:12:13.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.207e-06 2024-09-15T00:12:13.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277743+0000", 2024-09-15T00:12:13.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.639e-06 2024-09-15T00:12:13.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.301683+0000", 2024-09-15T00:12:13.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.023939750999999999 2024-09-15T00:12:13.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.301765+0000", 2024-09-15T00:12:13.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2566999999999997e-05 2024-09-15T00:12:13.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302342+0000", 2024-09-15T00:12:13.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00057671600000000001 2024-09-15T00:12:13.501 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.501 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.501 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.501 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:88238 3.a 3:52924f23:::rbd_data.11396cb1f0e4.0000000000000001:16 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-15T00:12:13.501 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:13.501 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.134320428000001, 2024-09-15T00:12:13.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024851132000000001, 2024-09-15T00:12:13.502 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:13.502 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:13.502 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:13.502 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:13.502 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:13.502 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:13.502 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88238 2024-09-15T00:12:13.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.502 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:13.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:13.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:13.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:13.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:13.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:13.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:13.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277750+0000", 2024-09-15T00:12:13.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3300000000000002e-07 2024-09-15T00:12:13.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:13.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277751+0000", 2024-09-15T00:12:13.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:12:13.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:13.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277757+0000", 2024-09-15T00:12:13.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.767e-06 2024-09-15T00:12:13.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:13.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277769+0000", 2024-09-15T00:12:13.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1874e-05 2024-09-15T00:12:13.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:13.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302374+0000", 2024-09-15T00:12:13.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024604669999999999 2024-09-15T00:12:13.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:13.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302460+0000", 2024-09-15T00:12:13.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6489999999999994e-05 2024-09-15T00:12:13.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:13.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:13.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:13.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302600+0000", 2024-09-15T00:12:13.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013985700000000001 2024-09-15T00:12:13.505 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.505 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.505 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.506 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:13.506 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:13.506 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:13.506 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:13.439+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:13.507 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:13.508 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:13.505+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:13.517 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:13.518 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:13.534 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:13.534 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:13.550 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:12:13.551 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:12:13.566 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:13.566 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:13.582 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:13.582 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:13.608 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:13.607+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:13.652 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:13.652 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:13.652 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:13.652 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:13.661 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:13.709 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:13.707+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:13.763 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:13.763 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:13.763 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:13.763 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:13.763 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:13.773 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:13.799 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:13.800 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:13.810 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:13.808+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:13.815 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:13.816 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:13.833 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:13.833 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:13.911 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:13.909+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:13.916 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:13.917 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:13.926 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:14.012 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:14.011+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:14.033 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:14.034 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:14.043 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:14.050 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:12:14.084 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone24 from 0xc58debd (overlap 0x60e131) to 0x9b01e 2024-09-15T00:12:14.113 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:14.111+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:14.196 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-15T00:12:14.199 INFO:teuthology.orchestra.run.smithi171.stdout:3507 read 0x5b8db6d thru 0x5b9960b (0xba9f bytes) 2024-09-15T00:12:14.199 INFO:teuthology.orchestra.run.smithi171.stdout:3508 punch from 0x7353dc9 to 0x73614f9, (0xd730 bytes) 2024-09-15T00:12:14.203 INFO:teuthology.orchestra.run.smithi171.stdout:3509 read 0x7dc70e thru 0x7e758f (0xae82 bytes) 2024-09-15T00:12:14.203 INFO:teuthology.orchestra.run.smithi171.stdout:3510 read 0xabdbfbe thru 0xabde3fe (0x2441 bytes) 2024-09-15T00:12:14.204 INFO:teuthology.orchestra.run.smithi171.stdout:3511 write 0x49facf3 thru 0x4a05b21 (0xae2f bytes) 2024-09-15T00:12:14.205 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:14.205 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:14.213 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:14.212+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:14.218 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:14.222 INFO:teuthology.orchestra.run.smithi171.stdout:3512 writesame 0x1bb4bcf thru 0x1bbddfb data_size 0x1(0x922d bytes) 2024-09-15T00:12:14.296 INFO:teuthology.orchestra.run.smithi171.stdout:3513 write 0xef2781 thru 0xef71db (0x4a5b bytes) 2024-09-15T00:12:14.308 INFO:teuthology.orchestra.run.smithi171.stdout:3515 clone 26 order 18 su 32768 sc 7 2024-09-15T00:12:14.314 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:14.313+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:14.334 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:14.334 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:14.344 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:14.415 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:14.413+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:14.448 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:14.448 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:14.456 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:14.516 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:14.514+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:14.598 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:14.599 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:14.607 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:14.651 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:14.650+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:14.710 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:14.710 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:14.717 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:14.716+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:14.719 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:12:14.819 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:14.817+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:14.863 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:14.864 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:14.872 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:12:14.920 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:14.919+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:14.975 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:14.975 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:14.983 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:12:15.025 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is unset 2024-09-15T00:12:15.026 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:15.024+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:15.097 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:15.097 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:15.106 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:15.114 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone25 from 0xc58debd (overlap 0xc58debd) to 0x2782c8c 2024-09-15T00:12:15.126 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:15.120+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:15.222 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:15.220+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:15.258 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:15.259 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:15.274 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:15.323 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:15.321+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:15.424 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:15.423+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:15.449 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:15.449 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:15.459 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:15.485 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-15T00:12:15.489 INFO:teuthology.orchestra.run.smithi171.stdout:3516 write 0x9adf82e thru 0x9aec39e (0xcb71 bytes) 2024-09-15T00:12:15.505 INFO:teuthology.orchestra.run.smithi171.stdout:3517 write 0x14a71c6 thru 0x14aa0d9 (0x2f14 bytes) 2024-09-15T00:12:15.511 INFO:teuthology.orchestra.run.smithi171.stdout:3520 read 0x84c3b18 thru 0x84d219e (0xe687 bytes) 2024-09-15T00:12:15.518 INFO:teuthology.orchestra.run.smithi171.stdout:3521 trunc from 0xc58debd to 0x77eb241 2024-09-15T00:12:15.525 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:15.524+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:15.536 INFO:teuthology.orchestra.run.smithi171.stdout:3522 writesame 0x1720bf5 thru 0x172524a data_size 0x3(0x4656 bytes) 2024-09-15T00:12:15.550 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:15.550 INFO:teuthology.orchestra.run.smithi171.stdout:3525 punch from 0x75e6409 to 0x75ea1ba, (0x3db1 bytes) 2024-09-15T00:12:15.562 INFO:teuthology.orchestra.run.smithi171.stdout:3526 writesame 0xd89d81b thru 0xd8a89c5 data_size 0x1(0xb1ab bytes) 2024-09-15T00:12:15.563 INFO:teuthology.orchestra.run.smithi171.stdout:3528 write 0x407a193 thru 0x40859d6 (0xb844 bytes) 2024-09-15T00:12:15.577 INFO:teuthology.orchestra.run.smithi171.stdout:3529 writesame 0xe674434 thru 0xe67c119 data_size 0x49(0x7ce6 bytes) 2024-09-15T00:12:15.577 INFO:teuthology.orchestra.run.smithi171.stdout:3530 writesame 0x6a87759 thru 0x6a8be81 data_size 0x1(0x4729 bytes) 2024-09-15T00:12:15.599 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:15.599 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:15.605 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:15.610 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:15.621 INFO:teuthology.orchestra.run.smithi171.stdout:3533 trunc from 0xe67c11a to 0x15a0fbf 2024-09-15T00:12:15.642 INFO:teuthology.orchestra.run.smithi171.stdout:3534 punch from 0xcf0b73 to 0xcf1505, (0x992 bytes) 2024-09-15T00:12:15.654 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:15.653+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:15.656 INFO:teuthology.orchestra.run.smithi171.stdout:3535 trunc from 0x15a0fbf to 0xcc94016 2024-09-15T00:12:15.668 INFO:teuthology.orchestra.run.smithi171.stdout:3536 trunc from 0xcc94016 to 0x79aaf59 2024-09-15T00:12:15.676 INFO:teuthology.orchestra.run.smithi171.stdout:3537 trunc from 0x79aaf59 to 0x9976b7a 2024-09-15T00:12:15.682 INFO:teuthology.orchestra.run.smithi171.stdout:3538 punch from 0x58331bf to 0x5841219, (0xe05a bytes) 2024-09-15T00:12:15.682 INFO:teuthology.orchestra.run.smithi171.stdout:3539 read 0x1f25344 thru 0x1f2a825 (0x54e2 bytes) 2024-09-15T00:12:15.682 INFO:teuthology.orchestra.run.smithi171.stdout:3540 punch from 0x7b92cb0 to 0x7b99a81, (0x6dd1 bytes) 2024-09-15T00:12:15.682 INFO:teuthology.orchestra.run.smithi171.stdout:3541 read 0x4df9aa5 thru 0x4e05c73 (0xc1cf bytes) 2024-09-15T00:12:15.682 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:15.682 INFO:teuthology.orchestra.run.smithi171.stdout:3544 read 0x27e079a thru 0x27ef36e (0xebd5 bytes) 2024-09-15T00:12:15.691 INFO:teuthology.orchestra.run.smithi171.stdout:3545 trunc from 0x9976b7a to 0x256e24d 2024-09-15T00:12:15.696 INFO:teuthology.orchestra.run.smithi171.stdout:3546 punch from 0x224b8e4 to 0x2252c0b, (0x7327 bytes) 2024-09-15T00:12:15.708 INFO:teuthology.orchestra.run.smithi171.stdout:3548 write 0x6aad0f9 thru 0x6ab3300 (0x6208 bytes) 2024-09-15T00:12:15.711 INFO:teuthology.orchestra.run.smithi171.stdout:3552 read 0x3d18bad thru 0x3d261e9 (0xd63d bytes) 2024-09-15T00:12:15.711 INFO:teuthology.orchestra.run.smithi171.stdout:3554 write 0xd29603 thru 0xd37b5f (0xe55d bytes) 2024-09-15T00:12:15.720 INFO:teuthology.orchestra.run.smithi171.stdout:3555 write 0x398f2f2 thru 0x399e558 (0xf267 bytes) 2024-09-15T00:12:15.721 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:15.721 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:15.727 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:15.725+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:15.729 INFO:teuthology.orchestra.run.smithi171.stdout:3556 writesame 0x3bc1b3f thru 0x3bc8937 data_size 0x1(0x6df9 bytes) 2024-09-15T00:12:15.732 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:15.776 INFO:teuthology.orchestra.run.smithi171.stdout:3558 read 0x2dbbaa4 thru 0x2dc178c (0x5ce9 bytes) 2024-09-15T00:12:15.776 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:15.785 INFO:teuthology.orchestra.run.smithi171.stdout:3560 trunc from 0x6ab3301 to 0xea1ee73 2024-09-15T00:12:15.795 INFO:teuthology.orchestra.run.smithi171.stdout:3561 trunc from 0xea1ee73 to 0xb9eaf4d 2024-09-15T00:12:15.802 INFO:teuthology.orchestra.run.smithi171.stdout:3562 write 0x22a8194 thru 0x22b59df (0xd84c bytes) 2024-09-15T00:12:15.808 INFO:teuthology.orchestra.run.smithi171.stdout:3563 write 0x8d407dc thru 0x8d43c97 (0x34bc bytes) 2024-09-15T00:12:15.811 INFO:teuthology.orchestra.run.smithi171.stdout:3564 trunc from 0xb9eaf4d to 0xb8b9117 2024-09-15T00:12:15.816 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:15.824 INFO:teuthology.orchestra.run.smithi171.stdout:3569 trunc from 0xb8b9117 to 0x57fdc12 2024-09-15T00:12:15.828 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:15.826+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:15.840 INFO:teuthology.orchestra.run.smithi171.stdout:3570 trunc from 0x57fdc12 to 0xf52d5f 2024-09-15T00:12:15.845 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:12:15.868 INFO:teuthology.orchestra.run.smithi171.stdout:3572 read 0x64dbc9 thru 0x657d47 (0xa17f bytes) 2024-09-15T00:12:15.869 INFO:teuthology.orchestra.run.smithi171.stdout:3573 punch from 0x5f0233 to 0x5fcafd, (0xc8ca bytes) 2024-09-15T00:12:15.877 INFO:teuthology.orchestra.run.smithi171.stdout:3576 writesame 0x360193b thru 0x3607dc2 data_size 0x4(0x6488 bytes) 2024-09-15T00:12:15.886 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:15.886 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:15.896 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:15.897 INFO:teuthology.orchestra.run.smithi171.stdout:3579 write 0xe4c99d3 thru 0xe4d409b (0xa6c9 bytes) 2024-09-15T00:12:15.900 INFO:teuthology.orchestra.run.smithi171.stdout:3580 write 0x40d32be thru 0x40dc58f (0x92d2 bytes) 2024-09-15T00:12:15.907 INFO:teuthology.orchestra.run.smithi171.stdout:3581 write 0xcc6a540 thru 0xcc75e2d (0xb8ee bytes) 2024-09-15T00:12:15.912 INFO:teuthology.orchestra.run.smithi171.stdout:3582 read 0xdfdc128 thru 0xdfdd879 (0x1752 bytes) 2024-09-15T00:12:15.912 INFO:teuthology.orchestra.run.smithi171.stdout:3583 writesame 0x889f0c2 thru 0x88a3b26 data_size 0xd(0x4a65 bytes) 2024-09-15T00:12:15.919 INFO:teuthology.orchestra.run.smithi171.stdout:3584 write 0x1d3249b thru 0x1d3de2c (0xb992 bytes) 2024-09-15T00:12:15.927 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:15.927 INFO:teuthology.orchestra.run.smithi171.stdout:3586 writesame 0x2958522 thru 0x295fa3c data_size 0x3(0x751b bytes) 2024-09-15T00:12:15.939 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:15.938+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:15.952 INFO:teuthology.orchestra.run.smithi171.stdout:3588 trunc from 0xe4d409c to 0xa39009b 2024-09-15T00:12:15.962 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:15.962 INFO:teuthology.orchestra.run.smithi171.stdout:3590 punch from 0x47b6a06 to 0x47c59dd, (0xefd7 bytes) 2024-09-15T00:12:15.962 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:15.962 INFO:teuthology.orchestra.run.smithi171.stdout:3592 read 0x39116a8 thru 0x391a628 (0x8f81 bytes) 2024-09-15T00:12:15.962 INFO:teuthology.orchestra.run.smithi171.stdout:3594 write 0x337b2a7 thru 0x3388bc4 (0xd91e bytes) 2024-09-15T00:12:15.969 INFO:teuthology.orchestra.run.smithi171.stdout:3595 write 0x1d3c7ef thru 0x1d48a19 (0xc22b bytes) 2024-09-15T00:12:15.981 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:15.982 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:15.983 INFO:teuthology.orchestra.run.smithi171.stdout:3598 writesame 0xd1207d3 thru 0xd12e445 data_size 0x1(0xdc73 bytes) 2024-09-15T00:12:15.994 INFO:teuthology.orchestra.run.smithi171.stdout:3599 trunc from 0xd12e446 to 0x49bb7ae 2024-09-15T00:12:15.999 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:15.999 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:16.010 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:16.010 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:16.013 INFO:teuthology.orchestra.run.smithi171.stdout:3600 write 0x4c10099 thru 0x4c134a4 (0x340c bytes) 2024-09-15T00:12:16.016 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:12:16.016 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:12:16.016 INFO:teuthology.orchestra.run.smithi171.stdout:3602 read 0x23ceb60 thru 0x23d421a (0x56bb bytes) 2024-09-15T00:12:16.016 INFO:teuthology.orchestra.run.smithi171.stdout:3603 writesame 0x380ebc3 thru 0x380edfc data_size 0x1(0x23a bytes) 2024-09-15T00:12:16.019 INFO:teuthology.orchestra.run.smithi171.stdout:3605 punch from 0x4a3cefd to 0x4a436d0, (0x67d3 bytes) 2024-09-15T00:12:16.021 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:16.024 INFO:teuthology.orchestra.run.smithi171.stdout:3606 writesame 0x4dbdc38 thru 0x4dcc735 data_size 0x7(0xeafe bytes) 2024-09-15T00:12:16.024 INFO:teuthology.orchestra.run.smithi171.stdout:3607 read 0x3e508bb thru 0x3e57020 (0x6766 bytes) 2024-09-15T00:12:16.025 INFO:teuthology.orchestra.run.smithi171.stdout:3609 punch from 0x24a3ced to 0x24b10f6, (0xd409 bytes) 2024-09-15T00:12:16.025 INFO:teuthology.orchestra.run.smithi171.stdout:3610 punch from 0x13bccf0 to 0x13c1285, (0x4595 bytes) 2024-09-15T00:12:16.025 INFO:teuthology.orchestra.run.smithi171.stdout:3611 writesame 0x47bf6aa thru 0x47c3ee8 data_size 0xf(0x483f bytes) 2024-09-15T00:12:16.028 INFO:teuthology.orchestra.run.smithi171.stdout:3612 punch from 0x2f5972a to 0x2f66962, (0xd238 bytes) 2024-09-15T00:12:16.028 INFO:teuthology.orchestra.run.smithi171.stdout:3613 write 0x1830407 thru 0x18395b9 (0x91b3 bytes) 2024-09-15T00:12:16.035 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:16.036 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:16.038 INFO:teuthology.orchestra.run.smithi171.stdout:3614 writesame 0x56f42b9 thru 0x56feec2 data_size 0x3d(0xac0a bytes) 2024-09-15T00:12:16.048 INFO:teuthology.orchestra.run.smithi171.stdout:3616 trunc from 0x56feec3 to 0xbc608ee 2024-09-15T00:12:16.054 INFO:teuthology.orchestra.run.smithi171.stdout:3617 punch from 0x47b704f to 0x47bd4c4, (0x6475 bytes) 2024-09-15T00:12:16.061 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:16.061 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:16.062 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:16.060+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:16.062 INFO:teuthology.orchestra.run.smithi171.stdout:3618 write 0xce72708 thru 0xce7462a (0x1f23 bytes) 2024-09-15T00:12:16.065 INFO:teuthology.orchestra.run.smithi171.stdout:3620 writesame 0x6443e32 thru 0x64455cd data_size 0x2(0x179c bytes) 2024-09-15T00:12:16.068 INFO:teuthology.orchestra.run.smithi171.stdout:3621 read 0x63017e1 thru 0x6304bbb (0x33db bytes) 2024-09-15T00:12:16.068 INFO:teuthology.orchestra.run.smithi171.stdout:3622 read 0x1251dad thru 0x125cbc0 (0xae14 bytes) 2024-09-15T00:12:16.069 INFO:teuthology.orchestra.run.smithi171.stdout:3625 write 0x1873943 thru 0x187e230 (0xa8ee bytes) 2024-09-15T00:12:16.072 INFO:teuthology.orchestra.run.smithi171.stdout:3626 write 0x9a8e6dd thru 0x9a903a6 (0x1cca bytes) 2024-09-15T00:12:16.079 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:16.080 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:16.080 INFO:teuthology.orchestra.run.smithi171.stdout:3627 trunc from 0xce7462b to 0x9f3de68 2024-09-15T00:12:16.101 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:16.101 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:16.102 INFO:teuthology.orchestra.run.smithi171.stdout:3628 write 0xe70bb07 thru 0xe712846 (0x6d40 bytes) 2024-09-15T00:12:16.106 INFO:teuthology.orchestra.run.smithi171.stdout:3629 writesame 0xa093244 thru 0xa0932bd data_size 0x1(0x7a bytes) 2024-09-15T00:12:16.109 INFO:teuthology.orchestra.run.smithi171.stdout:3631 read 0x219899a thru 0x21a4406 (0xba6d bytes) 2024-09-15T00:12:16.109 INFO:teuthology.orchestra.run.smithi171.stdout:3632 write 0x929e141 thru 0x92a27d4 (0x4694 bytes) 2024-09-15T00:12:16.113 INFO:teuthology.orchestra.run.smithi171.stdout:3633 write 0x122544b thru 0x1235445 (0xfffb bytes) 2024-09-15T00:12:16.121 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:16.121 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:16.130 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:16.128+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:16.135 INFO:teuthology.orchestra.run.smithi171.stdout:3634 trunc from 0xe712847 to 0xfa8688 2024-09-15T00:12:16.162 INFO:teuthology.orchestra.run.smithi171.stdout:3635 trunc from 0xfa8688 to 0x7d04716 2024-09-15T00:12:16.169 INFO:teuthology.orchestra.run.smithi171.stdout:3636 punch from 0x3b30374 to 0x3b3fe86, (0xfb12 bytes) 2024-09-15T00:12:16.169 INFO:teuthology.orchestra.run.smithi171.stdout:3637 read 0x79f00e0 thru 0x79fab01 (0xaa22 bytes) 2024-09-15T00:12:16.169 INFO:teuthology.orchestra.run.smithi171.stdout:3638 punch from 0x75ea835 to 0x75f4312, (0x9add bytes) 2024-09-15T00:12:16.169 INFO:teuthology.orchestra.run.smithi171.stdout:3639 writesame 0x79b513d thru 0x79bb78c data_size 0x8(0x6650 bytes) 2024-09-15T00:12:16.176 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:16.176 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:16.176 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:16.176 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:16.183 INFO:teuthology.orchestra.run.smithi171.stdout:3642 write 0xa6e83ae thru 0xa6ef939 (0x758c bytes) 2024-09-15T00:12:16.185 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:16.186 INFO:teuthology.orchestra.run.smithi171.stdout:3645 writesame 0x3a9bc5a thru 0x3aa10e8 data_size 0x1(0x548f bytes) 2024-09-15T00:12:16.232 INFO:teuthology.orchestra.run.smithi171.stdout:3646 writesame 0xcdcf440 thru 0xcdd8f56 data_size 0x1(0x9b17 bytes) 2024-09-15T00:12:16.232 INFO:teuthology.orchestra.run.smithi171.stdout:3648 read 0x991ee21 thru 0x99204a1 (0x1681 bytes) 2024-09-15T00:12:16.232 INFO:teuthology.orchestra.run.smithi171.stdout:3649 read 0x7d1b53 thru 0x7df9a0 (0xde4e bytes) 2024-09-15T00:12:16.233 INFO:teuthology.orchestra.run.smithi171.stdout:3650 writesame 0x1abbf62 thru 0x1ac4abd data_size 0x12(0x8b5c bytes) 2024-09-15T00:12:16.238 INFO:teuthology.orchestra.run.smithi171.stdout:3651 read 0x1b8e53b thru 0x1b8f358 (0xe1e bytes) 2024-09-15T00:12:16.238 INFO:teuthology.orchestra.run.smithi171.stdout:3652 write 0x3f7c5e0 thru 0x3f8236b (0x5d8c bytes) 2024-09-15T00:12:16.242 INFO:teuthology.orchestra.run.smithi171.stdout:3654 writesame 0x5d9645f thru 0x5d9a173 data_size 0x1(0x3d15 bytes) 2024-09-15T00:12:16.265 INFO:teuthology.orchestra.run.smithi171.stdout:3655 trunc from 0xcdd8f57 to 0xd26d1fd 2024-09-15T00:12:16.269 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:16.268+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:16.270 INFO:teuthology.orchestra.run.smithi171.stdout:3656 writesame 0xca9f093 thru 0xcaad515 data_size 0x3d(0xe483 bytes) 2024-09-15T00:12:16.275 INFO:teuthology.orchestra.run.smithi171.stdout:3657 write 0x22ed42c thru 0x22f469e (0x7273 bytes) 2024-09-15T00:12:16.280 INFO:teuthology.orchestra.run.smithi171.stdout:3658 writesame 0x84d870a thru 0x84e1d65 data_size 0x2(0x965c bytes) 2024-09-15T00:12:16.297 INFO:teuthology.orchestra.run.smithi171.stdout:3662 writesame 0x4494bfc thru 0x4496b2b data_size 0x8(0x1f30 bytes) 2024-09-15T00:12:16.301 INFO:teuthology.orchestra.run.smithi171.stdout:3663 trunc from 0xd26d1fd to 0xd29d579 2024-09-15T00:12:16.306 INFO:teuthology.orchestra.run.smithi171.stdout:3664 write 0x3d45800 thru 0x3d5173e (0xbf3f bytes) 2024-09-15T00:12:16.313 INFO:teuthology.orchestra.run.smithi171.stdout:3665 write 0x83fb305 thru 0x8405b47 (0xa843 bytes) 2024-09-15T00:12:16.316 INFO:teuthology.orchestra.run.smithi171.stdout:3666 write 0x1705cd0 thru 0x1709cd2 (0x4003 bytes) 2024-09-15T00:12:16.332 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:16.329+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:16.334 INFO:teuthology.orchestra.run.smithi171.stdout:3669 trunc from 0xd29d579 to 0x1d3a9d1 2024-09-15T00:12:16.340 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:16.340 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:16.340 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:16.340 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:16.341 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:16.353 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:16.353 INFO:teuthology.orchestra.run.smithi171.stdout:3671 read 0x7ceec7 thru 0x7d553c (0x6676 bytes) 2024-09-15T00:12:16.353 INFO:teuthology.orchestra.run.smithi171.stdout:3672 read 0x14347d7 thru 0x143b879 (0x70a3 bytes) 2024-09-15T00:12:16.353 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:16.353 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:16.353 INFO:teuthology.orchestra.run.smithi171.stdout:3675 trunc from 0x1d3a9d1 to 0xa62abf 2024-09-15T00:12:16.354 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:16.388 INFO:teuthology.orchestra.run.smithi171.stdout:3676 writesame 0x4d233ec thru 0x4d2f782 data_size 0x17(0xc397 bytes) 2024-09-15T00:12:16.388 INFO:teuthology.orchestra.run.smithi171.stdout:3678 punch from 0x205a408 to 0x205c6f2, (0x22ea bytes) 2024-09-15T00:12:16.388 INFO:teuthology.orchestra.run.smithi171.stdout:3679 read 0x3d220b9 thru 0x3d2edf9 (0xcd41 bytes) 2024-09-15T00:12:16.389 INFO:teuthology.orchestra.run.smithi171.stdout:3680 trunc from 0x4d2f783 to 0x6007d20 2024-09-15T00:12:16.399 INFO:teuthology.orchestra.run.smithi171.stdout:3681 trunc from 0x6007d20 to 0x9d01ebe 2024-09-15T00:12:16.405 INFO:teuthology.orchestra.run.smithi171.stdout:3682 write 0x695f9d6 thru 0x6962b3c (0x3167 bytes) 2024-09-15T00:12:16.409 INFO:teuthology.orchestra.run.smithi171.stdout:3683 punch from 0x7a42d58 to 0x7a4c0fb, (0x93a3 bytes) 2024-09-15T00:12:16.409 INFO:teuthology.orchestra.run.smithi171.stdout:3685 read 0x827c0e8 thru 0x8287180 (0xb099 bytes) 2024-09-15T00:12:16.409 INFO:teuthology.orchestra.run.smithi171.stdout:3686 clone 27 order 18 su 65536 sc 9 2024-09-15T00:12:16.439 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:16.437+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:16.500 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:16.501 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:16.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:16.501 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:16.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.501 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:47986 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.501 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:16.501 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.026681046, 2024-09-15T00:12:16.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045287635, 2024-09-15T00:12:16.501 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.502 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.502 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:16.502 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.502 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.502 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.502 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 47986 2024-09-15T00:12:16.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.502 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:16.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:16.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:16.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8400000000000004e-07 2024-09-15T00:12:16.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474928+0000", 2024-09-15T00:12:16.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033009430000000002 2024-09-15T00:12:16.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474943+0000", 2024-09-15T00:12:16.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5172e-05 2024-09-15T00:12:16.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474951+0000", 2024-09-15T00:12:16.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1779999999999997e-06 2024-09-15T00:12:16.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475058+0000", 2024-09-15T00:12:16.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000107595 2024-09-15T00:12:16.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475200+0000", 2024-09-15T00:12:16.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014180400000000001 2024-09-15T00:12:16.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:16.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.478046+0000", 2024-09-15T00:12:16.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028464150000000001 2024-09-15T00:12:16.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:16.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516790+0000", 2024-09-15T00:12:16.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038743278999999999 2024-09-15T00:12:16.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:16.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516832+0000", 2024-09-15T00:12:16.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1974000000000002e-05 2024-09-15T00:12:16.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:16.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516872+0000", 2024-09-15T00:12:16.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9928999999999997e-05 2024-09-15T00:12:16.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516914+0000", 2024-09-15T00:12:16.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2762000000000002e-05 2024-09-15T00:12:16.507 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.507 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.507 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.508 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48957 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 4456448~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.508 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:12:16.508 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.889242534000001, 2024-09-15T00:12:16.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044512426000000001, 2024-09-15T00:12:16.508 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.508 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.508 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.508 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.508 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.508 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.508 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48957 2024-09-15T00:12:16.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.508 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609066+0000", 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609068+0000", 2024-09-15T00:12:16.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9700000000000002e-06 2024-09-15T00:12:16.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609073+0000", 2024-09-15T00:12:16.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7939999999999999e-06 2024-09-15T00:12:16.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609078+0000", 2024-09-15T00:12:16.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3209999999999998e-06 2024-09-15T00:12:16.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653082+0000", 2024-09-15T00:12:16.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044003852000000003 2024-09-15T00:12:16.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653136+0000", 2024-09-15T00:12:16.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4024000000000003e-05 2024-09-15T00:12:16.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653578+0000", 2024-09-15T00:12:16.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044211599999999999 2024-09-15T00:12:16.512 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.512 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.512 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.512 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48958 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.512 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:16.512 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.889224895000002, 2024-09-15T00:12:16.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044724939999999998, 2024-09-15T00:12:16.512 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.512 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.512 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.513 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.513 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.513 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.513 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48958 2024-09-15T00:12:16.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.513 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:16.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:16.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:16.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:12:16.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609085+0000", 2024-09-15T00:12:16.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6610000000000001e-06 2024-09-15T00:12:16.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609090+0000", 2024-09-15T00:12:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2719999999999997e-06 2024-09-15T00:12:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609094+0000", 2024-09-15T00:12:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.907e-06 2024-09-15T00:12:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653618+0000", 2024-09-15T00:12:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044523759000000003 2024-09-15T00:12:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653703+0000", 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5550000000000006e-05 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653808+0000", 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010445700000000001 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48959 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.889208956000001, 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044884741999999998, 2024-09-15T00:12:16.516 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.517 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.517 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.517 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.517 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.517 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.517 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48959 2024-09-15T00:12:16.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.517 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:16.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:16.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:16.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2899999999999999e-07 2024-09-15T00:12:16.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609101+0000", 2024-09-15T00:12:16.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.826e-06 2024-09-15T00:12:16.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609108+0000", 2024-09-15T00:12:16.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.165e-06 2024-09-15T00:12:16.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609113+0000", 2024-09-15T00:12:16.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7849999999999999e-06 2024-09-15T00:12:16.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653842+0000", 2024-09-15T00:12:16.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044728954000000001 2024-09-15T00:12:16.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653896+0000", 2024-09-15T00:12:16.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4593999999999999e-05 2024-09-15T00:12:16.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653984+0000", 2024-09-15T00:12:16.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7089000000000006e-05 2024-09-15T00:12:16.520 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.520 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.520 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.521 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48960 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 9961472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.521 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:16.521 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.889188437000001, 2024-09-15T00:12:16.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045084194000000001, 2024-09-15T00:12:16.521 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.521 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.521 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.521 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.521 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.521 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.521 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48960 2024-09-15T00:12:16.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.521 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609120+0000", 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7099999999999997e-07 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609122+0000", 2024-09-15T00:12:16.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:12:16.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609129+0000", 2024-09-15T00:12:16.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4159999999999998e-06 2024-09-15T00:12:16.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609134+0000", 2024-09-15T00:12:16.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7210000000000002e-06 2024-09-15T00:12:16.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654023+0000", 2024-09-15T00:12:16.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044888979000000002 2024-09-15T00:12:16.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654104+0000", 2024-09-15T00:12:16.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1108999999999999e-05 2024-09-15T00:12:16.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654204+0000", 2024-09-15T00:12:16.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9630000000000007e-05 2024-09-15T00:12:16.525 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.525 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.525 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.525 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48961 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.525 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:16.525 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.889167616000002, 2024-09-15T00:12:16.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045197821999999999, 2024-09-15T00:12:16.525 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.525 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.525 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.525 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.525 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.525 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.526 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48961 2024-09-15T00:12:16.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.526 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:16.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:16.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609141+0000", 2024-09-15T00:12:16.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:12:16.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609143+0000", 2024-09-15T00:12:16.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-15T00:12:16.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609150+0000", 2024-09-15T00:12:16.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2420000000000003e-06 2024-09-15T00:12:16.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609155+0000", 2024-09-15T00:12:16.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.028e-06 2024-09-15T00:12:16.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654232+0000", 2024-09-15T00:12:16.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045076702000000003 2024-09-15T00:12:16.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654281+0000", 2024-09-15T00:12:16.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9798e-05 2024-09-15T00:12:16.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654338+0000", 2024-09-15T00:12:16.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6498e-05 2024-09-15T00:12:16.529 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.529 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.529 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.529 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48962 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 13631488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.529 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:16.530 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.889146566000001, 2024-09-15T00:12:16.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045312458, 2024-09-15T00:12:16.530 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.530 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.530 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.530 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.530 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.530 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.530 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48962 2024-09-15T00:12:16.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.530 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:16.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:16.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609162+0000", 2024-09-15T00:12:16.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:16.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609164+0000", 2024-09-15T00:12:16.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0099999999999998e-06 2024-09-15T00:12:16.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609171+0000", 2024-09-15T00:12:16.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7530000000000001e-06 2024-09-15T00:12:16.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609178+0000", 2024-09-15T00:12:16.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1709999999999999e-06 2024-09-15T00:12:16.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654365+0000", 2024-09-15T00:12:16.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045187867999999999 2024-09-15T00:12:16.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654420+0000", 2024-09-15T00:12:16.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4376000000000003e-05 2024-09-15T00:12:16.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654474+0000", 2024-09-15T00:12:16.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3879999999999999e-05 2024-09-15T00:12:16.533 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.534 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.534 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.534 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48971 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.534 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:16.534 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.889115572000001, 2024-09-15T00:12:16.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045896151000000003, 2024-09-15T00:12:16.534 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.534 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.534 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.534 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.534 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.534 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.535 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48971 2024-09-15T00:12:16.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.535 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:16.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:16.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609193+0000", 2024-09-15T00:12:16.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7599999999999997e-07 2024-09-15T00:12:16.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609195+0000", 2024-09-15T00:12:16.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:12:16.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609210+0000", 2024-09-15T00:12:16.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4956999999999999e-05 2024-09-15T00:12:16.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609505+0000", 2024-09-15T00:12:16.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000295078 2024-09-15T00:12:16.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654498+0000", 2024-09-15T00:12:16.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044992277999999997 2024-09-15T00:12:16.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654541+0000", 2024-09-15T00:12:16.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3525000000000001e-05 2024-09-15T00:12:16.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655088+0000", 2024-09-15T00:12:16.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054726799999999995 2024-09-15T00:12:16.538 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.538 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.538 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.538 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48972 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 2097152~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.538 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:16.538 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.888788997999999, 2024-09-15T00:12:16.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046150885000000003, 2024-09-15T00:12:16.539 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.539 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.539 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.539 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.539 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.539 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.539 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48972 2024-09-15T00:12:16.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.539 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:16.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:16.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:16.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6400000000000002e-07 2024-09-15T00:12:16.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609523+0000", 2024-09-15T00:12:16.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5300000000000001e-06 2024-09-15T00:12:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609533+0000", 2024-09-15T00:12:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0023999999999999e-05 2024-09-15T00:12:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609539+0000", 2024-09-15T00:12:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3119999999999999e-06 2024-09-15T00:12:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655115+0000", 2024-09-15T00:12:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045576094999999997 2024-09-15T00:12:16.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655197+0000", 2024-09-15T00:12:16.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1129000000000002e-05 2024-09-15T00:12:16.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655670+0000", 2024-09-15T00:12:16.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00047313100000000001 2024-09-15T00:12:16.542 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.542 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.542 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.543 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.543 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:16.543 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.888760736000002, 2024-09-15T00:12:16.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:12:16.543 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.543 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.543 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.543 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.543 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.543 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.543 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:12:16.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.543 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:16.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:16.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:16.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:12:16.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:12:16.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:12:16.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:12:16.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:12:16.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:12:16.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:12:16.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:12:16.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:12:16.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:12:16.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:12:16.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:12:16.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:12:16.547 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.547 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.547 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.547 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.547 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:16.547 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.888739507, 2024-09-15T00:12:16.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:12:16.547 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:12:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:16.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:12:16.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:12:16.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:12:16.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:12:16.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:12:16.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:12:16.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:12:16.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:12:16.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:12:16.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.551 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.552 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:16.552 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.888713966000001, 2024-09-15T00:12:16.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:12:16.552 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.552 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.552 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.552 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.552 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.552 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.552 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:12:16.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.552 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:16.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:16.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:16.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:12:16.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:12:16.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:12:16.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:12:16.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:12:16.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:12:16.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:12:16.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:12:16.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:12:16.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:12:16.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:12:16.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:12:16.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:12:16.556 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.556 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.556 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.556 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.556 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:16.556 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.888691490999999, 2024-09-15T00:12:16.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:12:16.556 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.556 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.556 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.556 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.556 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.557 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.557 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:12:16.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.557 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:16.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:16.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:12:16.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:12:16.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:12:16.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:12:16.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:12:16.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:12:16.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:12:16.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:12:16.560 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.560 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.560 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.560 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.560 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:16.560 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.888670981000001, 2024-09-15T00:12:16.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:12:16.560 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.560 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.561 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.561 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.561 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.561 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.561 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:12:16.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.561 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:16.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:16.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:16.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:12:16.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:12:16.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:12:16.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:12:16.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:12:16.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:12:16.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:12:16.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:12:16.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:12:16.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:12:16.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:12:16.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:12:16.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:12:16.564 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.564 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.565 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.565 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.565 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:16.565 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.888650666, 2024-09-15T00:12:16.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:12:16.565 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.565 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.565 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.565 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.565 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.565 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.565 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:12:16.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.566 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:16.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:16.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:12:16.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:12:16.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:12:16.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:12:16.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:12:16.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:12:16.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:12:16.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:12:16.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:12:16.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:12:16.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:12:16.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:12:16.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:12:16.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:12:16.569 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.569 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.569 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.569 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.569 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:16.569 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.88862589, 2024-09-15T00:12:16.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:12:16.569 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.570 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.570 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.570 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.570 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.570 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.570 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:12:16.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.570 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:16.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:16.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:16.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:12:16.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:12:16.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:12:16.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:12:16.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:12:16.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:12:16.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:12:16.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:12:16.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:12:16.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:12:16.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:12:16.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:12:16.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:12:16.574 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.574 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.574 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.574 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:16.574 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:16.574 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.709284502999999, 2024-09-15T00:12:16.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:12:16.574 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.574 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.574 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:16.575 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.575 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.575 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.575 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:12:16.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.575 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:16.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:16.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:12:16.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:12:16.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:12:16.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:12:16.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:12:16.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:12:16.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:12:16.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:12:16.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:12:16.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:12:16.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:12:16.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:12:16.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:16.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:12:16.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:12:16.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:16.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:12:16.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:12:16.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:16.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:12:16.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:12:16.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:16.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:12:16.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:12:16.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:12:16.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:12:16.580 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.580 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.580 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.580 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:12:16.580 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:16.580 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.692331051, 2024-09-15T00:12:16.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:12:16.580 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.580 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.580 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.580 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:16.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:16.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:12:16.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:12:16.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:12:16.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:12:16.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:12:16.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:12:16.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:12:16.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:12:16.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:12:16.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:12:16.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:12:16.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:12:16.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:12:16.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:12:16.584 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.584 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.584 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.584 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:12:16.584 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:16.584 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.397900144000001, 2024-09-15T00:12:16.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:12:16.584 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.584 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.584 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:16.584 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:16.585 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:16.585 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:16.585 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:12:16.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.585 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:16.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:16.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:12:16.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:12:16.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:12:16.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:12:16.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:12:16.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:12:16.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:12:16.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:12:16.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:12:16.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:12:16.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:12:16.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:12:16.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:16.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:12:16.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:12:16.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:16.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:12:16.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:12:16.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:16.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:12:16.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:12:16.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:16.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:12:16.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:12:16.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:12:16.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:12:16.590 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.590 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.590 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.590 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:12:16.590 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:16.590 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.385444329999999, 2024-09-15T00:12:16.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:12:16.590 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:16.591 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:16.591 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:16.591 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:16.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:16.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:16.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:16.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:16.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:16.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:16.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:12:16.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:12:16.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:16.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:12:16.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:12:16.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:16.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:12:16.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:12:16.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:16.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:12:16.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:12:16.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:16.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:12:16.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:12:16.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:16.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:12:16.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:12:16.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:16.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:16.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:16.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:12:16.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:12:16.594 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.594 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.594 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.594 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:16.594 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:16.594 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:16.596 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:16.596 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:16.593+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:16.696 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:16.694+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:16.738 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:16.738 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:16.738 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:16.738 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:16.746 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:16.797 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:16.795+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:16.849 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:16.850 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:16.850 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:16.850 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:16.850 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:16.858 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:16.898 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:16.896+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:16.998 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:16.996+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:17.003 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:17.003 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:17.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:17.003 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:17.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.004 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:19308 3.10 3:0ee1524d:::rbd_data.11398e1a091a.0000000000000017:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-15T00:12:17.004 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:17.004 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.016011444, 2024-09-15T00:12:17.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02547147, 2024-09-15T00:12:17.004 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.004 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.004 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:17.004 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.004 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.004 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.004 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 19308 2024-09-15T00:12:17.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.004 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:17.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:17.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985182+0000", 2024-09-15T00:12:17.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6500000000000001e-06 2024-09-15T00:12:17.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985189+0000", 2024-09-15T00:12:17.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9949999999999999e-06 2024-09-15T00:12:17.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985205+0000", 2024-09-15T00:12:17.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6055000000000001e-05 2024-09-15T00:12:17.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985212+0000", 2024-09-15T00:12:17.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:12:17.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985261+0000", 2024-09-15T00:12:17.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8380000000000001e-05 2024-09-15T00:12:17.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985311+0000", 2024-09-15T00:12:17.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0763999999999997e-05 2024-09-15T00:12:17.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:17.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.986489+0000", 2024-09-15T00:12:17.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0011773420000000001 2024-09-15T00:12:17.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:17.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010514+0000", 2024-09-15T00:12:17.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024025392999999999 2024-09-15T00:12:17.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:17.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010535+0000", 2024-09-15T00:12:17.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0652000000000001e-05 2024-09-15T00:12:17.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:17.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010597+0000", 2024-09-15T00:12:17.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2411999999999998e-05 2024-09-15T00:12:17.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010652+0000", 2024-09-15T00:12:17.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5074999999999997e-05 2024-09-15T00:12:17.009 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.010 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.010 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.010 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:17.010 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:17.010 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.333834476, 2024-09-15T00:12:17.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:12:17.010 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.010 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.010 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:17.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.011 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.011 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:12:17.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.011 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:17.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:17.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:12:17.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:12:17.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:12:17.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:12:17.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:12:17.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:12:17.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:12:17.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:12:17.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:12:17.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:12:17.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:12:17.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:12:17.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:17.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:12:17.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:12:17.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:17.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:12:17.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:12:17.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:17.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:12:17.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:12:17.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:17.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:12:17.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:12:17.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:12:17.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:12:17.016 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.016 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.016 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.016 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65520 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.016 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:17.016 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.118779420999999, 2024-09-15T00:12:17.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024513282000000001, 2024-09-15T00:12:17.016 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.016 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.016 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:17.017 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.017 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.017 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.017 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65520 2024-09-15T00:12:17.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.017 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:17.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:17.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:17.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.39e-07 2024-09-15T00:12:17.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882415+0000", 2024-09-15T00:12:17.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8020000000000001e-06 2024-09-15T00:12:17.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882421+0000", 2024-09-15T00:12:17.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7119999999999997e-06 2024-09-15T00:12:17.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882425+0000", 2024-09-15T00:12:17.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2660000000000003e-06 2024-09-15T00:12:17.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906919+0000", 2024-09-15T00:12:17.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024494438 2024-09-15T00:12:17.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906926+0000", 2024-09-15T00:12:17.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.725e-06 2024-09-15T00:12:17.020 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.020 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.020 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.020 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65522 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.020 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882465+0000", 2024-09-15T00:12:17.020 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.118726994999999, 2024-09-15T00:12:17.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024507998, 2024-09-15T00:12:17.020 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.021 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.021 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:17.021 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.021 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.021 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.021 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65522 2024-09-15T00:12:17.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.021 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882465+0000", 2024-09-15T00:12:17.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882465+0000", 2024-09-15T00:12:17.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882466+0000", 2024-09-15T00:12:17.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2399999999999999e-07 2024-09-15T00:12:17.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882472+0000", 2024-09-15T00:12:17.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7819999999999996e-06 2024-09-15T00:12:17.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882479+0000", 2024-09-15T00:12:17.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2859999999999996e-06 2024-09-15T00:12:17.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882484+0000", 2024-09-15T00:12:17.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0329999999999999e-06 2024-09-15T00:12:17.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906966+0000", 2024-09-15T00:12:17.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024481975 2024-09-15T00:12:17.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906973+0000", 2024-09-15T00:12:17.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.498e-06 2024-09-15T00:12:17.024 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.024 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.024 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.024 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65523 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.024 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882490+0000", 2024-09-15T00:12:17.024 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.118701759, 2024-09-15T00:12:17.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024506971999999998, 2024-09-15T00:12:17.025 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.025 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.025 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:17.025 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.025 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.025 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.025 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65523 2024-09-15T00:12:17.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.025 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882490+0000", 2024-09-15T00:12:17.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882490+0000", 2024-09-15T00:12:17.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882491+0000", 2024-09-15T00:12:17.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:12:17.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882493+0000", 2024-09-15T00:12:17.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9439999999999999e-06 2024-09-15T00:12:17.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882499+0000", 2024-09-15T00:12:17.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1489999999999999e-06 2024-09-15T00:12:17.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882503+0000", 2024-09-15T00:12:17.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.617e-06 2024-09-15T00:12:17.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906991+0000", 2024-09-15T00:12:17.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024487143999999999 2024-09-15T00:12:17.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906997+0000", 2024-09-15T00:12:17.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:12:17.028 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.028 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.028 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.029 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65524 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.029 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882509+0000", 2024-09-15T00:12:17.029 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.118682695, 2024-09-15T00:12:17.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024511689999999999, 2024-09-15T00:12:17.029 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.029 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.029 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:17.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.029 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65524 2024-09-15T00:12:17.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.029 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882509+0000", 2024-09-15T00:12:17.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882509+0000", 2024-09-15T00:12:17.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882510+0000", 2024-09-15T00:12:17.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:12:17.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882512+0000", 2024-09-15T00:12:17.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.818e-06 2024-09-15T00:12:17.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882518+0000", 2024-09-15T00:12:17.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8919999999999997e-06 2024-09-15T00:12:17.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882522+0000", 2024-09-15T00:12:17.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3939999999999998e-06 2024-09-15T00:12:17.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907015+0000", 2024-09-15T00:12:17.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024492726999999999 2024-09-15T00:12:17.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907021+0000", 2024-09-15T00:12:17.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5250000000000002e-06 2024-09-15T00:12:17.033 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.033 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.033 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.033 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65525 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.033 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:17.033 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.118664552999999, 2024-09-15T00:12:17.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024517753, 2024-09-15T00:12:17.033 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.033 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.033 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:17.033 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.034 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.034 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.034 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65525 2024-09-15T00:12:17.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.034 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:17.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:17.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:17.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:12:17.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882530+0000", 2024-09-15T00:12:17.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.804e-06 2024-09-15T00:12:17.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882535+0000", 2024-09-15T00:12:17.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5500000000000002e-06 2024-09-15T00:12:17.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882540+0000", 2024-09-15T00:12:17.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6909999999999996e-06 2024-09-15T00:12:17.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907039+0000", 2024-09-15T00:12:17.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024498809 2024-09-15T00:12:17.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907045+0000", 2024-09-15T00:12:17.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5459999999999997e-06 2024-09-15T00:12:17.037 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.037 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.037 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.037 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66541 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1449984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.038 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:12:17.038 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.087056963999999, 2024-09-15T00:12:17.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024574262999999999, 2024-09-15T00:12:17.038 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.038 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.038 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.038 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.038 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.038 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.038 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66541 2024-09-15T00:12:17.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.038 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:12:17.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:12:17.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914136+0000", 2024-09-15T00:12:17.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3599999999999999e-07 2024-09-15T00:12:17.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914137+0000", 2024-09-15T00:12:17.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9309999999999998e-06 2024-09-15T00:12:17.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914143+0000", 2024-09-15T00:12:17.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0120000000000002e-06 2024-09-15T00:12:17.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914148+0000", 2024-09-15T00:12:17.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.707e-06 2024-09-15T00:12:17.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938536+0000", 2024-09-15T00:12:17.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024387671999999999 2024-09-15T00:12:17.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938618+0000", 2024-09-15T00:12:17.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1811999999999995e-05 2024-09-15T00:12:17.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938709+0000", 2024-09-15T00:12:17.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1792999999999993e-05 2024-09-15T00:12:17.042 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.042 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.042 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.042 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66542 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1560576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.042 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:17.042 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.087037822999999, 2024-09-15T00:12:17.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024740410000000001, 2024-09-15T00:12:17.042 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.042 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.043 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.043 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.043 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.043 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.043 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66542 2024-09-15T00:12:17.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.043 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:17.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:17.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914155+0000", 2024-09-15T00:12:17.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:12:17.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914157+0000", 2024-09-15T00:12:17.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:12:17.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914162+0000", 2024-09-15T00:12:17.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:17.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914167+0000", 2024-09-15T00:12:17.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3789999999999999e-06 2024-09-15T00:12:17.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938730+0000", 2024-09-15T00:12:17.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024562694 2024-09-15T00:12:17.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938803+0000", 2024-09-15T00:12:17.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3325999999999996e-05 2024-09-15T00:12:17.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938895+0000", 2024-09-15T00:12:17.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1884000000000003e-05 2024-09-15T00:12:17.046 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.046 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.047 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.047 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66543 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1572864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.047 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:17.047 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.087007564, 2024-09-15T00:12:17.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024893976000000002, 2024-09-15T00:12:17.047 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.047 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.047 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.047 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.047 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.048 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.048 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66543 2024-09-15T00:12:17.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.048 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:17.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:17.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:17.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:17.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914188+0000", 2024-09-15T00:12:17.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:12:17.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914193+0000", 2024-09-15T00:12:17.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4129999999999997e-06 2024-09-15T00:12:17.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914198+0000", 2024-09-15T00:12:17.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9309999999999999e-06 2024-09-15T00:12:17.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938914+0000", 2024-09-15T00:12:17.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024716380999999999 2024-09-15T00:12:17.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938993+0000", 2024-09-15T00:12:17.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8763000000000005e-05 2024-09-15T00:12:17.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939079+0000", 2024-09-15T00:12:17.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5372999999999997e-05 2024-09-15T00:12:17.051 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.052 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.052 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.052 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66544 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1683456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.052 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:17.052 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.086985161000001, 2024-09-15T00:12:17.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025051225, 2024-09-15T00:12:17.052 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.052 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.052 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.052 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.052 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.053 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.053 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66544 2024-09-15T00:12:17.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.053 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:17.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:17.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:17.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:12:17.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914210+0000", 2024-09-15T00:12:17.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2450000000000001e-06 2024-09-15T00:12:17.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914215+0000", 2024-09-15T00:12:17.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7869999999999998e-06 2024-09-15T00:12:17.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914220+0000", 2024-09-15T00:12:17.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7899999999999999e-06 2024-09-15T00:12:17.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939105+0000", 2024-09-15T00:12:17.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024884381000000001 2024-09-15T00:12:17.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939177+0000", 2024-09-15T00:12:17.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2138999999999995e-05 2024-09-15T00:12:17.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939258+0000", 2024-09-15T00:12:17.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1462e-05 2024-09-15T00:12:17.056 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.056 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.056 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.056 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66545 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1695744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.057 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:17.057 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.086962886999999, 2024-09-15T00:12:17.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025209788, 2024-09-15T00:12:17.057 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.057 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.057 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.057 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.057 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.057 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.057 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66545 2024-09-15T00:12:17.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.057 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:17.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:17.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914230+0000", 2024-09-15T00:12:17.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1099999999999996e-07 2024-09-15T00:12:17.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914232+0000", 2024-09-15T00:12:17.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3080000000000002e-06 2024-09-15T00:12:17.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914238+0000", 2024-09-15T00:12:17.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.02e-06 2024-09-15T00:12:17.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914243+0000", 2024-09-15T00:12:17.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1200000000000001e-06 2024-09-15T00:12:17.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939285+0000", 2024-09-15T00:12:17.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025042212000000001 2024-09-15T00:12:17.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939361+0000", 2024-09-15T00:12:17.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5558999999999999e-05 2024-09-15T00:12:17.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939439+0000", 2024-09-15T00:12:17.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8058000000000004e-05 2024-09-15T00:12:17.061 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.061 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.061 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.061 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66546 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.061 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:17.061 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.086942912000001, 2024-09-15T00:12:17.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025375960999999999, 2024-09-15T00:12:17.062 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.062 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.062 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.062 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.062 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.062 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.062 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66546 2024-09-15T00:12:17.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.062 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:17.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:17.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914250+0000", 2024-09-15T00:12:17.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:12:17.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914252+0000", 2024-09-15T00:12:17.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.052e-06 2024-09-15T00:12:17.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914258+0000", 2024-09-15T00:12:17.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9039999999999997e-06 2024-09-15T00:12:17.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914262+0000", 2024-09-15T00:12:17.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4719999999999997e-06 2024-09-15T00:12:17.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939466+0000", 2024-09-15T00:12:17.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025203550000000002 2024-09-15T00:12:17.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939537+0000", 2024-09-15T00:12:17.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1780999999999996e-05 2024-09-15T00:12:17.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939625+0000", 2024-09-15T00:12:17.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7828e-05 2024-09-15T00:12:17.066 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.066 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.066 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.066 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66547 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1818624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.066 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:17.066 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.086923239000001, 2024-09-15T00:12:17.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025552638999999999, 2024-09-15T00:12:17.066 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.067 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.067 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.067 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.067 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.067 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.067 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66547 2024-09-15T00:12:17.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.067 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:17.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:17.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:17.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:17.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914271+0000", 2024-09-15T00:12:17.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8920000000000001e-06 2024-09-15T00:12:17.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914277+0000", 2024-09-15T00:12:17.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:17.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914282+0000", 2024-09-15T00:12:17.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7190000000000001e-06 2024-09-15T00:12:17.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939654+0000", 2024-09-15T00:12:17.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025372372000000001 2024-09-15T00:12:17.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939734+0000", 2024-09-15T00:12:17.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9946999999999999e-05 2024-09-15T00:12:17.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939822+0000", 2024-09-15T00:12:17.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7680000000000001e-05 2024-09-15T00:12:17.070 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.071 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.071 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.071 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66548 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.071 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:17.071 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.086892851999998, 2024-09-15T00:12:17.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025712727000000001, 2024-09-15T00:12:17.071 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.071 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.071 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.071 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.071 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.072 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.072 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66548 2024-09-15T00:12:17.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.072 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:17.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:17.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914300+0000", 2024-09-15T00:12:17.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3599999999999999e-07 2024-09-15T00:12:17.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914302+0000", 2024-09-15T00:12:17.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3369999999999998e-06 2024-09-15T00:12:17.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914313+0000", 2024-09-15T00:12:17.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1399999999999999e-05 2024-09-15T00:12:17.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914319+0000", 2024-09-15T00:12:17.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9100000000000002e-06 2024-09-15T00:12:17.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939850+0000", 2024-09-15T00:12:17.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025531087000000001 2024-09-15T00:12:17.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939932+0000", 2024-09-15T00:12:17.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1534999999999998e-05 2024-09-15T00:12:17.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940012+0000", 2024-09-15T00:12:17.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0022000000000001e-05 2024-09-15T00:12:17.075 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.075 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.075 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.076 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66549 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1941504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.076 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:17.076 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.086827760999999, 2024-09-15T00:12:17.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025813836999999999, 2024-09-15T00:12:17.076 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.076 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.076 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.076 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.076 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.076 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.076 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66549 2024-09-15T00:12:17.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.077 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:17.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:17.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914365+0000", 2024-09-15T00:12:17.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2799999999999995e-07 2024-09-15T00:12:17.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914368+0000", 2024-09-15T00:12:17.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.92e-06 2024-09-15T00:12:17.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914376+0000", 2024-09-15T00:12:17.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8110000000000003e-06 2024-09-15T00:12:17.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914381+0000", 2024-09-15T00:12:17.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5119999999999999e-06 2024-09-15T00:12:17.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940040+0000", 2024-09-15T00:12:17.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025658271999999999 2024-09-15T00:12:17.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940103+0000", 2024-09-15T00:12:17.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2891000000000005e-05 2024-09-15T00:12:17.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940178+0000", 2024-09-15T00:12:17.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5703000000000003e-05 2024-09-15T00:12:17.080 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.080 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.080 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.080 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66550 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.080 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:17.080 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.086684180999999, 2024-09-15T00:12:17.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025820701000000001, 2024-09-15T00:12:17.080 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.080 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.081 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.081 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.081 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.081 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.081 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66550 2024-09-15T00:12:17.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.081 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:17.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:17.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914509+0000", 2024-09-15T00:12:17.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1999999999999999e-07 2024-09-15T00:12:17.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914513+0000", 2024-09-15T00:12:17.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1640000000000003e-06 2024-09-15T00:12:17.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914522+0000", 2024-09-15T00:12:17.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7639999999999994e-06 2024-09-15T00:12:17.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914527+0000", 2024-09-15T00:12:17.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9170000000000003e-06 2024-09-15T00:12:17.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940199+0000", 2024-09-15T00:12:17.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025672037000000002 2024-09-15T00:12:17.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940269+0000", 2024-09-15T00:12:17.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9208999999999994e-05 2024-09-15T00:12:17.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940329+0000", 2024-09-15T00:12:17.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.999e-05 2024-09-15T00:12:17.084 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.084 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.084 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.085 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66551 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:17.085 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:17.085 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.086627434, 2024-09-15T00:12:17.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025917345000000001, 2024-09-15T00:12:17.085 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.085 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.085 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.085 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.085 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.085 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.085 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66551 2024-09-15T00:12:17.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.085 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:17.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:17.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:17.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1699999999999997e-07 2024-09-15T00:12:17.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914569+0000", 2024-09-15T00:12:17.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.225e-06 2024-09-15T00:12:17.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914582+0000", 2024-09-15T00:12:17.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3648999999999999e-05 2024-09-15T00:12:17.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914588+0000", 2024-09-15T00:12:17.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8449999999999999e-06 2024-09-15T00:12:17.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940355+0000", 2024-09-15T00:12:17.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025767110999999999 2024-09-15T00:12:17.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940423+0000", 2024-09-15T00:12:17.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7574e-05 2024-09-15T00:12:17.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940482+0000", 2024-09-15T00:12:17.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9224000000000001e-05 2024-09-15T00:12:17.089 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.089 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.089 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.089 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:88237 3.a 3:52924f23:::rbd_data.11396cb1f0e4.0000000000000001:16 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-15T00:12:17.089 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:17.089 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.723462470999999, 2024-09-15T00:12:17.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024612426999999999, 2024-09-15T00:12:17.089 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.089 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.089 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.090 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.090 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.090 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.090 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88237 2024-09-15T00:12:17.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.090 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:17.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:17.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:17.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7899999999999999e-07 2024-09-15T00:12:17.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277732+0000", 2024-09-15T00:12:17.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1679999999999998e-06 2024-09-15T00:12:17.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277738+0000", 2024-09-15T00:12:17.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.207e-06 2024-09-15T00:12:17.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277743+0000", 2024-09-15T00:12:17.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.639e-06 2024-09-15T00:12:17.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.301683+0000", 2024-09-15T00:12:17.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.023939750999999999 2024-09-15T00:12:17.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.301765+0000", 2024-09-15T00:12:17.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2566999999999997e-05 2024-09-15T00:12:17.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302342+0000", 2024-09-15T00:12:17.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00057671600000000001 2024-09-15T00:12:17.093 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.093 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.093 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.094 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:88238 3.a 3:52924f23:::rbd_data.11396cb1f0e4.0000000000000001:16 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-15T00:12:17.094 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:17.094 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.723443137, 2024-09-15T00:12:17.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024851132000000001, 2024-09-15T00:12:17.094 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.094 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.094 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.094 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.094 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.094 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.094 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88238 2024-09-15T00:12:17.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.094 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:17.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:17.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277750+0000", 2024-09-15T00:12:17.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3300000000000002e-07 2024-09-15T00:12:17.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277751+0000", 2024-09-15T00:12:17.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:12:17.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277757+0000", 2024-09-15T00:12:17.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.767e-06 2024-09-15T00:12:17.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277769+0000", 2024-09-15T00:12:17.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1874e-05 2024-09-15T00:12:17.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302374+0000", 2024-09-15T00:12:17.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024604669999999999 2024-09-15T00:12:17.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302460+0000", 2024-09-15T00:12:17.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6489999999999994e-05 2024-09-15T00:12:17.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302600+0000", 2024-09-15T00:12:17.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013985700000000001 2024-09-15T00:12:17.098 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.098 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.098 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.098 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.098 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.098 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:17.098 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:12:17.099 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 0] out_osds: [2, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T00:12:17.099 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:12:17.099 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-15T00:12:17.099 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T00:12:17.100 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:17.101 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:17.098+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:17.116 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone26 from 0x9d01ebe (overlap 0xa62abf) to 0x958cdf 2024-09-15T00:12:17.201 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:17.199+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:17.231 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-15T00:12:17.263 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:17.264 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:17.264 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:17.264 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:17.277 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:17.302 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:17.300+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:17.349 INFO:teuthology.orchestra.run.smithi171.stdout:3687 read 0x649cb98 thru 0x64a62c3 (0x972c bytes) 2024-09-15T00:12:17.354 INFO:teuthology.orchestra.run.smithi171.stdout:3688 trunc from 0x9d01ebe to 0xd77d6e4 2024-09-15T00:12:17.363 INFO:teuthology.orchestra.run.smithi171.stdout:3689 writesame 0x542f5de thru 0x543f215 data_size 0x1c(0xfc38 bytes) 2024-09-15T00:12:17.379 INFO:teuthology.orchestra.run.smithi171.stdout:3690 trunc from 0xd77d6e4 to 0x49aadae 2024-09-15T00:12:17.393 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:17.393 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:17.393 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:17.393 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:17.393 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:17.402 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:17.404 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:17.401+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:17.413 INFO:teuthology.orchestra.run.smithi171.stdout:3691 writesame 0xc74db86 thru 0xc75442a data_size 0x1(0x68a5 bytes) 2024-09-15T00:12:17.413 INFO:teuthology.orchestra.run.smithi171.stdout:3692 punch from 0xa0c480a to 0xa0d182e, (0xd024 bytes) 2024-09-15T00:12:17.414 INFO:teuthology.orchestra.run.smithi171.stdout:3694 write 0xa3ff7ab thru 0xa401f85 (0x27db bytes) 2024-09-15T00:12:17.419 INFO:teuthology.orchestra.run.smithi171.stdout:3696 write 0x9904cd6 thru 0x9907ac0 (0x2deb bytes) 2024-09-15T00:12:17.422 INFO:teuthology.orchestra.run.smithi171.stdout:3698 read 0x1f401b5 thru 0x1f40a12 (0x85e bytes) 2024-09-15T00:12:17.422 INFO:teuthology.orchestra.run.smithi171.stdout:3700 punch from 0x242c469 to 0x2430d67, (0x48fe bytes) 2024-09-15T00:12:17.423 INFO:teuthology.orchestra.run.smithi171.stdout:3701 trunc from 0xc75442b to 0xcda799f 2024-09-15T00:12:17.428 INFO:teuthology.orchestra.run.smithi171.stdout:3702 read 0x9123db4 thru 0x9132c4c (0xee99 bytes) 2024-09-15T00:12:17.428 INFO:teuthology.orchestra.run.smithi171.stdout:3703 write 0x7f10868 thru 0x7f137e1 (0x2f7a bytes) 2024-09-15T00:12:17.432 INFO:teuthology.orchestra.run.smithi171.stdout:3704 read 0x2b07a07 thru 0x2b08cb6 (0x12b0 bytes) 2024-09-15T00:12:17.432 INFO:teuthology.orchestra.run.smithi171.stdout:3707 punch from 0x4585d25 to 0x4585d82, (0x5d bytes) 2024-09-15T00:12:17.432 INFO:teuthology.orchestra.run.smithi171.stdout:3708 read 0xbcbf5be thru 0xbcc46f5 (0x5138 bytes) 2024-09-15T00:12:17.432 INFO:teuthology.orchestra.run.smithi171.stdout:3710 read 0x64c7612 thru 0x64ce066 (0x6a55 bytes) 2024-09-15T00:12:17.432 INFO:teuthology.orchestra.run.smithi171.stdout:3711 write 0x8c8021f thru 0x8c8b046 (0xae28 bytes) 2024-09-15T00:12:17.436 INFO:teuthology.orchestra.run.smithi171.stdout:3712 write 0x50b338c thru 0x50b5910 (0x2585 bytes) 2024-09-15T00:12:17.439 INFO:teuthology.orchestra.run.smithi171.stdout:3714 read 0x5835a59 thru 0x5839eb4 (0x445c bytes) 2024-09-15T00:12:17.440 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:17.442 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:12:17.442 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:17.442 INFO:teuthology.orchestra.run.smithi171.stdout: "epoch": 73, 2024-09-15T00:12:17.442 INFO:teuthology.orchestra.run.smithi171.stdout: "fsid": "bad2533c-9c09-4af4-ab93-147afbacb705", 2024-09-15T00:12:17.442 INFO:teuthology.orchestra.run.smithi171.stdout: "created": "2024-09-15T00:10:50.203030+0000", 2024-09-15T00:12:17.443 INFO:teuthology.orchestra.run.smithi171.stdout: "modified": "2024-09-15T00:12:17.026096+0000", 2024-09-15T00:12:17.443 INFO:teuthology.orchestra.run.smithi171.stdout: "last_up_change": "2024-09-15T00:11:03.441129+0000", 2024-09-15T00:12:17.443 INFO:teuthology.orchestra.run.smithi171.stdout: "last_in_change": "2024-09-15T00:12:06.204759+0000", 2024-09-15T00:12:17.443 INFO:teuthology.orchestra.run.smithi171.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T00:12:17.443 INFO:teuthology.orchestra.run.smithi171.stdout: "flags_num": 5799936, 2024-09-15T00:12:17.443 INFO:teuthology.orchestra.run.smithi171.stdout: "flags_set": [ 2024-09-15T00:12:17.443 INFO:teuthology.orchestra.run.smithi171.stdout: "pglog_hardlimit", 2024-09-15T00:12:17.443 INFO:teuthology.orchestra.run.smithi171.stdout: "purged_snapdirs", 2024-09-15T00:12:17.443 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_deletes", 2024-09-15T00:12:17.443 INFO:teuthology.orchestra.run.smithi171.stdout: "sortbitwise" 2024-09-15T00:12:17.443 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:12:17.443 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_version": 5, 2024-09-15T00:12:17.443 INFO:teuthology.orchestra.run.smithi171.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T00:12:17.443 INFO:teuthology.orchestra.run.smithi171.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T00:12:17.444 INFO:teuthology.orchestra.run.smithi171.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T00:12:17.444 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_snapshot": "", 2024-09-15T00:12:17.444 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_max": 3, 2024-09-15T00:12:17.444 INFO:teuthology.orchestra.run.smithi171.stdout: "max_osd": 8, 2024-09-15T00:12:17.444 INFO:teuthology.orchestra.run.smithi171.stdout: "require_min_compat_client": "luminous", 2024-09-15T00:12:17.444 INFO:teuthology.orchestra.run.smithi171.stdout: "min_compat_client": "jewel", 2024-09-15T00:12:17.444 INFO:teuthology.orchestra.run.smithi171.stdout: "require_osd_release": "squid", 2024-09-15T00:12:17.444 INFO:teuthology.orchestra.run.smithi171.stdout: "allow_crimson": false, 2024-09-15T00:12:17.444 INFO:teuthology.orchestra.run.smithi171.stdout: "pools": [ 2024-09-15T00:12:17.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.444 INFO:teuthology.orchestra.run.smithi171.stdout: "pool": 1, 2024-09-15T00:12:17.444 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_name": ".mgr", 2024-09-15T00:12:17.444 INFO:teuthology.orchestra.run.smithi171.stdout: "create_time": "2024-09-15T00:11:04.326356+0000", 2024-09-15T00:12:17.445 INFO:teuthology.orchestra.run.smithi171.stdout: "flags": 1, 2024-09-15T00:12:17.445 INFO:teuthology.orchestra.run.smithi171.stdout: "flags_names": "hashpspool", 2024-09-15T00:12:17.445 INFO:teuthology.orchestra.run.smithi171.stdout: "type": 1, 2024-09-15T00:12:17.445 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 2, 2024-09-15T00:12:17.445 INFO:teuthology.orchestra.run.smithi171.stdout: "min_size": 1, 2024-09-15T00:12:17.445 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_rule": 0, 2024-09-15T00:12:17.445 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:12:17.445 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:12:17.445 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:12:17.445 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:12:17.445 INFO:teuthology.orchestra.run.smithi171.stdout: "is_stretch_pool": false, 2024-09-15T00:12:17.445 INFO:teuthology.orchestra.run.smithi171.stdout: "object_hash": 2, 2024-09-15T00:12:17.445 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:12:17.446 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num": 1, 2024-09-15T00:12:17.446 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num": 1, 2024-09-15T00:12:17.446 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num_target": 1, 2024-09-15T00:12:17.446 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_target": 1, 2024-09-15T00:12:17.446 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_pending": 1, 2024-09-15T00:12:17.446 INFO:teuthology.orchestra.run.smithi171.stdout: "last_pg_merge_meta": { 2024-09-15T00:12:17.446 INFO:teuthology.orchestra.run.smithi171.stdout: "source_pgid": "0.0", 2024-09-15T00:12:17.446 INFO:teuthology.orchestra.run.smithi171.stdout: "ready_epoch": 0, 2024-09-15T00:12:17.446 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_started": 0, 2024-09-15T00:12:17.446 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_clean": 0, 2024-09-15T00:12:17.446 INFO:teuthology.orchestra.run.smithi171.stdout: "source_version": "0'0", 2024-09-15T00:12:17.446 INFO:teuthology.orchestra.run.smithi171.stdout: "target_version": "0'0" 2024-09-15T00:12:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "last_change": "17", 2024-09-15T00:12:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend": "0", 2024-09-15T00:12:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:12:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:12:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "auid": 0, 2024-09-15T00:12:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:12:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_seq": 0, 2024-09-15T00:12:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_epoch": 0, 2024-09-15T00:12:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_snaps": [], 2024-09-15T00:12:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "removed_snaps": "[]", 2024-09-15T00:12:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_bytes": 0, 2024-09-15T00:12:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_objects": 0, 2024-09-15T00:12:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "tiers": [], 2024-09-15T00:12:17.448 INFO:teuthology.orchestra.run.smithi171.stdout: "tier_of": -1, 2024-09-15T00:12:17.448 INFO:teuthology.orchestra.run.smithi171.stdout: "read_tier": -1, 2024-09-15T00:12:17.448 INFO:teuthology.orchestra.run.smithi171.stdout: "write_tier": -1, 2024-09-15T00:12:17.448 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_mode": "none", 2024-09-15T00:12:17.448 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_bytes": 0, 2024-09-15T00:12:17.448 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_objects": 0, 2024-09-15T00:12:17.448 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:12:17.448 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:12:17.448 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:12:17.448 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_flush_age": 0, 2024-09-15T00:12:17.448 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_evict_age": 0, 2024-09-15T00:12:17.448 INFO:teuthology.orchestra.run.smithi171.stdout: "erasure_code_profile": "", 2024-09-15T00:12:17.449 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_params": { 2024-09-15T00:12:17.449 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "none" 2024-09-15T00:12:17.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.449 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_period": 0, 2024-09-15T00:12:17.449 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_count": 0, 2024-09-15T00:12:17.449 INFO:teuthology.orchestra.run.smithi171.stdout: "use_gmt_hitset": true, 2024-09-15T00:12:17.449 INFO:teuthology.orchestra.run.smithi171.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:12:17.449 INFO:teuthology.orchestra.run.smithi171.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:12:17.449 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:12:17.449 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:12:17.449 INFO:teuthology.orchestra.run.smithi171.stdout: "grade_table": [], 2024-09-15T00:12:17.449 INFO:teuthology.orchestra.run.smithi171.stdout: "stripe_width": 0, 2024-09-15T00:12:17.449 INFO:teuthology.orchestra.run.smithi171.stdout: "expected_num_objects": 0, 2024-09-15T00:12:17.449 INFO:teuthology.orchestra.run.smithi171.stdout: "fast_read": false, 2024-09-15T00:12:17.450 INFO:teuthology.orchestra.run.smithi171.stdout: "options": { 2024-09-15T00:12:17.450 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_max": 32, 2024-09-15T00:12:17.450 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_min": 1 2024-09-15T00:12:17.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.450 INFO:teuthology.orchestra.run.smithi171.stdout: "application_metadata": { 2024-09-15T00:12:17.450 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr": {} 2024-09-15T00:12:17.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.450 INFO:teuthology.orchestra.run.smithi171.stdout: "read_balance": { 2024-09-15T00:12:17.450 INFO:teuthology.orchestra.run.smithi171.stdout: "score_type": "Fair distribution", 2024-09-15T00:12:17.450 INFO:teuthology.orchestra.run.smithi171.stdout: "score_acting": 6.9000000953674316, 2024-09-15T00:12:17.450 INFO:teuthology.orchestra.run.smithi171.stdout: "score_stable": 6.9000000953674316, 2024-09-15T00:12:17.450 INFO:teuthology.orchestra.run.smithi171.stdout: "optimal_score": 0.28999999165534973, 2024-09-15T00:12:17.450 INFO:teuthology.orchestra.run.smithi171.stdout: "raw_score_acting": 2, 2024-09-15T00:12:17.451 INFO:teuthology.orchestra.run.smithi171.stdout: "raw_score_stable": 2, 2024-09-15T00:12:17.451 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity_weighted": 0.875, 2024-09-15T00:12:17.451 INFO:teuthology.orchestra.run.smithi171.stdout: "average_primary_affinity": 1, 2024-09-15T00:12:17.451 INFO:teuthology.orchestra.run.smithi171.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-15T00:12:17.451 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.451 INFO:teuthology.orchestra.run.smithi171.stdout: "pool": 2, 2024-09-15T00:12:17.451 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_name": "rbd", 2024-09-15T00:12:17.451 INFO:teuthology.orchestra.run.smithi171.stdout: "create_time": "2024-09-15T00:11:08.005967+0000", 2024-09-15T00:12:17.451 INFO:teuthology.orchestra.run.smithi171.stdout: "flags": 8193, 2024-09-15T00:12:17.451 INFO:teuthology.orchestra.run.smithi171.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:12:17.451 INFO:teuthology.orchestra.run.smithi171.stdout: "type": 1, 2024-09-15T00:12:17.452 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 2, 2024-09-15T00:12:17.452 INFO:teuthology.orchestra.run.smithi171.stdout: "min_size": 1, 2024-09-15T00:12:17.452 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_rule": 0, 2024-09-15T00:12:17.452 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:12:17.452 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:12:17.452 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:12:17.452 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:12:17.452 INFO:teuthology.orchestra.run.smithi171.stdout: "is_stretch_pool": false, 2024-09-15T00:12:17.452 INFO:teuthology.orchestra.run.smithi171.stdout: "object_hash": 2, 2024-09-15T00:12:17.452 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:12:17.452 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num": 8, 2024-09-15T00:12:17.452 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num": 8, 2024-09-15T00:12:17.452 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num_target": 8, 2024-09-15T00:12:17.453 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_target": 8, 2024-09-15T00:12:17.453 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_pending": 8, 2024-09-15T00:12:17.453 INFO:teuthology.orchestra.run.smithi171.stdout: "last_pg_merge_meta": { 2024-09-15T00:12:17.453 INFO:teuthology.orchestra.run.smithi171.stdout: "source_pgid": "0.0", 2024-09-15T00:12:17.453 INFO:teuthology.orchestra.run.smithi171.stdout: "ready_epoch": 0, 2024-09-15T00:12:17.453 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_started": 0, 2024-09-15T00:12:17.453 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_clean": 0, 2024-09-15T00:12:17.453 INFO:teuthology.orchestra.run.smithi171.stdout: "source_version": "0'0", 2024-09-15T00:12:17.453 INFO:teuthology.orchestra.run.smithi171.stdout: "target_version": "0'0" 2024-09-15T00:12:17.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.453 INFO:teuthology.orchestra.run.smithi171.stdout: "last_change": "21", 2024-09-15T00:12:17.453 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend": "0", 2024-09-15T00:12:17.453 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:12:17.453 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:12:17.454 INFO:teuthology.orchestra.run.smithi171.stdout: "auid": 0, 2024-09-15T00:12:17.454 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:12:17.454 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_seq": 2, 2024-09-15T00:12:17.454 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_epoch": 21, 2024-09-15T00:12:17.454 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_snaps": [], 2024-09-15T00:12:17.454 INFO:teuthology.orchestra.run.smithi171.stdout: "removed_snaps": "[]", 2024-09-15T00:12:17.454 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_bytes": 0, 2024-09-15T00:12:17.454 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_objects": 0, 2024-09-15T00:12:17.454 INFO:teuthology.orchestra.run.smithi171.stdout: "tiers": [], 2024-09-15T00:12:17.454 INFO:teuthology.orchestra.run.smithi171.stdout: "tier_of": -1, 2024-09-15T00:12:17.454 INFO:teuthology.orchestra.run.smithi171.stdout: "read_tier": -1, 2024-09-15T00:12:17.454 INFO:teuthology.orchestra.run.smithi171.stdout: "write_tier": -1, 2024-09-15T00:12:17.454 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_mode": "none", 2024-09-15T00:12:17.455 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_bytes": 0, 2024-09-15T00:12:17.455 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_objects": 0, 2024-09-15T00:12:17.455 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:12:17.455 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:12:17.455 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:12:17.455 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_flush_age": 0, 2024-09-15T00:12:17.455 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_evict_age": 0, 2024-09-15T00:12:17.455 INFO:teuthology.orchestra.run.smithi171.stdout: "erasure_code_profile": "", 2024-09-15T00:12:17.455 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_params": { 2024-09-15T00:12:17.455 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "none" 2024-09-15T00:12:17.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.455 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_period": 0, 2024-09-15T00:12:17.456 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_count": 0, 2024-09-15T00:12:17.456 INFO:teuthology.orchestra.run.smithi171.stdout: "use_gmt_hitset": true, 2024-09-15T00:12:17.456 INFO:teuthology.orchestra.run.smithi171.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:12:17.456 INFO:teuthology.orchestra.run.smithi171.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:12:17.456 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:12:17.456 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:12:17.456 INFO:teuthology.orchestra.run.smithi171.stdout: "grade_table": [], 2024-09-15T00:12:17.456 INFO:teuthology.orchestra.run.smithi171.stdout: "stripe_width": 0, 2024-09-15T00:12:17.456 INFO:teuthology.orchestra.run.smithi171.stdout: "expected_num_objects": 0, 2024-09-15T00:12:17.456 INFO:teuthology.orchestra.run.smithi171.stdout: "fast_read": false, 2024-09-15T00:12:17.456 INFO:teuthology.orchestra.run.smithi171.stdout: "options": {}, 2024-09-15T00:12:17.456 INFO:teuthology.orchestra.run.smithi171.stdout: "application_metadata": { 2024-09-15T00:12:17.456 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd": {} 2024-09-15T00:12:17.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.456 INFO:teuthology.orchestra.run.smithi171.stdout: "read_balance": { 2024-09-15T00:12:17.457 INFO:teuthology.orchestra.run.smithi171.stdout: "score_type": "Fair distribution", 2024-09-15T00:12:17.457 INFO:teuthology.orchestra.run.smithi171.stdout: "score_acting": 2.619999885559082, 2024-09-15T00:12:17.457 INFO:teuthology.orchestra.run.smithi171.stdout: "score_stable": 2.619999885559082, 2024-09-15T00:12:17.457 INFO:teuthology.orchestra.run.smithi171.stdout: "optimal_score": 0.86000001430511475, 2024-09-15T00:12:17.457 INFO:teuthology.orchestra.run.smithi171.stdout: "raw_score_acting": 2.25, 2024-09-15T00:12:17.457 INFO:teuthology.orchestra.run.smithi171.stdout: "raw_score_stable": 2.25, 2024-09-15T00:12:17.457 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity_weighted": 0.875, 2024-09-15T00:12:17.457 INFO:teuthology.orchestra.run.smithi171.stdout: "average_primary_affinity": 1, 2024-09-15T00:12:17.457 INFO:teuthology.orchestra.run.smithi171.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-15T00:12:17.457 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.457 INFO:teuthology.orchestra.run.smithi171.stdout: "pool": 3, 2024-09-15T00:12:17.458 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_name": "pool_client.0", 2024-09-15T00:12:17.458 INFO:teuthology.orchestra.run.smithi171.stdout: "create_time": "2024-09-15T00:11:17.676102+0000", 2024-09-15T00:12:17.458 INFO:teuthology.orchestra.run.smithi171.stdout: "flags": 8193, 2024-09-15T00:12:17.458 INFO:teuthology.orchestra.run.smithi171.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:12:17.458 INFO:teuthology.orchestra.run.smithi171.stdout: "type": 1, 2024-09-15T00:12:17.458 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 2, 2024-09-15T00:12:17.458 INFO:teuthology.orchestra.run.smithi171.stdout: "min_size": 1, 2024-09-15T00:12:17.458 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_rule": 0, 2024-09-15T00:12:17.458 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:12:17.458 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:12:17.458 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:12:17.458 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:12:17.458 INFO:teuthology.orchestra.run.smithi171.stdout: "is_stretch_pool": false, 2024-09-15T00:12:17.458 INFO:teuthology.orchestra.run.smithi171.stdout: "object_hash": 2, 2024-09-15T00:12:17.458 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:12:17.459 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num": 32, 2024-09-15T00:12:17.459 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num": 32, 2024-09-15T00:12:17.459 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num_target": 32, 2024-09-15T00:12:17.459 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_target": 32, 2024-09-15T00:12:17.459 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_pending": 32, 2024-09-15T00:12:17.459 INFO:teuthology.orchestra.run.smithi171.stdout: "last_pg_merge_meta": { 2024-09-15T00:12:17.459 INFO:teuthology.orchestra.run.smithi171.stdout: "source_pgid": "0.0", 2024-09-15T00:12:17.459 INFO:teuthology.orchestra.run.smithi171.stdout: "ready_epoch": 0, 2024-09-15T00:12:17.459 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_started": 0, 2024-09-15T00:12:17.459 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_clean": 0, 2024-09-15T00:12:17.459 INFO:teuthology.orchestra.run.smithi171.stdout: "source_version": "0'0", 2024-09-15T00:12:17.459 INFO:teuthology.orchestra.run.smithi171.stdout: "target_version": "0'0" 2024-09-15T00:12:17.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.460 INFO:teuthology.orchestra.run.smithi171.stdout: "last_change": "73", 2024-09-15T00:12:17.460 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend": "0", 2024-09-15T00:12:17.460 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:12:17.460 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:12:17.460 INFO:teuthology.orchestra.run.smithi171.stdout: "auid": 0, 2024-09-15T00:12:17.460 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:12:17.460 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_seq": 29, 2024-09-15T00:12:17.460 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_epoch": 73, 2024-09-15T00:12:17.460 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_snaps": [], 2024-09-15T00:12:17.460 INFO:teuthology.orchestra.run.smithi171.stdout: "removed_snaps": "[]", 2024-09-15T00:12:17.460 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_bytes": 0, 2024-09-15T00:12:17.460 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_objects": 0, 2024-09-15T00:12:17.460 INFO:teuthology.orchestra.run.smithi171.stdout: "tiers": [], 2024-09-15T00:12:17.461 INFO:teuthology.orchestra.run.smithi171.stdout: "tier_of": -1, 2024-09-15T00:12:17.461 INFO:teuthology.orchestra.run.smithi171.stdout: "read_tier": -1, 2024-09-15T00:12:17.461 INFO:teuthology.orchestra.run.smithi171.stdout: "write_tier": -1, 2024-09-15T00:12:17.461 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_mode": "none", 2024-09-15T00:12:17.461 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_bytes": 0, 2024-09-15T00:12:17.461 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_objects": 0, 2024-09-15T00:12:17.461 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:12:17.461 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:12:17.461 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:12:17.461 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_flush_age": 0, 2024-09-15T00:12:17.461 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_evict_age": 0, 2024-09-15T00:12:17.461 INFO:teuthology.orchestra.run.smithi171.stdout: "erasure_code_profile": "", 2024-09-15T00:12:17.461 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_params": { 2024-09-15T00:12:17.462 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "none" 2024-09-15T00:12:17.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.462 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_period": 0, 2024-09-15T00:12:17.462 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_count": 0, 2024-09-15T00:12:17.462 INFO:teuthology.orchestra.run.smithi171.stdout: "use_gmt_hitset": true, 2024-09-15T00:12:17.462 INFO:teuthology.orchestra.run.smithi171.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:12:17.462 INFO:teuthology.orchestra.run.smithi171.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:12:17.462 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:12:17.462 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:12:17.462 INFO:teuthology.orchestra.run.smithi171.stdout: "grade_table": [], 2024-09-15T00:12:17.462 INFO:teuthology.orchestra.run.smithi171.stdout: "stripe_width": 0, 2024-09-15T00:12:17.462 INFO:teuthology.orchestra.run.smithi171.stdout: "expected_num_objects": 0, 2024-09-15T00:12:17.462 INFO:teuthology.orchestra.run.smithi171.stdout: "fast_read": false, 2024-09-15T00:12:17.462 INFO:teuthology.orchestra.run.smithi171.stdout: "options": {}, 2024-09-15T00:12:17.463 INFO:teuthology.orchestra.run.smithi171.stdout: "application_metadata": { 2024-09-15T00:12:17.463 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd": {} 2024-09-15T00:12:17.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.463 INFO:teuthology.orchestra.run.smithi171.stdout: "read_balance": { 2024-09-15T00:12:17.463 INFO:teuthology.orchestra.run.smithi171.stdout: "score_type": "Fair distribution", 2024-09-15T00:12:17.463 INFO:teuthology.orchestra.run.smithi171.stdout: "score_acting": 2.190000057220459, 2024-09-15T00:12:17.463 INFO:teuthology.orchestra.run.smithi171.stdout: "score_stable": 2.190000057220459, 2024-09-15T00:12:17.463 INFO:teuthology.orchestra.run.smithi171.stdout: "optimal_score": 0.86000001430511475, 2024-09-15T00:12:17.463 INFO:teuthology.orchestra.run.smithi171.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-15T00:12:17.463 INFO:teuthology.orchestra.run.smithi171.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-15T00:12:17.463 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity_weighted": 0.875, 2024-09-15T00:12:17.463 INFO:teuthology.orchestra.run.smithi171.stdout: "average_primary_affinity": 1, 2024-09-15T00:12:17.463 INFO:teuthology.orchestra.run.smithi171.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-15T00:12:17.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.464 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.464 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:12:17.464 INFO:teuthology.orchestra.run.smithi171.stdout: "osds": [ 2024-09-15T00:12:17.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.464 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 0, 2024-09-15T00:12:17.464 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "7337111f-0e02-490d-976b-3814a03371f0", 2024-09-15T00:12:17.464 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:12:17.464 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:12:17.464 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:12:17.464 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 0, 2024-09-15T00:12:17.464 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:12:17.464 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:12:17.465 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:12:17.465 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 0, 2024-09-15T00:12:17.465 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:12:17.465 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:12:17.465 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:12:17.465 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.465 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.465 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6824", 2024-09-15T00:12:17.465 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:12:17.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.465 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.466 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6825", 2024-09-15T00:12:17.466 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:12:17.466 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.466 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.466 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:12:17.466 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.466 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.466 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6826", 2024-09-15T00:12:17.466 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:12:17.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.467 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.467 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6827", 2024-09-15T00:12:17.467 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:12:17.467 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.467 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.467 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:12:17.467 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.467 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.467 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6830", 2024-09-15T00:12:17.467 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:12:17.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.468 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.468 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6831", 2024-09-15T00:12:17.468 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:12:17.468 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.468 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.468 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:12:17.468 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.468 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.468 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6828", 2024-09-15T00:12:17.468 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:12:17.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.469 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.469 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6829", 2024-09-15T00:12:17.469 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:12:17.469 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.469 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.469 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.171:6825/2906515165", 2024-09-15T00:12:17.469 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.171:6827/2906515165", 2024-09-15T00:12:17.469 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.171:6831/2906515165", 2024-09-15T00:12:17.469 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.171:6829/2906515165", 2024-09-15T00:12:17.469 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:12:17.469 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:12:17.469 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:12:17.469 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.470 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 1, 2024-09-15T00:12:17.470 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "64412633-4dd4-4b8d-b3f1-ed3c664ddf48", 2024-09-15T00:12:17.470 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:12:17.470 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:12:17.470 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:12:17.470 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:12:17.470 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:12:17.470 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:12:17.470 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:12:17.470 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 67, 2024-09-15T00:12:17.470 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:12:17.471 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:12:17.471 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:12:17.471 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.471 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.471 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6816", 2024-09-15T00:12:17.471 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:12:17.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.471 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.471 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6817", 2024-09-15T00:12:17.471 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:12:17.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.471 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:12:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6818", 2024-09-15T00:12:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:12:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6819", 2024-09-15T00:12:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:12:17.472 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:12:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6822", 2024-09-15T00:12:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:12:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6823", 2024-09-15T00:12:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:12:17.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:12:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6820", 2024-09-15T00:12:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:12:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6821", 2024-09-15T00:12:17.474 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:12:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.171:6817/3039831960", 2024-09-15T00:12:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.171:6819/3039831960", 2024-09-15T00:12:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.171:6823/3039831960", 2024-09-15T00:12:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.171:6821/3039831960", 2024-09-15T00:12:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:12:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:12:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:12:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.475 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 2, 2024-09-15T00:12:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "0300adee-5ce3-43ce-997b-2c0e970edbc7", 2024-09-15T00:12:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:12:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 0, 2024-09-15T00:12:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 0, 2024-09-15T00:12:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:12:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:12:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:12:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:12:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 18, 2024-09-15T00:12:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:12:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:12:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:12:17.476 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6808", 2024-09-15T00:12:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:12:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6809", 2024-09-15T00:12:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:12:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:12:17.477 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6810", 2024-09-15T00:12:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:12:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6811", 2024-09-15T00:12:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:12:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.478 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:12:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6814", 2024-09-15T00:12:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:12:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6815", 2024-09-15T00:12:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:12:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6812", 2024-09-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6813", 2024-09-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3950855930 2024-09-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.171:6809/3950855930", 2024-09-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.171:6811/3950855930", 2024-09-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.171:6815/3950855930", 2024-09-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.171:6813/3950855930", 2024-09-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 3, 2024-09-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "ec668801-9b8f-401a-829c-0b1f8d8f08fb", 2024-09-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 69, 2024-09-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6800", 2024-09-15T00:12:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:12:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6801", 2024-09-15T00:12:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:12:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:12:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6802", 2024-09-15T00:12:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:12:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6803", 2024-09-15T00:12:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:12:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:12:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6806", 2024-09-15T00:12:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:12:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6807", 2024-09-15T00:12:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:12:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:12:17.485 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6804", 2024-09-15T00:12:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:12:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6805", 2024-09-15T00:12:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:12:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.486 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.171:6801/1819098683", 2024-09-15T00:12:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.171:6803/1819098683", 2024-09-15T00:12:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.171:6807/1819098683", 2024-09-15T00:12:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.171:6805/1819098683", 2024-09-15T00:12:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:12:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:12:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:12:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 4, 2024-09-15T00:12:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "9cc48936-5644-44c6-84bb-06510e933e8d", 2024-09-15T00:12:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:12:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:12:17.487 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:12:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:12:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:12:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:12:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:12:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 69, 2024-09-15T00:12:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:12:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:12:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:12:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6800", 2024-09-15T00:12:17.488 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:12:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6801", 2024-09-15T00:12:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:12:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:12:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.489 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6802", 2024-09-15T00:12:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:12:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6803", 2024-09-15T00:12:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:12:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:12:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.490 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6806", 2024-09-15T00:12:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:12:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6807", 2024-09-15T00:12:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:12:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:12:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.491 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6804", 2024-09-15T00:12:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:12:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6805", 2024-09-15T00:12:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:12:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.184:6801/2801626173", 2024-09-15T00:12:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.184:6803/2801626173", 2024-09-15T00:12:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.184:6807/2801626173", 2024-09-15T00:12:17.492 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.184:6805/2801626173", 2024-09-15T00:12:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:12:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:12:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:12:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 5, 2024-09-15T00:12:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "f6c53297-9b00-476f-9be8-e76ff762b044", 2024-09-15T00:12:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:12:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:12:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:12:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:12:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:12:17.493 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:12:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:12:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 68, 2024-09-15T00:12:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:12:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:12:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:12:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6808", 2024-09-15T00:12:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:12:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.494 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6809", 2024-09-15T00:12:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:12:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:12:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6810", 2024-09-15T00:12:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:12:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.495 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6811", 2024-09-15T00:12:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:12:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:12:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6814", 2024-09-15T00:12:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:12:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6815", 2024-09-15T00:12:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:12:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:12:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6812", 2024-09-15T00:12:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:12:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6813", 2024-09-15T00:12:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:12:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.184:6809/3271066302", 2024-09-15T00:12:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.184:6811/3271066302", 2024-09-15T00:12:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.184:6815/3271066302", 2024-09-15T00:12:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.184:6813/3271066302", 2024-09-15T00:12:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:12:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:12:17.498 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:12:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 6, 2024-09-15T00:12:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "737f08a6-2223-46b3-b9fe-6ae18b067ec3", 2024-09-15T00:12:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:12:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:12:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:12:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:12:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:12:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:12:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:12:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 68, 2024-09-15T00:12:17.499 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:12:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:12:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:12:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6816", 2024-09-15T00:12:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:12:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6817", 2024-09-15T00:12:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:12:17.500 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:12:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6818", 2024-09-15T00:12:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:12:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6819", 2024-09-15T00:12:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:12:17.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:12:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6822", 2024-09-15T00:12:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:12:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6823", 2024-09-15T00:12:17.502 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:12:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:12:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6820", 2024-09-15T00:12:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:12:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6821", 2024-09-15T00:12:17.503 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:12:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.184:6817/1903994256", 2024-09-15T00:12:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.184:6819/1903994256", 2024-09-15T00:12:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.184:6823/1903994256", 2024-09-15T00:12:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.184:6821/1903994256", 2024-09-15T00:12:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:12:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:12:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:12:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 7, 2024-09-15T00:12:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "5a5fba64-9249-426d-814b-fd251e8aa910", 2024-09-15T00:12:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:12:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 0, 2024-09-15T00:12:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 0, 2024-09-15T00:12:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:12:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:12:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:12:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:12:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 31, 2024-09-15T00:12:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:12:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:12:17.505 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:12:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6824", 2024-09-15T00:12:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:12:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6825", 2024-09-15T00:12:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:12:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.506 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:12:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6826", 2024-09-15T00:12:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:12:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6827", 2024-09-15T00:12:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:12:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:12:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6830", 2024-09-15T00:12:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:12:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6831", 2024-09-15T00:12:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:12:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.508 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:12:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:12:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:12:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6828", 2024-09-15T00:12:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:12:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:12:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6829", 2024-09-15T00:12:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:12:17.509 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.184:6825/978565004", 2024-09-15T00:12:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.184:6827/978565004", 2024-09-15T00:12:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.184:6831/978565004", 2024-09-15T00:12:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.184:6829/978565004", 2024-09-15T00:12:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:12:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:12:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:12:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:17.510 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:12:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_xinfo": [ 2024-09-15T00:12:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 0, 2024-09-15T00:12:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:12:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:12:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:12:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:12:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:12:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:59.070337+0000", 2024-09-15T00:12:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:12:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.511 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 1, 2024-09-15T00:12:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:12:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:12:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:12:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:12:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:12:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.884683+0000", 2024-09-15T00:12:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:12:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 2, 2024-09-15T00:12:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:12:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:12:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:12:17.512 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:12:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 65536, 2024-09-15T00:12:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.998882+0000", 2024-09-15T00:12:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:12:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 3, 2024-09-15T00:12:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:12:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:12:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:12:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:12:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:12:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.656630+0000", 2024-09-15T00:12:17.513 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:12:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 4, 2024-09-15T00:12:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:12:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:12:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:12:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:12:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:12:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.802744+0000", 2024-09-15T00:12:17.514 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:12:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 5, 2024-09-15T00:12:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:12:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:12:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:12:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:12:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:12:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.567865+0000", 2024-09-15T00:12:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:12:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 6, 2024-09-15T00:12:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:12:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:12:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:12:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:12:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:12:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.739964+0000", 2024-09-15T00:12:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:12:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 7, 2024-09-15T00:12:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:12:17.516 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:12:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:12:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:12:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 65536, 2024-09-15T00:12:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:59.565310+0000", 2024-09-15T00:12:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:12:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:12:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_upmap": [], 2024-09-15T00:12:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_upmap_items": [], 2024-09-15T00:12:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_upmap_primaries": [], 2024-09-15T00:12:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_temp": [], 2024-09-15T00:12:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_temp": [], 2024-09-15T00:12:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "blocklist": {}, 2024-09-15T00:12:17.517 INFO:teuthology.orchestra.run.smithi171.stdout: "range_blocklist": {}, 2024-09-15T00:12:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "erasure_code_profiles": { 2024-09-15T00:12:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "default": { 2024-09-15T00:12:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "crush-failure-domain": "osd", 2024-09-15T00:12:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "k": "2", 2024-09-15T00:12:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "m": "1", 2024-09-15T00:12:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "plugin": "jerasure", 2024-09-15T00:12:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "technique": "reed_sol_van" 2024-09-15T00:12:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "removed_snaps_queue": [], 2024-09-15T00:12:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "new_removed_snaps": [], 2024-09-15T00:12:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "new_purged_snaps": [], 2024-09-15T00:12:17.518 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_node_flags": {}, 2024-09-15T00:12:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: "device_class_flags": {}, 2024-09-15T00:12:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: "stretch_mode": { 2024-09-15T00:12:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: "stretch_mode_enabled": false, 2024-09-15T00:12:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: "stretch_bucket_count": 0, 2024-09-15T00:12:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: "degraded_stretch_mode": 0, 2024-09-15T00:12:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: "recovering_stretch_mode": 0, 2024-09-15T00:12:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: "stretch_mode_bucket": 0 2024-09-15T00:12:17.519 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:17.519 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:17.521 INFO:teuthology.orchestra.run.smithi171.stdout:3716 write 0xe20e5fe thru 0xe21c4ca (0xdecd bytes) 2024-09-15T00:12:17.521 INFO:teuthology.orchestra.run.smithi171.stdout:3717 write 0xc35d18a thru 0xc36b5c4 (0xe43b bytes) 2024-09-15T00:12:17.521 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:17.521 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:17.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:17.522 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:17.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.522 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:17.522 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:17.522 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 51.439404547000002, 2024-09-15T00:12:17.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:12:17.522 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.522 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.522 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:17.522 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.522 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.522 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.522 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:12:17.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.523 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:17.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:17.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:12:17.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:12:17.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:12:17.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:12:17.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:12:17.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:12:17.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:12:17.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:12:17.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:12:17.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:12:17.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:12:17.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:12:17.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:17.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:12:17.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:12:17.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:17.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:12:17.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:12:17.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:17.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:12:17.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:12:17.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:17.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:12:17.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:12:17.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:12:17.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:12:17.528 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.528 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.528 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.528 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:12:17.528 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:17.528 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 51.426664004999999, 2024-09-15T00:12:17.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:12:17.528 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.528 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.529 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.529 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:17.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:17.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:12:17.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:12:17.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:12:17.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:12:17.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:12:17.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:12:17.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:12:17.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:12:17.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:12:17.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:12:17.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:12:17.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:12:17.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:12:17.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:12:17.532 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.532 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.532 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.532 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:17.533 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:17.533 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 51.294308246, 2024-09-15T00:12:17.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:12:17.533 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.533 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.533 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:17.533 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.533 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.533 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.533 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:12:17.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.533 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:17.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:17.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:12:17.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:12:17.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:12:17.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:12:17.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:12:17.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:12:17.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:12:17.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:12:17.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:12:17.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:12:17.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:12:17.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:12:17.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:17.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:12:17.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:12:17.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:17.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:12:17.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:12:17.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:17.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:12:17.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:12:17.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:17.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:12:17.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:12:17.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:12:17.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:12:17.538 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.538 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.538 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.539 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:12:17.539 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:17.539 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 51.280846601, 2024-09-15T00:12:17.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:12:17.539 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.539 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.539 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.539 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:17.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:17.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:12:17.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:12:17.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:12:17.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:12:17.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:12:17.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:12:17.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:12:17.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:12:17.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:12:17.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:12:17.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:12:17.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:12:17.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:12:17.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:12:17.542 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.543 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.543 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.543 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:17.543 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:17.543 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.095845118, 2024-09-15T00:12:17.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:12:17.543 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.543 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.543 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:17.543 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.543 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.544 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.544 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:12:17.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.544 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:17.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:17.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:12:17.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:12:17.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:12:17.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:12:17.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:12:17.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:12:17.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:12:17.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:12:17.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:12:17.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:12:17.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:12:17.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:12:17.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:17.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:12:17.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:12:17.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:17.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:12:17.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:12:17.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:17.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:12:17.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:12:17.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:17.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:12:17.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:12:17.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:12:17.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:12:17.549 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.549 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.549 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.549 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:12:17.549 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:17.549 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.082968850999997, 2024-09-15T00:12:17.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:12:17.549 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.549 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.550 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.550 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:17.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:17.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:12:17.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:12:17.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:12:17.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:12:17.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:12:17.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:12:17.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:12:17.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:12:17.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:12:17.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:12:17.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:12:17.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:12:17.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:12:17.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:12:17.553 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.553 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.553 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.553 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:12:17.554 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:17.554 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.624453707000001, 2024-09-15T00:12:17.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:12:17.554 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.554 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.554 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:17.554 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.554 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.554 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.554 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:12:17.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.554 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:17.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:17.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:12:17.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:17.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:12:17.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:12:17.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:12:17.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:12:17.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:12:17.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:12:17.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:12:17.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:12:17.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:12:17.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:12:17.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:17.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:12:17.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:12:17.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:17.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:12:17.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:12:17.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:17.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:12:17.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:12:17.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:17.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:12:17.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:12:17.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:12:17.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:12:17.559 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.559 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.559 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.559 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:17.559 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:17.560 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.906943147, 2024-09-15T00:12:17.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:12:17.560 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.560 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.560 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.560 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.560 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.560 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.560 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:12:17.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.560 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:17.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:17.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:12:17.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:12:17.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:12:17.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:12:17.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:12:17.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:12:17.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:12:17.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:12:17.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:12:17.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:12:17.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:12:17.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:12:17.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:12:17.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:12:17.564 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.564 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.564 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.564 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:17.564 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:17.564 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.906924537999998, 2024-09-15T00:12:17.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:12:17.564 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.564 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.565 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.565 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.565 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.565 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.565 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:12:17.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.565 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:17.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:17.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:17.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:17.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:12:17.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:12:17.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:12:17.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:12:17.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:12:17.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:12:17.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:12:17.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:12:17.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:12:17.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:12:17.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:12:17.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:12:17.568 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.568 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.569 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.569 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:17.569 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:17.569 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.906899213999999, 2024-09-15T00:12:17.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:12:17.569 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.569 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.569 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.569 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.569 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.569 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.569 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:12:17.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.570 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:17.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:17.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:17.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:12:17.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:12:17.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:12:17.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:12:17.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:12:17.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:12:17.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:12:17.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:12:17.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:12:17.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:12:17.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:12:17.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:12:17.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:12:17.573 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.573 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.573 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.573 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:17.573 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:17.573 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.906880889999996, 2024-09-15T00:12:17.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:12:17.574 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.574 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.574 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.574 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.574 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.574 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.574 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:12:17.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.574 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:17.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:17.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:12:17.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:12:17.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:12:17.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:12:17.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:12:17.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:17.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:12:17.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:12:17.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:12:17.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:12:17.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:12:17.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:12:17.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:12:17.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:12:17.578 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.578 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.578 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.580 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:17.580 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:17.580 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.906698317, 2024-09-15T00:12:17.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:12:17.580 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.580 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.580 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.580 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.580 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.580 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.580 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:12:17.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.581 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:17.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:17.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:17.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:12:17.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:12:17.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:12:17.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:12:17.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:12:17.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:12:17.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:17.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:12:17.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:12:17.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:12:17.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:12:17.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:12:17.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:12:17.584 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.584 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.584 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.584 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:17.584 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:17.585 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.906669475999998, 2024-09-15T00:12:17.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:12:17.585 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.585 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.585 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.585 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.585 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.585 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.585 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:12:17.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.585 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:17.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:17.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:17.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:12:17.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:12:17.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:12:17.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:12:17.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:12:17.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:12:17.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:12:17.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:12:17.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:12:17.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:12:17.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:12:17.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:12:17.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:12:17.589 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.589 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.589 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.589 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:17.589 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:17.589 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.906647264999997, 2024-09-15T00:12:17.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:12:17.590 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.590 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.590 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.590 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.590 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.590 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.590 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:12:17.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.590 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:17.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:17.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:17.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:12:17.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:12:17.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:12:17.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:12:17.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:12:17.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:12:17.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:17.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:12:17.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:12:17.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:12:17.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:12:17.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:12:17.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:12:17.593 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.594 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.594 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.594 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:17.594 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:17.594 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.906626105999997, 2024-09-15T00:12:17.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:12:17.594 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.594 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.594 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.594 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.594 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.594 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.595 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:12:17.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.595 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:17.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:17.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:17.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:17.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:12:17.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:12:17.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:12:17.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:12:17.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:12:17.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:12:17.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:12:17.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:12:17.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:12:17.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:12:17.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:12:17.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:12:17.598 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.598 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.598 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.598 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:17.599 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:17.599 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.906596157999999, 2024-09-15T00:12:17.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:12:17.599 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.599 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.599 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.599 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.599 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.599 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.599 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:12:17.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.599 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:17.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:17.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:17.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:12:17.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:12:17.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:12:17.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:12:17.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:12:17.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:12:17.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:12:17.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:12:17.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:12:17.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:12:17.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:12:17.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:12:17.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:12:17.603 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.603 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.603 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.603 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:17.603 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:17.603 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.906574194000001, 2024-09-15T00:12:17.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:12:17.603 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.603 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.604 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.604 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.604 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.604 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.604 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:12:17.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.604 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:17.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:17.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:17.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:12:17.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:12:17.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:12:17.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:12:17.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:12:17.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:12:17.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:12:17.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:12:17.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:12:17.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:12:17.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:12:17.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:12:17.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:12:17.607 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.607 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.608 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.608 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:17.608 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:17.608 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.906556766999998, 2024-09-15T00:12:17.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:12:17.608 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.608 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.608 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.609 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:12:17.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.609 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:17.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:17.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:12:17.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:12:17.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:12:17.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:12:17.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:12:17.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:17.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:12:17.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:12:17.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:12:17.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:12:17.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:12:17.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:12:17.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:12:17.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:12:17.612 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.612 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.612 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.613 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:17.613 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:17.613 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.906534968000003, 2024-09-15T00:12:17.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:12:17.613 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.613 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.613 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:17.613 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.613 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.613 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.613 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:12:17.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.613 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:17.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:17.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:12:17.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:12:17.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:12:17.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:12:17.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:12:17.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:12:17.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:12:17.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:12:17.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:12:17.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:12:17.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:12:17.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:12:17.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:12:17.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:12:17.617 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.617 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.617 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.617 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:17.617 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:17.617 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.857306608999998, 2024-09-15T00:12:17.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:12:17.618 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:17.618 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:17.618 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:17.618 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:17.618 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:17.618 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:17.618 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:12:17.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.618 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:17.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:17.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:17.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:17.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:17.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:17.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:17.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:12:17.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:12:17.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:17.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:12:17.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:12:17.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:17.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:12:17.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:12:17.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:17.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:12:17.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:12:17.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:17.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:12:17.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:12:17.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:17.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:12:17.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:12:17.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:17.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:12:17.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:12:17.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:17.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:12:17.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:12:17.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:17.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:12:17.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:12:17.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:17.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:12:17.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:12:17.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:17.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:17.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:17.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:12:17.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:12:17.623 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.623 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.623 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.623 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:17.623 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:17.623 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:17.625 INFO:tasks.thrashosds.thrasher:j is {'epoch': 73, 'fsid': 'bad2533c-9c09-4af4-ab93-147afbacb705', 'created': '2024-09-15T00:10:50.203030+0000', 'modified': '2024-09-15T00:12:17.026096+0000', 'last_up_change': '2024-09-15T00:11:03.441129+0000', 'last_in_change': '2024-09-15T00:12:06.204759+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T00:11:04.326356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T00:11:08.005967+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.619999885559082, 'score_stable': 2.619999885559082, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-15T00:11:17.676102+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 29, 'snap_epoch': 73, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.190000057220459, 'score_stable': 2.190000057220459, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '7337111f-0e02-490d-976b-3814a03371f0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6824', 'nonce': 2906515165}, {'type': 'v1', 'addr': '172.21.15.171:6825', 'nonce': 2906515165}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6826', 'nonce': 2906515165}, {'type': 'v1', 'addr': '172.21.15.171:6827', 'nonce': 2906515165}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6830', 'nonce': 2906515165}, {'type': 'v1', 'addr': '172.21.15.171:6831', 'nonce': 2906515165}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6828', 'nonce': 2906515165}, {'type': 'v1', 'addr': '172.21.15.171:6829', 'nonce': 2906515165}]}, 'public_addr': '172.21.15.171:6825/2906515165', 'cluster_addr': '172.21.15.171:6827/2906515165', 'heartbeat_back_addr': '172.21.15.171:6831/2906515165', 'heartbeat_front_addr': '172.21.15.171:6829/2906515165', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '64412633-4dd4-4b8d-b3f1-ed3c664ddf48', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6816', 'nonce': 3039831960}, {'type': 'v1', 'addr': '172.21.15.171:6817', 'nonce': 3039831960}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6818', 'nonce': 3039831960}, {'type': 'v1', 'addr': '172.21.15.171:6819', 'nonce': 3039831960}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6822', 'nonce': 3039831960}, {'type': 'v1', 'addr': '172.21.15.171:6823', 'nonce': 3039831960}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6820', 'nonce': 3039831960}, {'type': 'v1', 'addr': '172.21.15.171:6821', 'nonce': 3039831960}]}, 'public_addr': '172.21.15.171:6817/3039831960', 'cluster_addr': '172.21.15.171:6819/3039831960', 'heartbeat_back_addr': '172.21.15.171:6823/3039831960', 'heartbeat_front_addr': '172.21.15.171:6821/3039831960', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0300adee-5ce3-43ce-997b-2c0e970edbc7', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6808', 'nonce': 3950855930}, {'type': 'v1', 'addr': '172.21.15.171:6809', 'nonce': 3950855930}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6810', 'nonce': 3950855930}, {'type': 'v1', 'addr': '172.21.15.171:6811', 'nonce': 3950855930}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6814', 'nonce': 3950855930}, {'type': 'v1', 'addr': '172.21.15.171:6815', 'nonce': 3950855930}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6812', 'nonce': 3950855930}, {'type': 'v1', 'addr': '172.21.15.171:6813', 'nonce': 3950855930}]}, 'public_addr': '172.21.15.171:6809/3950855930', 'cluster_addr': '172.21.15.171:6811/3950855930', 'heartbeat_back_addr': '172.21.15.171:6815/3950855930', 'heartbeat_front_addr': '172.21.15.171:6813/3950855930', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ec668801-9b8f-401a-829c-0b1f8d8f08fb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6800', 'nonce': 1819098683}, {'type': 'v1', 'addr': '172.21.15.171:6801', 'nonce': 1819098683}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6802', 'nonce': 1819098683}, {'type': 'v1', 'addr': '172.21.15.171:6803', 'nonce': 1819098683}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6806', 'nonce': 1819098683}, {'type': 'v1', 'addr': '172.21.15.171:6807', 'nonce': 1819098683}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6804', 'nonce': 1819098683}, {'type': 'v1', 'addr': '172.21.15.171:6805', 'nonce': 1819098683}]}, 'public_addr': '172.21.15.171:6801/1819098683', 'cluster_addr': '172.21.15.171:6803/1819098683', 'heartbeat_back_addr': '172.21.15.171:6807/1819098683', 'heartbeat_front_addr': '172.21.15.171:6805/1819098683', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '9cc48936-5644-44c6-84bb-06510e933e8d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6800', 'nonce': 2801626173}, {'type': 'v1', 'addr': '172.21.15.184:6801', 'nonce': 2801626173}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6802', 'nonce': 2801626173}, {'type': 'v1', 'addr': '172.21.15.184:6803', 'nonce': 2801626173}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6806', 'nonce': 2801626173}, {'type': 'v1', 'addr': '172.21.15.184:6807', 'nonce': 2801626173}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6804', 'nonce': 2801626173}, {'type': 'v1', 'addr': '172.21.15.184:6805', 'nonce': 2801626173}]}, 'public_addr': '172.21.15.184:6801/2801626173', 'cluster_addr': '172.21.15.184:6803/2801626173', 'heartbeat_back_addr': '172.21.15.184:6807/2801626173', 'heartbeat_front_addr': '172.21.15.184:6805/2801626173', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f6c53297-9b00-476f-9be8-e76ff762b044', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 68, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6808', 'nonce': 3271066302}, {'type': 'v1', 'addr': '172.21.15.184:6809', 'nonce': 3271066302}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6810', 'nonce': 3271066302}, {'type': 'v1', 'addr': '172.21.15.184:6811', 'nonce': 3271066302}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6814', 'nonce': 3271066302}, {'type': 'v1', 'addr': '172.21.15.184:6815', 'nonce': 3271066302}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6812', 'nonce': 3271066302}, {'type': 'v1', 'addr': '172.21.15.184:6813', 'nonce': 3271066302}]}, 'public_addr': '172.21.15.184:6809/3271066302', 'cluster_addr': '172.21.15.184:6811/3271066302', 'heartbeat_back_addr': '172.21.15.184:6815/3271066302', 'heartbeat_front_addr': '172.21.15.184:6813/3271066302', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '737f08a6-2223-46b3-b9fe-6ae18b067ec3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 68, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6816', 'nonce': 1903994256}, {'type': 'v1', 'addr': '172.21.15.184:6817', 'nonce': 1903994256}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6818', 'nonce': 1903994256}, {'type': 'v1', 'addr': '172.21.15.184:6819', 'nonce': 1903994256}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6822', 'nonce': 1903994256}, {'type': 'v1', 'addr': '172.21.15.184:6823', 'nonce': 1903994256}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6820', 'nonce': 1903994256}, {'type': 'v1', 'addr': '172.21.15.184:6821', 'nonce': 1903994256}]}, 'public_addr': '172.21.15.184:6817/1903994256', 'cluster_addr': '172.21.15.184:6819/1903994256', 'heartbeat_back_addr': '172.21.15.184:6823/1903994256', 'heartbeat_front_addr': '172.21.15.184:6821/1903994256', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '5a5fba64-9249-426d-814b-fd251e8aa910', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6824', 'nonce': 978565004}, {'type': 'v1', 'addr': '172.21.15.184:6825', 'nonce': 978565004}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6826', 'nonce': 978565004}, {'type': 'v1', 'addr': '172.21.15.184:6827', 'nonce': 978565004}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6830', 'nonce': 978565004}, {'type': 'v1', 'addr': '172.21.15.184:6831', 'nonce': 978565004}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6828', 'nonce': 978565004}, {'type': 'v1', 'addr': '172.21.15.184:6829', 'nonce': 978565004}]}, 'public_addr': '172.21.15.184:6825/978565004', 'cluster_addr': '172.21.15.184:6827/978565004', 'heartbeat_back_addr': '172.21.15.184:6831/978565004', 'heartbeat_front_addr': '172.21.15.184:6829/978565004', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:59.070337+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.884683+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.998882+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.656630+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.802744+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.567865+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.739964+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-15T00:10:59.565310+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T00:12:17.625 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:12:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:3718 read 0xd3dbf84 thru 0xd3eb7f6 (0xf873 bytes) 2024-09-15T00:12:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:3721 write 0x7bed3dc thru 0x7bfba9f (0xe6c4 bytes) 2024-09-15T00:12:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:3723 read 0x479e2c1 thru 0x47a9ae5 (0xb825 bytes) 2024-09-15T00:12:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:3725 trunc from 0xe21c4cb to 0x6215df5 2024-09-15T00:12:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:3726 read 0xa32934 thru 0xa32a64 (0x131 bytes) 2024-09-15T00:12:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:3728 write 0x2e863e5 thru 0x2e89555 (0x3171 bytes) 2024-09-15T00:12:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:3730 read 0x4f0a0ad thru 0x4f194c7 (0xf41b bytes) 2024-09-15T00:12:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:3731 read 0x24beb3a thru 0x24c1296 (0x275d bytes) 2024-09-15T00:12:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:3735 read 0x31627bf thru 0x31634ea (0xd2c bytes) 2024-09-15T00:12:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:3738 trunc from 0x6215df5 to 0xbb940fa 2024-09-15T00:12:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:3739 write 0x2a79519 thru 0x2a7e661 (0x5149 bytes) 2024-09-15T00:12:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:3740 write 0x73af3b9 thru 0x73b6d4d (0x7995 bytes) 2024-09-15T00:12:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:3741 punch from 0x50bd5e9 to 0x50c68c5, (0x92dc bytes) 2024-09-15T00:12:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:3742 writesame 0x5c611c5 thru 0x5c6c0b0 data_size 0xa(0xaeec bytes) 2024-09-15T00:12:17.629 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:17.518+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:17.629 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:17.624+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:17.630 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:3743 read 0x3b3d0de thru 0x3b470d5 (0x9ff8 bytes) 2024-09-15T00:12:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:3744 punch from 0x58ecac7 to 0x58f93e1, (0xc91a bytes) 2024-09-15T00:12:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:3746 read 0x1ee573c thru 0x1ef1166 (0xba2b bytes) 2024-09-15T00:12:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:3747 write 0x9491101 thru 0x9492a91 (0x1991 bytes) 2024-09-15T00:12:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:3751 punch from 0x1e42d8d to 0x1e4d415, (0xa688 bytes) 2024-09-15T00:12:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:3753 trunc from 0xbb940fa to 0xc9294af 2024-09-15T00:12:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:3754 write 0x2e7c883 thru 0x2e86709 (0x9e87 bytes) 2024-09-15T00:12:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:3756 read 0x2a606a8 thru 0x2a64ef8 (0x4851 bytes) 2024-09-15T00:12:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:3757 write 0x5f34599 thru 0x5f41fce (0xda36 bytes) 2024-09-15T00:12:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:3759 trunc from 0xc9294af to 0xe397553 2024-09-15T00:12:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:3764 read 0x2e18c8e thru 0x2e1b4f0 (0x2863 bytes) 2024-09-15T00:12:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:3765 trunc from 0xe397553 to 0x7b2c401 2024-09-15T00:12:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:3766 read 0x75045bf thru 0x750bb47 (0x7589 bytes) 2024-09-15T00:12:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:3767 trunc from 0x7b2c401 to 0x95095e3 2024-09-15T00:12:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:3770 write 0x2cbc94f thru 0x2cbf666 (0x2d18 bytes) 2024-09-15T00:12:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:3773 read 0x420e6ac thru 0x421b4e4 (0xce39 bytes) 2024-09-15T00:12:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:3775 read 0x3680291 thru 0x36842bb (0x402b bytes) 2024-09-15T00:12:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:3778 punch from 0x1e39980 to 0x1e45876, (0xbef6 bytes) 2024-09-15T00:12:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:3779 write 0x1786949 thru 0x17871cb (0x883 bytes) 2024-09-15T00:12:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:3780 writesame 0x443499c thru 0x4436739 data_size 0x11(0x1d9e bytes) 2024-09-15T00:12:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:3782 read 0x550e254 thru 0x551bf3a (0xdce7 bytes) 2024-09-15T00:12:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:3783 read 0x494cc6f thru 0x4959549 (0xc8db bytes) 2024-09-15T00:12:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:3784 punch from 0x1b8a445 to 0x1b945f8, (0xa1b3 bytes) 2024-09-15T00:12:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:3786 trunc from 0x95095e3 to 0x7edc6e2 2024-09-15T00:12:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:3787 write 0x8345508 thru 0x834d203 (0x7cfc bytes) 2024-09-15T00:12:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:3789 trunc from 0x834d204 to 0x408f92d 2024-09-15T00:12:17.650 INFO:teuthology.orchestra.run.smithi171.stdout:3791 write 0x7326528 thru 0x732947b (0x2f54 bytes) 2024-09-15T00:12:17.667 INFO:teuthology.orchestra.run.smithi171.stdout:3792 write 0xdcf0056 thru 0xdcf07ba (0x765 bytes) 2024-09-15T00:12:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:3796 punch from 0x46d40ed to 0x46e24d6, (0xe3e9 bytes) 2024-09-15T00:12:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:3798 writesame 0x368754b thru 0x368c661 data_size 0x1(0x5117 bytes) 2024-09-15T00:12:17.707 INFO:teuthology.orchestra.run.smithi171.stdout:3799 read 0x4c66fa8 thru 0x4c73184 (0xc1dd bytes) 2024-09-15T00:12:17.707 INFO:teuthology.orchestra.run.smithi171.stdout:3800 read 0x396b437 thru 0x3970997 (0x5561 bytes) 2024-09-15T00:12:17.707 INFO:teuthology.orchestra.run.smithi171.stdout:3802 write 0x8122924 thru 0x81261bf (0x389c bytes) 2024-09-15T00:12:17.711 INFO:teuthology.orchestra.run.smithi171.stdout:3803 write 0xb3bd34a thru 0xb3c3607 (0x62be bytes) 2024-09-15T00:12:17.719 INFO:teuthology.orchestra.run.smithi171.stdout:3804 writesame 0x7e5c291 thru 0x7e5fcef data_size 0x11(0x3a5f bytes) 2024-09-15T00:12:17.723 INFO:teuthology.orchestra.run.smithi171.stdout:3805 write 0x1651311 thru 0x165bdc3 (0xaab3 bytes) 2024-09-15T00:12:17.728 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:17.725+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:17.729 INFO:teuthology.orchestra.run.smithi171.stdout:3807 writesame 0x39d2f37 thru 0x39d8cf3 data_size 0x13(0x5dbd bytes) 2024-09-15T00:12:17.733 INFO:teuthology.orchestra.run.smithi171.stdout:3808 punch from 0x23b65bb to 0x23c5275, (0xecba bytes) 2024-09-15T00:12:17.734 INFO:teuthology.orchestra.run.smithi171.stdout:3809 write 0x73276b3 thru 0x733218f (0xaadd bytes) 2024-09-15T00:12:17.737 INFO:teuthology.orchestra.run.smithi171.stdout:3810 write 0x13d5508 thru 0x13dbf36 (0x6a2f bytes) 2024-09-15T00:12:17.741 INFO:teuthology.orchestra.run.smithi171.stdout:3811 punch from 0x23e8bfd to 0x23eda25, (0x4e28 bytes) 2024-09-15T00:12:17.741 INFO:teuthology.orchestra.run.smithi171.stdout:3812 write 0xd6ff777 thru 0xd706d63 (0x75ed bytes) 2024-09-15T00:12:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:3814 read 0xb8a5274 thru 0xb8a8eb1 (0x3c3e bytes) 2024-09-15T00:12:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:3817 write 0x5580980 thru 0x55874a3 (0x6b24 bytes) 2024-09-15T00:12:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:17.764 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:17.764 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:17.765 INFO:teuthology.orchestra.run.smithi171.stdout:3818 trunc from 0xdcf07bb to 0x5269972 2024-09-15T00:12:17.775 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:3819 read 0xfa8468 thru 0xfa8f11 (0xaaa bytes) 2024-09-15T00:12:17.791 INFO:teuthology.orchestra.run.smithi171.stdout:3825 write 0xc028e21 thru 0xc034382 (0xb562 bytes) 2024-09-15T00:12:17.795 INFO:teuthology.orchestra.run.smithi171.stdout:3826 punch from 0x2861da4 to 0x286eb7b, (0xcdd7 bytes) 2024-09-15T00:12:17.795 INFO:teuthology.orchestra.run.smithi171.stdout:3827 clone 28 order 23 su 1048576 sc 3 2024-09-15T00:12:17.829 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:17.826+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:17.883 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:17.883 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:17.883 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:17.884 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:17.884 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:17.892 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:17.930 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:17.928+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:17.946 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:12:17.946 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:12:17.958 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-09-15T00:12:17.034878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":980156791,"num_objects":1520,"num_object_clones":552,"num_object_copies":3040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1520,"num_whiteouts":595,"num_read":136777,"num_read_kb":596825,"num_write":16138,"num_write_kb":311933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1385,"num_bytes_recovered":122991278,"num_keys_recovered":171,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12923,"ondisk_log_size":12923,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":285648,"kb_used_data":45788,"kb_used_omap":9,"kb_used_meta":239734,"kb_avail":565945392,"statfs":{"total":579820584960,"available":579528081408,"internally_reserved":0,"allocated":46886912,"data_stored":617063904,"data_compressed":12312174,"data_compressed_allocated":34832384,"data_compressed_original":608831994,"omap_allocated":9566,"internal_metadata":245488290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,12],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51851824,"num_objects":173,"num_object_clones":78,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-318,"num_objects_misplaced":-12,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":75,"num_read":6098,"num_read_kb":13952,"num_write":1976,"num_write_kb":9747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":29914626,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.017790"},"pg_stats":[{"pgid":"3.1f","version":"71'325","reported_seq":3345,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.092661+0000","last_change":"2024-09-15T00:12:11.014105+0000","last_active":"2024-09-15T00:12:14.092661+0000","last_peered":"2024-09-15T00:12:14.092661+0000","last_clean":"2024-09-15T00:12:14.092661+0000","last_became_active":"2024-09-15T00:12:11.013266+0000","last_became_peered":"2024-09-15T00:12:11.013266+0000","last_unstale":"2024-09-15T00:12:14.092661+0000","last_undegraded":"2024-09-15T00:12:14.092661+0000","last_fullsized":"2024-09-15T00:12:14.092661+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:21:30.909501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012175910000000001,"stat_sum":{"num_bytes":16592310,"num_objects":41,"num_object_clones":15,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":18,"num_read":3619,"num_read_kb":11589,"num_write":329,"num_write_kb":4397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":5475412,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"71'416","reported_seq":4801,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.298883+0000","last_change":"2024-09-15T00:11:21.594579+0000","last_active":"2024-09-15T00:12:14.298883+0000","last_peered":"2024-09-15T00:12:14.298883+0000","last_clean":"2024-09-15T00:12:14.298883+0000","last_became_active":"2024-09-15T00:11:19.598640+0000","last_became_peered":"2024-09-15T00:11:19.598640+0000","last_unstale":"2024-09-15T00:12:14.298883+0000","last_undegraded":"2024-09-15T00:12:14.298883+0000","last_fullsized":"2024-09-15T00:12:14.298883+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:27:25.641378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065049699999999999,"stat_sum":{"num_bytes":33490296,"num_objects":63,"num_object_clones":24,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":25,"num_read":5419,"num_read_kb":23046,"num_write":569,"num_write_kb":8811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"71'223","reported_seq":1628,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.109488+0000","last_change":"2024-09-15T00:11:21.603457+0000","last_active":"2024-09-15T00:12:14.109488+0000","last_peered":"2024-09-15T00:12:14.109488+0000","last_clean":"2024-09-15T00:12:14.109488+0000","last_became_active":"2024-09-15T00:11:19.595954+0000","last_became_peered":"2024-09-15T00:11:19.595954+0000","last_unstale":"2024-09-15T00:12:14.109488+0000","last_undegraded":"2024-09-15T00:12:14.109488+0000","last_fullsized":"2024-09-15T00:12:14.109488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:35:44.444049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017890510000000001,"stat_sum":{"num_bytes":15790710,"num_objects":32,"num_object_clones":11,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":1331,"num_read_kb":9608,"num_write":253,"num_write_kb":9060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"71'214","reported_seq":1743,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.297464+0000","last_change":"2024-09-15T00:11:21.602612+0000","last_active":"2024-09-15T00:12:14.297464+0000","last_peered":"2024-09-15T00:12:14.297464+0000","last_clean":"2024-09-15T00:12:14.297464+0000","last_became_active":"2024-09-15T00:11:19.602583+0000","last_became_peered":"2024-09-15T00:11:19.602583+0000","last_unstale":"2024-09-15T00:12:14.297464+0000","last_undegraded":"2024-09-15T00:12:14.297464+0000","last_fullsized":"2024-09-15T00:12:14.297464+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:21:26.088020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011501599999999999,"stat_sum":{"num_bytes":17465295,"num_objects":42,"num_object_clones":16,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":16,"num_read":1546,"num_read_kb":11182,"num_write":215,"num_write_kb":5491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"72'305","reported_seq":5212,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T00:12:15.319961+0000","last_change":"2024-09-15T00:12:12.020404+0000","last_active":"2024-09-15T00:12:15.319961+0000","last_peered":"2024-09-15T00:12:15.319961+0000","last_clean":"2024-09-15T00:12:15.319961+0000","last_became_active":"2024-09-15T00:12:12.019886+0000","last_became_peered":"2024-09-15T00:12:12.019886+0000","last_unstale":"2024-09-15T00:12:15.319961+0000","last_undegraded":"2024-09-15T00:12:15.319961+0000","last_fullsized":"2024-09-15T00:12:15.319961+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:48:00.068673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074638899999999997,"stat_sum":{"num_bytes":24998461,"num_objects":65,"num_object_clones":27,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":28,"num_read":4917,"num_read_kb":19745,"num_write":278,"num_write_kb":7624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":10954706,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"72'307","reported_seq":3670,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T00:12:15.304083+0000","last_change":"2024-09-15T00:11:21.595070+0000","last_active":"2024-09-15T00:12:15.304083+0000","last_peered":"2024-09-15T00:12:15.304083+0000","last_clean":"2024-09-15T00:12:15.304083+0000","last_became_active":"2024-09-15T00:11:19.600452+0000","last_became_peered":"2024-09-15T00:11:19.600452+0000","last_unstale":"2024-09-15T00:12:15.304083+0000","last_undegraded":"2024-09-15T00:12:15.304083+0000","last_fullsized":"2024-09-15T00:12:15.304083+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:13:16.732963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048699700000000003,"stat_sum":{"num_bytes":58087005,"num_objects":38,"num_object_clones":12,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":10,"num_read":4775,"num_read_kb":39981,"num_write":323,"num_write_kb":20562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"71'370","reported_seq":2475,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.183963+0000","last_change":"2024-09-15T00:11:21.595707+0000","last_active":"2024-09-15T00:12:14.183963+0000","last_peered":"2024-09-15T00:12:14.183963+0000","last_clean":"2024-09-15T00:12:14.183963+0000","last_became_active":"2024-09-15T00:11:19.595311+0000","last_became_peered":"2024-09-15T00:11:19.595311+0000","last_unstale":"2024-09-15T00:12:14.183963+0000","last_undegraded":"2024-09-15T00:12:14.183963+0000","last_fullsized":"2024-09-15T00:12:14.183963+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:16:49.905442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015502809999999999,"stat_sum":{"num_bytes":30917179,"num_objects":51,"num_object_clones":18,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":20,"num_read":2522,"num_read_kb":15440,"num_write":424,"num_write_kb":7790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"71'404","reported_seq":4205,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.297953+0000","last_change":"2024-09-15T00:12:12.021741+0000","last_active":"2024-09-15T00:12:14.297953+0000","last_peered":"2024-09-15T00:12:14.297953+0000","last_clean":"2024-09-15T00:12:14.297953+0000","last_became_active":"2024-09-15T00:12:12.020924+0000","last_became_peered":"2024-09-15T00:12:12.020924+0000","last_unstale":"2024-09-15T00:12:14.297953+0000","last_undegraded":"2024-09-15T00:12:14.297953+0000","last_fullsized":"2024-09-15T00:12:14.297953+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:53:57.200457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081289999999999997,"stat_sum":{"num_bytes":22180626,"num_objects":59,"num_object_clones":21,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":24,"num_read":4441,"num_read_kb":17140,"num_write":509,"num_write_kb":4862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4986328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"71'345","reported_seq":2963,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.092583+0000","last_change":"2024-09-15T00:12:11.014149+0000","last_active":"2024-09-15T00:12:14.092583+0000","last_peered":"2024-09-15T00:12:14.092583+0000","last_clean":"2024-09-15T00:12:14.092583+0000","last_became_active":"2024-09-15T00:12:11.013261+0000","last_became_peered":"2024-09-15T00:12:11.013261+0000","last_unstale":"2024-09-15T00:12:14.092583+0000","last_undegraded":"2024-09-15T00:12:14.092583+0000","last_fullsized":"2024-09-15T00:12:14.092583+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:43:46.283467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00163887,"stat_sum":{"num_bytes":45142964,"num_objects":52,"num_object_clones":22,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":20,"num_read":3346,"num_read_kb":26674,"num_write":390,"num_write_kb":13426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":12460957,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"71'639","reported_seq":3515,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.297734+0000","last_change":"2024-09-15T00:11:53.012391+0000","last_active":"2024-09-15T00:12:14.297734+0000","last_peered":"2024-09-15T00:12:14.297734+0000","last_clean":"2024-09-15T00:12:14.297734+0000","last_became_active":"2024-09-15T00:11:53.011806+0000","last_became_peered":"2024-09-15T00:11:53.011806+0000","last_unstale":"2024-09-15T00:12:14.297734+0000","last_undegraded":"2024-09-15T00:12:14.297734+0000","last_fullsized":"2024-09-15T00:12:14.297734+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:23:08.821216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014330530000000001,"stat_sum":{"num_bytes":40253788,"num_objects":39,"num_object_clones":14,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":4694,"num_read_kb":18226,"num_write":908,"num_write_kb":12115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":3122028,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"71'249","reported_seq":1977,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.301371+0000","last_change":"2024-09-15T00:11:53.010457+0000","last_active":"2024-09-15T00:12:14.301371+0000","last_peered":"2024-09-15T00:12:14.301371+0000","last_clean":"2024-09-15T00:12:14.301371+0000","last_became_active":"2024-09-15T00:11:53.010039+0000","last_became_peered":"2024-09-15T00:11:53.010039+0000","last_unstale":"2024-09-15T00:12:14.301371+0000","last_undegraded":"2024-09-15T00:12:14.301371+0000","last_fullsized":"2024-09-15T00:12:14.301371+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:16:19.320521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085960799999999999,"stat_sum":{"num_bytes":37464658,"num_objects":41,"num_object_clones":15,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":1755,"num_read_kb":20409,"num_write":274,"num_write_kb":12303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":9563692,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"72'790","reported_seq":5361,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T00:12:15.315999+0000","last_change":"2024-09-15T00:11:53.007713+0000","last_active":"2024-09-15T00:12:15.315999+0000","last_peered":"2024-09-15T00:12:15.315999+0000","last_clean":"2024-09-15T00:12:15.315999+0000","last_became_active":"2024-09-15T00:11:53.007266+0000","last_became_peered":"2024-09-15T00:11:53.007266+0000","last_unstale":"2024-09-15T00:12:15.315999+0000","last_undegraded":"2024-09-15T00:12:15.315999+0000","last_fullsized":"2024-09-15T00:12:15.315999+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:47:14.571507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077114700000000002,"stat_sum":{"num_bytes":32515264,"num_objects":54,"num_object_clones":18,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":24,"num_read":8205,"num_read_kb":27093,"num_write":1265,"num_write_kb":13907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":3041694,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":123,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.020464+0000","last_change":"2024-09-15T00:11:50.004384+0000","last_active":"2024-09-15T00:12:14.020464+0000","last_peered":"2024-09-15T00:12:14.020464+0000","last_clean":"2024-09-15T00:12:14.020464+0000","last_became_active":"2024-09-15T00:11:50.004088+0000","last_became_peered":"2024-09-15T00:11:50.004088+0000","last_unstale":"2024-09-15T00:12:14.020464+0000","last_undegraded":"2024-09-15T00:12:14.020464+0000","last_fullsized":"2024-09-15T00:12:14.020464+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:29:29.359137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"71'327","reported_seq":4769,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.211134+0000","last_change":"2024-09-15T00:12:10.012838+0000","last_active":"2024-09-15T00:12:14.211134+0000","last_peered":"2024-09-15T00:12:14.211134+0000","last_clean":"2024-09-15T00:12:14.211134+0000","last_became_active":"2024-09-15T00:12:10.009168+0000","last_became_peered":"2024-09-15T00:12:10.009168+0000","last_unstale":"2024-09-15T00:12:14.211134+0000","last_undegraded":"2024-09-15T00:12:14.211134+0000","last_fullsized":"2024-09-15T00:12:14.211134+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:06:11.512093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014370050000000001,"stat_sum":{"num_bytes":41566721,"num_objects":53,"num_object_clones":22,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":24,"num_read":4260,"num_read_kb":27747,"num_write":363,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":8585078,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":119,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.061666+0000","last_change":"2024-09-15T00:11:11.545630+0000","last_active":"2024-09-15T00:12:14.061666+0000","last_peered":"2024-09-15T00:12:14.061666+0000","last_clean":"2024-09-15T00:12:14.061666+0000","last_became_active":"2024-09-15T00:11:09.536615+0000","last_became_peered":"2024-09-15T00:11:09.536615+0000","last_unstale":"2024-09-15T00:12:14.061666+0000","last_undegraded":"2024-09-15T00:12:14.061666+0000","last_fullsized":"2024-09-15T00:12:14.061666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:51:38.340257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035555,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"71'378","reported_seq":3331,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.299075+0000","last_change":"2024-09-15T00:12:10.011410+0000","last_active":"2024-09-15T00:12:14.299075+0000","last_peered":"2024-09-15T00:12:14.299075+0000","last_clean":"2024-09-15T00:12:14.299075+0000","last_became_active":"2024-09-15T00:12:10.010014+0000","last_became_peered":"2024-09-15T00:12:10.010014+0000","last_unstale":"2024-09-15T00:12:14.299075+0000","last_undegraded":"2024-09-15T00:12:14.299075+0000","last_fullsized":"2024-09-15T00:12:14.299075+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:47.743844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058512600000000005,"stat_sum":{"num_bytes":17161677,"num_objects":57,"num_object_clones":22,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":23,"num_read":3489,"num_read_kb":16838,"num_write":444,"num_write_kb":12375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":11200702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T00:12:15.039925+0000","last_change":"2024-09-15T00:12:07.216196+0000","last_active":"2024-09-15T00:12:15.039925+0000","last_peered":"2024-09-15T00:12:15.039925+0000","last_clean":"2024-09-15T00:12:15.039925+0000","last_became_active":"2024-09-15T00:12:07.215917+0000","last_became_peered":"2024-09-15T00:12:07.215917+0000","last_unstale":"2024-09-15T00:12:15.039925+0000","last_undegraded":"2024-09-15T00:12:15.039925+0000","last_fullsized":"2024-09-15T00:12:15.039925+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:31:37.723306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"71'301","reported_seq":8295,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.061594+0000","last_change":"2024-09-15T00:11:40.073377+0000","last_active":"2024-09-15T00:12:14.061594+0000","last_peered":"2024-09-15T00:12:14.061594+0000","last_clean":"2024-09-15T00:12:14.061594+0000","last_became_active":"2024-09-15T00:11:40.072803+0000","last_became_peered":"2024-09-15T00:11:40.072803+0000","last_unstale":"2024-09-15T00:12:14.061594+0000","last_undegraded":"2024-09-15T00:12:14.061594+0000","last_fullsized":"2024-09-15T00:12:14.061594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:34:57.955682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001489752,"stat_sum":{"num_bytes":55407800,"num_objects":54,"num_object_clones":20,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":20,"num_read":9759,"num_read_kb":32480,"num_write":295,"num_write_kb":11558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":482924,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":134,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.061512+0000","last_change":"2024-09-15T00:12:07.218010+0000","last_active":"2024-09-15T00:12:14.061512+0000","last_peered":"2024-09-15T00:12:14.061512+0000","last_clean":"2024-09-15T00:12:14.061512+0000","last_became_active":"2024-09-15T00:12:07.217723+0000","last_became_peered":"2024-09-15T00:12:07.217723+0000","last_unstale":"2024-09-15T00:12:14.061512+0000","last_undegraded":"2024-09-15T00:12:14.061512+0000","last_fullsized":"2024-09-15T00:12:14.061512+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:16:30.572035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"70'241","reported_seq":1945,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.034596+0000","last_change":"2024-09-15T00:11:21.603338+0000","last_active":"2024-09-15T00:12:14.034596+0000","last_peered":"2024-09-15T00:12:14.034596+0000","last_clean":"2024-09-15T00:12:14.034596+0000","last_became_active":"2024-09-15T00:11:19.595633+0000","last_became_peered":"2024-09-15T00:11:19.595633+0000","last_unstale":"2024-09-15T00:12:14.034596+0000","last_undegraded":"2024-09-15T00:12:14.034596+0000","last_fullsized":"2024-09-15T00:12:14.034596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:02:22.620600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017065330000000001,"stat_sum":{"num_bytes":9685942,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":2502,"num_read_kb":6432,"num_write":225,"num_write_kb":1097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"72'393","reported_seq":3516,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T00:12:15.314983+0000","last_change":"2024-09-15T00:12:11.018217+0000","last_active":"2024-09-15T00:12:15.314983+0000","last_peered":"2024-09-15T00:12:15.314983+0000","last_clean":"2024-09-15T00:12:15.314983+0000","last_became_active":"2024-09-15T00:12:11.013156+0000","last_became_peered":"2024-09-15T00:12:11.013156+0000","last_unstale":"2024-09-15T00:12:15.314983+0000","last_undegraded":"2024-09-15T00:12:15.314983+0000","last_fullsized":"2024-09-15T00:12:15.314983+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:54:15.788600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075923799999999997,"stat_sum":{"num_bytes":46582839,"num_objects":47,"num_object_clones":19,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":3631,"num_read_kb":22590,"num_write":407,"num_write_kb":19337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":9273484,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":120,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.034326+0000","last_change":"2024-09-15T00:11:11.543519+0000","last_active":"2024-09-15T00:12:14.034326+0000","last_peered":"2024-09-15T00:12:14.034326+0000","last_clean":"2024-09-15T00:12:14.034326+0000","last_became_active":"2024-09-15T00:11:09.533896+0000","last_became_peered":"2024-09-15T00:11:09.533896+0000","last_unstale":"2024-09-15T00:12:14.034326+0000","last_undegraded":"2024-09-15T00:12:14.034326+0000","last_fullsized":"2024-09-15T00:12:14.034326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:47.613931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"71'440","reported_seq":4192,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.061539+0000","last_change":"2024-09-15T00:11:37.317374+0000","last_active":"2024-09-15T00:12:14.061539+0000","last_peered":"2024-09-15T00:12:14.061539+0000","last_clean":"2024-09-15T00:12:14.061539+0000","last_became_active":"2024-09-15T00:11:37.238849+0000","last_became_peered":"2024-09-15T00:11:37.238849+0000","last_unstale":"2024-09-15T00:12:14.061539+0000","last_undegraded":"2024-09-15T00:12:14.061539+0000","last_fullsized":"2024-09-15T00:12:14.061539+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:00:42.263777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241242,"stat_sum":{"num_bytes":18517623,"num_objects":51,"num_object_clones":15,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":19,"num_read":5342,"num_read_kb":13979,"num_write":521,"num_write_kb":6467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1846940,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":79,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.020561+0000","last_change":"2024-09-15T00:12:07.215690+0000","last_active":"2024-09-15T00:12:14.020561+0000","last_peered":"2024-09-15T00:12:14.020561+0000","last_clean":"2024-09-15T00:12:14.020561+0000","last_became_active":"2024-09-15T00:12:07.215396+0000","last_became_peered":"2024-09-15T00:12:07.215396+0000","last_unstale":"2024-09-15T00:12:14.020561+0000","last_undegraded":"2024-09-15T00:12:14.020561+0000","last_fullsized":"2024-09-15T00:12:14.020561+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:03:59.283399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"72'266","reported_seq":2775,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T00:12:15.319898+0000","last_change":"2024-09-15T00:12:10.010199+0000","last_active":"2024-09-15T00:12:15.319898+0000","last_peered":"2024-09-15T00:12:15.319898+0000","last_clean":"2024-09-15T00:12:15.319898+0000","last_became_active":"2024-09-15T00:12:10.009654+0000","last_became_peered":"2024-09-15T00:12:10.009654+0000","last_unstale":"2024-09-15T00:12:15.319898+0000","last_undegraded":"2024-09-15T00:12:15.319898+0000","last_fullsized":"2024-09-15T00:12:15.319898+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:03.396049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069791600000000003,"stat_sum":{"num_bytes":37246322,"num_objects":43,"num_object_clones":17,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":12,"num_read":2438,"num_read_kb":13671,"num_write":279,"num_write_kb":8654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":6058433,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.061557+0000","last_change":"2024-09-15T00:12:07.216951+0000","last_active":"2024-09-15T00:12:14.061557+0000","last_peered":"2024-09-15T00:12:14.061557+0000","last_clean":"2024-09-15T00:12:14.061557+0000","last_became_active":"2024-09-15T00:12:07.216646+0000","last_became_peered":"2024-09-15T00:12:07.216646+0000","last_unstale":"2024-09-15T00:12:14.061557+0000","last_undegraded":"2024-09-15T00:12:14.061557+0000","last_fullsized":"2024-09-15T00:12:14.061557+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:30:36.276256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"71'299","reported_seq":2169,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.298562+0000","last_change":"2024-09-15T00:11:21.595240+0000","last_active":"2024-09-15T00:12:14.298562+0000","last_peered":"2024-09-15T00:12:14.298562+0000","last_clean":"2024-09-15T00:12:14.298562+0000","last_became_active":"2024-09-15T00:11:19.594528+0000","last_became_peered":"2024-09-15T00:11:19.594528+0000","last_unstale":"2024-09-15T00:12:14.298562+0000","last_undegraded":"2024-09-15T00:12:14.298562+0000","last_fullsized":"2024-09-15T00:12:14.298562+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:46:07.458347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080299099999999997,"stat_sum":{"num_bytes":26304988,"num_objects":46,"num_object_clones":17,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":16,"num_read":2222,"num_read_kb":12574,"num_write":350,"num_write_kb":6773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":137,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.034317+0000","last_change":"2024-09-15T00:11:50.010414+0000","last_active":"2024-09-15T00:12:14.034317+0000","last_peered":"2024-09-15T00:12:14.034317+0000","last_clean":"2024-09-15T00:12:14.034317+0000","last_became_active":"2024-09-15T00:11:50.007204+0000","last_became_peered":"2024-09-15T00:11:50.007204+0000","last_unstale":"2024-09-15T00:12:14.034317+0000","last_undegraded":"2024-09-15T00:12:14.034317+0000","last_fullsized":"2024-09-15T00:12:14.034317+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:13:14.255774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":251,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.020507+0000","last_change":"2024-09-15T00:12:10.022102+0000","last_active":"2024-09-15T00:12:14.020507+0000","last_peered":"2024-09-15T00:12:14.020507+0000","last_clean":"2024-09-15T00:12:14.020507+0000","last_became_active":"2024-09-15T00:12:10.021831+0000","last_became_peered":"2024-09-15T00:12:10.021831+0000","last_unstale":"2024-09-15T00:12:14.020507+0000","last_undegraded":"2024-09-15T00:12:14.020507+0000","last_fullsized":"2024-09-15T00:12:14.020507+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_clean_scrub_stamp":"2024-09-15T00:11:04.453672+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:54:25.088117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"71'498","reported_seq":3541,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.195563+0000","last_change":"2024-09-15T00:11:21.595379+0000","last_active":"2024-09-15T00:12:14.195563+0000","last_peered":"2024-09-15T00:12:14.195563+0000","last_clean":"2024-09-15T00:12:14.195563+0000","last_became_active":"2024-09-15T00:11:19.594476+0000","last_became_peered":"2024-09-15T00:11:19.594476+0000","last_unstale":"2024-09-15T00:12:14.195563+0000","last_undegraded":"2024-09-15T00:12:14.195563+0000","last_fullsized":"2024-09-15T00:12:14.195563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:53.728473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089807799999999996,"stat_sum":{"num_bytes":33316980,"num_objects":55,"num_object_clones":20,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":18,"num_read":4671,"num_read_kb":18445,"num_write":611,"num_write_kb":10246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"72'835","reported_seq":6914,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T00:12:15.311413+0000","last_change":"2024-09-15T00:11:53.010850+0000","last_active":"2024-09-15T00:12:15.311413+0000","last_peered":"2024-09-15T00:12:15.311413+0000","last_clean":"2024-09-15T00:12:15.311413+0000","last_became_active":"2024-09-15T00:11:53.008775+0000","last_became_peered":"2024-09-15T00:11:53.008775+0000","last_unstale":"2024-09-15T00:12:15.311413+0000","last_undegraded":"2024-09-15T00:12:15.311413+0000","last_fullsized":"2024-09-15T00:12:15.311413+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:16:21.218203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053468500000000002,"stat_sum":{"num_bytes":28268237,"num_objects":48,"num_object_clones":15,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":17,"num_read":10507,"num_read_kb":30195,"num_write":1225,"num_write_kb":10190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":3263210,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"71'389","reported_seq":4440,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.092543+0000","last_change":"2024-09-15T00:11:21.608019+0000","last_active":"2024-09-15T00:12:14.092543+0000","last_peered":"2024-09-15T00:12:14.092543+0000","last_clean":"2024-09-15T00:12:14.092543+0000","last_became_active":"2024-09-15T00:11:19.602625+0000","last_became_peered":"2024-09-15T00:11:19.602625+0000","last_unstale":"2024-09-15T00:12:14.092543+0000","last_undegraded":"2024-09-15T00:12:14.092543+0000","last_fullsized":"2024-09-15T00:12:14.092543+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:53:04.096378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036072699999999998,"stat_sum":{"num_bytes":23225220,"num_objects":48,"num_object_clones":15,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":18,"num_read":4972,"num_read_kb":13979,"num_write":413,"num_write_kb":7516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"71'225","reported_seq":2713,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.020851+0000","last_change":"2024-09-15T00:12:12.021697+0000","last_active":"2024-09-15T00:12:14.020851+0000","last_peered":"2024-09-15T00:12:14.020851+0000","last_clean":"2024-09-15T00:12:14.020851+0000","last_became_active":"2024-09-15T00:12:12.020929+0000","last_became_peered":"2024-09-15T00:12:12.020929+0000","last_unstale":"2024-09-15T00:12:14.020851+0000","last_undegraded":"2024-09-15T00:12:14.020851+0000","last_fullsized":"2024-09-15T00:12:14.020851+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:08:20.732936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082326600000000002,"stat_sum":{"num_bytes":40600821,"num_objects":41,"num_object_clones":15,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":13,"num_read":2322,"num_read_kb":23957,"num_write":285,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":10092575,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"71'556","reported_seq":2190,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.092587+0000","last_change":"2024-09-15T00:11:21.603031+0000","last_active":"2024-09-15T00:12:14.092587+0000","last_peered":"2024-09-15T00:12:14.092587+0000","last_clean":"2024-09-15T00:12:14.092587+0000","last_became_active":"2024-09-15T00:11:19.595088+0000","last_became_peered":"2024-09-15T00:11:19.595088+0000","last_unstale":"2024-09-15T00:12:14.092587+0000","last_undegraded":"2024-09-15T00:12:14.092587+0000","last_fullsized":"2024-09-15T00:12:14.092587+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:38:40.255977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537407,"stat_sum":{"num_bytes":25803235,"num_objects":34,"num_object_clones":14,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":4197,"num_read_kb":16410,"num_write":810,"num_write_kb":9173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"71'293","reported_seq":3691,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.297271+0000","last_change":"2024-09-15T00:11:50.060453+0000","last_active":"2024-09-15T00:12:14.297271+0000","last_peered":"2024-09-15T00:12:14.297271+0000","last_clean":"2024-09-15T00:12:14.297271+0000","last_became_active":"2024-09-15T00:11:50.008449+0000","last_became_peered":"2024-09-15T00:11:50.008449+0000","last_unstale":"2024-09-15T00:12:14.297271+0000","last_undegraded":"2024-09-15T00:12:14.297271+0000","last_fullsized":"2024-09-15T00:12:14.297271+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:23:15.473597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069298099999999996,"stat_sum":{"num_bytes":35136551,"num_objects":54,"num_object_clones":23,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":20,"num_read":3817,"num_read_kb":11652,"num_write":262,"num_write_kb":3660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1062007,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"71'1100","reported_seq":3592,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T00:12:15.305503+0000","last_change":"2024-09-15T00:11:53.009720+0000","last_active":"2024-09-15T00:12:15.305503+0000","last_peered":"2024-09-15T00:12:15.305503+0000","last_clean":"2024-09-15T00:12:15.305503+0000","last_became_active":"2024-09-15T00:11:53.009138+0000","last_became_peered":"2024-09-15T00:11:53.009138+0000","last_unstale":"2024-09-15T00:12:15.305503+0000","last_undegraded":"2024-09-15T00:12:15.305503+0000","last_fullsized":"2024-09-15T00:12:15.305503+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:02:11.614434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076259299999999999,"stat_sum":{"num_bytes":28955109,"num_objects":49,"num_object_clones":17,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":19,"num_read":7017,"num_read_kb":21632,"num_write":1829,"num_write_kb":9712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":647422,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"72'229","reported_seq":4382,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T00:12:15.319810+0000","last_change":"2024-09-15T00:12:11.009737+0000","last_active":"2024-09-15T00:12:15.319810+0000","last_peered":"2024-09-15T00:12:15.319810+0000","last_clean":"2024-09-15T00:12:15.319810+0000","last_became_active":"2024-09-15T00:12:11.009246+0000","last_became_peered":"2024-09-15T00:12:11.009246+0000","last_unstale":"2024-09-15T00:12:15.319810+0000","last_undegraded":"2024-09-15T00:12:15.319810+0000","last_fullsized":"2024-09-15T00:12:15.319810+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:36:48.522874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055547799999999998,"stat_sum":{"num_bytes":30829983,"num_objects":45,"num_object_clones":14,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":22,"num_read":3938,"num_read_kb":11365,"num_write":216,"num_write_kb":7014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":4060758,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"71'436","reported_seq":4207,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.308226+0000","last_change":"2024-09-15T00:12:11.014001+0000","last_active":"2024-09-15T00:12:14.308226+0000","last_peered":"2024-09-15T00:12:14.308226+0000","last_clean":"2024-09-15T00:12:14.308226+0000","last_became_active":"2024-09-15T00:12:11.013203+0000","last_became_peered":"2024-09-15T00:12:11.013203+0000","last_unstale":"2024-09-15T00:12:14.308226+0000","last_undegraded":"2024-09-15T00:12:14.308226+0000","last_fullsized":"2024-09-15T00:12:14.308226+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:12:54.476646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093430300000000004,"stat_sum":{"num_bytes":32227095,"num_objects":46,"num_object_clones":14,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":16,"num_read":4645,"num_read_kb":23223,"num_write":591,"num_write_kb":12675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":8339863,"num_keys_recovered":18,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"72'361","reported_seq":2912,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T00:12:15.314129+0000","last_change":"2024-09-15T00:11:53.010310+0000","last_active":"2024-09-15T00:12:15.314129+0000","last_peered":"2024-09-15T00:12:15.314129+0000","last_clean":"2024-09-15T00:12:15.314129+0000","last_became_active":"2024-09-15T00:11:53.009725+0000","last_became_peered":"2024-09-15T00:11:53.009725+0000","last_unstale":"2024-09-15T00:12:15.314129+0000","last_undegraded":"2024-09-15T00:12:15.314129+0000","last_fullsized":"2024-09-15T00:12:15.314129+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:44:12.452671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042917200000000002,"stat_sum":{"num_bytes":33327156,"num_objects":54,"num_object_clones":20,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":2990,"num_read_kb":11160,"num_write":410,"num_write_kb":6197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":758528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"71'363","reported_seq":3147,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T00:12:14.297736+0000","last_change":"2024-09-15T00:12:11.013515+0000","last_active":"2024-09-15T00:12:14.297736+0000","last_peered":"2024-09-15T00:12:14.297736+0000","last_clean":"2024-09-15T00:12:14.297736+0000","last_became_active":"2024-09-15T00:12:11.012806+0000","last_became_peered":"2024-09-15T00:12:11.012806+0000","last_unstale":"2024-09-15T00:12:14.297736+0000","last_undegraded":"2024-09-15T00:12:14.297736+0000","last_fullsized":"2024-09-15T00:12:14.297736+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:33:25.380203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042453400000000001,"stat_sum":{"num_bytes":19631098,"num_objects":39,"num_object_clones":15,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":2846,"num_read_kb":11268,"num_write":431,"num_write_kb":10428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":4214788,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"72'286","reported_seq":4489,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T00:12:15.319572+0000","last_change":"2024-09-15T00:11:53.010010+0000","last_active":"2024-09-15T00:12:15.319572+0000","last_peered":"2024-09-15T00:12:15.319572+0000","last_clean":"2024-09-15T00:12:15.319572+0000","last_became_active":"2024-09-15T00:11:53.009403+0000","last_became_peered":"2024-09-15T00:11:53.009403+0000","last_unstale":"2024-09-15T00:12:15.319572+0000","last_undegraded":"2024-09-15T00:12:15.319572+0000","last_fullsized":"2024-09-15T00:12:15.319572+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:09:37.077090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062135599999999999,"stat_sum":{"num_bytes":20872451,"num_objects":49,"num_object_clones":18,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":23,"num_read":4526,"num_read_kb":16995,"num_write":273,"num_write_kb":7716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":2319013,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":979566404,"num_objects":1516,"num_object_clones":552,"num_object_copies":3032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1516,"num_whiteouts":595,"num_read":136661,"num_read_kb":596725,"num_write":15977,"num_write_kb":309515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1379,"num_bytes_recovered":121810542,"num_keys_recovered":171,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45805568,"data_stored":671178238,"data_compressed":13215064,"data_compressed_allocated":38043648,"data_compressed_original":664737650,"omap_allocated":0,"internal_metadata":0},"log_size":12803,"ondisk_log_size":12803,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":2361472,"data_compressed":24608,"data_compressed_allocated":147456,"data_compressed_original":2342912,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542165,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59292,"kb_used_data":10060,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94312548,"statfs":{"total":96636764160,"available":96576049152,"internally_reserved":0,"allocated":10301440,"data_stored":141793041,"data_compressed":2674708,"data_compressed_allocated":7938048,"data_compressed_original":140114960,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,12],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34412,"kb_used_data":7384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337428,"statfs":{"total":96636764160,"available":96601526272,"internally_reserved":0,"allocated":7561216,"data_stored":102693297,"data_compressed":1925653,"data_compressed_allocated":5541888,"data_compressed_original":101368080,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":0,"seq":60129542162,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44728,"kb_used_data":5224,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327112,"statfs":{"total":96636764160,"available":96590962688,"internally_reserved":0,"allocated":5349376,"data_stored":58807610,"data_compressed":1235805,"data_compressed_allocated":3731456,"data_compressed_original":57864586,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50092,"kb_used_data":6424,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94321748,"statfs":{"total":96636764160,"available":96585469952,"internally_reserved":0,"allocated":6578176,"data_stored":82091846,"data_compressed":1787539,"data_compressed_allocated":4816896,"data_compressed_original":80942998,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":60129542162,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52576,"kb_used_data":7504,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94319264,"statfs":{"total":96636764160,"available":96582926336,"internally_reserved":0,"allocated":7684096,"data_stored":102922040,"data_compressed":2131611,"data_compressed_allocated":5713920,"data_compressed_original":101504784,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44548,"kb_used_data":9192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94327292,"statfs":{"total":96636764160,"available":96591147008,"internally_reserved":0,"allocated":9412608,"data_stored":128756070,"data_compressed":2556858,"data_compressed_allocated":7090176,"data_compressed_original":127036586,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8544256,"data_stored":128058843,"data_compressed":2568849,"data_compressed_allocated":7053312,"data_compressed_original":126717128,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6893568,"data_stored":102662387,"data_compressed":2151180,"data_compressed_allocated":5730304,"data_compressed_original":101615376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":542788,"data_compressed":51639,"data_compressed_allocated":77824,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6733824,"data_stored":102316072,"data_compressed":1925653,"data_compressed_allocated":5541888,"data_compressed_original":101368080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9703424,"data_stored":144066651,"data_compressed":2721836,"data_compressed_allocated":8146944,"data_compressed_original":142760976,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5898240,"data_stored":82186860,"data_compressed":1810074,"data_compressed_allocated":4841472,"data_compressed_original":81307542,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4517888,"data_stored":58008792,"data_compressed":1251376,"data_compressed_allocated":3710976,"data_compressed_original":57434506,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3395584,"data_stored":53335845,"data_compressed":734457,"data_compressed_allocated":2940928,"data_compressed_original":53009754,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:12:17.960 INFO:tasks.thrashosds.thrasher:Setting 3.9 to [3, 0] 2024-09-15T00:12:17.961 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.9', '3', '0'] 2024-09-15T00:12:17.961 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.9 3 0 2024-09-15T00:12:18.031 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:18.028+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:18.041 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:18.041 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:18.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:18.041 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:18.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.041 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:70 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-09-15T00:12:18.041 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:12:18.041 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 72.502316004999997, 2024-09-15T00:12:18.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057871490000000001, 2024-09-15T00:12:18.042 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.042 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.042 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:18.042 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:12:18.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:12:18.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536110+0000", 2024-09-15T00:12:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-15T00:12:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536115+0000", 2024-09-15T00:12:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8960000000000002e-06 2024-09-15T00:12:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536123+0000", 2024-09-15T00:12:18.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3979999999999999e-06 2024-09-15T00:12:18.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536128+0000", 2024-09-15T00:12:18.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5569999999999997e-06 2024-09-15T00:12:18.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537011+0000", 2024-09-15T00:12:18.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088276299999999995 2024-09-15T00:12:18.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537034+0000", 2024-09-15T00:12:18.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3448999999999999e-05 2024-09-15T00:12:18.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:18.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541867+0000", 2024-09-15T00:12:18.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048332949999999996 2024-09-15T00:12:18.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541896+0000", 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9179999999999998e-05 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:154 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'108, mlcod=16'105)", 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 71.514165544999997, 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053187770000000002, 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.046 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:18.047 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:12:18.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:12:18.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524260+0000", 2024-09-15T00:12:18.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:12:18.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524264+0000", 2024-09-15T00:12:18.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.738e-06 2024-09-15T00:12:18.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524270+0000", 2024-09-15T00:12:18.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9379999999999997e-06 2024-09-15T00:12:18.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524273+0000", 2024-09-15T00:12:18.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6370000000000001e-06 2024-09-15T00:12:18.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524310+0000", 2024-09-15T00:12:18.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6266999999999997e-05 2024-09-15T00:12:18.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524322+0000", 2024-09-15T00:12:18.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2741999999999999e-05 2024-09-15T00:12:18.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:18.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.529547+0000", 2024-09-15T00:12:18.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005225001 2024-09-15T00:12:18.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.529579+0000", 2024-09-15T00:12:18.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1065999999999997e-05 2024-09-15T00:12:18.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.050 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.051 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-15T00:12:18.051 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:18.051 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 71.513727907000003, 2024-09-15T00:12:18.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077021529999999998, 2024-09-15T00:12:18.051 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.051 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.051 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:18.051 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:18.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:18.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524698+0000", 2024-09-15T00:12:18.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6300000000000001e-07 2024-09-15T00:12:18.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524778+0000", 2024-09-15T00:12:18.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0598000000000003e-05 2024-09-15T00:12:18.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524785+0000", 2024-09-15T00:12:18.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6359999999999997e-06 2024-09-15T00:12:18.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524788+0000", 2024-09-15T00:12:18.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2540000000000001e-06 2024-09-15T00:12:18.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524817+0000", 2024-09-15T00:12:18.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8784000000000001e-05 2024-09-15T00:12:18.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524828+0000", 2024-09-15T00:12:18.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-15T00:12:18.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:18.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532360+0000", 2024-09-15T00:12:18.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007531757 2024-09-15T00:12:18.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532400+0000", 2024-09-15T00:12:18.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9835000000000001e-05 2024-09-15T00:12:18.055 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.055 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.055 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.055 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=16'106)", 2024-09-15T00:12:18.055 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:18.055 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 71.512785754000006, 2024-09-15T00:12:18.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067923050000000002, 2024-09-15T00:12:18.055 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.055 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.055 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:18.055 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-15T00:12:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525644+0000", 2024-09-15T00:12:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.709e-06 2024-09-15T00:12:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525651+0000", 2024-09-15T00:12:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7970000000000003e-06 2024-09-15T00:12:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525654+0000", 2024-09-15T00:12:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.433e-06 2024-09-15T00:12:18.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525685+0000", 2024-09-15T00:12:18.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0970000000000003e-05 2024-09-15T00:12:18.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525697+0000", 2024-09-15T00:12:18.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1759e-05 2024-09-15T00:12:18.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:18.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532401+0000", 2024-09-15T00:12:18.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006704425 2024-09-15T00:12:18.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532432+0000", 2024-09-15T00:12:18.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0709e-05 2024-09-15T00:12:18.059 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.059 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.059 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.059 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'106)", 2024-09-15T00:12:18.059 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:18.059 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 71.512323499999994, 2024-09-15T00:12:18.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063539190000000004, 2024-09-15T00:12:18.060 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.060 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.060 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:18.060 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:18.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:18.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:18.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:12:18.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526107+0000", 2024-09-15T00:12:18.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:12:18.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526117+0000", 2024-09-15T00:12:18.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7750000000000004e-06 2024-09-15T00:12:18.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526121+0000", 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526163+0000", 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1360999999999999e-05 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526175+0000", 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2729e-05 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532433+0000", 2024-09-15T00:12:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062576999999999997 2024-09-15T00:12:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532456+0000", 2024-09-15T00:12:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2773e-05 2024-09-15T00:12:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'106)", 2024-09-15T00:12:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:12:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 71.511854098000001, 2024-09-15T00:12:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059082689999999998, 2024-09-15T00:12:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:12:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:12:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526572+0000", 2024-09-15T00:12:18.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9500000000000003e-07 2024-09-15T00:12:18.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526641+0000", 2024-09-15T00:12:18.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8858999999999999e-05 2024-09-15T00:12:18.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526648+0000", 2024-09-15T00:12:18.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:12:18.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526654+0000", 2024-09-15T00:12:18.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7509999999999997e-06 2024-09-15T00:12:18.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526689+0000", 2024-09-15T00:12:18.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5800000000000003e-05 2024-09-15T00:12:18.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526700+0000", 2024-09-15T00:12:18.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0467e-05 2024-09-15T00:12:18.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:18.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532457+0000", 2024-09-15T00:12:18.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057568180000000004 2024-09-15T00:12:18.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532479+0000", 2024-09-15T00:12:18.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2739e-05 2024-09-15T00:12:18.067 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.067 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.067 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.068 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'106)", 2024-09-15T00:12:18.068 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:12:18.068 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 71.511197832999997, 2024-09-15T00:12:18.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005274801, 2024-09-15T00:12:18.068 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.068 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.068 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:18.068 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:12:18.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527227+0000", 2024-09-15T00:12:18.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527228+0000", 2024-09-15T00:12:18.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1600000000000001e-07 2024-09-15T00:12:18.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527232+0000", 2024-09-15T00:12:18.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8920000000000002e-06 2024-09-15T00:12:18.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527239+0000", 2024-09-15T00:12:18.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2880000000000004e-06 2024-09-15T00:12:18.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527243+0000", 2024-09-15T00:12:18.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.143e-06 2024-09-15T00:12:18.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527275+0000", 2024-09-15T00:12:18.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1866000000000003e-05 2024-09-15T00:12:18.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.527288+0000", 2024-09-15T00:12:18.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469999999999999e-05 2024-09-15T00:12:18.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:18.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532480+0000", 2024-09-15T00:12:18.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051927129999999998 2024-09-15T00:12:18.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532502+0000", 2024-09-15T00:12:18.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1912999999999998e-05 2024-09-15T00:12:18.072 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.072 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.072 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.073 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:18.073 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.504060+0000", 2024-09-15T00:12:18.073 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.5343650619999991, 2024-09-15T00:12:18.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060970119999999997, 2024-09-15T00:12:18.073 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.073 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.073 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:18.073 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.504060+0000", 2024-09-15T00:12:18.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.504060+0000", 2024-09-15T00:12:18.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.504061+0000", 2024-09-15T00:12:18.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3200000000000005e-07 2024-09-15T00:12:18.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505702+0000", 2024-09-15T00:12:18.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016412149999999999 2024-09-15T00:12:18.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505724+0000", 2024-09-15T00:12:18.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1900999999999999e-05 2024-09-15T00:12:18.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505743+0000", 2024-09-15T00:12:18.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8529000000000002e-05 2024-09-15T00:12:18.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505840+0000", 2024-09-15T00:12:18.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7533000000000003e-05 2024-09-15T00:12:18.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505854+0000", 2024-09-15T00:12:18.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4070000000000001e-05 2024-09-15T00:12:18.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.510157+0000", 2024-09-15T00:12:18.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043031320000000003 2024-09-15T00:12:18.077 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.077 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.077 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.077 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [2097152~420557,2629632~4096,2650112~4096,2670592~4096,2691072~4096,2711552~4096,2732032~4096,2756608~12288,2777088~12288,2797568~12288,2818048~12288,2838528~12288,2859008~12288,2879488~529715], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3409203, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:18.077 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:18.077 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.5248390240000003, 2024-09-15T00:12:18.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0071155660000000003, 2024-09-15T00:12:18.077 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.077 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.077 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:18.078 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:18.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:18.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513587+0000", 2024-09-15T00:12:18.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4899999999999995e-07 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514349+0000", 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00076184799999999997 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514368+0000", 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9507000000000001e-05 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514391+0000", 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2497000000000002e-05 2024-09-15T00:12:18.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514491+0000", 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9838000000000002e-05 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514500+0000", 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4860000000000003e-06 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.520702+0000", 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062017410000000002 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.080 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.081 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.081 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [3409203~654029,4194304~266240,4476928~4096,4497408~4096,4517888~4096,4538368~4096,4558848~4096,4579328~4096,4587520~103731], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4691251, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3409203, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:18.081 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:18.081 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.5139809470000003, 2024-09-15T00:12:18.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079084649999999999, 2024-09-15T00:12:18.081 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.081 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.081 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:18.081 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:18.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:18.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524446+0000", 2024-09-15T00:12:18.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.243e-06 2024-09-15T00:12:18.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525115+0000", 2024-09-15T00:12:18.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00066896999999999996 2024-09-15T00:12:18.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525135+0000", 2024-09-15T00:12:18.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0137999999999999e-05 2024-09-15T00:12:18.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525148+0000", 2024-09-15T00:12:18.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3591000000000001e-05 2024-09-15T00:12:18.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525242+0000", 2024-09-15T00:12:18.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3603999999999994e-05 2024-09-15T00:12:18.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525258+0000", 2024-09-15T00:12:18.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6150999999999998e-05 2024-09-15T00:12:18.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.532353+0000", 2024-09-15T00:12:18.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0070947680000000004 2024-09-15T00:12:18.084 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.084 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.084 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.085 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [4691251~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5739827, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4691251, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:18.085 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:18.085 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.5002973859999997, 2024-09-15T00:12:18.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0085554080000000005, 2024-09-15T00:12:18.085 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.085 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.085 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:18.085 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:18.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:18.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538129+0000", 2024-09-15T00:12:18.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:12:18.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539029+0000", 2024-09-15T00:12:18.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090036599999999999 2024-09-15T00:12:18.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539047+0000", 2024-09-15T00:12:18.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7776999999999999e-05 2024-09-15T00:12:18.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539058+0000", 2024-09-15T00:12:18.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1070000000000001e-05 2024-09-15T00:12:18.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539128+0000", 2024-09-15T00:12:18.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0531999999999995e-05 2024-09-15T00:12:18.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539138+0000", 2024-09-15T00:12:18.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3910000000000004e-06 2024-09-15T00:12:18.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.546683+0000", 2024-09-15T00:12:18.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075455519999999996 2024-09-15T00:12:18.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.088 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.089 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:6618a9b3:::rbd_data.113947ee3a89.0000000000000009:15, version: 59'176, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:6618a9b3:::rbd_data.113947ee3a89.0000000000000009:15@59'176, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:18.089 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:18.089 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.4309795899999997, 2024-09-15T00:12:18.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073968319999999999, 2024-09-15T00:12:18.089 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.089 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.089 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:18.089 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:18.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:18.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607447+0000", 2024-09-15T00:12:18.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2100000000000001e-06 2024-09-15T00:12:18.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607778+0000", 2024-09-15T00:12:18.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033101199999999999 2024-09-15T00:12:18.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607800+0000", 2024-09-15T00:12:18.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2323e-05 2024-09-15T00:12:18.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607815+0000", 2024-09-15T00:12:18.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5073e-05 2024-09-15T00:12:18.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607926+0000", 2024-09-15T00:12:18.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110538 2024-09-15T00:12:18.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607942+0000", 2024-09-15T00:12:18.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5662999999999999e-05 2024-09-15T00:12:18.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.614843+0000", 2024-09-15T00:12:18.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0069010130000000001 2024-09-15T00:12:18.093 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.093 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.093 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.093 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:62885eaf:::rbd_data.1139e6d65aff.0000000000000005:head, version: 62'226, data_included: [0~262144,393216~131072,655360~655360], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:62885eaf:::rbd_data.1139e6d65aff.0000000000000005:head@62'226, size: 1572864, copy_subset: [(0, 1572864)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1310720, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:18.093 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.639003+0000", 2024-09-15T00:12:18.093 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.399422028, 2024-09-15T00:12:18.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0070761069999999999, 2024-09-15T00:12:18.093 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.093 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.094 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:18.094 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639003+0000", 2024-09-15T00:12:18.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639003+0000", 2024-09-15T00:12:18.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639004+0000", 2024-09-15T00:12:18.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:12:18.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639567+0000", 2024-09-15T00:12:18.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00056333800000000003 2024-09-15T00:12:18.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639583+0000", 2024-09-15T00:12:18.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5906999999999999e-05 2024-09-15T00:12:18.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639594+0000", 2024-09-15T00:12:18.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1093e-05 2024-09-15T00:12:18.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639667+0000", 2024-09-15T00:12:18.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3089000000000004e-05 2024-09-15T00:12:18.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639680+0000", 2024-09-15T00:12:18.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2689e-05 2024-09-15T00:12:18.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.646079+0000", 2024-09-15T00:12:18.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063994990000000003 2024-09-15T00:12:18.097 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.097 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.097 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.097 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1 66/65 [PushOp(3:8182271c:::rbd_data.1139e2e18583.000000000000000a:9, version: 37'34, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:8182271c:::rbd_data.1139e2e18583.000000000000000a:9@37'34, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:18.097 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.692362+0000", 2024-09-15T00:12:18.097 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.3460629530000006, 2024-09-15T00:12:18.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00552842, 2024-09-15T00:12:18.098 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.098 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.098 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:18.098 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.692362+0000", 2024-09-15T00:12:18.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.692362+0000", 2024-09-15T00:12:18.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.692363+0000", 2024-09-15T00:12:18.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6100000000000001e-07 2024-09-15T00:12:18.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.693304+0000", 2024-09-15T00:12:18.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00094152000000000005 2024-09-15T00:12:18.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.693324+0000", 2024-09-15T00:12:18.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9922999999999999e-05 2024-09-15T00:12:18.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.693336+0000", 2024-09-15T00:12:18.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1861e-05 2024-09-15T00:12:18.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.693402+0000", 2024-09-15T00:12:18.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5938999999999993e-05 2024-09-15T00:12:18.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.693411+0000", 2024-09-15T00:12:18.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0240000000000003e-06 2024-09-15T00:12:18.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.697891+0000", 2024-09-15T00:12:18.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044795920000000001 2024-09-15T00:12:18.101 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.101 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.101 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.101 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1 66/65 [PushOp(3:8182271c:::rbd_data.1139e2e18583.000000000000000a:9, version: 37'34, data_included: [1048576~499712,1744896~180224], data_size: 679936, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:8182271c:::rbd_data.1139e2e18583.000000000000000a:9@37'34, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1925120, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:18.101 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.698673+0000", 2024-09-15T00:12:18.101 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.3397525609999992, 2024-09-15T00:12:18.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060406629999999999, 2024-09-15T00:12:18.102 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.102 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.102 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:18.102 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.698673+0000", 2024-09-15T00:12:18.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.698673+0000", 2024-09-15T00:12:18.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.698674+0000", 2024-09-15T00:12:18.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5600000000000004e-07 2024-09-15T00:12:18.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699307+0000", 2024-09-15T00:12:18.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00063296200000000004 2024-09-15T00:12:18.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699324+0000", 2024-09-15T00:12:18.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7115000000000001e-05 2024-09-15T00:12:18.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699336+0000", 2024-09-15T00:12:18.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2047e-05 2024-09-15T00:12:18.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699428+0000", 2024-09-15T00:12:18.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2305000000000005e-05 2024-09-15T00:12:18.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699443+0000", 2024-09-15T00:12:18.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4386000000000001e-05 2024-09-15T00:12:18.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.704713+0000", 2024-09-15T00:12:18.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052708920000000001 2024-09-15T00:12:18.105 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.105 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.105 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.105 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1 66/65 [PushOp(3:83414961:::rbd_data.11392b39946e.0000000000000005:18, version: 62'186, data_included: [0~655360,5144177~35006], data_size: 690366, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:83414961:::rbd_data.11392b39946e.0000000000000005:18@62'186, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:18.106 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.781708+0000", 2024-09-15T00:12:18.106 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.2567170680000004, 2024-09-15T00:12:18.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0064762520000000001, 2024-09-15T00:12:18.106 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.106 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.106 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:18.106 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.781708+0000", 2024-09-15T00:12:18.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.781708+0000", 2024-09-15T00:12:18.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.781709+0000", 2024-09-15T00:12:18.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7000000000000004e-07 2024-09-15T00:12:18.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782362+0000", 2024-09-15T00:12:18.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00065286900000000004 2024-09-15T00:12:18.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782394+0000", 2024-09-15T00:12:18.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1767999999999998e-05 2024-09-15T00:12:18.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782416+0000", 2024-09-15T00:12:18.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2833e-05 2024-09-15T00:12:18.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782497+0000", 2024-09-15T00:12:18.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0706999999999998e-05 2024-09-15T00:12:18.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782507+0000", 2024-09-15T00:12:18.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3540000000000006e-06 2024-09-15T00:12:18.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.788185+0000", 2024-09-15T00:12:18.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056779509999999997 2024-09-15T00:12:18.109 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.109 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.109 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.109 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1 66/65 [PushOp(3:84afde92:::rbd_data.11392b39946e.0000000000000007:18, version: 62'188, data_included: [0~786432,4718224~368,6422528~38621], data_size: 825421, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:84afde92:::rbd_data.11392b39946e.0000000000000007:18@62'188, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:18.109 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.791890+0000", 2024-09-15T00:12:18.109 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.2465350050000001, 2024-09-15T00:12:18.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006572825, 2024-09-15T00:12:18.110 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.110 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.110 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:18.110 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.791890+0000", 2024-09-15T00:12:18.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.791890+0000", 2024-09-15T00:12:18.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.791892+0000", 2024-09-15T00:12:18.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9630000000000001e-06 2024-09-15T00:12:18.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792684+0000", 2024-09-15T00:12:18.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00079155399999999998 2024-09-15T00:12:18.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792736+0000", 2024-09-15T00:12:18.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2490000000000001e-05 2024-09-15T00:12:18.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792774+0000", 2024-09-15T00:12:18.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7358999999999999e-05 2024-09-15T00:12:18.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792889+0000", 2024-09-15T00:12:18.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114987 2024-09-15T00:12:18.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792902+0000", 2024-09-15T00:12:18.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3684e-05 2024-09-15T00:12:18.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.798463+0000", 2024-09-15T00:12:18.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055607879999999997 2024-09-15T00:12:18.113 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.113 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.113 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.113 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:104300 3.7 e70/67 3:e1fa70c7:::rbd_data.113978285a61.0000000000000089:head v 70'331, mlcod=70'329)", 2024-09-15T00:12:18.113 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:12.648189+0000", 2024-09-15T00:12:18.114 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.3902362019999996, 2024-09-15T00:12:18.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059675149999999996, 2024-09-15T00:12:18.114 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.114 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.114 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:18.114 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648189+0000", 2024-09-15T00:12:18.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648189+0000", 2024-09-15T00:12:18.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648190+0000", 2024-09-15T00:12:18.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7899999999999997e-07 2024-09-15T00:12:18.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648203+0000", 2024-09-15T00:12:18.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2622e-05 2024-09-15T00:12:18.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648214+0000", 2024-09-15T00:12:18.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1291000000000001e-05 2024-09-15T00:12:18.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648221+0000", 2024-09-15T00:12:18.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-15T00:12:18.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648306+0000", 2024-09-15T00:12:18.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5142999999999994e-05 2024-09-15T00:12:18.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648327+0000", 2024-09-15T00:12:18.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1270000000000001e-05 2024-09-15T00:12:18.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:18.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.654107+0000", 2024-09-15T00:12:18.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057798279999999999 2024-09-15T00:12:18.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.654157+0000", 2024-09-15T00:12:18.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9648999999999998e-05 2024-09-15T00:12:18.118 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.118 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.118 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.118 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:104297 3.1f e70/68 3:ffd85812:::rbd_data.113978285a61.0000000000000086:head v 70'310, mlcod=70'308)", 2024-09-15T00:12:18.118 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:12.648365+0000", 2024-09-15T00:12:18.118 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.3900604129999996, 2024-09-15T00:12:18.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057393929999999998, 2024-09-15T00:12:18.119 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.119 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.119 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:18.119 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648365+0000", 2024-09-15T00:12:18.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648365+0000", 2024-09-15T00:12:18.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648366+0000", 2024-09-15T00:12:18.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3099999999999997e-07 2024-09-15T00:12:18.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648380+0000", 2024-09-15T00:12:18.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4392999999999999e-05 2024-09-15T00:12:18.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648389+0000", 2024-09-15T00:12:18.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9090000000000006e-06 2024-09-15T00:12:18.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648395+0000", 2024-09-15T00:12:18.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0000000000000002e-06 2024-09-15T00:12:18.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648443+0000", 2024-09-15T00:12:18.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8087999999999998e-05 2024-09-15T00:12:18.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648460+0000", 2024-09-15T00:12:18.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7414999999999998e-05 2024-09-15T00:12:18.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:18.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.654069+0000", 2024-09-15T00:12:18.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005608498 2024-09-15T00:12:18.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.654104+0000", 2024-09-15T00:12:18.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5358999999999998e-05 2024-09-15T00:12:18.122 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.122 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.123 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.123 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:104302 3.12 e70/68 3:4fc50744:::rbd_data.113978285a61.000000000000008b:head v 70'332, mlcod=70'330)", 2024-09-15T00:12:18.123 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:12.648533+0000", 2024-09-15T00:12:18.123 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.3898922770000004, 2024-09-15T00:12:18.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056143149999999999, 2024-09-15T00:12:18.123 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:18.123 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:18.123 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:18.123 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:18.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:18.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648533+0000", 2024-09-15T00:12:18.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:18.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648533+0000", 2024-09-15T00:12:18.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:18.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:18.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648533+0000", 2024-09-15T00:12:18.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999998e-07 2024-09-15T00:12:18.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:18.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648543+0000", 2024-09-15T00:12:18.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8740000000000007e-06 2024-09-15T00:12:18.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:18.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648552+0000", 2024-09-15T00:12:18.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6240000000000008e-06 2024-09-15T00:12:18.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:18.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648558+0000", 2024-09-15T00:12:18.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5309999999999998e-06 2024-09-15T00:12:18.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:18.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648609+0000", 2024-09-15T00:12:18.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0643000000000003e-05 2024-09-15T00:12:18.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:18.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648626+0000", 2024-09-15T00:12:18.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7218e-05 2024-09-15T00:12:18.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:18.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.654103+0000", 2024-09-15T00:12:18.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054770599999999997 2024-09-15T00:12:18.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:18.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:18.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:18.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.654147+0000", 2024-09-15T00:12:18.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3983999999999998e-05 2024-09-15T00:12:18.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.127 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:18.127 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:18.127 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:18.128 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:18.129 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is set 2024-09-15T00:12:18.138 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:12:18.170 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone27 from 0xc034383 (overlap 0x408f92d) to 0x3a1ad0e 2024-09-15T00:12:18.170 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:18.167+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:18.233 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:18.230+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:18.288 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:18.289 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:18.289 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:18.289 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:18.298 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:18.308 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:18.308 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:18.329 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:18.330 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:18.334 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:18.331+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:18.336 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-15T00:12:18.356 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:12:18.356 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:12:18.399 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:18.400 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:18.430 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:18.431 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:18.435 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:18.432+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:18.450 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:18.450 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:18.455 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:18.455 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:18.460 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:18.462 INFO:teuthology.orchestra.run.smithi171.stdout:3828 trunc from 0xc034383 to 0x9f1d47b 2024-09-15T00:12:18.474 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:18.474 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:18.479 INFO:teuthology.orchestra.run.smithi171.stdout:3830 write 0x9098e1b thru 0x909eecd (0x60b3 bytes) 2024-09-15T00:12:18.481 INFO:teuthology.orchestra.run.smithi171.stdout:3833 punch from 0x96a64d1 to 0x96a686b, (0x39a bytes) 2024-09-15T00:12:18.481 INFO:teuthology.orchestra.run.smithi171.stdout:3834 write 0x1aa01bf thru 0x1aaaef4 (0xad36 bytes) 2024-09-15T00:12:18.492 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:18.492 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:18.536 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:18.533+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:18.556 INFO:teuthology.orchestra.run.smithi171.stdout:3835 trunc from 0x9f1d47b to 0x11f2420 2024-09-15T00:12:18.595 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:18.595 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:18.608 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:12:18.654 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:18.650+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:18.708 INFO:teuthology.orchestra.run.smithi171.stdout:3840 write 0x1dd06e5 thru 0x1ddc93e (0xc25a bytes) 2024-09-15T00:12:18.711 INFO:teuthology.orchestra.run.smithi171.stdout:3842 punch from 0xbd56e to 0xbd7b7, (0x249 bytes) 2024-09-15T00:12:18.711 INFO:teuthology.orchestra.run.smithi171.stdout:3843 writesame 0xb464d2 thru 0xb55630 data_size 0x2b(0xf15f bytes) 2024-09-15T00:12:18.713 INFO:teuthology.orchestra.run.smithi171.stdout:3844 read 0x1d822c0 thru 0x1d8cd0a (0xaa4b bytes) 2024-09-15T00:12:18.714 INFO:teuthology.orchestra.run.smithi171.stdout:3845 read 0x1162ef5 thru 0x11635d2 (0x6de bytes) 2024-09-15T00:12:18.714 INFO:teuthology.orchestra.run.smithi171.stdout:3846 read 0xd3a25f thru 0xd49e1f (0xfbc1 bytes) 2024-09-15T00:12:18.733 INFO:teuthology.orchestra.run.smithi171.stdout:3847 writesame 0xb58f6c5 thru 0xb596109 data_size 0x1(0x6a45 bytes) 2024-09-15T00:12:18.733 INFO:teuthology.orchestra.run.smithi171.stdout:3848 punch from 0x168605 to 0x16dece, (0x58c9 bytes) 2024-09-15T00:12:18.733 INFO:teuthology.orchestra.run.smithi171.stdout:3849 punch from 0x48c8d24 to 0x48ca377, (0x1653 bytes) 2024-09-15T00:12:18.738 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:18.735+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:18.746 INFO:teuthology.orchestra.run.smithi171.stdout:3850 trunc from 0xb59610a to 0x1f6a738 2024-09-15T00:12:18.753 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:18.753 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:18.758 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:18.758 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:18.765 INFO:teuthology.orchestra.run.smithi171.stdout:3853 write 0x6e4a8c7 thru 0x6e53d76 (0x94b0 bytes) 2024-09-15T00:12:18.768 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:12:18.768 INFO:teuthology.orchestra.run.smithi171.stdout:3854 read 0x1a3e2fd thru 0x1a488ce (0xa5d2 bytes) 2024-09-15T00:12:18.776 INFO:teuthology.orchestra.run.smithi171.stdout:3855 write 0x92602e0 thru 0x92657b8 (0x54d9 bytes) 2024-09-15T00:12:18.779 INFO:teuthology.orchestra.run.smithi171.stdout:3856 write 0x1a5501f thru 0x1a581f0 (0x31d2 bytes) 2024-09-15T00:12:18.790 INFO:teuthology.orchestra.run.smithi171.stdout:3857 writesame 0xd7dd4d9 thru 0xd7e1ec5 data_size 0x5(0x49ed bytes) 2024-09-15T00:12:18.791 INFO:teuthology.orchestra.run.smithi171.stdout:3858 writesame 0xa1314a4 thru 0xa136c8b data_size 0x3a(0x57e8 bytes) 2024-09-15T00:12:18.793 INFO:teuthology.orchestra.run.smithi171.stdout:3859 read 0x33a962e thru 0x33b6e94 (0xd867 bytes) 2024-09-15T00:12:18.793 INFO:teuthology.orchestra.run.smithi171.stdout:3860 punch from 0x1197b33 to 0x11a0163, (0x8630 bytes) 2024-09-15T00:12:18.793 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:18.802 INFO:teuthology.orchestra.run.smithi171.stdout:3862 trunc from 0xd7e1ec6 to 0x626150b 2024-09-15T00:12:18.812 INFO:teuthology.orchestra.run.smithi171.stdout:3863 read 0x2205ef1 thru 0x2206af5 (0xc05 bytes) 2024-09-15T00:12:18.812 INFO:teuthology.orchestra.run.smithi171.stdout:3864 writesame 0x23ba4e4 thru 0x23c86c8 data_size 0x1(0xe1e5 bytes) 2024-09-15T00:12:18.838 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:18.836+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:18.889 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:18.889 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:18.899 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:12:18.924 INFO:teuthology.orchestra.run.smithi171.stdout:3865 trunc from 0x626150b to 0xe51950f 2024-09-15T00:12:18.930 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:18.930 INFO:teuthology.orchestra.run.smithi171.stdout:3868 punch from 0x9f1972f to 0x9f1b581, (0x1e52 bytes) 2024-09-15T00:12:18.930 INFO:teuthology.orchestra.run.smithi171.stdout:3869 writesame 0x93f4711 thru 0x9403a2e data_size 0x42(0xf31e bytes) 2024-09-15T00:12:18.934 INFO:teuthology.orchestra.run.smithi171.stdout:3872 read 0x605e47b thru 0x606d675 (0xf1fb bytes) 2024-09-15T00:12:18.934 INFO:teuthology.orchestra.run.smithi171.stdout:3873 read 0x88e9b14 thru 0x88efadc (0x5fc9 bytes) 2024-09-15T00:12:18.935 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:18.935 INFO:teuthology.orchestra.run.smithi171.stdout:3875 read 0x40cf15e thru 0x40d500d (0x5eb0 bytes) 2024-09-15T00:12:18.939 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:18.937+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:18.948 INFO:teuthology.orchestra.run.smithi171.stdout:3878 trunc from 0xe51950f to 0x2a3c9d7 2024-09-15T00:12:18.974 INFO:teuthology.orchestra.run.smithi171.stdout:3879 writesame 0x992a9db thru 0x992bfeb data_size 0x7(0x1611 bytes) 2024-09-15T00:12:18.974 INFO:teuthology.orchestra.run.smithi171.stdout:3881 writesame 0x536143c thru 0x536f4e3 data_size 0x9e(0xe0a8 bytes) 2024-09-15T00:12:18.977 INFO:teuthology.orchestra.run.smithi171.stdout:3882 read 0x4ea954b thru 0x4ead8ee (0x43a4 bytes) 2024-09-15T00:12:18.977 INFO:teuthology.orchestra.run.smithi171.stdout:3886 write 0x70751e thru 0x70dbe0 (0x66c3 bytes) 2024-09-15T00:12:18.979 INFO:teuthology.orchestra.run.smithi171.stdout:3888 read 0x566ec3b thru 0x5675063 (0x6429 bytes) 2024-09-15T00:12:18.980 INFO:teuthology.orchestra.run.smithi171.stdout:3890 read 0x21f3f1 thru 0x222eca (0x3ada bytes) 2024-09-15T00:12:18.980 INFO:teuthology.orchestra.run.smithi171.stdout:3891 punch from 0xe81116 to 0xe88824, (0x770e bytes) 2024-09-15T00:12:18.981 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:18.984 INFO:teuthology.orchestra.run.smithi171.stdout:3894 trunc from 0x992bfec to 0xcebb777 2024-09-15T00:12:18.989 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:18.989 INFO:teuthology.orchestra.run.smithi171.stdout:3896 writesame 0x7be84f8 thru 0x7becf7a data_size 0x23(0x4a83 bytes) 2024-09-15T00:12:18.993 INFO:teuthology.orchestra.run.smithi171.stdout:3897 read 0xae7f38b thru 0xae7f87a (0x4f0 bytes) 2024-09-15T00:12:18.993 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:18.993 INFO:teuthology.orchestra.run.smithi171.stdout:3900 read 0x4bb6083 thru 0x4bbb145 (0x50c3 bytes) 2024-09-15T00:12:18.993 INFO:teuthology.orchestra.run.smithi171.stdout:3901 write 0x43785b8 thru 0x437b0a0 (0x2ae9 bytes) 2024-09-15T00:12:18.995 INFO:teuthology.orchestra.run.smithi171.stdout:3902 punch from 0x8d84b96 to 0x8d8e781, (0x9beb bytes) 2024-09-15T00:12:19.004 INFO:teuthology.orchestra.run.smithi171.stdout:3904 trunc from 0xcebb777 to 0x615abb9 2024-09-15T00:12:19.013 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:19.013 INFO:teuthology.orchestra.run.smithi171.stdout:3906 writesame 0x46baba2 thru 0x46bca40 data_size 0x3(0x1e9f bytes) 2024-09-15T00:12:19.016 INFO:teuthology.orchestra.run.smithi171.stdout:3908 write 0x50bce2c thru 0x50cb51c (0xe6f1 bytes) 2024-09-15T00:12:19.021 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:19.027 INFO:teuthology.orchestra.run.smithi171.stdout:3911 writesame 0x78972d9 thru 0x789b547 data_size 0x1(0x426f bytes) 2024-09-15T00:12:19.028 INFO:teuthology.orchestra.run.smithi171.stdout:3912 clone 29 order 20 su 8192 sc 15 2024-09-15T00:12:19.041 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:19.039+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:19.043 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:19.044 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:19.053 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:19.096 INFO:teuthology.orchestra.run.smithi171.stderr:set 3.9 pg_upmap mapping to [3,0] 2024-09-15T00:12:19.153 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:19.150+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:19.201 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:19.201 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:19.211 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:19.243 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:19.240+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:19.343 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:19.341+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:19.355 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:19.355 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:19.363 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:19.444 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:19.442+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:19.507 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:19.507 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:19.515 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:19.545 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:19.543+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:19.620 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:19.620 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:19.629 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:19.673 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:19.670+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:19.731 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:19.731 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:19.739 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:19.781 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:19.778+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:19.848 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:19.846+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:19.883 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:19.883 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:19.892 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:19.949 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:19.946+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:19.995 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:19.995 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:20.004 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:20.088 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:20.085+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:20.130 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:12:20.148 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:20.148 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:20.157 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:20.174 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:20.172+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:20.251 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:20.248+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:20.302 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:20.303 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:20.311 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:20.353 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:20.350+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:20.453 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:20.451+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:20.455 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:20.455 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:20.463 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:20.506 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:12:20.554 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:20.551+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:20.566 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:20.566 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:20.575 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:20.627 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:20.627 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:20.643 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:20.644 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:20.656 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:20.654+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:20.660 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:12:20.660 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:12:20.676 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:20.676 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:20.693 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:20.693 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:20.709 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:20.709 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:20.720 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:20.720 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:20.726 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:20.726 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:20.729 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:20.743 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:20.743 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:20.772 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:20.770+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:20.831 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:20.831 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:20.831 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:20.831 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:20.840 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:20.881 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:20.878+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:20.961 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:20.958+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:20.982 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:20.983 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:20.983 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:20.983 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:20.983 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:20.991 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:21.005 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is set 2024-09-15T00:12:21.076 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:21.073+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:21.138 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:21.138 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:21.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:21.138 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:21.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.139 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:21.139 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:21.139 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.057347542999999, 2024-09-15T00:12:21.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:12:21.139 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.139 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.139 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:21.139 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.139 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.139 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.139 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:12:21.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.140 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:21.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:21.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:12:21.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:12:21.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:12:21.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:12:21.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:12:21.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:12:21.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:12:21.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:12:21.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:12:21.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:12:21.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:12:21.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:12:21.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:21.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:12:21.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:12:21.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:21.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:12:21.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:12:21.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:21.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:12:21.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:12:21.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:21.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:12:21.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:12:21.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:12:21.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:12:21.144 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.144 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.144 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.145 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:12:21.145 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:21.145 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.044607001000003, 2024-09-15T00:12:21.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:12:21.145 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.145 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.145 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:21.145 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:21.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:21.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:12:21.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:12:21.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:12:21.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:12:21.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:12:21.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:12:21.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:12:21.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:12:21.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:12:21.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:12:21.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:12:21.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:12:21.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:12:21.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:12:21.148 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.148 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.148 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.149 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:21.149 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:21.149 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 54.912251241999996, 2024-09-15T00:12:21.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:12:21.149 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.149 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.149 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:21.149 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.149 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.149 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.149 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:12:21.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.149 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:21.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:21.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:12:21.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:12:21.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:12:21.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:12:21.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:12:21.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:12:21.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:21.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:12:21.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:12:21.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:21.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:12:21.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:12:21.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:21.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:12:21.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:12:21.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:21.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:12:21.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:12:21.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:12:21.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:12:21.154 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.154 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.154 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.154 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:12:21.154 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:21.154 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 54.898789596999997, 2024-09-15T00:12:21.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:12:21.155 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.155 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.155 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:21.155 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:21.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:21.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:12:21.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:12:21.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:12:21.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:12:21.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:12:21.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:12:21.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:12:21.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:12:21.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:12:21.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:12:21.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:12:21.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:12:21.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:12:21.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:12:21.158 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.158 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.158 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.158 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:21.158 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:21.158 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.713788114000003, 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:21.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:12:21.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:12:21.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:12:21.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:12:21.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:12:21.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:12:21.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:12:21.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:12:21.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:12:21.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:12:21.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:12:21.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:12:21.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:21.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:12:21.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:12:21.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:21.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:12:21.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:12:21.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:21.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:12:21.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:12:21.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:21.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:12:21.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:12:21.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:12:21.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:12:21.164 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.164 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.164 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.164 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:12:21.164 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:21.164 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.700911847, 2024-09-15T00:12:21.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:12:21.164 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.164 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.164 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:21.164 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:21.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:21.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:12:21.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:12:21.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:12:21.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:12:21.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:12:21.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:12:21.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:12:21.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:12:21.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:12:21.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:12:21.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:12:21.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:12:21.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:12:21.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:12:21.168 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.168 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.168 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.168 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:12:21.168 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:21.168 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 40.242396702999997, 2024-09-15T00:12:21.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:12:21.168 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.168 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.168 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:21.169 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.169 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.169 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.169 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:12:21.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.169 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:21.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:21.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:12:21.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:21.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:12:21.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:12:21.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:12:21.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:12:21.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:12:21.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:12:21.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:12:21.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:12:21.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:12:21.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:12:21.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:21.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:12:21.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:12:21.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:21.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:12:21.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:12:21.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:21.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:12:21.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:12:21.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:21.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:12:21.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:12:21.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:12:21.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:12:21.173 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.173 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.174 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.174 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:21.174 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:21.174 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.524886143000003, 2024-09-15T00:12:21.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:12:21.174 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.174 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.174 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:21.174 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.174 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.174 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.175 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:12:21.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.175 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:21.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:21.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:12:21.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:12:21.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:12:21.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:12:21.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:12:21.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:12:21.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:12:21.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:12:21.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:12:21.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:12:21.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:12:21.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:12:21.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:12:21.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:12:21.178 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.178 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.178 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.178 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:21.178 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:21.178 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.524867534000002, 2024-09-15T00:12:21.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:12:21.179 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.179 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.179 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:21.179 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.179 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.179 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.179 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:12:21.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.179 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:21.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:21.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:21.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:21.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:12:21.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:12:21.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:12:21.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:12:21.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:12:21.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:12:21.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:12:21.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:12:21.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:12:21.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:12:21.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:12:21.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:12:21.182 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.182 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.182 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.183 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:21.183 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:21.183 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.524842210000003, 2024-09-15T00:12:21.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:12:21.183 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.183 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.183 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:21.183 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.183 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.183 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.183 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:12:21.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.183 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:21.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:21.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:12:21.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:12:21.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:12:21.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:12:21.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:12:21.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:12:21.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:12:21.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:12:21.187 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.187 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.187 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.187 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:21.187 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:21.187 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.524823886, 2024-09-15T00:12:21.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:12:21.187 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.187 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.187 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:21.187 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.187 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.188 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.188 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:12:21.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.188 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.188 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:21.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.188 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:21.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:12:21.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:12:21.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:12:21.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:12:21.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:12:21.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:21.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:12:21.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:12:21.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:12:21.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:12:21.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:12:21.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:12:21.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:12:21.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:12:21.191 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.191 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.191 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.191 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:21.191 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:21.191 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.524641313000004, 2024-09-15T00:12:21.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:12:21.192 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.192 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.192 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:21.192 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.192 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.192 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.192 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:12:21.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.192 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.192 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.192 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:21.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.193 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:21.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.193 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:21.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:12:21.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.193 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:12:21.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:12:21.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:12:21.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:12:21.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:12:21.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:21.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:12:21.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:12:21.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:12:21.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:12:21.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:12:21.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:12:21.195 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.195 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.196 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.196 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:21.196 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:21.196 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.524612472000001, 2024-09-15T00:12:21.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:12:21.196 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.196 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.196 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:21.196 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.196 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.196 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:12:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:12:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:12:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:12:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:12:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:12:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:12:21.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:12:21.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:12:21.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:12:21.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:12:21.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:12:21.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:12:21.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:12:21.200 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.200 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.200 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.200 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:21.200 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:21.200 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.524590261, 2024-09-15T00:12:21.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:12:21.201 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.201 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.201 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:21.201 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.201 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.201 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.201 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:12:21.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.201 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:21.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:21.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:21.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:12:21.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:12:21.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:12:21.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:12:21.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:12:21.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:12:21.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:21.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:12:21.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:12:21.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:12:21.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:12:21.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:12:21.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:12:21.204 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.205 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.205 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.205 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:21.205 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:21.205 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.524569102000001, 2024-09-15T00:12:21.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:12:21.205 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.205 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.205 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:21.205 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.205 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.205 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.206 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:12:21.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.206 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:21.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:21.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:21.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:21.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:12:21.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:12:21.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:12:21.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:12:21.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:12:21.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:12:21.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:12:21.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:12:21.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:12:21.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:12:21.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:12:21.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:12:21.209 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.209 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.209 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.209 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:21.209 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:21.209 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.524539154000003, 2024-09-15T00:12:21.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:12:21.209 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.210 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.210 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:21.210 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.210 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.210 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.210 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:12:21.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.210 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:21.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:21.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:21.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:12:21.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:12:21.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:12:21.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:12:21.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:12:21.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:12:21.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:12:21.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:12:21.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:12:21.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:12:21.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:12:21.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:12:21.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:12:21.213 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.213 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.213 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.214 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:21.214 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:21.214 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.524517189999997, 2024-09-15T00:12:21.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:12:21.214 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.214 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.214 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:21.214 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.214 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.214 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.214 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:12:21.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.214 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:21.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:21.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:21.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:12:21.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:12:21.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:12:21.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:12:21.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:12:21.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:12:21.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:12:21.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:12:21.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:12:21.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:12:21.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:12:21.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:12:21.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:12:21.218 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.218 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.218 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.218 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:21.218 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:21.218 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.524499763000001, 2024-09-15T00:12:21.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:12:21.218 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.218 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.218 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:21.218 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.219 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.219 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.219 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:12:21.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.219 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:21.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:21.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:12:21.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:12:21.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:12:21.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:12:21.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:12:21.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:21.221 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:12:21.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:12:21.221 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:12:21.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:12:21.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:12:21.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:12:21.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:12:21.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:12:21.222 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.222 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.222 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.223 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:21.223 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:21.223 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.524477963999999, 2024-09-15T00:12:21.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:12:21.223 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.223 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.223 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:21.223 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.223 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.223 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.224 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:12:21.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.224 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:21.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.225 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:21.225 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.225 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.225 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.225 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:12:21.225 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:12:21.225 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.225 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.225 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:12:21.225 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:12:21.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:12:21.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:12:21.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:12:21.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:12:21.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:12:21.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:12:21.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:12:21.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:12:21.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:12:21.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:12:21.227 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.228 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.228 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.228 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:21.228 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:21.228 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.475249605000002, 2024-09-15T00:12:21.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:12:21.228 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:21.228 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:21.228 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:21.228 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:21.228 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:21.228 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:21.229 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:12:21.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.229 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:21.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:21.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:21.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:21.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:21.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:21.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:21.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:12:21.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:12:21.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:21.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:12:21.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:12:21.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:21.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:12:21.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:12:21.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:12:21.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:12:21.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:21.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:12:21.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:12:21.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:21.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:12:21.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:12:21.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:21.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:12:21.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:12:21.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:21.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:12:21.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:12:21.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:21.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:12:21.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:12:21.233 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.233 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:21.233 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:12:21.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:12:21.233 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:21.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:21.233 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:21.233 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:12:21.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:12:21.233 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.233 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.233 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.233 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:21.234 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:21.234 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:21.235 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:21.236 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:21.232+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:21.312 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone28 from 0x789b548 (overlap 0x11f2420) to 0x0 2024-09-15T00:12:21.335 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:21.333+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:21.362 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:21.362 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:21.373 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:21.437 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:21.434+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:21.459 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-15T00:12:21.520 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:21.520 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:21.520 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:21.521 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:21.521 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:21.534 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:21.537 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:21.535+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:21.638 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:21.636+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:21.681 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:21.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:70 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-09-15T00:12:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:12:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 76.142397367000001, 2024-09-15T00:12:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057871490000000001, 2024-09-15T00:12:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:12:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536109+0000", 2024-09-15T00:12:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536110+0000", 2024-09-15T00:12:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-15T00:12:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536115+0000", 2024-09-15T00:12:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8960000000000002e-06 2024-09-15T00:12:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536123+0000", 2024-09-15T00:12:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3979999999999999e-06 2024-09-15T00:12:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.536128+0000", 2024-09-15T00:12:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5569999999999997e-06 2024-09-15T00:12:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537011+0000", 2024-09-15T00:12:21.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088276299999999995 2024-09-15T00:12:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.537034+0000", 2024-09-15T00:12:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3448999999999999e-05 2024-09-15T00:12:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541867+0000", 2024-09-15T00:12:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048332949999999996 2024-09-15T00:12:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:05.541896+0000", 2024-09-15T00:12:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9179999999999998e-05 2024-09-15T00:12:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-15T00:12:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.153809269000007, 2024-09-15T00:12:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077021529999999998, 2024-09-15T00:12:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524698+0000", 2024-09-15T00:12:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6300000000000001e-07 2024-09-15T00:12:21.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524778+0000", 2024-09-15T00:12:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0598000000000003e-05 2024-09-15T00:12:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524785+0000", 2024-09-15T00:12:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6359999999999997e-06 2024-09-15T00:12:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524788+0000", 2024-09-15T00:12:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2540000000000001e-06 2024-09-15T00:12:21.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524817+0000", 2024-09-15T00:12:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8784000000000001e-05 2024-09-15T00:12:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524828+0000", 2024-09-15T00:12:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-15T00:12:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532360+0000", 2024-09-15T00:12:21.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007531757 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532400+0000", 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9835000000000001e-05 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=16'106)", 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.152867115999996, 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067923050000000002, 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.690 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-15T00:12:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525644+0000", 2024-09-15T00:12:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.709e-06 2024-09-15T00:12:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525651+0000", 2024-09-15T00:12:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7970000000000003e-06 2024-09-15T00:12:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525654+0000", 2024-09-15T00:12:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.433e-06 2024-09-15T00:12:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525685+0000", 2024-09-15T00:12:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0970000000000003e-05 2024-09-15T00:12:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525697+0000", 2024-09-15T00:12:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1759e-05 2024-09-15T00:12:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532401+0000", 2024-09-15T00:12:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006704425 2024-09-15T00:12:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532432+0000", 2024-09-15T00:12:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0709e-05 2024-09-15T00:12:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'106)", 2024-09-15T00:12:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:21.694 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.152404861999997, 2024-09-15T00:12:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063539190000000004, 2024-09-15T00:12:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:21.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:12:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526107+0000", 2024-09-15T00:12:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:12:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526117+0000", 2024-09-15T00:12:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7750000000000004e-06 2024-09-15T00:12:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526121+0000", 2024-09-15T00:12:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:12:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526163+0000", 2024-09-15T00:12:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1360999999999999e-05 2024-09-15T00:12:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526175+0000", 2024-09-15T00:12:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2729e-05 2024-09-15T00:12:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532433+0000", 2024-09-15T00:12:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062576999999999997 2024-09-15T00:12:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532456+0000", 2024-09-15T00:12:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2773e-05 2024-09-15T00:12:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.698 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'106)", 2024-09-15T00:12:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:12:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.151935460000004, 2024-09-15T00:12:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059082689999999998, 2024-09-15T00:12:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:12:21.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526571+0000", 2024-09-15T00:12:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526572+0000", 2024-09-15T00:12:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9500000000000003e-07 2024-09-15T00:12:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526641+0000", 2024-09-15T00:12:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8858999999999999e-05 2024-09-15T00:12:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526648+0000", 2024-09-15T00:12:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:12:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526654+0000", 2024-09-15T00:12:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7509999999999997e-06 2024-09-15T00:12:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526689+0000", 2024-09-15T00:12:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5800000000000003e-05 2024-09-15T00:12:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526700+0000", 2024-09-15T00:12:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0467e-05 2024-09-15T00:12:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532457+0000", 2024-09-15T00:12:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057568180000000004 2024-09-15T00:12:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532479+0000", 2024-09-15T00:12:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2739e-05 2024-09-15T00:12:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.504060+0000", 2024-09-15T00:12:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.174446423999999, 2024-09-15T00:12:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060970119999999997, 2024-09-15T00:12:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.703 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.504060+0000", 2024-09-15T00:12:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.504060+0000", 2024-09-15T00:12:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.504061+0000", 2024-09-15T00:12:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3200000000000005e-07 2024-09-15T00:12:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505702+0000", 2024-09-15T00:12:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016412149999999999 2024-09-15T00:12:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505724+0000", 2024-09-15T00:12:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1900999999999999e-05 2024-09-15T00:12:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505743+0000", 2024-09-15T00:12:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8529000000000002e-05 2024-09-15T00:12:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505840+0000", 2024-09-15T00:12:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7533000000000003e-05 2024-09-15T00:12:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505854+0000", 2024-09-15T00:12:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4070000000000001e-05 2024-09-15T00:12:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.510157+0000", 2024-09-15T00:12:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043031320000000003 2024-09-15T00:12:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [2097152~420557,2629632~4096,2650112~4096,2670592~4096,2691072~4096,2711552~4096,2732032~4096,2756608~12288,2777088~12288,2797568~12288,2818048~12288,2838528~12288,2859008~12288,2879488~529715], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3409203, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.164920386, 2024-09-15T00:12:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0071155660000000003, 2024-09-15T00:12:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.707 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513587+0000", 2024-09-15T00:12:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4899999999999995e-07 2024-09-15T00:12:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514349+0000", 2024-09-15T00:12:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00076184799999999997 2024-09-15T00:12:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514368+0000", 2024-09-15T00:12:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9507000000000001e-05 2024-09-15T00:12:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514391+0000", 2024-09-15T00:12:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2497000000000002e-05 2024-09-15T00:12:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514491+0000", 2024-09-15T00:12:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9838000000000002e-05 2024-09-15T00:12:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514500+0000", 2024-09-15T00:12:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4860000000000003e-06 2024-09-15T00:12:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.520702+0000", 2024-09-15T00:12:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062017410000000002 2024-09-15T00:12:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [3409203~654029,4194304~266240,4476928~4096,4497408~4096,4517888~4096,4538368~4096,4558848~4096,4579328~4096,4587520~103731], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4691251, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3409203, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.154062309, 2024-09-15T00:12:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079084649999999999, 2024-09-15T00:12:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524446+0000", 2024-09-15T00:12:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.243e-06 2024-09-15T00:12:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525115+0000", 2024-09-15T00:12:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00066896999999999996 2024-09-15T00:12:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525135+0000", 2024-09-15T00:12:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0137999999999999e-05 2024-09-15T00:12:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525148+0000", 2024-09-15T00:12:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3591000000000001e-05 2024-09-15T00:12:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525242+0000", 2024-09-15T00:12:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3603999999999994e-05 2024-09-15T00:12:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525258+0000", 2024-09-15T00:12:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6150999999999998e-05 2024-09-15T00:12:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.532353+0000", 2024-09-15T00:12:21.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0070947680000000004 2024-09-15T00:12:21.715 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.715 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.715 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.715 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [4691251~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5739827, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4691251, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:21.715 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:21.715 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.140378748, 2024-09-15T00:12:21.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0085554080000000005, 2024-09-15T00:12:21.715 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.715 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.715 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:21.715 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:21.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:21.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538129+0000", 2024-09-15T00:12:21.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:12:21.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539029+0000", 2024-09-15T00:12:21.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090036599999999999 2024-09-15T00:12:21.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539047+0000", 2024-09-15T00:12:21.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7776999999999999e-05 2024-09-15T00:12:21.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539058+0000", 2024-09-15T00:12:21.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1070000000000001e-05 2024-09-15T00:12:21.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539128+0000", 2024-09-15T00:12:21.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0531999999999995e-05 2024-09-15T00:12:21.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539138+0000", 2024-09-15T00:12:21.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3910000000000004e-06 2024-09-15T00:12:21.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.546683+0000", 2024-09-15T00:12:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075455519999999996 2024-09-15T00:12:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:6618a9b3:::rbd_data.113947ee3a89.0000000000000009:15, version: 59'176, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:6618a9b3:::rbd_data.113947ee3a89.0000000000000009:15@59'176, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.071060952, 2024-09-15T00:12:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073968319999999999, 2024-09-15T00:12:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.719 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:21.720 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:21.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:21.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607447+0000", 2024-09-15T00:12:21.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2100000000000001e-06 2024-09-15T00:12:21.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607778+0000", 2024-09-15T00:12:21.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033101199999999999 2024-09-15T00:12:21.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607800+0000", 2024-09-15T00:12:21.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2323e-05 2024-09-15T00:12:21.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607815+0000", 2024-09-15T00:12:21.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5073e-05 2024-09-15T00:12:21.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607926+0000", 2024-09-15T00:12:21.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110538 2024-09-15T00:12:21.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607942+0000", 2024-09-15T00:12:21.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5662999999999999e-05 2024-09-15T00:12:21.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.614843+0000", 2024-09-15T00:12:21.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0069010130000000001 2024-09-15T00:12:21.723 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.723 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.723 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.723 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:62885eaf:::rbd_data.1139e6d65aff.0000000000000005:head, version: 62'226, data_included: [0~262144,393216~131072,655360~655360], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:62885eaf:::rbd_data.1139e6d65aff.0000000000000005:head@62'226, size: 1572864, copy_subset: [(0, 1572864)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1310720, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:21.723 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.639003+0000", 2024-09-15T00:12:21.723 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.03950339, 2024-09-15T00:12:21.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0070761069999999999, 2024-09-15T00:12:21.723 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.723 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.723 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:21.723 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639003+0000", 2024-09-15T00:12:21.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639003+0000", 2024-09-15T00:12:21.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639004+0000", 2024-09-15T00:12:21.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:12:21.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639567+0000", 2024-09-15T00:12:21.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00056333800000000003 2024-09-15T00:12:21.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639583+0000", 2024-09-15T00:12:21.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5906999999999999e-05 2024-09-15T00:12:21.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639594+0000", 2024-09-15T00:12:21.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1093e-05 2024-09-15T00:12:21.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639667+0000", 2024-09-15T00:12:21.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3089000000000004e-05 2024-09-15T00:12:21.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639680+0000", 2024-09-15T00:12:21.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2689e-05 2024-09-15T00:12:21.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.646079+0000", 2024-09-15T00:12:21.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063994990000000003 2024-09-15T00:12:21.727 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.727 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.727 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.727 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1 66/65 [PushOp(3:8182271c:::rbd_data.1139e2e18583.000000000000000a:9, version: 37'34, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:8182271c:::rbd_data.1139e2e18583.000000000000000a:9@37'34, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:21.727 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.692362+0000", 2024-09-15T00:12:21.727 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.986144315000001, 2024-09-15T00:12:21.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00552842, 2024-09-15T00:12:21.727 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.727 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.727 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:21.727 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.692362+0000", 2024-09-15T00:12:21.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.692362+0000", 2024-09-15T00:12:21.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.692363+0000", 2024-09-15T00:12:21.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6100000000000001e-07 2024-09-15T00:12:21.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.693304+0000", 2024-09-15T00:12:21.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00094152000000000005 2024-09-15T00:12:21.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.693324+0000", 2024-09-15T00:12:21.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9922999999999999e-05 2024-09-15T00:12:21.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.693336+0000", 2024-09-15T00:12:21.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1861e-05 2024-09-15T00:12:21.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.693402+0000", 2024-09-15T00:12:21.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5938999999999993e-05 2024-09-15T00:12:21.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.693411+0000", 2024-09-15T00:12:21.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0240000000000003e-06 2024-09-15T00:12:21.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.697891+0000", 2024-09-15T00:12:21.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044795920000000001 2024-09-15T00:12:21.731 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.731 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.731 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.731 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1 66/65 [PushOp(3:8182271c:::rbd_data.1139e2e18583.000000000000000a:9, version: 37'34, data_included: [1048576~499712,1744896~180224], data_size: 679936, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:8182271c:::rbd_data.1139e2e18583.000000000000000a:9@37'34, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1925120, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:21.731 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.698673+0000", 2024-09-15T00:12:21.731 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.979833922999999, 2024-09-15T00:12:21.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060406629999999999, 2024-09-15T00:12:21.731 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.731 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.731 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:21.731 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.698673+0000", 2024-09-15T00:12:21.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.698673+0000", 2024-09-15T00:12:21.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.698674+0000", 2024-09-15T00:12:21.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5600000000000004e-07 2024-09-15T00:12:21.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699307+0000", 2024-09-15T00:12:21.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00063296200000000004 2024-09-15T00:12:21.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699324+0000", 2024-09-15T00:12:21.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7115000000000001e-05 2024-09-15T00:12:21.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699336+0000", 2024-09-15T00:12:21.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2047e-05 2024-09-15T00:12:21.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699428+0000", 2024-09-15T00:12:21.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2305000000000005e-05 2024-09-15T00:12:21.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699443+0000", 2024-09-15T00:12:21.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4386000000000001e-05 2024-09-15T00:12:21.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.704713+0000", 2024-09-15T00:12:21.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052708920000000001 2024-09-15T00:12:21.734 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.735 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.735 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.735 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1 66/65 [PushOp(3:83414961:::rbd_data.11392b39946e.0000000000000005:18, version: 62'186, data_included: [0~655360,5144177~35006], data_size: 690366, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:83414961:::rbd_data.11392b39946e.0000000000000005:18@62'186, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:21.735 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.781708+0000", 2024-09-15T00:12:21.735 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.89679843, 2024-09-15T00:12:21.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0064762520000000001, 2024-09-15T00:12:21.735 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.735 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.735 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:21.735 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.781708+0000", 2024-09-15T00:12:21.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.781708+0000", 2024-09-15T00:12:21.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.781709+0000", 2024-09-15T00:12:21.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7000000000000004e-07 2024-09-15T00:12:21.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782362+0000", 2024-09-15T00:12:21.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00065286900000000004 2024-09-15T00:12:21.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782394+0000", 2024-09-15T00:12:21.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1767999999999998e-05 2024-09-15T00:12:21.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782416+0000", 2024-09-15T00:12:21.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2833e-05 2024-09-15T00:12:21.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782497+0000", 2024-09-15T00:12:21.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0706999999999998e-05 2024-09-15T00:12:21.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782507+0000", 2024-09-15T00:12:21.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3540000000000006e-06 2024-09-15T00:12:21.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.788185+0000", 2024-09-15T00:12:21.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056779509999999997 2024-09-15T00:12:21.739 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.739 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.739 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.739 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1 66/65 [PushOp(3:84afde92:::rbd_data.11392b39946e.0000000000000007:18, version: 62'188, data_included: [0~786432,4718224~368,6422528~38621], data_size: 825421, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:84afde92:::rbd_data.11392b39946e.0000000000000007:18@62'188, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:21.739 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.791890+0000", 2024-09-15T00:12:21.739 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.886616367, 2024-09-15T00:12:21.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006572825, 2024-09-15T00:12:21.739 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.739 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.739 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:21.740 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.791890+0000", 2024-09-15T00:12:21.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.791890+0000", 2024-09-15T00:12:21.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.791892+0000", 2024-09-15T00:12:21.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9630000000000001e-06 2024-09-15T00:12:21.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792684+0000", 2024-09-15T00:12:21.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00079155399999999998 2024-09-15T00:12:21.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792736+0000", 2024-09-15T00:12:21.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2490000000000001e-05 2024-09-15T00:12:21.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792774+0000", 2024-09-15T00:12:21.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7358999999999999e-05 2024-09-15T00:12:21.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792889+0000", 2024-09-15T00:12:21.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114987 2024-09-15T00:12:21.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792902+0000", 2024-09-15T00:12:21.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3684e-05 2024-09-15T00:12:21.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.798463+0000", 2024-09-15T00:12:21.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055607879999999997 2024-09-15T00:12:21.743 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.743 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.743 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.743 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:104300 3.7 e70/67 3:e1fa70c7:::rbd_data.113978285a61.0000000000000089:head v 70'331, mlcod=70'329)", 2024-09-15T00:12:21.743 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:12.648189+0000", 2024-09-15T00:12:21.743 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.0303175640000006, 2024-09-15T00:12:21.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059675149999999996, 2024-09-15T00:12:21.743 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.743 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.743 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:21.744 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648189+0000", 2024-09-15T00:12:21.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648189+0000", 2024-09-15T00:12:21.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648190+0000", 2024-09-15T00:12:21.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7899999999999997e-07 2024-09-15T00:12:21.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648203+0000", 2024-09-15T00:12:21.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2622e-05 2024-09-15T00:12:21.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648214+0000", 2024-09-15T00:12:21.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1291000000000001e-05 2024-09-15T00:12:21.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648221+0000", 2024-09-15T00:12:21.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9330000000000004e-06 2024-09-15T00:12:21.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648306+0000", 2024-09-15T00:12:21.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5142999999999994e-05 2024-09-15T00:12:21.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648327+0000", 2024-09-15T00:12:21.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1270000000000001e-05 2024-09-15T00:12:21.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:21.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.654107+0000", 2024-09-15T00:12:21.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057798279999999999 2024-09-15T00:12:21.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.654157+0000", 2024-09-15T00:12:21.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9648999999999998e-05 2024-09-15T00:12:21.747 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.747 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.747 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.747 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:104297 3.1f e70/68 3:ffd85812:::rbd_data.113978285a61.0000000000000086:head v 70'310, mlcod=70'308)", 2024-09-15T00:12:21.747 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:12.648365+0000", 2024-09-15T00:12:21.747 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.0301417750000006, 2024-09-15T00:12:21.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057393929999999998, 2024-09-15T00:12:21.748 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.748 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.748 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:21.748 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648365+0000", 2024-09-15T00:12:21.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648365+0000", 2024-09-15T00:12:21.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648366+0000", 2024-09-15T00:12:21.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3099999999999997e-07 2024-09-15T00:12:21.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648380+0000", 2024-09-15T00:12:21.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4392999999999999e-05 2024-09-15T00:12:21.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648389+0000", 2024-09-15T00:12:21.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9090000000000006e-06 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648395+0000", 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0000000000000002e-06 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648443+0000", 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8087999999999998e-05 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648460+0000", 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7414999999999998e-05 2024-09-15T00:12:21.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:21.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.654069+0000", 2024-09-15T00:12:21.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005608498 2024-09-15T00:12:21.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.654104+0000", 2024-09-15T00:12:21.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5358999999999998e-05 2024-09-15T00:12:21.751 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.751 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.751 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.751 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:104302 3.12 e70/68 3:4fc50744:::rbd_data.113978285a61.000000000000008b:head v 70'332, mlcod=70'330)", 2024-09-15T00:12:21.752 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:12.648533+0000", 2024-09-15T00:12:21.752 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.0299736389999996, 2024-09-15T00:12:21.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056143149999999999, 2024-09-15T00:12:21.752 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.752 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.752 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:21.752 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648533+0000", 2024-09-15T00:12:21.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648533+0000", 2024-09-15T00:12:21.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648533+0000", 2024-09-15T00:12:21.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999998e-07 2024-09-15T00:12:21.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648543+0000", 2024-09-15T00:12:21.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8740000000000007e-06 2024-09-15T00:12:21.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648552+0000", 2024-09-15T00:12:21.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6240000000000008e-06 2024-09-15T00:12:21.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648558+0000", 2024-09-15T00:12:21.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5309999999999998e-06 2024-09-15T00:12:21.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648609+0000", 2024-09-15T00:12:21.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0643000000000003e-05 2024-09-15T00:12:21.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.648626+0000", 2024-09-15T00:12:21.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7218e-05 2024-09-15T00:12:21.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:21.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.654103+0000", 2024-09-15T00:12:21.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054770599999999997 2024-09-15T00:12:21.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:12.654147+0000", 2024-09-15T00:12:21.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3983999999999998e-05 2024-09-15T00:12:21.755 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.756 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.756 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.756 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.9 77/76 [PushOp(3:9229c9d9:::rbd_data.1139766160da.0000000000000000:13, version: 57'689, data_included: [0~65536,229376~65536,385024~5116,458752~32768,589824~131072,851968~131072,1081344~32768,1146880~65536,1310720~32768,1409024~32768,1507328~49152,1572864~32768,1638400~32768,1802240~32768], data_size: 742396, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:9229c9d9:::rbd_data.1139766160da.0000000000000000:13@57'689, size: 1835008, copy_subset: [(0, 1835008)], clone_subset: {}, snapset: 13=[]:{13: [13]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1835008, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:21.756 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.386155+0000", 2024-09-15T00:12:21.756 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.29235214100000001, 2024-09-15T00:12:21.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062156169999999997, 2024-09-15T00:12:21.756 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.756 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.756 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:21.756 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386155+0000", 2024-09-15T00:12:21.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386155+0000", 2024-09-15T00:12:21.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386155+0000", 2024-09-15T00:12:21.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0999999999999997e-07 2024-09-15T00:12:21.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386629+0000", 2024-09-15T00:12:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047386200000000002 2024-09-15T00:12:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386660+0000", 2024-09-15T00:12:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0834999999999999e-05 2024-09-15T00:12:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386686+0000", 2024-09-15T00:12:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5488999999999999e-05 2024-09-15T00:12:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386804+0000", 2024-09-15T00:12:21.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011882700000000001 2024-09-15T00:12:21.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386820+0000", 2024-09-15T00:12:21.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5330000000000001e-05 2024-09-15T00:12:21.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.392370+0000", 2024-09-15T00:12:21.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055503640000000003 2024-09-15T00:12:21.759 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.759 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.760 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.760 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.9 77/76 [PushOp(3:95907793:::rbd_data.11396cb1f0e4.0000000000000007:head, version: 59'729, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:95907793:::rbd_data.11396cb1f0e4.0000000000000007:head@59'729, size: 5701632, copy_subset: [(0, 5701632)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:21.760 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.428602+0000", 2024-09-15T00:12:21.760 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.24990512300000001, 2024-09-15T00:12:21.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006605606, 2024-09-15T00:12:21.760 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:21.760 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:21.760 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:21.760 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:21.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:21.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.428602+0000", 2024-09-15T00:12:21.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:21.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.428602+0000", 2024-09-15T00:12:21.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:21.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:21.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.428603+0000", 2024-09-15T00:12:21.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1000000000000001e-06 2024-09-15T00:12:21.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:21.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.429252+0000", 2024-09-15T00:12:21.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064978799999999999 2024-09-15T00:12:21.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:21.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.429282+0000", 2024-09-15T00:12:21.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9352e-05 2024-09-15T00:12:21.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:21.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.429301+0000", 2024-09-15T00:12:21.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9436000000000002e-05 2024-09-15T00:12:21.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:21.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.429400+0000", 2024-09-15T00:12:21.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8862000000000003e-05 2024-09-15T00:12:21.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:21.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.429415+0000", 2024-09-15T00:12:21.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5145e-05 2024-09-15T00:12:21.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:21.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:21.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:21.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.435207+0000", 2024-09-15T00:12:21.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005791923 2024-09-15T00:12:21.763 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.763 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.763 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.764 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:21.764 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:21.764 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:21.765 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:21.762+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:21.766 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:21.766 INFO:teuthology.orchestra.run.smithi171.stdout:3913 write 0xbd89b12 thru 0xbd93a54 (0x9f43 bytes) 2024-09-15T00:12:21.766 INFO:teuthology.orchestra.run.smithi171.stdout:3914 read 0x63caa98 thru 0x63cfde1 (0x534a bytes) 2024-09-15T00:12:21.766 INFO:teuthology.orchestra.run.smithi171.stdout:3916 writesame 0x422b9a thru 0x431a68 data_size 0x1(0xeecf bytes) 2024-09-15T00:12:21.865 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:21.863+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:21.885 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:21.886 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:21.895 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:21.912 INFO:teuthology.orchestra.run.smithi171.stdout:3917 writesame 0xd1260c8 thru 0xd12bbea data_size 0x4d(0x5b23 bytes) 2024-09-15T00:12:21.912 INFO:teuthology.orchestra.run.smithi171.stdout:3918 read 0xd019222 thru 0xd0231e0 (0x9fbf bytes) 2024-09-15T00:12:21.912 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:21.912 INFO:teuthology.orchestra.run.smithi171.stdout:3920 read 0x161c166 thru 0x162a70f (0xe5aa bytes) 2024-09-15T00:12:21.927 INFO:teuthology.orchestra.run.smithi171.stdout:3922 trunc from 0xd12bbeb to 0x1d61ed3 2024-09-15T00:12:21.966 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:21.963+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:22.038 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:22.038 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:22.038 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:22.038 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:22.038 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:22.047 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:22.053 INFO:teuthology.orchestra.run.smithi171.stdout:3923 write 0x528b0d thru 0x53182b (0x8d1f bytes) 2024-09-15T00:12:22.096 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:22.094+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:22.132 INFO:teuthology.orchestra.run.smithi171.stdout:3925 read 0x1c5dd52 thru 0x1c613a6 (0x3655 bytes) 2024-09-15T00:12:22.132 INFO:teuthology.orchestra.run.smithi171.stdout:3926 read 0x5e40d6 thru 0x5efc49 (0xbb74 bytes) 2024-09-15T00:12:22.152 INFO:teuthology.orchestra.run.smithi171.stdout:3927 write 0xc13a756 thru 0xc143e5a (0x9705 bytes) 2024-09-15T00:12:22.159 INFO:teuthology.orchestra.run.smithi171.stdout:3928 read 0x57a7333 thru 0x57b5f27 (0xebf5 bytes) 2024-09-15T00:12:22.159 INFO:teuthology.orchestra.run.smithi171.stdout:3929 punch from 0x58b0562 to 0x58beb6f, (0xe60d bytes) 2024-09-15T00:12:22.159 INFO:teuthology.orchestra.run.smithi171.stdout:3930 writesame 0xaca3eb6 thru 0xacb25b6 data_size 0x1(0xe701 bytes) 2024-09-15T00:12:22.167 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:22.165+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:22.178 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:22.178 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:22.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:22.178 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:22.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.178 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:22.178 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:22.179 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.995983764000002, 2024-09-15T00:12:22.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:12:22.179 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.179 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.179 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.179 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.179 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.179 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.179 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:12:22.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.179 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:12:22.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:12:22.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:12:22.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:12:22.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:12:22.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:12:22.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:12:22.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:12:22.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:12:22.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:12:22.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:12:22.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:22.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:12:22.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:12:22.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:22.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:12:22.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:12:22.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:22.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:12:22.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:12:22.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:12:22.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:12:22.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:12:22.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:12:22.185 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.185 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.185 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.186 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:22.186 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:22.186 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.266142584999997, 2024-09-15T00:12:22.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:12:22.186 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.186 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.186 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.186 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.186 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.186 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.186 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:12:22.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.187 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:22.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:22.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:12:22.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:12:22.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:12:22.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:12:22.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:12:22.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:12:22.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:12:22.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:12:22.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:12:22.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:12:22.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:12:22.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:12:22.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:22.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:12:22.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:12:22.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:22.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:12:22.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:12:22.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:22.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:12:22.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:12:22.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:12:22.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:12:22.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:12:22.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:12:22.192 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.192 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.192 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.192 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:12:22.192 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:22.192 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.259184222000002, 2024-09-15T00:12:22.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:12:22.192 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.192 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.192 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:22.192 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:22.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:22.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:12:22.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:12:22.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:12:22.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:12:22.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:12:22.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:12:22.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:12:22.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:12:22.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:12:22.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:12:22.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:12:22.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:12:22.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:12:22.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:12:22.196 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.196 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.196 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.196 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:22.196 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:22.196 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.511995384999999, 2024-09-15T00:12:22.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:12:22.196 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.196 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.197 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.197 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.197 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.197 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.197 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:12:22.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.197 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:22.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:12:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:12:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:12:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:12:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:12:22.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:12:22.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:12:22.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:12:22.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:12:22.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:12:22.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:12:22.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:12:22.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:22.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:12:22.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:12:22.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:22.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:12:22.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:12:22.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:22.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:12:22.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:12:22.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:12:22.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:12:22.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:12:22.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:12:22.202 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.202 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.202 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.202 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97493 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:22.202 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:22.202 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.322412665, 2024-09-15T00:12:22.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033259141999999998, 2024-09-15T00:12:22.202 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.202 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.202 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:22.203 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.203 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.203 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.203 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97493 2024-09-15T00:12:22.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.203 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:22.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:22.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:22.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-15T00:12:22.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853401+0000", 2024-09-15T00:12:22.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:12:22.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853407+0000", 2024-09-15T00:12:22.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-15T00:12:22.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853411+0000", 2024-09-15T00:12:22.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.481e-06 2024-09-15T00:12:22.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886544+0000", 2024-09-15T00:12:22.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033132861999999999 2024-09-15T00:12:22.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886587+0000", 2024-09-15T00:12:22.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2942000000000003e-05 2024-09-15T00:12:22.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886655+0000", 2024-09-15T00:12:22.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7711000000000003e-05 2024-09-15T00:12:22.206 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.207 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.207 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.207 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97494 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:22.207 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:22.207 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.322391542, 2024-09-15T00:12:22.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033392390000000001, 2024-09-15T00:12:22.207 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.208 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.208 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:22.208 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.208 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.208 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.208 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97494 2024-09-15T00:12:22.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.208 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:22.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:22.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:22.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-15T00:12:22.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853419+0000", 2024-09-15T00:12:22.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.917e-06 2024-09-15T00:12:22.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853425+0000", 2024-09-15T00:12:22.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:12:22.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853433+0000", 2024-09-15T00:12:22.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8120000000000008e-06 2024-09-15T00:12:22.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886680+0000", 2024-09-15T00:12:22.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033247207000000001 2024-09-15T00:12:22.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886726+0000", 2024-09-15T00:12:22.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5902000000000002e-05 2024-09-15T00:12:22.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886809+0000", 2024-09-15T00:12:22.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3475999999999998e-05 2024-09-15T00:12:22.212 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.212 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.212 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.212 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97495 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:22.212 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:22.212 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.322370589, 2024-09-15T00:12:22.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033640503000000002, 2024-09-15T00:12:22.212 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.212 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.212 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:22.213 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.213 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.213 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.213 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97495 2024-09-15T00:12:22.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.213 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:22.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:22.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:22.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:12:22.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853440+0000", 2024-09-15T00:12:22.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.804e-06 2024-09-15T00:12:22.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853446+0000", 2024-09-15T00:12:22.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5559999999999998e-06 2024-09-15T00:12:22.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853450+0000", 2024-09-15T00:12:22.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9519999999999996e-06 2024-09-15T00:12:22.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886904+0000", 2024-09-15T00:12:22.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033454207999999999 2024-09-15T00:12:22.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887007+0000", 2024-09-15T00:12:22.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102904 2024-09-15T00:12:22.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887078+0000", 2024-09-15T00:12:22.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1719999999999998e-05 2024-09-15T00:12:22.216 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.216 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.216 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.216 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97496 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:22.216 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:22.217 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.322353490999999, 2024-09-15T00:12:22.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033802431000000001, 2024-09-15T00:12:22.217 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.217 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.217 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:22.217 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.217 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.217 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.217 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97496 2024-09-15T00:12:22.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.217 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:22.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:22.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:22.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.53e-07 2024-09-15T00:12:22.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853462+0000", 2024-09-15T00:12:22.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-15T00:12:22.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853468+0000", 2024-09-15T00:12:22.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3609999999999997e-06 2024-09-15T00:12:22.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853473+0000", 2024-09-15T00:12:22.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-15T00:12:22.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887111+0000", 2024-09-15T00:12:22.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033638376999999997 2024-09-15T00:12:22.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887178+0000", 2024-09-15T00:12:22.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6877000000000004e-05 2024-09-15T00:12:22.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887257+0000", 2024-09-15T00:12:22.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9616999999999993e-05 2024-09-15T00:12:22.221 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.221 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.221 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.221 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97587 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:22.221 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:22.221 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.322330257000001, 2024-09-15T00:12:22.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033973642999999998, 2024-09-15T00:12:22.221 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.221 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.221 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:22.221 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.222 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.222 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.222 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97587 2024-09-15T00:12:22.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.222 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:22.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:22.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853479+0000", 2024-09-15T00:12:22.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:12:22.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853481+0000", 2024-09-15T00:12:22.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.136e-06 2024-09-15T00:12:22.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853487+0000", 2024-09-15T00:12:22.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2439999999999998e-06 2024-09-15T00:12:22.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853492+0000", 2024-09-15T00:12:22.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:12:22.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887284+0000", 2024-09-15T00:12:22.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033791827000000003 2024-09-15T00:12:22.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887361+0000", 2024-09-15T00:12:22.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6983000000000005e-05 2024-09-15T00:12:22.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887452+0000", 2024-09-15T00:12:22.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1236999999999994e-05 2024-09-15T00:12:22.225 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.225 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.226 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.226 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97588 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:22.226 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:22.226 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.322310099999999, 2024-09-15T00:12:22.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034134617999999999, 2024-09-15T00:12:22.226 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.226 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.226 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:22.226 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.226 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.226 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.226 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97588 2024-09-15T00:12:22.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.227 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:22.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:22.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853499+0000", 2024-09-15T00:12:22.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:12:22.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853501+0000", 2024-09-15T00:12:22.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.26e-06 2024-09-15T00:12:22.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853507+0000", 2024-09-15T00:12:22.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2899999999999999e-06 2024-09-15T00:12:22.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853512+0000", 2024-09-15T00:12:22.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8330000000000001e-06 2024-09-15T00:12:22.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887495+0000", 2024-09-15T00:12:22.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033982529999999997 2024-09-15T00:12:22.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887548+0000", 2024-09-15T00:12:22.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3445000000000001e-05 2024-09-15T00:12:22.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887633+0000", 2024-09-15T00:12:22.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4890999999999997e-05 2024-09-15T00:12:22.230 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.230 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.230 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.230 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126866 3.6 3.973c9fa6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:22.230 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:22.230 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.21898815499999999, 2024-09-15T00:12:22.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153301000000003, 2024-09-15T00:12:22.230 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.230 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.231 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:22.231 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.231 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.231 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.231 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126866 2024-09-15T00:12:22.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.231 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:22.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956821+0000", 2024-09-15T00:12:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.91e-07 2024-09-15T00:12:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956822+0000", 2024-09-15T00:12:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.705e-06 2024-09-15T00:12:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956828+0000", 2024-09-15T00:12:22.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6760000000000004e-06 2024-09-15T00:12:22.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956833+0000", 2024-09-15T00:12:22.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5379999999999996e-06 2024-09-15T00:12:22.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989970+0000", 2024-09-15T00:12:22.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033137560000000003 2024-09-15T00:12:22.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989974+0000", 2024-09-15T00:12:22.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-15T00:12:22.235 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.235 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.235 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.235 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126951 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:22.235 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:22.235 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.218862472, 2024-09-15T00:12:22.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033152585999999998, 2024-09-15T00:12:22.235 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.236 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.236 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:22.236 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.236 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.236 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.236 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126951 2024-09-15T00:12:22.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.236 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:22.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:22.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:22.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-07 2024-09-15T00:12:22.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956953+0000", 2024-09-15T00:12:22.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9310000000000002e-06 2024-09-15T00:12:22.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956957+0000", 2024-09-15T00:12:22.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5379999999999998e-06 2024-09-15T00:12:22.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956959+0000", 2024-09-15T00:12:22.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6599999999999999e-06 2024-09-15T00:12:22.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990095+0000", 2024-09-15T00:12:22.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033135855999999998 2024-09-15T00:12:22.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990099+0000", 2024-09-15T00:12:22.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4010000000000001e-06 2024-09-15T00:12:22.239 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.239 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.239 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.239 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126952 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:22.239 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:22.240 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.21884548300000001, 2024-09-15T00:12:22.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033156066999999997, 2024-09-15T00:12:22.240 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.240 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.240 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:22.240 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.240 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.240 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.240 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126952 2024-09-15T00:12:22.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.241 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:22.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:22.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:22.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:12:22.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956965+0000", 2024-09-15T00:12:22.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3379999999999999e-06 2024-09-15T00:12:22.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956968+0000", 2024-09-15T00:12:22.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0419999999999998e-06 2024-09-15T00:12:22.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956970+0000", 2024-09-15T00:12:22.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4899999999999999e-06 2024-09-15T00:12:22.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990113+0000", 2024-09-15T00:12:22.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033142642 2024-09-15T00:12:22.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990119+0000", 2024-09-15T00:12:22.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2979999999999997e-06 2024-09-15T00:12:22.243 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.244 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.244 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.244 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126954 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:22.244 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:22.244 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.218799988, 2024-09-15T00:12:22.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033158449, 2024-09-15T00:12:22.244 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.245 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.245 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:22.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.245 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126954 2024-09-15T00:12:22.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.245 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:22.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:22.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:22.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-15T00:12:22.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957010+0000", 2024-09-15T00:12:22.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3659999999999999e-06 2024-09-15T00:12:22.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957013+0000", 2024-09-15T00:12:22.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1140000000000002e-06 2024-09-15T00:12:22.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957016+0000", 2024-09-15T00:12:22.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1209999999999998e-06 2024-09-15T00:12:22.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990163+0000", 2024-09-15T00:12:22.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033146451 2024-09-15T00:12:22.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990167+0000", 2024-09-15T00:12:22.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0579999999999999e-06 2024-09-15T00:12:22.248 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.248 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.248 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.248 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126955 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:22.248 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:22.248 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.21878877399999999, 2024-09-15T00:12:22.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033160617000000003, 2024-09-15T00:12:22.249 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.249 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.249 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:22.249 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.249 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.249 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.249 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126955 2024-09-15T00:12:22.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.249 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:22.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:22.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:22.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-15T00:12:22.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957021+0000", 2024-09-15T00:12:22.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-15T00:12:22.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957024+0000", 2024-09-15T00:12:22.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1599999999999998e-06 2024-09-15T00:12:22.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957027+0000", 2024-09-15T00:12:22.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000001e-06 2024-09-15T00:12:22.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990177+0000", 2024-09-15T00:12:22.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033149741000000003 2024-09-15T00:12:22.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990180+0000", 2024-09-15T00:12:22.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4910000000000001e-06 2024-09-15T00:12:22.252 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.252 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.252 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.252 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126956 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:22.252 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:22.252 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.21877865299999999, 2024-09-15T00:12:22.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033163633999999997, 2024-09-15T00:12:22.253 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.253 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.253 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:22.253 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.253 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.253 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.253 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126956 2024-09-15T00:12:22.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.253 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:22.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:22.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:22.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:22.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957031+0000", 2024-09-15T00:12:22.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-06 2024-09-15T00:12:22.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957034+0000", 2024-09-15T00:12:22.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9960000000000002e-06 2024-09-15T00:12:22.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957037+0000", 2024-09-15T00:12:22.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6419999999999999e-06 2024-09-15T00:12:22.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990190+0000", 2024-09-15T00:12:22.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153152999999998 2024-09-15T00:12:22.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990193+0000", 2024-09-15T00:12:22.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6069999999999999e-06 2024-09-15T00:12:22.256 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.256 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.256 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.256 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126957 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:22.256 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:22.256 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.21876900299999999, 2024-09-15T00:12:22.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033166442999999997, 2024-09-15T00:12:22.257 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.257 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.257 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:22.257 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.257 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.257 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.257 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126957 2024-09-15T00:12:22.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.257 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:22.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:22.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957040+0000", 2024-09-15T00:12:22.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:12:22.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957041+0000", 2024-09-15T00:12:22.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0440000000000001e-06 2024-09-15T00:12:22.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957044+0000", 2024-09-15T00:12:22.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0170000000000001e-06 2024-09-15T00:12:22.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957048+0000", 2024-09-15T00:12:22.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0319999999999997e-06 2024-09-15T00:12:22.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990202+0000", 2024-09-15T00:12:22.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033154640999999999 2024-09-15T00:12:22.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990206+0000", 2024-09-15T00:12:22.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.507e-06 2024-09-15T00:12:22.260 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.260 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.260 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.260 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126958 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:22.260 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:22.260 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.21875777299999999, 2024-09-15T00:12:22.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033167613999999998, 2024-09-15T00:12:22.261 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.261 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.261 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:22.261 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.261 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.261 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.261 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126958 2024-09-15T00:12:22.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.261 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:22.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:22.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:22.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:12:22.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957053+0000", 2024-09-15T00:12:22.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.756e-06 2024-09-15T00:12:22.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957059+0000", 2024-09-15T00:12:22.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2770000000000002e-06 2024-09-15T00:12:22.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957063+0000", 2024-09-15T00:12:22.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2459999999999997e-06 2024-09-15T00:12:22.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990215+0000", 2024-09-15T00:12:22.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033151509000000003 2024-09-15T00:12:22.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990218+0000", 2024-09-15T00:12:22.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5099999999999999e-06 2024-09-15T00:12:22.264 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.264 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.264 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.265 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126959 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:22.265 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:22.265 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.218738769, 2024-09-15T00:12:22.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033161673000000003, 2024-09-15T00:12:22.265 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.265 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.265 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:22.265 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.265 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.265 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.265 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126959 2024-09-15T00:12:22.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.265 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:22.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:22.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:22.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:12:22.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957072+0000", 2024-09-15T00:12:22.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2239999999999998e-06 2024-09-15T00:12:22.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957079+0000", 2024-09-15T00:12:22.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.19e-06 2024-09-15T00:12:22.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957083+0000", 2024-09-15T00:12:22.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:12:22.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990228+0000", 2024-09-15T00:12:22.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033144818 2024-09-15T00:12:22.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990231+0000", 2024-09-15T00:12:22.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:12:22.268 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.268 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.268 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126960 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.21871996199999999, 2024-09-15T00:12:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033155166, 2024-09-15T00:12:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:22.269 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126960 2024-09-15T00:12:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:12:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957091+0000", 2024-09-15T00:12:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:12:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957097+0000", 2024-09-15T00:12:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6050000000000003e-06 2024-09-15T00:12:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957102+0000", 2024-09-15T00:12:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5539999999999999e-06 2024-09-15T00:12:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990240+0000", 2024-09-15T00:12:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033138289000000001 2024-09-15T00:12:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990244+0000", 2024-09-15T00:12:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4740000000000001e-06 2024-09-15T00:12:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.272 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.273 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.273 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:22.274 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:12:22.275 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:22.275 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:22.275 INFO:teuthology.orchestra.run.smithi171.stdout:3934 writesame 0x47dd171 thru 0x47e50c9 data_size 0x1(0x7f59 bytes) 2024-09-15T00:12:22.275 INFO:teuthology.orchestra.run.smithi171.stdout:3935 write 0xddd16a1 thru 0xdddb93d (0xa29d bytes) 2024-09-15T00:12:22.276 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:22.272+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:22.375 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:22.373+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:22.418 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:22.418 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:22.427 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:12:22.476 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:22.474+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:22.571 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:22.571 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:22.576 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:22.574+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:22.580 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:12:22.677 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:22.675+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:22.726 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:22.885 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:22.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:22.886 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:22.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.886 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:462 3.0 e29/23 3:04479396:::rbd_id.image_client.0-clone2:head v 29'14, mlcod=27'12)", 2024-09-15T00:12:22.886 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:22.886 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.707753951000001, 2024-09-15T00:12:22.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030963029999999999, 2024-09-15T00:12:22.886 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.886 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.886 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.886 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:22.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:22.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016200+0000", 2024-09-15T00:12:22.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:12:22.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016208+0000", 2024-09-15T00:12:22.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3669999999999997e-06 2024-09-15T00:12:22.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016250+0000", 2024-09-15T00:12:22.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1845999999999999e-05 2024-09-15T00:12:22.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016260+0000", 2024-09-15T00:12:22.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9750000000000002e-06 2024-09-15T00:12:22.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016342+0000", 2024-09-15T00:12:22.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1595999999999996e-05 2024-09-15T00:12:22.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.016363+0000", 2024-09-15T00:12:22.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1073e-05 2024-09-15T00:12:22.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019246+0000", 2024-09-15T00:12:22.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028835240000000002 2024-09-15T00:12:22.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.019296+0000", 2024-09-15T00:12:22.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9533999999999997e-05 2024-09-15T00:12:22.890 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.890 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.890 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.890 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:463 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'9, mlcod=27'7)", 2024-09-15T00:12:22.890 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:22.890 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.703140947000001, 2024-09-15T00:12:22.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040015709999999998, 2024-09-15T00:12:22.890 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.890 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.891 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.891 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:22.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:22.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020813+0000", 2024-09-15T00:12:22.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7599999999999999e-07 2024-09-15T00:12:22.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020818+0000", 2024-09-15T00:12:22.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6609999999999999e-06 2024-09-15T00:12:22.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020826+0000", 2024-09-15T00:12:22.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4770000000000003e-06 2024-09-15T00:12:22.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020833+0000", 2024-09-15T00:12:22.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0809999999999996e-06 2024-09-15T00:12:22.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020878+0000", 2024-09-15T00:12:22.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4752000000000001e-05 2024-09-15T00:12:22.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.020892+0000", 2024-09-15T00:12:22.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3899e-05 2024-09-15T00:12:22.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024767+0000", 2024-09-15T00:12:22.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038752259999999998 2024-09-15T00:12:22.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.024814+0000", 2024-09-15T00:12:22.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7299000000000002e-05 2024-09-15T00:12:22.894 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.894 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.894 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.894 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:464 3.9 e29/23 3:942069cf:::rbd_header.11397bff333d:head v 29'10, mlcod=27'8)", 2024-09-15T00:12:22.895 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:22.895 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.698221105000002, 2024-09-15T00:12:22.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048109290000000002, 2024-09-15T00:12:22.895 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.895 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.895 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.895 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:22.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:22.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025733+0000", 2024-09-15T00:12:22.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-15T00:12:22.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025743+0000", 2024-09-15T00:12:22.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5789999999999993e-06 2024-09-15T00:12:22.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025752+0000", 2024-09-15T00:12:22.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8440000000000004e-06 2024-09-15T00:12:22.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025757+0000", 2024-09-15T00:12:22.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2610000000000001e-06 2024-09-15T00:12:22.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025796+0000", 2024-09-15T00:12:22.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9428999999999998e-05 2024-09-15T00:12:22.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.025810+0000", 2024-09-15T00:12:22.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3129000000000001e-05 2024-09-15T00:12:22.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030514+0000", 2024-09-15T00:12:22.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047040099999999998 2024-09-15T00:12:22.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.030544+0000", 2024-09-15T00:12:22.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9966000000000001e-05 2024-09-15T00:12:22.898 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.898 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.899 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.899 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:610 3.4 e29/23 3:255b0913:::rbd_data.11397bff333d.0000000000000039:head v 29'2, mlcod=0'0)", 2024-09-15T00:12:22.899 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:12:22.899 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.495477510000001, 2024-09-15T00:12:22.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003163014, 2024-09-15T00:12:22.899 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.899 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.899 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.899 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:12:22.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228476+0000", 2024-09-15T00:12:22.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228477+0000", 2024-09-15T00:12:22.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1100000000000004e-07 2024-09-15T00:12:22.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228483+0000", 2024-09-15T00:12:22.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3720000000000001e-06 2024-09-15T00:12:22.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228497+0000", 2024-09-15T00:12:22.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3944000000000001e-05 2024-09-15T00:12:22.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228505+0000", 2024-09-15T00:12:22.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8140000000000001e-06 2024-09-15T00:12:22.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228549+0000", 2024-09-15T00:12:22.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4116000000000002e-05 2024-09-15T00:12:22.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.228568+0000", 2024-09-15T00:12:22.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8473000000000001e-05 2024-09-15T00:12:22.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231599+0000", 2024-09-15T00:12:22.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030315590000000001 2024-09-15T00:12:22.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.231639+0000", 2024-09-15T00:12:22.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9824999999999999e-05 2024-09-15T00:12:22.904 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.904 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.904 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.904 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:766 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'81, mlcod=29'78)", 2024-09-15T00:12:22.904 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:12:22.905 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.290928432999998, 2024-09-15T00:12:22.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029025880000000002, 2024-09-15T00:12:22.905 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.905 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.905 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.905 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:12:22.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433025+0000", 2024-09-15T00:12:22.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433026+0000", 2024-09-15T00:12:22.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:12:22.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433030+0000", 2024-09-15T00:12:22.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6280000000000004e-06 2024-09-15T00:12:22.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433038+0000", 2024-09-15T00:12:22.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.072e-06 2024-09-15T00:12:22.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433045+0000", 2024-09-15T00:12:22.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1510000000000001e-06 2024-09-15T00:12:22.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433101+0000", 2024-09-15T00:12:22.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6336999999999999e-05 2024-09-15T00:12:22.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.433113+0000", 2024-09-15T00:12:22.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1639e-05 2024-09-15T00:12:22.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435897+0000", 2024-09-15T00:12:22.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027845779999999998 2024-09-15T00:12:22.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.435928+0000", 2024-09-15T00:12:22.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0689999999999999e-05 2024-09-15T00:12:22.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.909 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.909 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:832 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'100, mlcod=29'97)", 2024-09-15T00:12:22.909 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:22.909 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.198682933000001, 2024-09-15T00:12:22.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005498925, 2024-09-15T00:12:22.909 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.909 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.909 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.910 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:22.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:22.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525271+0000", 2024-09-15T00:12:22.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7999999999999995e-07 2024-09-15T00:12:22.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525276+0000", 2024-09-15T00:12:22.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6739999999999996e-06 2024-09-15T00:12:22.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525287+0000", 2024-09-15T00:12:22.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0906e-05 2024-09-15T00:12:22.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525293+0000", 2024-09-15T00:12:22.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4409999999999998e-06 2024-09-15T00:12:22.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525329+0000", 2024-09-15T00:12:22.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5237000000000002e-05 2024-09-15T00:12:22.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.525340+0000", 2024-09-15T00:12:22.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1782e-05 2024-09-15T00:12:22.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530710+0000", 2024-09-15T00:12:22.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053695460000000002 2024-09-15T00:12:22.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.530770+0000", 2024-09-15T00:12:22.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9658999999999999e-05 2024-09-15T00:12:22.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.913 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.914 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:876 3.9 e29/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 29'114, mlcod=29'111)", 2024-09-15T00:12:22.914 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:22.914 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.027643933, 2024-09-15T00:12:22.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028465830000000002, 2024-09-15T00:12:22.914 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.914 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.914 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.914 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:22.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:22.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696310+0000", 2024-09-15T00:12:22.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2399999999999998e-07 2024-09-15T00:12:22.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696316+0000", 2024-09-15T00:12:22.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2800000000000003e-06 2024-09-15T00:12:22.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696326+0000", 2024-09-15T00:12:22.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0546e-05 2024-09-15T00:12:22.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696333+0000", 2024-09-15T00:12:22.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7660000000000001e-06 2024-09-15T00:12:22.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696369+0000", 2024-09-15T00:12:22.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5933000000000003e-05 2024-09-15T00:12:22.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.696381+0000", 2024-09-15T00:12:22.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2417e-05 2024-09-15T00:12:22.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699124+0000", 2024-09-15T00:12:22.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027428000000000001 2024-09-15T00:12:22.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:25.699156+0000", 2024-09-15T00:12:22.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2217e-05 2024-09-15T00:12:22.918 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.918 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.918 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.918 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1313 3.1f e30/28 3:fa9cde43:::rbd_data.11397bff333d.000000000000003a:head v 30'3, mlcod=26'1)", 2024-09-15T00:12:22.918 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:12:22.918 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.439804295999998, 2024-09-15T00:12:22.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044064359999999997, 2024-09-15T00:12:22.918 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.918 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.918 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.918 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:12:22.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284149+0000", 2024-09-15T00:12:22.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284150+0000", 2024-09-15T00:12:22.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3799999999999999e-07 2024-09-15T00:12:22.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284228+0000", 2024-09-15T00:12:22.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:12:22.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284235+0000", 2024-09-15T00:12:22.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:12:22.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284239+0000", 2024-09-15T00:12:22.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3449999999999999e-06 2024-09-15T00:12:22.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284282+0000", 2024-09-15T00:12:22.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2506000000000003e-05 2024-09-15T00:12:22.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.284297+0000", 2024-09-15T00:12:22.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5523999999999999e-05 2024-09-15T00:12:22.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288517+0000", 2024-09-15T00:12:22.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042194019999999997 2024-09-15T00:12:22.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.288556+0000", 2024-09-15T00:12:22.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8952000000000002e-05 2024-09-15T00:12:22.922 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.922 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.922 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.922 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1321 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'7, mlcod=29'5)", 2024-09-15T00:12:22.922 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:22.922 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.415711037000001, 2024-09-15T00:12:22.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003131254, 2024-09-15T00:12:22.923 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.923 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.923 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.923 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:22.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:22.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308243+0000", 2024-09-15T00:12:22.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:12:22.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308250+0000", 2024-09-15T00:12:22.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6109999999999999e-06 2024-09-15T00:12:22.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308264+0000", 2024-09-15T00:12:22.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4473e-05 2024-09-15T00:12:22.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308272+0000", 2024-09-15T00:12:22.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3540000000000002e-06 2024-09-15T00:12:22.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308340+0000", 2024-09-15T00:12:22.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8305000000000004e-05 2024-09-15T00:12:22.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.308361+0000", 2024-09-15T00:12:22.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0409999999999999e-05 2024-09-15T00:12:22.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311332+0000", 2024-09-15T00:12:22.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029715950000000001 2024-09-15T00:12:22.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.311374+0000", 2024-09-15T00:12:22.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1786000000000003e-05 2024-09-15T00:12:22.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.926 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.927 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.927 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.9 epoch 30/23 rep_tid 277 entries 30'217 (0'0) error 3:942069cf:::rbd_header.11397bff333d:head by client.4409.0:1373 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'214)", 2024-09-15T00:12:22.927 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:12:22.927 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.370158439999997, 2024-09-15T00:12:22.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034617960000000001, 2024-09-15T00:12:22.927 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.927 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.927 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:22.927 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:12:22.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353795+0000", 2024-09-15T00:12:22.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353796+0000", 2024-09-15T00:12:22.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6699999999999999e-07 2024-09-15T00:12:22.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353803+0000", 2024-09-15T00:12:22.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.832e-06 2024-09-15T00:12:22.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353814+0000", 2024-09-15T00:12:22.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1815e-05 2024-09-15T00:12:22.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353822+0000", 2024-09-15T00:12:22.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7440000000000007e-06 2024-09-15T00:12:22.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.353910+0000", 2024-09-15T00:12:22.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7491000000000006e-05 2024-09-15T00:12:22.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.357257+0000", 2024-09-15T00:12:22.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033474469999999999 2024-09-15T00:12:22.930 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.930 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.930 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.931 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1374 3.9 e30/23 3:942069cf:::rbd_header.11397bff333d:head v 30'218, mlcod=30'215)", 2024-09-15T00:12:22.931 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:12:22.931 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.368758798000002, 2024-09-15T00:12:22.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047299830000000001, 2024-09-15T00:12:22.931 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.931 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.931 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.931 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:12:22.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355195+0000", 2024-09-15T00:12:22.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355196+0000", 2024-09-15T00:12:22.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3900000000000004e-07 2024-09-15T00:12:22.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355200+0000", 2024-09-15T00:12:22.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5700000000000003e-06 2024-09-15T00:12:22.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355210+0000", 2024-09-15T00:12:22.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6050000000000004e-06 2024-09-15T00:12:22.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355215+0000", 2024-09-15T00:12:22.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.933e-06 2024-09-15T00:12:22.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355249+0000", 2024-09-15T00:12:22.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4110999999999999e-05 2024-09-15T00:12:22.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.355260+0000", 2024-09-15T00:12:22.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1235e-05 2024-09-15T00:12:22.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359895+0000", 2024-09-15T00:12:22.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046345550000000003 2024-09-15T00:12:22.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.359925+0000", 2024-09-15T00:12:22.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0335e-05 2024-09-15T00:12:22.935 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.935 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.935 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.935 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1385 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'221, mlcod=30'218)", 2024-09-15T00:12:22.935 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:12:22.935 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.356302263000003, 2024-09-15T00:12:22.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030443950000000001, 2024-09-15T00:12:22.935 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.935 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.935 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.935 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:12:22.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367651+0000", 2024-09-15T00:12:22.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367652+0000", 2024-09-15T00:12:22.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:12:22.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367657+0000", 2024-09-15T00:12:22.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5410000000000002e-06 2024-09-15T00:12:22.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367666+0000", 2024-09-15T00:12:22.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9280000000000005e-06 2024-09-15T00:12:22.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367671+0000", 2024-09-15T00:12:22.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7870000000000001e-06 2024-09-15T00:12:22.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367707+0000", 2024-09-15T00:12:22.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6102000000000001e-05 2024-09-15T00:12:22.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.367719+0000", 2024-09-15T00:12:22.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1571e-05 2024-09-15T00:12:22.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370666+0000", 2024-09-15T00:12:22.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029474750000000002 2024-09-15T00:12:22.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.370696+0000", 2024-09-15T00:12:22.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9402000000000001e-05 2024-09-15T00:12:22.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.939 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.939 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1662 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'300, mlcod=30'297)", 2024-09-15T00:12:22.939 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:12:22.940 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.935937625999998, 2024-09-15T00:12:22.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031296090000000002, 2024-09-15T00:12:22.940 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.940 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.940 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.940 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:12:22.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788016+0000", 2024-09-15T00:12:22.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788017+0000", 2024-09-15T00:12:22.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8999999999999996e-07 2024-09-15T00:12:22.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788021+0000", 2024-09-15T00:12:22.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4970000000000003e-06 2024-09-15T00:12:22.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788027+0000", 2024-09-15T00:12:22.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1840000000000004e-06 2024-09-15T00:12:22.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788032+0000", 2024-09-15T00:12:22.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.135e-06 2024-09-15T00:12:22.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788070+0000", 2024-09-15T00:12:22.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7073999999999998e-05 2024-09-15T00:12:22.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.788081+0000", 2024-09-15T00:12:22.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0987e-05 2024-09-15T00:12:22.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791116+0000", 2024-09-15T00:12:22.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030355209999999998 2024-09-15T00:12:22.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.791146+0000", 2024-09-15T00:12:22.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9621e-05 2024-09-15T00:12:22.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.944 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.944 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1677 3.9 e30/23 3:90ee4fa6:::rbd_object_map.11397bff333d:head v 30'305, mlcod=30'302)", 2024-09-15T00:12:22.944 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:12:22.944 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.912209263999998, 2024-09-15T00:12:22.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031364320000000002, 2024-09-15T00:12:22.945 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.945 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.945 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.945 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:12:22.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811744+0000", 2024-09-15T00:12:22.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811745+0000", 2024-09-15T00:12:22.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:12:22.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811749+0000", 2024-09-15T00:12:22.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9369999999999997e-06 2024-09-15T00:12:22.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811756+0000", 2024-09-15T00:12:22.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:12:22.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811765+0000", 2024-09-15T00:12:22.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7930000000000003e-06 2024-09-15T00:12:22.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811797+0000", 2024-09-15T00:12:22.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2203999999999999e-05 2024-09-15T00:12:22.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.811810+0000", 2024-09-15T00:12:22.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2359e-05 2024-09-15T00:12:22.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814852+0000", 2024-09-15T00:12:22.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003042276 2024-09-15T00:12:22.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.814881+0000", 2024-09-15T00:12:22.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9037999999999999e-05 2024-09-15T00:12:22.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.949 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.949 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1724 3.4 e30/23 3:2714a3a2:::rbd_data.11397bff333d.000000000000001a:head v 30'9, mlcod=30'7)", 2024-09-15T00:12:22.949 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:12:22.949 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.843283579999998, 2024-09-15T00:12:22.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003115769, 2024-09-15T00:12:22.949 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.949 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.949 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.949 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:12:22.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880670+0000", 2024-09-15T00:12:22.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880671+0000", 2024-09-15T00:12:22.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4900000000000005e-07 2024-09-15T00:12:22.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880678+0000", 2024-09-15T00:12:22.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0979999999999996e-06 2024-09-15T00:12:22.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880691+0000", 2024-09-15T00:12:22.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3288999999999999e-05 2024-09-15T00:12:22.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880698+0000", 2024-09-15T00:12:22.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9240000000000001e-06 2024-09-15T00:12:22.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880757+0000", 2024-09-15T00:12:22.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8866000000000002e-05 2024-09-15T00:12:22.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.880790+0000", 2024-09-15T00:12:22.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2926999999999998e-05 2024-09-15T00:12:22.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883738+0000", 2024-09-15T00:12:22.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002948418 2024-09-15T00:12:22.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.883786+0000", 2024-09-15T00:12:22.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7398000000000003e-05 2024-09-15T00:12:22.953 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.953 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.953 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.953 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:1790 3.9 e30/23 3:9676e01b:::rbd_data.11397bff333d.0000000000000030:head v 30'338, mlcod=30'335)", 2024-09-15T00:12:22.953 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:12:22.953 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.764401210000003, 2024-09-15T00:12:22.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044496980000000002, 2024-09-15T00:12:22.954 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.954 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.954 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.954 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:12:22.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959552+0000", 2024-09-15T00:12:22.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959553+0000", 2024-09-15T00:12:22.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3600000000000004e-07 2024-09-15T00:12:22.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959569+0000", 2024-09-15T00:12:22.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6152999999999999e-05 2024-09-15T00:12:22.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959577+0000", 2024-09-15T00:12:22.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5179999999999998e-06 2024-09-15T00:12:22.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959581+0000", 2024-09-15T00:12:22.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3669999999999999e-06 2024-09-15T00:12:22.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959619+0000", 2024-09-15T00:12:22.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8209e-05 2024-09-15T00:12:22.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.959632+0000", 2024-09-15T00:12:22.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2405000000000001e-05 2024-09-15T00:12:22.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.963954+0000", 2024-09-15T00:12:22.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043226660000000002 2024-09-15T00:12:22.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.964002+0000", 2024-09-15T00:12:22.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7843999999999998e-05 2024-09-15T00:12:22.957 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.957 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.957 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.958 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:3177 3.4 e33/23 3:23818c00:::rbd_data.1139b531b717.0000000000000022:head v 33'58, mlcod=33'56)", 2024-09-15T00:12:22.958 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:12:22.958 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.066112644, 2024-09-15T00:12:22.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030564120000000001, 2024-09-15T00:12:22.958 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.958 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.958 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.958 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:12:22.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657841+0000", 2024-09-15T00:12:22.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657842+0000", 2024-09-15T00:12:22.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.19e-06 2024-09-15T00:12:22.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657853+0000", 2024-09-15T00:12:22.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:12:22.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657871+0000", 2024-09-15T00:12:22.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8233e-05 2024-09-15T00:12:22.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657880+0000", 2024-09-15T00:12:22.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8319999999999995e-06 2024-09-15T00:12:22.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657949+0000", 2024-09-15T00:12:22.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8847e-05 2024-09-15T00:12:22.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.657977+0000", 2024-09-15T00:12:22.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8308000000000001e-05 2024-09-15T00:12:22.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.660851+0000", 2024-09-15T00:12:22.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028743190000000002 2024-09-15T00:12:22.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:29.660897+0000", 2024-09-15T00:12:22.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6142e-05 2024-09-15T00:12:22.962 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.962 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.962 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.962 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:3602 3.1f e34/28 3:fd28af01:::rbd_data.1139b531b717.0000000000000032:head v 34'24, mlcod=33'22)", 2024-09-15T00:12:22.962 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:12:22.962 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 52.678715249, 2024-09-15T00:12:22.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034159709999999999, 2024-09-15T00:12:22.962 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.962 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.962 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.963 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:12:22.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045238+0000", 2024-09-15T00:12:22.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045239+0000", 2024-09-15T00:12:22.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:12:22.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045252+0000", 2024-09-15T00:12:22.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2891000000000001e-05 2024-09-15T00:12:22.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045263+0000", 2024-09-15T00:12:22.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0875e-05 2024-09-15T00:12:22.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045271+0000", 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3529999999999995e-06 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045338+0000", 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7377000000000002e-05 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.045359+0000", 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1078e-05 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.048621+0000", 2024-09-15T00:12:22.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003261543 2024-09-15T00:12:22.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:30.048654+0000", 2024-09-15T00:12:22.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3550999999999998e-05 2024-09-15T00:12:22.966 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.966 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.966 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.966 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:4253 3.4 e35/23 3:242f5431:::rbd_data.11396389f8f2.0000000000000000:head v 35'87, mlcod=35'85)", 2024-09-15T00:12:22.966 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:12:22.967 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.331226391000001, 2024-09-15T00:12:22.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002944997, 2024-09-15T00:12:22.967 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.967 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.967 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.967 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:12:22.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392727+0000", 2024-09-15T00:12:22.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392728+0000", 2024-09-15T00:12:22.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.06e-07 2024-09-15T00:12:22.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392782+0000", 2024-09-15T00:12:22.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.448e-05 2024-09-15T00:12:22.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392794+0000", 2024-09-15T00:12:22.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1768e-05 2024-09-15T00:12:22.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392801+0000", 2024-09-15T00:12:22.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9589999999999998e-06 2024-09-15T00:12:22.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392845+0000", 2024-09-15T00:12:22.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3884999999999997e-05 2024-09-15T00:12:22.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.392860+0000", 2024-09-15T00:12:22.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5364999999999999e-05 2024-09-15T00:12:22.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.395641+0000", 2024-09-15T00:12:22.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027807629999999999 2024-09-15T00:12:22.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:31.395672+0000", 2024-09-15T00:12:22.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1170999999999997e-05 2024-09-15T00:12:22.970 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.970 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.971 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.971 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:18791 3.0 e38/23 3:05ecfc97:::rbd_data.11398e1a091a.0000000000000030:head v 38'66, mlcod=38'64)", 2024-09-15T00:12:22.971 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:12:22.971 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.688310854999997, 2024-09-15T00:12:22.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029447940000000001, 2024-09-15T00:12:22.971 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:22.971 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:22.971 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:22.971 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:22.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:22.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:12:22.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:22.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035643+0000", 2024-09-15T00:12:22.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:22.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:22.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035644+0000", 2024-09-15T00:12:22.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4e-07 2024-09-15T00:12:22.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:22.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035684+0000", 2024-09-15T00:12:22.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9857999999999997e-05 2024-09-15T00:12:22.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:22.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035694+0000", 2024-09-15T00:12:22.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0259e-05 2024-09-15T00:12:22.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:22.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035699+0000", 2024-09-15T00:12:22.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6389999999999997e-06 2024-09-15T00:12:22.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:22.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035745+0000", 2024-09-15T00:12:22.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5640999999999999e-05 2024-09-15T00:12:22.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:22.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.035764+0000", 2024-09-15T00:12:22.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9368000000000002e-05 2024-09-15T00:12:22.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:22.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.038551+0000", 2024-09-15T00:12:22.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027860879999999999 2024-09-15T00:12:22.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:22.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:22.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:22.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:36.038588+0000", 2024-09-15T00:12:22.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7101000000000003e-05 2024-09-15T00:12:22.975 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.975 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.975 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.975 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:22.975 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:22.975 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:22.976 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:12:22.978 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:22.978 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:22.974+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:23.022 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:12:23.105 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:23.103+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:23.129 INFO:teuthology.orchestra.run.smithi171.stdout:3936 read 0x8dc6bba thru 0x8dd31a2 (0xc5e9 bytes) 2024-09-15T00:12:23.137 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:23.138 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:23.138 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:23.138 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:23.146 INFO:teuthology.orchestra.run.smithi171.stdout:3937 trunc from 0xdddb93e to 0x21ea9b7 2024-09-15T00:12:23.153 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:23.153 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:23.156 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:23.174 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:23.174 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:23.175 INFO:teuthology.orchestra.run.smithi171.stdout:3938 punch from 0xc6699b to 0xc669af, (0x14 bytes) 2024-09-15T00:12:23.175 INFO:teuthology.orchestra.run.smithi171.stdout:3939 read 0x4d963c thru 0x4e4a33 (0xb3f8 bytes) 2024-09-15T00:12:23.179 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:23.177+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:23.188 INFO:teuthology.orchestra.run.smithi171.stdout:3941 write 0x54ebd83 thru 0x54f8beb (0xce69 bytes) 2024-09-15T00:12:23.197 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:12:23.197 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:12:23.202 INFO:teuthology.orchestra.run.smithi171.stdout:3942 read 0xb2afd6 thru 0xb37bff (0xcc2a bytes) 2024-09-15T00:12:23.203 INFO:teuthology.orchestra.run.smithi171.stdout:3944 read 0xe7180 thru 0xf5d74 (0xebf5 bytes) 2024-09-15T00:12:23.204 INFO:teuthology.orchestra.run.smithi171.stdout:3945 read 0x8cfe8b thru 0x8dfd17 (0xfe8d bytes) 2024-09-15T00:12:23.206 INFO:teuthology.orchestra.run.smithi171.stdout:3946 write 0x49f170e thru 0x49fac5d (0x9550 bytes) 2024-09-15T00:12:23.214 INFO:teuthology.orchestra.run.smithi171.stdout:3947 writesame 0x30e9948 thru 0x30ebc43 data_size 0x2(0x22fc bytes) 2024-09-15T00:12:23.218 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:23.218 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:23.220 INFO:teuthology.orchestra.run.smithi171.stdout:3948 punch from 0x41c891 to 0x42af89, (0xe6f8 bytes) 2024-09-15T00:12:23.220 INFO:teuthology.orchestra.run.smithi171.stdout:3949 write 0x186a5f1 thru 0x1874340 (0x9d50 bytes) 2024-09-15T00:12:23.223 INFO:teuthology.orchestra.run.smithi171.stdout:3950 clone 30 order 25 su 2097152 sc 2 2024-09-15T00:12:23.250 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:23.251 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:23.268 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:23.268 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:23.280 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:23.278+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:23.287 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:23.288 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:23.297 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:23.299 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:23.300 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:23.322 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:23.322 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:23.382 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:23.380+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:23.405 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:23.405 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:23.414 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:23.482 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:23.480+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:23.522 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:23.522 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:23.531 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:23.614 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:23.613+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:23.675 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:23.675 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:23.684 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:23.684 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:23.681+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:23.783 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:23.782+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:23.827 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:23.827 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:23.835 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:23.920 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:23.918+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:23.981 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:23.981 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:23.985 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:23.983+0000 7f00348bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33240) UID: 0 2024-09-15T00:12:23.989 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:24.086 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:24.084+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:24.113 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 0] out_osds: [2, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T00:12:24.113 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:12:24.113 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T00:12:24.114 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T00:12:24.114 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:12:24.127 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is unset 2024-09-15T00:12:24.138 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:24.138 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:24.147 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:24.150 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T00:12:24.150 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.2 2024-09-15T00:12:24.150 INFO:tasks.thrashosds.thrasher:remote for osd 2 is ubuntu@smithi171.front.sepia.ceph.com 2024-09-15T00:12:24.150 DEBUG:teuthology.orchestra.run.smithi171:> type ceph-objectstore-tool 2024-09-15T00:12:24.189 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:24.187+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:24.206 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T00:12:24.206 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op list-pgs 2024-09-15T00:12:24.232 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone29 from 0x54f8bec (overlap 0x1d61ed3) to 0xbc0c54 2024-09-15T00:12:24.287 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:24.285+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:24.306 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:24.306 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:24.316 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:24.388 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:24.386+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:24.390 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-15T00:12:24.391 INFO:teuthology.orchestra.run.smithi171.stdout:3951 punch from 0x4577eaa to 0x4579b07, (0x1c5d bytes) 2024-09-15T00:12:24.394 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:24.394 INFO:teuthology.orchestra.run.smithi171.stdout:3954 write 0x29363c5 thru 0x2941f42 (0xbb7e bytes) 2024-09-15T00:12:24.431 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:24.431 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:24.443 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:24.490 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:24.487+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:24.497 INFO:teuthology.orchestra.run.smithi171.stdout:3955 write 0xaaba687 thru 0xaac92d7 (0xec51 bytes) 2024-09-15T00:12:24.500 INFO:teuthology.orchestra.run.smithi171.stdout:3956 writesame 0x1a5402c thru 0x1a57e21 data_size 0x4d(0x3df6 bytes) 2024-09-15T00:12:24.553 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:24.553 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:24.562 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:24.576 INFO:teuthology.orchestra.run.smithi171.stdout:3959 read 0x5fb8b50 thru 0x5fbad0a (0x21bb bytes) 2024-09-15T00:12:24.576 INFO:teuthology.orchestra.run.smithi171.stdout:3960 read 0x11f97af thru 0x120970c (0xff5e bytes) 2024-09-15T00:12:24.576 INFO:teuthology.orchestra.run.smithi171.stdout:3961 writesame 0x7bbe1de thru 0x7bc14d7 data_size 0x3a(0x32fa bytes) 2024-09-15T00:12:24.582 INFO:teuthology.orchestra.run.smithi171.stdout:3963 write 0x695f5f thru 0x696207 (0x2a9 bytes) 2024-09-15T00:12:24.583 INFO:teuthology.orchestra.run.smithi171.stdout:3965 read 0xc95408 thru 0xc961e7 (0xde0 bytes) 2024-09-15T00:12:24.591 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:24.588+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:24.591 INFO:teuthology.orchestra.run.smithi171.stdout:3966 write 0xbb2d618 thru 0xbb2dbec (0x5d5 bytes) 2024-09-15T00:12:24.593 INFO:teuthology.orchestra.run.smithi171.stdout:3967 write 0xf267d8 thru 0xf2ed53 (0x857c bytes) 2024-09-15T00:12:24.603 INFO:teuthology.orchestra.run.smithi171.stdout:3970 write 0xd53192d thru 0xd5414a3 (0xfb77 bytes) 2024-09-15T00:12:24.606 INFO:teuthology.orchestra.run.smithi171.stdout:3972 write 0x2cfbdf2 thru 0x2cfcdfe (0x100d bytes) 2024-09-15T00:12:24.608 INFO:teuthology.orchestra.run.smithi171.stdout:3973 flatten 2024-09-15T00:12:24.622 INFO:teuthology.orchestra.run.smithi171.stdout:3974 writesame 0x5f072de thru 0x5f0e37f data_size 0xd(0x70a2 bytes) 2024-09-15T00:12:24.626 INFO:teuthology.orchestra.run.smithi171.stdout:3975 writesame 0x1796429 thru 0x17a5fce data_size 0x1b(0xfba6 bytes) 2024-09-15T00:12:24.630 INFO:teuthology.orchestra.run.smithi171.stdout:3977 write 0x73bf8af thru 0x73ca79a (0xaeec bytes) 2024-09-15T00:12:24.634 INFO:teuthology.orchestra.run.smithi171.stdout:3978 read 0x955e88c thru 0x9568bf3 (0xa368 bytes) 2024-09-15T00:12:24.634 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:24.634 INFO:teuthology.orchestra.run.smithi171.stdout:3980 write 0x9efc3ff thru 0x9f00fb5 (0x4bb7 bytes) 2024-09-15T00:12:24.635 INFO:teuthology.orchestra.run.smithi171.stdout:3982 punch from 0x25fc20 to 0x267f69, (0x8349 bytes) 2024-09-15T00:12:24.636 INFO:teuthology.orchestra.run.smithi171.stdout:3984 write 0x2f732a1 thru 0x2f78fa4 (0x5d04 bytes) 2024-09-15T00:12:24.637 INFO:teuthology.orchestra.run.smithi171.stdout:3985 writesame 0x6250c09 thru 0x6254156 data_size 0x1(0x354e bytes) 2024-09-15T00:12:24.664 INFO:teuthology.orchestra.run.smithi171.stdout:3986 write 0x3bfbee thru 0x3c2d76 (0x3189 bytes) 2024-09-15T00:12:24.666 INFO:teuthology.orchestra.run.smithi171.stdout:3988 punch from 0x320919a to 0x3209961, (0x7c7 bytes) 2024-09-15T00:12:24.666 INFO:teuthology.orchestra.run.smithi171.stdout:3989 punch from 0x6c133a8 to 0x6c13a1b, (0x673 bytes) 2024-09-15T00:12:24.670 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:24.670 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:24.678 INFO:teuthology.orchestra.run.smithi171.stdout:3990 trunc from 0xd5414a4 to 0x3872baa 2024-09-15T00:12:24.679 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:24.700 INFO:teuthology.orchestra.run.smithi171.stdout:3992 trunc from 0x3872baa to 0xbb6c727 2024-09-15T00:12:24.701 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:24.698+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:24.708 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:24.708 INFO:teuthology.orchestra.run.smithi171.stdout:3995 write 0x2136fa0 thru 0x213e03f (0x70a0 bytes) 2024-09-15T00:12:24.708 INFO:teuthology.orchestra.run.smithi171.stdout:3996 read 0x8cf8fda thru 0x8cfa4fa (0x1521 bytes) 2024-09-15T00:12:24.709 INFO:teuthology.orchestra.run.smithi171.stdout:3997 write 0x93a262 thru 0x94173f (0x74de bytes) 2024-09-15T00:12:24.725 INFO:teuthology.orchestra.run.smithi171.stdout:3998 trunc from 0xbb6c727 to 0x2b89db 2024-09-15T00:12:24.749 INFO:teuthology.orchestra.run.smithi171.stdout:4000 writesame 0x80a491f thru 0x80a6282 data_size 0x34(0x1964 bytes) 2024-09-15T00:12:24.749 INFO:teuthology.orchestra.run.smithi171.stdout:4006 write 0x30ad623 thru 0x30b9dae (0xc78c bytes) 2024-09-15T00:12:24.765 INFO:teuthology.orchestra.run.smithi171.stdout:4009 writesame 0xda91571 thru 0xdaa0b0c data_size 0x16(0xf59c bytes) 2024-09-15T00:12:24.765 INFO:teuthology.orchestra.run.smithi171.stdout:4013 read 0x3bc26c1 thru 0x3bd1ba1 (0xf4e1 bytes) 2024-09-15T00:12:24.766 INFO:teuthology.orchestra.run.smithi171.stdout:4014 writesame 0xc7aaedf thru 0xc7b90c0 data_size 0x1d(0xe1e2 bytes) 2024-09-15T00:12:24.771 INFO:teuthology.orchestra.run.smithi171.stdout:4015 clone 31 order 20 su 131072 sc 14 2024-09-15T00:12:24.792 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:24.789+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:24.803 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:24.803 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:24.812 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:24.896 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:24.894+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:24.961 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:24.961 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:24.973 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:24.994 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:24.992+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:25.095 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:25.093+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:25.117 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:25.117 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:25.126 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:25.210 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:25.208+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:25.279 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:25.280 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:25.289 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:25.297 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:25.295+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:25.334 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:12:25.378 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-15T00:12:25.398 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:25.396+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:25.408 INFO:teuthology.orchestra.run.smithi171.stdout:4016 punch from 0x335edf3 to 0x336ecb4, (0xfec1 bytes) 2024-09-15T00:12:25.412 INFO:teuthology.orchestra.run.smithi171.stdout:4018 writesame 0x9fdfd00 thru 0x9fee7da data_size 0x31(0xeadb bytes) 2024-09-15T00:12:25.418 INFO:teuthology.orchestra.run.smithi171.stdout:4019 read 0x4aeb726 thru 0x4af541e (0x9cf9 bytes) 2024-09-15T00:12:25.418 INFO:teuthology.orchestra.run.smithi171.stdout:4020 write 0x63f4652 thru 0x6402771 (0xe120 bytes) 2024-09-15T00:12:25.424 INFO:teuthology.orchestra.run.smithi171.stdout:4022 read 0xb2193ac thru 0xb22310d (0x9d62 bytes) 2024-09-15T00:12:25.424 INFO:teuthology.orchestra.run.smithi171.stdout:4023 writesame 0x608ed3a thru 0x609899a data_size 0x7(0x9c61 bytes) 2024-09-15T00:12:25.435 INFO:teuthology.orchestra.run.smithi171.stdout:4025 read 0x1574533 thru 0x15810c3 (0xcb91 bytes) 2024-09-15T00:12:25.436 INFO:teuthology.orchestra.run.smithi171.stdout:4026 writesame 0x75b09b3 thru 0x75b95fe data_size 0x29(0x8c4c bytes) 2024-09-15T00:12:25.441 INFO:teuthology.orchestra.run.smithi171.stdout:4027 punch from 0x2626e9e to 0x26329b3, (0xbb15 bytes) 2024-09-15T00:12:25.441 INFO:teuthology.orchestra.run.smithi171.stdout:4028 punch from 0xc188742 to 0xc18bf27, (0x37e5 bytes) 2024-09-15T00:12:25.441 INFO:teuthology.orchestra.run.smithi171.stdout:4030 punch from 0xabf116a to 0xabfe696, (0xd52c bytes) 2024-09-15T00:12:25.441 INFO:teuthology.orchestra.run.smithi171.stdout:4031 writesame 0x3be4902 thru 0x3bf331a data_size 0x1(0xea19 bytes) 2024-09-15T00:12:25.447 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:25.447 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:25.457 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:25.499 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:25.497+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:25.500 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:25.501 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:25.519 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:25.519 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:25.519 INFO:teuthology.orchestra.run.smithi171.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T00:12:25.535 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:25.536 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:25.552 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:25.553 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:25.555 INFO:teuthology.orchestra.run.smithi171.stdout:4032 punch from 0x5c44ca9 to 0x5c4e642, (0x9999 bytes) 2024-09-15T00:12:25.555 INFO:teuthology.orchestra.run.smithi171.stdout:4033 punch from 0xd1c758f to 0xd1d3d92, (0xc803 bytes) 2024-09-15T00:12:25.555 INFO:teuthology.orchestra.run.smithi171.stdout:4034 punch from 0xa89e51f to 0xa8a0e51, (0x2932 bytes) 2024-09-15T00:12:25.555 INFO:teuthology.orchestra.run.smithi171.stdout:4035 write 0xad1bb34 thru 0xad25b90 (0xa05d bytes) 2024-09-15T00:12:25.566 INFO:teuthology.orchestra.run.smithi171.stdout:4036 writesame 0xe0d1328 thru 0xe0d62c2 data_size 0x1(0x4f9b bytes) 2024-09-15T00:12:25.566 INFO:teuthology.orchestra.run.smithi171.stdout:4039 punch from 0xc923288 to 0xc92ef98, (0xbd10 bytes) 2024-09-15T00:12:25.576 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:25.576 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:25.581 INFO:teuthology.orchestra.run.smithi171.stdout:4040 trunc from 0xe0d62c3 to 0x37fb5ea 2024-09-15T00:12:25.597 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:25.597 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:25.600 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:25.598+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:25.603 INFO:teuthology.orchestra.run.smithi171.stdout:4041 write 0x62982e1 thru 0x629a65c (0x237c bytes) 2024-09-15T00:12:25.606 INFO:teuthology.orchestra.run.smithi171.stdout:4042 read 0x3200e48 thru 0x320a3da (0x9593 bytes) 2024-09-15T00:12:25.606 INFO:teuthology.orchestra.run.smithi171.stdout:4045 write 0x386c544 thru 0x386f45a (0x2f17 bytes) 2024-09-15T00:12:25.610 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:25.610 INFO:teuthology.orchestra.run.smithi171.stdout:4048 read 0x22c9970 thru 0x22d18ef (0x7f80 bytes) 2024-09-15T00:12:25.613 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.430370044999997, 2024-09-15T00:12:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:12:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:12:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:12:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:12:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:12:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:12:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:12:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:12:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:12:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:12:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:12:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:12:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:12:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:12:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:12:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:12:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:12:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:12:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:12:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:12:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:12:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:12:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:12:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:12:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.700528865999999, 2024-09-15T00:12:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:12:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:12:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:12:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:12:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:12:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:12:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:12:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:12:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:12:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:12:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:12:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:12:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:12:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:12:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:12:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:12:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:12:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:12:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:12:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.693570502999997, 2024-09-15T00:12:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:12:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:12:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:12:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:12:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:12:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:12:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:12:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:12:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:12:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.946381666000001, 2024-09-15T00:12:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:12:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:12:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:12:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:12:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:12:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:12:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:12:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:12:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:12:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:12:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:12:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:12:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:12:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:12:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:12:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:12:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:12:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:12:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:12:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:12:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:12:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:12:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:12:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:12:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97493 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.756798946, 2024-09-15T00:12:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033259141999999998, 2024-09-15T00:12:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97493 2024-09-15T00:12:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-15T00:12:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853401+0000", 2024-09-15T00:12:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:12:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853407+0000", 2024-09-15T00:12:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-15T00:12:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853411+0000", 2024-09-15T00:12:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.481e-06 2024-09-15T00:12:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886544+0000", 2024-09-15T00:12:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033132861999999999 2024-09-15T00:12:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886587+0000", 2024-09-15T00:12:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2942000000000003e-05 2024-09-15T00:12:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886655+0000", 2024-09-15T00:12:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7711000000000003e-05 2024-09-15T00:12:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97494 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.756777823, 2024-09-15T00:12:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033392390000000001, 2024-09-15T00:12:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97494 2024-09-15T00:12:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-15T00:12:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853419+0000", 2024-09-15T00:12:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.917e-06 2024-09-15T00:12:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853425+0000", 2024-09-15T00:12:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:12:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853433+0000", 2024-09-15T00:12:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8120000000000008e-06 2024-09-15T00:12:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886680+0000", 2024-09-15T00:12:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033247207000000001 2024-09-15T00:12:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886726+0000", 2024-09-15T00:12:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5902000000000002e-05 2024-09-15T00:12:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886809+0000", 2024-09-15T00:12:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3475999999999998e-05 2024-09-15T00:12:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97495 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.75675687, 2024-09-15T00:12:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033640503000000002, 2024-09-15T00:12:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97495 2024-09-15T00:12:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:12:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853440+0000", 2024-09-15T00:12:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.804e-06 2024-09-15T00:12:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853446+0000", 2024-09-15T00:12:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5559999999999998e-06 2024-09-15T00:12:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853450+0000", 2024-09-15T00:12:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9519999999999996e-06 2024-09-15T00:12:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886904+0000", 2024-09-15T00:12:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033454207999999999 2024-09-15T00:12:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887007+0000", 2024-09-15T00:12:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102904 2024-09-15T00:12:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887078+0000", 2024-09-15T00:12:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1719999999999998e-05 2024-09-15T00:12:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97496 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.756739772, 2024-09-15T00:12:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033802431000000001, 2024-09-15T00:12:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97496 2024-09-15T00:12:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.53e-07 2024-09-15T00:12:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853462+0000", 2024-09-15T00:12:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-15T00:12:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853468+0000", 2024-09-15T00:12:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3609999999999997e-06 2024-09-15T00:12:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853473+0000", 2024-09-15T00:12:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-15T00:12:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887111+0000", 2024-09-15T00:12:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033638376999999997 2024-09-15T00:12:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887178+0000", 2024-09-15T00:12:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6877000000000004e-05 2024-09-15T00:12:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887257+0000", 2024-09-15T00:12:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9616999999999993e-05 2024-09-15T00:12:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97587 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.756716537999999, 2024-09-15T00:12:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033973642999999998, 2024-09-15T00:12:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97587 2024-09-15T00:12:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853479+0000", 2024-09-15T00:12:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:12:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853481+0000", 2024-09-15T00:12:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.136e-06 2024-09-15T00:12:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853487+0000", 2024-09-15T00:12:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2439999999999998e-06 2024-09-15T00:12:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853492+0000", 2024-09-15T00:12:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:12:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887284+0000", 2024-09-15T00:12:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033791827000000003 2024-09-15T00:12:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887361+0000", 2024-09-15T00:12:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6983000000000005e-05 2024-09-15T00:12:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887452+0000", 2024-09-15T00:12:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1236999999999994e-05 2024-09-15T00:12:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97588 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.756696381000001, 2024-09-15T00:12:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034134617999999999, 2024-09-15T00:12:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97588 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853499+0000", 2024-09-15T00:12:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:12:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853501+0000", 2024-09-15T00:12:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.26e-06 2024-09-15T00:12:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853507+0000", 2024-09-15T00:12:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2899999999999999e-06 2024-09-15T00:12:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853512+0000", 2024-09-15T00:12:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8330000000000001e-06 2024-09-15T00:12:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887495+0000", 2024-09-15T00:12:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033982529999999997 2024-09-15T00:12:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887548+0000", 2024-09-15T00:12:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3445000000000001e-05 2024-09-15T00:12:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887633+0000", 2024-09-15T00:12:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4890999999999997e-05 2024-09-15T00:12:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126866 3.6 3.973c9fa6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.653374436, 2024-09-15T00:12:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153301000000003, 2024-09-15T00:12:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126866 2024-09-15T00:12:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956821+0000", 2024-09-15T00:12:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.91e-07 2024-09-15T00:12:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956822+0000", 2024-09-15T00:12:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.705e-06 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956828+0000", 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6760000000000004e-06 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956833+0000", 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5379999999999996e-06 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989970+0000", 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033137560000000003 2024-09-15T00:12:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989974+0000", 2024-09-15T00:12:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-15T00:12:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126951 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.6532487529999997, 2024-09-15T00:12:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033152585999999998, 2024-09-15T00:12:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126951 2024-09-15T00:12:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-07 2024-09-15T00:12:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956953+0000", 2024-09-15T00:12:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9310000000000002e-06 2024-09-15T00:12:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956957+0000", 2024-09-15T00:12:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5379999999999998e-06 2024-09-15T00:12:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956959+0000", 2024-09-15T00:12:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6599999999999999e-06 2024-09-15T00:12:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990095+0000", 2024-09-15T00:12:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033135855999999998 2024-09-15T00:12:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990099+0000", 2024-09-15T00:12:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4010000000000001e-06 2024-09-15T00:12:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126952 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.653231764, 2024-09-15T00:12:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033156066999999997, 2024-09-15T00:12:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126952 2024-09-15T00:12:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:12:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956965+0000", 2024-09-15T00:12:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3379999999999999e-06 2024-09-15T00:12:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956968+0000", 2024-09-15T00:12:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0419999999999998e-06 2024-09-15T00:12:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956970+0000", 2024-09-15T00:12:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4899999999999999e-06 2024-09-15T00:12:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990113+0000", 2024-09-15T00:12:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033142642 2024-09-15T00:12:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990119+0000", 2024-09-15T00:12:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2979999999999997e-06 2024-09-15T00:12:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126954 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.6531862689999999, 2024-09-15T00:12:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033158449, 2024-09-15T00:12:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126954 2024-09-15T00:12:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-15T00:12:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957010+0000", 2024-09-15T00:12:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3659999999999999e-06 2024-09-15T00:12:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957013+0000", 2024-09-15T00:12:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1140000000000002e-06 2024-09-15T00:12:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957016+0000", 2024-09-15T00:12:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1209999999999998e-06 2024-09-15T00:12:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990163+0000", 2024-09-15T00:12:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033146451 2024-09-15T00:12:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990167+0000", 2024-09-15T00:12:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0579999999999999e-06 2024-09-15T00:12:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126955 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.6531750550000002, 2024-09-15T00:12:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033160617000000003, 2024-09-15T00:12:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126955 2024-09-15T00:12:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-15T00:12:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957021+0000", 2024-09-15T00:12:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-15T00:12:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957024+0000", 2024-09-15T00:12:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1599999999999998e-06 2024-09-15T00:12:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957027+0000", 2024-09-15T00:12:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000001e-06 2024-09-15T00:12:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990177+0000", 2024-09-15T00:12:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033149741000000003 2024-09-15T00:12:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990180+0000", 2024-09-15T00:12:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4910000000000001e-06 2024-09-15T00:12:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126956 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:25.681 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.6531649339999999, 2024-09-15T00:12:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033163633999999997, 2024-09-15T00:12:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126956 2024-09-15T00:12:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957031+0000", 2024-09-15T00:12:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-06 2024-09-15T00:12:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957034+0000", 2024-09-15T00:12:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9960000000000002e-06 2024-09-15T00:12:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957037+0000", 2024-09-15T00:12:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6419999999999999e-06 2024-09-15T00:12:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990190+0000", 2024-09-15T00:12:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153152999999998 2024-09-15T00:12:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990193+0000", 2024-09-15T00:12:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6069999999999999e-06 2024-09-15T00:12:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126957 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.6531552839999999, 2024-09-15T00:12:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033166442999999997, 2024-09-15T00:12:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126957 2024-09-15T00:12:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957040+0000", 2024-09-15T00:12:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:12:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957041+0000", 2024-09-15T00:12:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0440000000000001e-06 2024-09-15T00:12:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957044+0000", 2024-09-15T00:12:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0170000000000001e-06 2024-09-15T00:12:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957048+0000", 2024-09-15T00:12:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0319999999999997e-06 2024-09-15T00:12:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990202+0000", 2024-09-15T00:12:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033154640999999999 2024-09-15T00:12:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990206+0000", 2024-09-15T00:12:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.507e-06 2024-09-15T00:12:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.689 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126958 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.6531440540000002, 2024-09-15T00:12:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033167613999999998, 2024-09-15T00:12:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126958 2024-09-15T00:12:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:12:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957053+0000", 2024-09-15T00:12:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.756e-06 2024-09-15T00:12:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957059+0000", 2024-09-15T00:12:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2770000000000002e-06 2024-09-15T00:12:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957063+0000", 2024-09-15T00:12:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2459999999999997e-06 2024-09-15T00:12:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990215+0000", 2024-09-15T00:12:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033151509000000003 2024-09-15T00:12:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990218+0000", 2024-09-15T00:12:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5099999999999999e-06 2024-09-15T00:12:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126959 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:25.693 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.6531250499999999, 2024-09-15T00:12:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033161673000000003, 2024-09-15T00:12:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126959 2024-09-15T00:12:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:12:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957072+0000", 2024-09-15T00:12:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2239999999999998e-06 2024-09-15T00:12:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957079+0000", 2024-09-15T00:12:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.19e-06 2024-09-15T00:12:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957083+0000", 2024-09-15T00:12:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:12:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990228+0000", 2024-09-15T00:12:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033144818 2024-09-15T00:12:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990231+0000", 2024-09-15T00:12:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:12:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.697 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126960 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.6531062429999999, 2024-09-15T00:12:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033155166, 2024-09-15T00:12:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126960 2024-09-15T00:12:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:25.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957091+0000", 2024-09-15T00:12:25.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:12:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957097+0000", 2024-09-15T00:12:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6050000000000003e-06 2024-09-15T00:12:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957102+0000", 2024-09-15T00:12:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5539999999999999e-06 2024-09-15T00:12:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990240+0000", 2024-09-15T00:12:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033138289000000001 2024-09-15T00:12:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990244+0000", 2024-09-15T00:12:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4740000000000001e-06 2024-09-15T00:12:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:25.701 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:25.701 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:25.702 INFO:teuthology.orchestra.run.smithi171.stdout:4049 trunc from 0x629a65d to 0x4ba7485 2024-09-15T00:12:25.702 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:25.703 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:12:25.703 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:25.703 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:12:25.704 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:25.706 INFO:teuthology.orchestra.run.smithi171.stdout:4051 read 0x36e41d1 thru 0x36e8b56 (0x4986 bytes) 2024-09-15T00:12:25.707 INFO:teuthology.orchestra.run.smithi171.stdout:4052 read 0x8b8e18 thru 0x8bdfee (0x51d7 bytes) 2024-09-15T00:12:25.707 INFO:teuthology.orchestra.run.smithi171.stdout:4053 writesame 0xc0fff64 thru 0xc107c13 data_size 0xc(0x7cb0 bytes) 2024-09-15T00:12:25.707 INFO:teuthology.orchestra.run.smithi171.stdout:4054 read 0x98f0394 thru 0x98fc663 (0xc2d0 bytes) 2024-09-15T00:12:25.707 INFO:teuthology.orchestra.run.smithi171.stdout:4055 punch from 0x9df4eff to 0x9dfcf0f, (0x8010 bytes) 2024-09-15T00:12:25.707 INFO:teuthology.orchestra.run.smithi171.stdout:4056 write 0x5822eeb thru 0x5827d07 (0x4e1d bytes) 2024-09-15T00:12:25.707 INFO:teuthology.orchestra.run.smithi171.stdout:4057 write 0xc9fa80b thru 0xca0751c (0xcd12 bytes) 2024-09-15T00:12:25.707 INFO:teuthology.orchestra.run.smithi171.stdout:4058 trunc from 0xca0751d to 0x67b8c08 2024-09-15T00:12:25.707 INFO:teuthology.orchestra.run.smithi171.stdout:4059 read 0x558696 thru 0x5625a6 (0x9f11 bytes) 2024-09-15T00:12:25.707 INFO:teuthology.orchestra.run.smithi171.stdout:4060 write 0x68f35d7 thru 0x68f9091 (0x5abb bytes) 2024-09-15T00:12:25.707 INFO:teuthology.orchestra.run.smithi171.stdout:4061 write 0xa7a80ee thru 0xa7b04e6 (0x83f9 bytes) 2024-09-15T00:12:25.707 INFO:teuthology.orchestra.run.smithi171.stdout:4062 write 0x3e2943e thru 0x3e33664 (0xa227 bytes) 2024-09-15T00:12:25.707 INFO:teuthology.orchestra.run.smithi171.stdout:4063 write 0x9c38549 thru 0x9c44516 (0xbfce bytes) 2024-09-15T00:12:25.707 INFO:teuthology.orchestra.run.smithi171.stdout:4064 read 0x73b7a7a thru 0x73c4721 (0xcca8 bytes) 2024-09-15T00:12:25.708 INFO:teuthology.orchestra.run.smithi171.stdout:4066 write 0x89c4671 thru 0x89cab1f (0x64af bytes) 2024-09-15T00:12:25.708 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:25.704+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:25.710 INFO:teuthology.orchestra.run.smithi171.stdout:4068 write 0x34ce6aa thru 0x34d6eae (0x8805 bytes) 2024-09-15T00:12:25.725 INFO:teuthology.orchestra.run.smithi171.stdout:4070 write 0xdd33326 thru 0xdd42593 (0xf26e bytes) 2024-09-15T00:12:25.730 INFO:teuthology.orchestra.run.smithi171.stdout:4071 read 0xb18d937 thru 0xb193eb7 (0x6581 bytes) 2024-09-15T00:12:25.731 INFO:teuthology.orchestra.run.smithi171.stdout:4072 read 0x66a9859 thru 0x66aa2b8 (0xa60 bytes) 2024-09-15T00:12:25.731 INFO:teuthology.orchestra.run.smithi171.stdout:4073 read 0xaa03ca2 thru 0xaa06fc1 (0x3320 bytes) 2024-09-15T00:12:25.731 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:25.731 INFO:teuthology.orchestra.run.smithi171.stdout:4076 read 0x87d1be8 thru 0x87e12f3 (0xf70c bytes) 2024-09-15T00:12:25.731 INFO:teuthology.orchestra.run.smithi171.stdout:4077 writesame 0x7206012 thru 0x7206731 data_size 0x6(0x720 bytes) 2024-09-15T00:12:25.733 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:25.740 INFO:teuthology.orchestra.run.smithi171.stdout:4079 writesame 0xe80e956 thru 0xe8149b5 data_size 0x30(0x6060 bytes) 2024-09-15T00:12:25.740 INFO:teuthology.orchestra.run.smithi171.stdout:4080 write 0xd42cced thru 0xd43b590 (0xe8a4 bytes) 2024-09-15T00:12:25.745 INFO:teuthology.orchestra.run.smithi171.stdout:4081 read 0x2ba2bb3 thru 0x2ba4637 (0x1a85 bytes) 2024-09-15T00:12:25.745 INFO:teuthology.orchestra.run.smithi171.stdout:4082 read 0x88b59af thru 0x88bb15d (0x57af bytes) 2024-09-15T00:12:25.750 INFO:teuthology.orchestra.run.smithi171.stdout:4083 trunc from 0xe8149b6 to 0xac119f0 2024-09-15T00:12:25.759 INFO:teuthology.orchestra.run.smithi171.stdout:4084 write 0x2669faf thru 0x26755b2 (0xb604 bytes) 2024-09-15T00:12:25.772 INFO:teuthology.orchestra.run.smithi171.stdout:4085 trunc from 0xac119f0 to 0x4c11e55 2024-09-15T00:12:25.799 INFO:teuthology.orchestra.run.smithi171.stdout:4088 writesame 0xd38cfbd thru 0xd39110e data_size 0x9(0x4152 bytes) 2024-09-15T00:12:25.806 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:25.804+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:25.807 INFO:teuthology.orchestra.run.smithi171.stdout:4089 write 0xeb64c69 thru 0xeb6f5ef (0xa987 bytes) 2024-09-15T00:12:25.816 INFO:teuthology.orchestra.run.smithi171.stdout:4091 trunc from 0xeb6f5f0 to 0xa9dbb68 2024-09-15T00:12:25.823 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:25.823 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:25.823 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:25.825 INFO:teuthology.orchestra.run.smithi171.stdout:4092 write 0x15a3c80 thru 0x15a4787 (0xb08 bytes) 2024-09-15T00:12:25.833 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:25.840 INFO:teuthology.orchestra.run.smithi171.stdout:4093 trunc from 0xa9dbb68 to 0x1eae5be 2024-09-15T00:12:25.871 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:25.881 INFO:teuthology.orchestra.run.smithi171.stdout:4097 write 0x61f44ee thru 0x6200c7a (0xc78d bytes) 2024-09-15T00:12:25.884 INFO:teuthology.orchestra.run.smithi171.stdout:4098 punch from 0x4e3c497 to 0x4e3cc66, (0x7cf bytes) 2024-09-15T00:12:25.899 INFO:teuthology.orchestra.run.smithi171.stdout:4099 write 0xe24686e thru 0xe251cfe (0xb491 bytes) 2024-09-15T00:12:25.907 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:25.906+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:25.912 INFO:teuthology.orchestra.run.smithi171.stdout:4100 trunc from 0xe251cff to 0x67c2cd0 2024-09-15T00:12:25.923 INFO:teuthology.orchestra.run.smithi171.stdout:4102 read 0x40fe12 thru 0x41b5a0 (0xb78f bytes) 2024-09-15T00:12:25.924 INFO:teuthology.orchestra.run.smithi171.stdout:4103 read 0x5232bd0 thru 0x5235f41 (0x3372 bytes) 2024-09-15T00:12:25.924 INFO:teuthology.orchestra.run.smithi171.stdout:4104 read 0x6006d4b thru 0x600eb45 (0x7dfb bytes) 2024-09-15T00:12:25.924 INFO:teuthology.orchestra.run.smithi171.stdout:4105 read 0x490af54 thru 0x4910206 (0x52b3 bytes) 2024-09-15T00:12:25.924 INFO:teuthology.orchestra.run.smithi171.stdout:4107 punch from 0x207f026 to 0x207f546, (0x520 bytes) 2024-09-15T00:12:25.924 INFO:teuthology.orchestra.run.smithi171.stdout:4108 writesame 0x50db48b thru 0x50e08c2 data_size 0x8c(0x5438 bytes) 2024-09-15T00:12:25.931 INFO:teuthology.orchestra.run.smithi171.stdout:4109 trunc from 0x67c2cd0 to 0x34e9ff6 2024-09-15T00:12:25.941 INFO:teuthology.orchestra.run.smithi171.stdout:4110 punch from 0x232f995 to 0x233e13a, (0xe7a5 bytes) 2024-09-15T00:12:25.941 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:25.942 INFO:teuthology.orchestra.run.smithi171.stdout:4116 punch from 0x1b304d2 to 0x1b4032c, (0xfe5a bytes) 2024-09-15T00:12:25.951 INFO:teuthology.orchestra.run.smithi171.stdout:4117 write 0x70a468c thru 0x70a691b (0x2290 bytes) 2024-09-15T00:12:25.962 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:25.962 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:25.962 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:25.962 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:25.962 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:25.966 INFO:teuthology.orchestra.run.smithi171.stdout:4118 write 0xc1835e4 thru 0xc18e56f (0xaf8c bytes) 2024-09-15T00:12:25.971 INFO:teuthology.orchestra.run.smithi171.stdout:4119 punch from 0x6305a0 to 0x6385bc, (0x801c bytes) 2024-09-15T00:12:25.971 INFO:teuthology.orchestra.run.smithi171.stdout:4120 read 0x9e05997 thru 0x9e0d5c6 (0x7c30 bytes) 2024-09-15T00:12:25.972 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:25.977 INFO:teuthology.orchestra.run.smithi171.stdout:4122 write 0xc355176 thru 0xc360ad7 (0xb962 bytes) 2024-09-15T00:12:25.982 INFO:teuthology.orchestra.run.smithi171.stdout:4123 write 0xa77976a thru 0xa780852 (0x70e9 bytes) 2024-09-15T00:12:25.987 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:25.988 INFO:teuthology.orchestra.run.smithi171.stdout:4127 flatten 2024-09-15T00:12:26.008 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:26.006+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:26.018 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:26.018 INFO:teuthology.orchestra.run.smithi171.stdout:4129 read 0xc6a8dc thru 0xc6c33f (0x1a64 bytes) 2024-09-15T00:12:26.018 INFO:teuthology.orchestra.run.smithi171.stdout:4131 punch from 0x52d22a8 to 0x52e0a82, (0xe7da bytes) 2024-09-15T00:12:26.018 INFO:teuthology.orchestra.run.smithi171.stdout:4132 read 0x35291ce thru 0x352ce53 (0x3c86 bytes) 2024-09-15T00:12:26.018 INFO:teuthology.orchestra.run.smithi171.stdout:4133 read 0x4b525b2 thru 0x4b5789c (0x52eb bytes) 2024-09-15T00:12:26.018 INFO:teuthology.orchestra.run.smithi171.stdout:4134 write 0x77e0b73 thru 0x77eb805 (0xac93 bytes) 2024-09-15T00:12:26.022 INFO:teuthology.orchestra.run.smithi171.stdout:4136 read 0x77d666e thru 0x77ddaf1 (0x7484 bytes) 2024-09-15T00:12:26.029 INFO:teuthology.orchestra.run.smithi171.stdout:4137 trunc from 0xc360ad8 to 0x6ad5d4a 2024-09-15T00:12:26.052 INFO:teuthology.orchestra.run.smithi171.stdout:4138 write 0xc3cba4e thru 0xc3db732 (0xfce5 bytes) 2024-09-15T00:12:26.055 INFO:teuthology.orchestra.run.smithi171.stdout:4139 punch from 0xb9597b8 to 0xb961ed2, (0x871a bytes) 2024-09-15T00:12:26.067 INFO:teuthology.orchestra.run.smithi171.stdout:4140 trunc from 0xc3db733 to 0x3185556 2024-09-15T00:12:26.079 INFO:teuthology.orchestra.run.smithi171.stdout:4141 write 0xae9e67 thru 0xaf8837 (0xe9d1 bytes) 2024-09-15T00:12:26.096 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:12:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.850796785, 2024-09-15T00:12:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:12:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:12:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:12:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:12:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:12:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:12:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:12:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:12:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:12:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:12:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:12:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:12:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:12:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:12:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:12:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:12:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:12:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:12:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:12:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:12:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:12:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:12:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:12:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:12:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58878 3.8 3:12456482:::rbd_data.1139766160da.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.804491200999998, 2024-09-15T00:12:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061956199000000003, 2024-09-15T00:12:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58878 2024-09-15T00:12:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289151+0000", 2024-09-15T00:12:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:12:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289473+0000", 2024-09-15T00:12:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032183800000000002 2024-09-15T00:12:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289487+0000", 2024-09-15T00:12:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-05 2024-09-15T00:12:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289493+0000", 2024-09-15T00:12:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4489999999999996e-06 2024-09-15T00:12:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289542+0000", 2024-09-15T00:12:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8844999999999997e-05 2024-09-15T00:12:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289642+0000", 2024-09-15T00:12:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9607999999999999e-05 2024-09-15T00:12:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.293908+0000", 2024-09-15T00:12:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659350000000002 2024-09-15T00:12:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351022+0000", 2024-09-15T00:12:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057114535000000001 2024-09-15T00:12:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351033+0000", 2024-09-15T00:12:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0263e-05 2024-09-15T00:12:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351070+0000", 2024-09-15T00:12:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7666000000000002e-05 2024-09-15T00:12:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351106+0000", 2024-09-15T00:12:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999997e-05 2024-09-15T00:12:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58879 3.7 3:e169894d:::rbd_data.1139766160da.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.802139713000003, 2024-09-15T00:12:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043716294000000003, 2024-09-15T00:12:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58879 2024-09-15T00:12:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291502+0000", 2024-09-15T00:12:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-15T00:12:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291664+0000", 2024-09-15T00:12:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016216599999999999 2024-09-15T00:12:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291674+0000", 2024-09-15T00:12:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0276e-05 2024-09-15T00:12:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291680+0000", 2024-09-15T00:12:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2580000000000003e-06 2024-09-15T00:12:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291732+0000", 2024-09-15T00:12:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2114999999999998e-05 2024-09-15T00:12:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291866+0000", 2024-09-15T00:12:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013416900000000001 2024-09-15T00:12:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.296426+0000", 2024-09-15T00:12:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045595779999999999 2024-09-15T00:12:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335075+0000", 2024-09-15T00:12:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038649347000000001 2024-09-15T00:12:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335097+0000", 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2565000000000002e-05 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335161+0000", 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3928000000000002e-05 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335217+0000", 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6042000000000003e-05 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58880 3.8 3:13d91cfd:::rbd_data.1139766160da.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.798747605999999, 2024-09-15T00:12:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056331463999999998, 2024-09-15T00:12:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58880 2024-09-15T00:12:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294896+0000", 2024-09-15T00:12:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.356e-06 2024-09-15T00:12:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295288+0000", 2024-09-15T00:12:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039223200000000001 2024-09-15T00:12:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295304+0000", 2024-09-15T00:12:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5937e-05 2024-09-15T00:12:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295312+0000", 2024-09-15T00:12:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1489999999999994e-06 2024-09-15T00:12:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333162+0000", 2024-09-15T00:12:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037849840000000003 2024-09-15T00:12:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333269+0000", 2024-09-15T00:12:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010749800000000001 2024-09-15T00:12:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.336198+0000", 2024-09-15T00:12:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029284739999999999 2024-09-15T00:12:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351137+0000", 2024-09-15T00:12:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014939103 2024-09-15T00:12:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351150+0000", 2024-09-15T00:12:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3542e-05 2024-09-15T00:12:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351191+0000", 2024-09-15T00:12:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0791000000000003e-05 2024-09-15T00:12:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351225+0000", 2024-09-15T00:12:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3541999999999998e-05 2024-09-15T00:12:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58878 3.8 e56/23 ondisk, result = 0)", 2024-09-15T00:12:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.792217823000001, 2024-09-15T00:12:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049689096000000002, 2024-09-15T00:12:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301425+0000", 2024-09-15T00:12:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.782e-06 2024-09-15T00:12:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301433+0000", 2024-09-15T00:12:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689999999999994e-06 2024-09-15T00:12:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301449+0000", 2024-09-15T00:12:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5767e-05 2024-09-15T00:12:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301458+0000", 2024-09-15T00:12:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:12:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351009+0000", 2024-09-15T00:12:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049551523 2024-09-15T00:12:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351017+0000", 2024-09-15T00:12:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8359999999999992e-06 2024-09-15T00:12:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351112+0000", 2024-09-15T00:12:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5173999999999994e-05 2024-09-15T00:12:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127003 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 6758400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968309+0000", 2024-09-15T00:12:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.1253318080000003, 2024-09-15T00:12:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040647106000000002, 2024-09-15T00:12:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127003 2024-09-15T00:12:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968309+0000", 2024-09-15T00:12:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968309+0000", 2024-09-15T00:12:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968309+0000", 2024-09-15T00:12:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.03e-07 2024-09-15T00:12:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968312+0000", 2024-09-15T00:12:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2249999999999999e-06 2024-09-15T00:12:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968318+0000", 2024-09-15T00:12:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9050000000000002e-06 2024-09-15T00:12:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968322+0000", 2024-09-15T00:12:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8289999999999998e-06 2024-09-15T00:12:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008876+0000", 2024-09-15T00:12:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040553176000000003 2024-09-15T00:12:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008918+0000", 2024-09-15T00:12:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2147000000000002e-05 2024-09-15T00:12:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008956+0000", 2024-09-15T00:12:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8420999999999997e-05 2024-09-15T00:12:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127004 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 6881280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.1253123589999996, 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735720000000003, 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127004 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:12:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968331+0000", 2024-09-15T00:12:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.849e-06 2024-09-15T00:12:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968334+0000", 2024-09-15T00:12:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7689999999999998e-06 2024-09-15T00:12:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968338+0000", 2024-09-15T00:12:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2490000000000001e-06 2024-09-15T00:12:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008987+0000", 2024-09-15T00:12:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040649117999999998 2024-09-15T00:12:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009027+0000", 2024-09-15T00:12:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0357e-05 2024-09-15T00:12:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009064+0000", 2024-09-15T00:12:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7036000000000003e-05 2024-09-15T00:12:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127005 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7004160~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.1252998070000002, 2024-09-15T00:12:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040824358999999998, 2024-09-15T00:12:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127005 2024-09-15T00:12:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-15T00:12:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968343+0000", 2024-09-15T00:12:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3430000000000001e-06 2024-09-15T00:12:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968349+0000", 2024-09-15T00:12:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3080000000000004e-06 2024-09-15T00:12:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968355+0000", 2024-09-15T00:12:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9499999999999998e-06 2024-09-15T00:12:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009091+0000", 2024-09-15T00:12:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735645000000001 2024-09-15T00:12:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009130+0000", 2024-09-15T00:12:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8951e-05 2024-09-15T00:12:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009165+0000", 2024-09-15T00:12:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5926000000000001e-05 2024-09-15T00:12:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127006 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7127040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.1252827490000001, 2024-09-15T00:12:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040912783000000001, 2024-09-15T00:12:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127006 2024-09-15T00:12:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968359+0000", 2024-09-15T00:12:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0419999999999999e-06 2024-09-15T00:12:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968363+0000", 2024-09-15T00:12:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5999999999999998e-06 2024-09-15T00:12:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968366+0000", 2024-09-15T00:12:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5179999999999999e-06 2024-09-15T00:12:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009196+0000", 2024-09-15T00:12:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040830313 2024-09-15T00:12:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009234+0000", 2024-09-15T00:12:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8562000000000001e-05 2024-09-15T00:12:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009271+0000", 2024-09-15T00:12:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6538000000000002e-05 2024-09-15T00:12:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127007 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7249920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:12:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.12527255, 2024-09-15T00:12:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041019218000000003, 2024-09-15T00:12:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127007 2024-09-15T00:12:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:12:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:12:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968369+0000", 2024-09-15T00:12:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-15T00:12:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968370+0000", 2024-09-15T00:12:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.012e-06 2024-09-15T00:12:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968374+0000", 2024-09-15T00:12:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3839999999999999e-06 2024-09-15T00:12:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968376+0000", 2024-09-15T00:12:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5339999999999998e-06 2024-09-15T00:12:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009292+0000", 2024-09-15T00:12:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040915092 2024-09-15T00:12:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009333+0000", 2024-09-15T00:12:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1105e-05 2024-09-15T00:12:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009388+0000", 2024-09-15T00:12:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4868999999999999e-05 2024-09-15T00:12:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127008 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7364608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:12:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.125261429, 2024-09-15T00:12:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041115290999999998, 2024-09-15T00:12:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127008 2024-09-15T00:12:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:12:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:12:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968380+0000", 2024-09-15T00:12:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3999999999999998e-07 2024-09-15T00:12:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968381+0000", 2024-09-15T00:12:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8700000000000004e-07 2024-09-15T00:12:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968385+0000", 2024-09-15T00:12:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9260000000000002e-06 2024-09-15T00:12:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968389+0000", 2024-09-15T00:12:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.685e-06 2024-09-15T00:12:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009419+0000", 2024-09-15T00:12:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041029852999999998 2024-09-15T00:12:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009458+0000", 2024-09-15T00:12:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8538000000000003e-05 2024-09-15T00:12:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009495+0000", 2024-09-15T00:12:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7061999999999999e-05 2024-09-15T00:12:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127009 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7487488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.1252273119999998, 2024-09-15T00:12:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041233063, 2024-09-15T00:12:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127009 2024-09-15T00:12:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7199999999999999e-07 2024-09-15T00:12:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968417+0000", 2024-09-15T00:12:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3810000000000002e-06 2024-09-15T00:12:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968423+0000", 2024-09-15T00:12:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8820000000000003e-06 2024-09-15T00:12:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968433+0000", 2024-09-15T00:12:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0005e-05 2024-09-15T00:12:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009527+0000", 2024-09-15T00:12:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041093750999999998 2024-09-15T00:12:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009601+0000", 2024-09-15T00:12:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3979000000000004e-05 2024-09-15T00:12:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009647+0000", 2024-09-15T00:12:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5493e-05 2024-09-15T00:12:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127010 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7610368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.1251983079999999, 2024-09-15T00:12:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041325924999999999, 2024-09-15T00:12:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127010 2024-09-15T00:12:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:12:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968445+0000", 2024-09-15T00:12:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:12:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968452+0000", 2024-09-15T00:12:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-15T00:12:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968456+0000", 2024-09-15T00:12:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5210000000000004e-06 2024-09-15T00:12:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009664+0000", 2024-09-15T00:12:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041207404000000003 2024-09-15T00:12:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009723+0000", 2024-09-15T00:12:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9036000000000002e-05 2024-09-15T00:12:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009768+0000", 2024-09-15T00:12:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5578000000000003e-05 2024-09-15T00:12:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127011 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7733248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.1251800730000001, 2024-09-15T00:12:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041482441000000002, 2024-09-15T00:12:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127011 2024-09-15T00:12:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-15T00:12:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968466+0000", 2024-09-15T00:12:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:12:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968470+0000", 2024-09-15T00:12:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6469999999999997e-06 2024-09-15T00:12:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968473+0000", 2024-09-15T00:12:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.977e-06 2024-09-15T00:12:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009800+0000", 2024-09-15T00:12:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041326918999999997 2024-09-15T00:12:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009869+0000", 2024-09-15T00:12:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8935000000000003e-05 2024-09-15T00:12:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009943+0000", 2024-09-15T00:12:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4188000000000002e-05 2024-09-15T00:12:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127012 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7856128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:12:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.125164571, 2024-09-15T00:12:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041640756000000001, 2024-09-15T00:12:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127012 2024-09-15T00:12:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:12:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:12:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968477+0000", 2024-09-15T00:12:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968482+0000", 2024-09-15T00:12:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6180000000000003e-06 2024-09-15T00:12:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968486+0000", 2024-09-15T00:12:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.501e-06 2024-09-15T00:12:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968489+0000", 2024-09-15T00:12:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1530000000000001e-06 2024-09-15T00:12:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009983+0000", 2024-09-15T00:12:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041493980999999999 2024-09-15T00:12:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010051+0000", 2024-09-15T00:12:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8039000000000004e-05 2024-09-15T00:12:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010117+0000", 2024-09-15T00:12:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6254000000000006e-05 2024-09-15T00:12:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127013 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7979008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:12:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.1251487510000002, 2024-09-15T00:12:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041763863999999998, 2024-09-15T00:12:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127013 2024-09-15T00:12:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:12:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:12:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969767+0000", 2024-09-15T00:12:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001275006 2024-09-15T00:12:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969772+0000", 2024-09-15T00:12:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4769999999999997e-06 2024-09-15T00:12:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969781+0000", 2024-09-15T00:12:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1959999999999997e-06 2024-09-15T00:12:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969788+0000", 2024-09-15T00:12:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6359999999999994e-06 2024-09-15T00:12:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010162+0000", 2024-09-15T00:12:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040373291999999998 2024-09-15T00:12:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010212+0000", 2024-09-15T00:12:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0543e-05 2024-09-15T00:12:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010256+0000", 2024-09-15T00:12:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3714000000000002e-05 2024-09-15T00:12:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127014 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8101888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969802+0000", 2024-09-15T00:12:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.1238387479999998, 2024-09-15T00:12:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040616513, 2024-09-15T00:12:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127014 2024-09-15T00:12:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969802+0000", 2024-09-15T00:12:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969802+0000", 2024-09-15T00:12:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969803+0000", 2024-09-15T00:12:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6100000000000001e-07 2024-09-15T00:12:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969805+0000", 2024-09-15T00:12:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7300000000000001e-06 2024-09-15T00:12:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969813+0000", 2024-09-15T00:12:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8299999999999996e-06 2024-09-15T00:12:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969823+0000", 2024-09-15T00:12:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5829999999999996e-06 2024-09-15T00:12:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010290+0000", 2024-09-15T00:12:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040466886000000001 2024-09-15T00:12:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010367+0000", 2024-09-15T00:12:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7763999999999996e-05 2024-09-15T00:12:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010419+0000", 2024-09-15T00:12:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1159000000000003e-05 2024-09-15T00:12:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127015 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8224768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:12:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.1238098150000004, 2024-09-15T00:12:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040710186000000002, 2024-09-15T00:12:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127015 2024-09-15T00:12:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:12:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:12:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969832+0000", 2024-09-15T00:12:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:12:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969834+0000", 2024-09-15T00:12:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2919999999999998e-06 2024-09-15T00:12:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969843+0000", 2024-09-15T00:12:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0480000000000004e-06 2024-09-15T00:12:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969848+0000", 2024-09-15T00:12:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-15T00:12:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010457+0000", 2024-09-15T00:12:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040609054999999998 2024-09-15T00:12:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010501+0000", 2024-09-15T00:12:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3396000000000003e-05 2024-09-15T00:12:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010541+0000", 2024-09-15T00:12:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0599000000000002e-05 2024-09-15T00:12:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127016 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.1237591890000003, 2024-09-15T00:12:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040766216000000001, 2024-09-15T00:12:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127016 2024-09-15T00:12:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:12:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969886+0000", 2024-09-15T00:12:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8449999999999996e-06 2024-09-15T00:12:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969897+0000", 2024-09-15T00:12:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-15T00:12:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969904+0000", 2024-09-15T00:12:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1169999999999998e-06 2024-09-15T00:12:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010561+0000", 2024-09-15T00:12:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040657470000000001 2024-09-15T00:12:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010607+0000", 2024-09-15T00:12:26.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6108e-05 2024-09-15T00:12:26.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010648+0000", 2024-09-15T00:12:26.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0479000000000003e-05 2024-09-15T00:12:26.190 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.190 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.190 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.190 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127694 3.9 3:9390b205:::rbd_data.11396481e1ec.00000000000000df:head [set-alloc-hint object_size 1048576 write_size 1048576,write 819200~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T00:12:26.191 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:12:26.191 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.8690400719999998, 2024-09-15T00:12:26.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90376419399999997, 2024-09-15T00:12:26.191 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:26.191 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:26.191 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:26.191 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:26.191 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:26.191 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.191 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127694 2024-09-15T00:12:26.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.191 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:26.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:26.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:12:26.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:26.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:12:26.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:26.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:26.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224602+0000", 2024-09-15T00:12:26.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5899999999999995e-07 2024-09-15T00:12:26.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:26.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224610+0000", 2024-09-15T00:12:26.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9130000000000003e-06 2024-09-15T00:12:26.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:26.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224620+0000", 2024-09-15T00:12:26.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0426e-05 2024-09-15T00:12:26.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224627+0000", 2024-09-15T00:12:26.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4390000000000003e-06 2024-09-15T00:12:26.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224675+0000", 2024-09-15T00:12:26.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7636000000000003e-05 2024-09-15T00:12:26.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:12:26.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224681+0000", 2024-09-15T00:12:26.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1460000000000001e-06 2024-09-15T00:12:26.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:26.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126388+0000", 2024-09-15T00:12:26.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90170724999999996 2024-09-15T00:12:26.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:26.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126526+0000", 2024-09-15T00:12:26.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013730599999999999 2024-09-15T00:12:26.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:26.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126684+0000", 2024-09-15T00:12:26.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000157935 2024-09-15T00:12:26.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:26.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.127932+0000", 2024-09-15T00:12:26.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012479240000000001 2024-09-15T00:12:26.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:26.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128269+0000", 2024-09-15T00:12:26.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033761099999999998 2024-09-15T00:12:26.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:26.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128324+0000", 2024-09-15T00:12:26.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5244999999999997e-05 2024-09-15T00:12:26.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:26.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:26.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:26.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128365+0000", 2024-09-15T00:12:26.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0503999999999997e-05 2024-09-15T00:12:26.197 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.197 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.197 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.197 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:26.197 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:26.197 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:26.198 INFO:teuthology.orchestra.run.smithi171.stdout:4142 write 0x9bfbe78 thru 0x9c08ef8 (0xd081 bytes) 2024-09-15T00:12:26.199 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:26.201 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:12:26.202 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:26.200+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:26.203 INFO:teuthology.orchestra.run.smithi171.stdout:4143 writesame 0xe61e897 thru 0xe62deca data_size 0xe(0xf634 bytes) 2024-09-15T00:12:26.204 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:26.204 INFO:teuthology.orchestra.run.smithi171.stdout:4147 write 0x93c7430 thru 0x93d210e (0xacdf bytes) 2024-09-15T00:12:26.204 INFO:teuthology.orchestra.run.smithi171.stdout:4148 read 0x53d74e1 thru 0x53dc2bd (0x4ddd bytes) 2024-09-15T00:12:26.204 INFO:teuthology.orchestra.run.smithi171.stdout:4149 read 0xd8872ca thru 0xd890611 (0x9348 bytes) 2024-09-15T00:12:26.204 INFO:teuthology.orchestra.run.smithi171.stdout:4151 write 0x8ef2226 thru 0x8ef78e0 (0x56bb bytes) 2024-09-15T00:12:26.204 INFO:teuthology.orchestra.run.smithi171.stdout:4152 read 0xd2441ca thru 0xd24ae92 (0x6cc9 bytes) 2024-09-15T00:12:26.204 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:26.204 INFO:teuthology.orchestra.run.smithi171.stdout:4154 write 0x71eaad9 thru 0x71efe37 (0x535f bytes) 2024-09-15T00:12:26.204 INFO:teuthology.orchestra.run.smithi171.stdout:4155 trunc from 0xe62decb to 0xc350f6a 2024-09-15T00:12:26.204 INFO:teuthology.orchestra.run.smithi171.stdout:4156 write 0x6e1502e thru 0x6e1bade (0x6ab1 bytes) 2024-09-15T00:12:26.204 INFO:teuthology.orchestra.run.smithi171.stdout:4158 trunc from 0xc350f6a to 0x8036893 2024-09-15T00:12:26.204 INFO:teuthology.orchestra.run.smithi171.stdout:4160 punch from 0x58454f6 to 0x5848e3f, (0x3949 bytes) 2024-09-15T00:12:26.205 INFO:teuthology.orchestra.run.smithi171.stdout:4162 write 0xd38479b thru 0xd385295 (0xafb bytes) 2024-09-15T00:12:26.205 INFO:teuthology.orchestra.run.smithi171.stdout:4164 punch from 0x4fba14d to 0x4fc92ab, (0xf15e bytes) 2024-09-15T00:12:26.205 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:26.205 INFO:teuthology.orchestra.run.smithi171.stdout:4166 read 0x65bf0e2 thru 0x65c8445 (0x9364 bytes) 2024-09-15T00:12:26.205 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:26.205 INFO:teuthology.orchestra.run.smithi171.stdout:4168 punch from 0xbc319e7 to 0xbc403d3, (0xe9ec bytes) 2024-09-15T00:12:26.205 INFO:teuthology.orchestra.run.smithi171.stdout:4169 write 0xc7c9591 thru 0xc7d1b82 (0x85f2 bytes) 2024-09-15T00:12:26.205 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:26.205 INFO:teuthology.orchestra.run.smithi171.stdout:4173 write 0x712f72f thru 0x713bc2a (0xc4fc bytes) 2024-09-15T00:12:26.205 INFO:teuthology.orchestra.run.smithi171.stdout:4174 writesame 0x9c594ef thru 0x9c625b4 data_size 0x12(0x90c6 bytes) 2024-09-15T00:12:26.205 INFO:teuthology.orchestra.run.smithi171.stdout:4175 read 0x83f22b6 thru 0x84018b9 (0xf604 bytes) 2024-09-15T00:12:26.205 INFO:teuthology.orchestra.run.smithi171.stdout:4176 read 0x9e40f53 thru 0x9e4ae04 (0x9eb2 bytes) 2024-09-15T00:12:26.205 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:26.205 INFO:teuthology.orchestra.run.smithi171.stdout:4180 writesame 0xe93007d thru 0xe93d2b8 data_size 0x75(0xd23c bytes) 2024-09-15T00:12:26.206 INFO:teuthology.orchestra.run.smithi171.stdout:4181 punch from 0x36e43b9 to 0x36ee509, (0xa150 bytes) 2024-09-15T00:12:26.206 INFO:teuthology.orchestra.run.smithi171.stdout:4182 read 0x6aac79d thru 0x6aafa50 (0x32b4 bytes) 2024-09-15T00:12:26.206 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:26.206 INFO:teuthology.orchestra.run.smithi171.stdout:4184 write 0x9de6bfd thru 0x9dece78 (0x627c bytes) 2024-09-15T00:12:26.206 INFO:teuthology.orchestra.run.smithi171.stdout:4185 write 0xa522beb thru 0xa524a41 (0x1e57 bytes) 2024-09-15T00:12:26.210 INFO:teuthology.orchestra.run.smithi171.stdout:4186 trunc from 0xe93d2b9 to 0x63aed8c 2024-09-15T00:12:26.235 INFO:teuthology.orchestra.run.smithi171.stdout:4187 write 0xc0db817 thru 0xc0dc44a (0xc34 bytes) 2024-09-15T00:12:26.238 INFO:teuthology.orchestra.run.smithi171.stdout:4189 write 0x65123bb thru 0x651b52e (0x9174 bytes) 2024-09-15T00:12:26.242 INFO:teuthology.orchestra.run.smithi171.stdout:4190 clone 32 order 20 su 524288 sc 10 2024-09-15T00:12:26.302 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:26.300+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:26.306 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:26.307 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:26.315 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:26.403 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:26.401+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:26.417 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:26.418 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:26.418 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:26.418 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:26.418 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:26.426 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:26.487 INFO:tasks.thrashosds.thrasher:No PGs found for osd.2 2024-09-15T00:12:26.511 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:26.509+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:26.572 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:26.572 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:26.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:26.572 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:26.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.572 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:26.572 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:26.573 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 60.997564840999999, 2024-09-15T00:12:26.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:12:26.573 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.573 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.573 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:26.573 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.573 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.573 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.573 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:12:26.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.573 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:26.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:26.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:12:26.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:12:26.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:12:26.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:12:26.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:12:26.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:12:26.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:12:26.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:12:26.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:12:26.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:12:26.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:12:26.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:12:26.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:26.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:12:26.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:12:26.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:26.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:12:26.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:12:26.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:26.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:12:26.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:12:26.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:26.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:12:26.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:12:26.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:12:26.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:12:26.578 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.578 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.578 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.579 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:26.579 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:26.579 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 60.997530148999999, 2024-09-15T00:12:26.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:12:26.579 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.579 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.579 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:26.579 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.579 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.579 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.579 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:12:26.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.580 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:26.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:26.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:26.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:12:26.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:12:26.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:12:26.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:12:26.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:12:26.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:12:26.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:12:26.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:12:26.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:12:26.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:12:26.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:12:26.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:26.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:12:26.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:12:26.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:26.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:12:26.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:12:26.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:26.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:12:26.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:12:26.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:26.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:12:26.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:12:26.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:12:26.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:12:26.584 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.584 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.584 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.585 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48937 3.1a 3:5e7e71d0:::rbd_data.11395b9829f3.0000000000000000:e [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.585 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:12:26.585 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.957437194000001, 2024-09-15T00:12:26.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072322336000000001, 2024-09-15T00:12:26.585 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.585 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.585 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.585 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.585 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.585 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.585 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48937 2024-09-15T00:12:26.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.586 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:12:26.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:12:26.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612043+0000", 2024-09-15T00:12:26.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:12:26.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612045+0000", 2024-09-15T00:12:26.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2749999999999998e-06 2024-09-15T00:12:26.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612053+0000", 2024-09-15T00:12:26.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0439999999999996e-06 2024-09-15T00:12:26.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612062+0000", 2024-09-15T00:12:26.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6829999999999997e-06 2024-09-15T00:12:26.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684078+0000", 2024-09-15T00:12:26.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072016017000000002 2024-09-15T00:12:26.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684274+0000", 2024-09-15T00:12:26.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019636600000000001 2024-09-15T00:12:26.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684364+0000", 2024-09-15T00:12:26.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0420000000000005e-05 2024-09-15T00:12:26.589 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.589 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.589 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.589 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.589 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:26.589 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.956856461000001, 2024-09-15T00:12:26.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:12:26.590 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.590 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.590 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.590 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.590 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.590 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.590 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:12:26.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.590 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:26.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:26.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:26.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:12:26.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:12:26.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:12:26.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:12:26.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:12:26.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:12:26.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:12:26.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:12:26.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:12:26.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:12:26.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:12:26.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:12:26.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:12:26.594 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.594 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.594 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.594 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.594 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:26.594 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.956839930000001, 2024-09-15T00:12:26.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:12:26.594 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.594 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.594 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.595 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.595 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.595 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.595 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:12:26.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.595 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:26.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:26.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:12:26.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:12:26.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:12:26.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:12:26.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:12:26.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:12:26.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:12:26.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:12:26.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:12:26.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:12:26.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:12:26.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:12:26.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:12:26.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:12:26.601 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.601 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.601 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.601 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.601 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:26.601 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.956823151000002, 2024-09-15T00:12:26.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:12:26.602 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.602 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.602 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.602 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.602 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.602 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.602 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:12:26.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.602 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:26.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:26.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:26.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:12:26.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:12:26.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:12:26.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:12:26.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:12:26.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:12:26.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:12:26.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:12:26.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:12:26.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:12:26.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:12:26.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:12:26.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:12:26.605 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.606 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.606 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.606 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.606 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:26.606 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.956805420999999, 2024-09-15T00:12:26.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:12:26.606 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.606 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.606 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.607 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.607 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:12:26.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.607 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:26.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:26.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:26.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:12:26.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:12:26.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:12:26.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:12:26.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:12:26.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:12:26.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:12:26.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:12:26.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:12:26.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:12:26.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:12:26.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:12:26.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:12:26.610 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.610 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.610 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.611 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.611 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:26.611 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.956788256999999, 2024-09-15T00:12:26.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:12:26.611 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.611 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.611 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.611 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.611 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.611 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.611 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:12:26.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.611 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:26.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:26.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:26.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:12:26.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:12:26.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:12:26.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:12:26.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:12:26.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:12:26.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:12:26.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:12:26.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:12:26.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:12:26.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:12:26.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:12:26.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:12:26.615 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.615 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.615 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.615 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.615 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:26.616 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.956766580999997, 2024-09-15T00:12:26.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:12:26.616 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.616 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.616 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.616 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.616 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.616 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.616 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:12:26.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.616 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:26.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:26.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:26.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:12:26.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:12:26.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:12:26.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:12:26.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:12:26.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:12:26.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:12:26.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:12:26.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:12:26.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:12:26.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:12:26.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:12:26.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:12:26.620 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.620 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.620 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.620 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.620 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:26.620 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.956720457000003, 2024-09-15T00:12:26.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:12:26.620 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.620 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.621 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.621 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.621 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.621 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.621 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:12:26.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.621 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:26.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:26.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:26.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:12:26.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:12:26.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:12:26.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:12:26.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:12:26.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:12:26.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:12:26.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:12:26.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:12:26.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:12:26.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:12:26.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:12:26.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:12:26.624 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.624 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.624 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.956690700000003, 2024-09-15T00:12:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:12:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:12:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:12:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:12:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:12:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:12:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:12:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:12:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:12:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:12:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:12:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:12:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:12:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:12:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:12:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:12:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.956664381000003, 2024-09-15T00:12:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:12:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.630 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.630 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.630 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.630 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.630 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:12:26.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.630 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:26.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:12:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:12:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:12:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:12:26.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:12:26.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:12:26.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:12:26.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:12:26.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:12:26.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:12:26.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:12:26.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:12:26.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:12:26.633 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.633 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.633 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.634 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.634 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:26.634 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.956628518999999, 2024-09-15T00:12:26.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:12:26.634 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.634 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.634 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.634 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.634 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.634 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.634 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:12:26.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.635 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:26.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:26.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:26.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:12:26.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:12:26.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:12:26.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:12:26.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:12:26.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:12:26.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:12:26.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:12:26.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:12:26.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:12:26.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:12:26.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:12:26.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:12:26.638 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.638 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.638 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.638 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.638 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:26.638 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.956606801, 2024-09-15T00:12:26.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:12:26.639 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.639 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.639 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.639 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.639 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.639 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.639 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:12:26.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.639 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:26.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:26.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:12:26.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:12:26.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:12:26.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:12:26.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:12:26.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:12:26.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:12:26.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:12:26.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:12:26.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:12:26.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:12:26.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:12:26.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:12:26.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:12:26.642 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.642 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.642 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.642 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.643 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:26.643 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.956410333999997, 2024-09-15T00:12:26.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:12:26.643 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.643 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.643 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.643 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.643 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.643 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.643 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:12:26.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.644 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:26.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:26.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:26.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:12:26.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:12:26.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:12:26.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:12:26.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:12:26.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:12:26.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:12:26.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:12:26.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:12:26.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:12:26.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:12:26.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:12:26.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:12:26.647 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.647 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.647 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.647 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.647 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:26.647 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.956384495000002, 2024-09-15T00:12:26.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:12:26.648 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.648 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.648 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.648 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.648 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.648 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.648 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:12:26.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.648 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:26.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:26.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:26.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:12:26.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:12:26.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:12:26.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:12:26.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:12:26.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:12:26.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:12:26.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:12:26.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:12:26.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:12:26.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:12:26.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:12:26.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:12:26.651 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.651 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.651 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.652 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.652 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:26.652 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.956322849000003, 2024-09-15T00:12:26.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:12:26.652 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.652 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.652 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.652 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:12:26.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.652 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:26.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:26.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:12:26.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:12:26.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:12:26.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:12:26.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:12:26.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:12:26.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:12:26.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:12:26.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:12:26.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:12:26.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:12:26.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:12:26.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:12:26.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:12:26.656 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.656 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.656 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.656 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.656 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:26.656 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.956283521000003, 2024-09-15T00:12:26.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:12:26.656 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.656 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.656 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.656 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.657 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.657 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.657 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:12:26.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.657 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:26.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:26.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:12:26.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:12:26.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:12:26.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:12:26.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:12:26.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:12:26.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:12:26.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:12:26.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:12:26.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:12:26.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:12:26.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:12:26.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:12:26.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:12:26.660 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.660 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.660 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.661 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.661 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:26.661 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.954558120999998, 2024-09-15T00:12:26.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:12:26.661 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.661 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.661 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.661 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.661 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.661 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.662 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:12:26.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.662 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:26.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:26.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:12:26.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:12:26.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:12:26.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:12:26.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:12:26.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:12:26.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:12:26.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:12:26.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:12:26.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:12:26.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:12:26.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:12:26.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:12:26.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:12:26.665 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.665 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.665 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.665 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:26.665 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:26.665 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.954535960000001, 2024-09-15T00:12:26.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:12:26.666 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:26.666 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:26.666 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:26.666 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:26.666 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:26.666 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:26.666 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:12:26.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.666 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:26.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:26.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:26.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:26.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:26.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:26.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:26.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:26.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:12:26.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:26.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:12:26.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:12:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:12:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:12:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:12:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:12:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:12:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:12:26.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:26.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:12:26.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:12:26.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:26.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:26.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:26.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:12:26.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:12:26.669 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.669 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.669 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.669 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:26.669 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:26.670 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:26.671 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:26.671 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:26.669+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:26.772 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:26.770+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:26.814 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:26.814 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:26.823 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:26.872 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:26.870+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:26.927 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:26.927 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:26.927 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:26.927 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:26.927 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:26.936 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:26.973 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:26.971+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:27.074 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:27.072+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:27.081 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:27.081 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:27.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:27.082 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:27.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.082 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:47986 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.082 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:27.082 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.607640111999999, 2024-09-15T00:12:27.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045287635, 2024-09-15T00:12:27.082 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.082 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.082 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:27.082 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.082 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.082 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.082 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 47986 2024-09-15T00:12:27.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.083 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:27.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:27.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:27.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8400000000000004e-07 2024-09-15T00:12:27.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474928+0000", 2024-09-15T00:12:27.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033009430000000002 2024-09-15T00:12:27.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474943+0000", 2024-09-15T00:12:27.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5172e-05 2024-09-15T00:12:27.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474951+0000", 2024-09-15T00:12:27.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1779999999999997e-06 2024-09-15T00:12:27.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475058+0000", 2024-09-15T00:12:27.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000107595 2024-09-15T00:12:27.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475200+0000", 2024-09-15T00:12:27.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014180400000000001 2024-09-15T00:12:27.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:27.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.478046+0000", 2024-09-15T00:12:27.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028464150000000001 2024-09-15T00:12:27.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:27.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516790+0000", 2024-09-15T00:12:27.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038743278999999999 2024-09-15T00:12:27.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:27.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516832+0000", 2024-09-15T00:12:27.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1974000000000002e-05 2024-09-15T00:12:27.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:27.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516872+0000", 2024-09-15T00:12:27.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9928999999999997e-05 2024-09-15T00:12:27.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516914+0000", 2024-09-15T00:12:27.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2762000000000002e-05 2024-09-15T00:12:27.088 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.088 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.088 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.088 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48957 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 4456448~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.088 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:12:27.088 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.470201600000003, 2024-09-15T00:12:27.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044512426000000001, 2024-09-15T00:12:27.088 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.088 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.088 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.089 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.089 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.089 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.089 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48957 2024-09-15T00:12:27.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.089 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:12:27.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609066+0000", 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609068+0000", 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9700000000000002e-06 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609073+0000", 2024-09-15T00:12:27.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7939999999999999e-06 2024-09-15T00:12:27.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609078+0000", 2024-09-15T00:12:27.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3209999999999998e-06 2024-09-15T00:12:27.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653082+0000", 2024-09-15T00:12:27.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044003852000000003 2024-09-15T00:12:27.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653136+0000", 2024-09-15T00:12:27.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4024000000000003e-05 2024-09-15T00:12:27.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653578+0000", 2024-09-15T00:12:27.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044211599999999999 2024-09-15T00:12:27.092 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.092 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.092 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.092 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48958 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.092 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:27.093 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.470183960999996, 2024-09-15T00:12:27.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044724939999999998, 2024-09-15T00:12:27.093 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.093 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.093 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.093 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.093 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.093 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.093 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48958 2024-09-15T00:12:27.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.093 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609085+0000", 2024-09-15T00:12:27.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6610000000000001e-06 2024-09-15T00:12:27.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609090+0000", 2024-09-15T00:12:27.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2719999999999997e-06 2024-09-15T00:12:27.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609094+0000", 2024-09-15T00:12:27.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.907e-06 2024-09-15T00:12:27.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653618+0000", 2024-09-15T00:12:27.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044523759000000003 2024-09-15T00:12:27.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653703+0000", 2024-09-15T00:12:27.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5550000000000006e-05 2024-09-15T00:12:27.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653808+0000", 2024-09-15T00:12:27.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010445700000000001 2024-09-15T00:12:27.096 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.097 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.097 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.097 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48959 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.097 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:27.097 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.470168022000003, 2024-09-15T00:12:27.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044884741999999998, 2024-09-15T00:12:27.097 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.097 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.097 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.098 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48959 2024-09-15T00:12:27.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.098 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:27.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:27.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:27.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2899999999999999e-07 2024-09-15T00:12:27.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609101+0000", 2024-09-15T00:12:27.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.826e-06 2024-09-15T00:12:27.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609108+0000", 2024-09-15T00:12:27.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.165e-06 2024-09-15T00:12:27.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609113+0000", 2024-09-15T00:12:27.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7849999999999999e-06 2024-09-15T00:12:27.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653842+0000", 2024-09-15T00:12:27.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044728954000000001 2024-09-15T00:12:27.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653896+0000", 2024-09-15T00:12:27.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4593999999999999e-05 2024-09-15T00:12:27.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653984+0000", 2024-09-15T00:12:27.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7089000000000006e-05 2024-09-15T00:12:27.101 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.101 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.101 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.102 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48960 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 9961472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.102 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:27.102 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.470147503, 2024-09-15T00:12:27.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045084194000000001, 2024-09-15T00:12:27.102 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.102 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.102 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.102 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.102 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.102 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.102 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48960 2024-09-15T00:12:27.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.103 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:27.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:27.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609120+0000", 2024-09-15T00:12:27.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7099999999999997e-07 2024-09-15T00:12:27.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609122+0000", 2024-09-15T00:12:27.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:12:27.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609129+0000", 2024-09-15T00:12:27.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4159999999999998e-06 2024-09-15T00:12:27.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609134+0000", 2024-09-15T00:12:27.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7210000000000002e-06 2024-09-15T00:12:27.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654023+0000", 2024-09-15T00:12:27.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044888979000000002 2024-09-15T00:12:27.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654104+0000", 2024-09-15T00:12:27.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1108999999999999e-05 2024-09-15T00:12:27.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654204+0000", 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9630000000000007e-05 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48961 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.470126682, 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045197821999999999, 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.106 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.107 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.107 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.107 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.107 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48961 2024-09-15T00:12:27.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.107 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:27.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:27.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609141+0000", 2024-09-15T00:12:27.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:12:27.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609143+0000", 2024-09-15T00:12:27.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-15T00:12:27.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609150+0000", 2024-09-15T00:12:27.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2420000000000003e-06 2024-09-15T00:12:27.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609155+0000", 2024-09-15T00:12:27.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.028e-06 2024-09-15T00:12:27.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654232+0000", 2024-09-15T00:12:27.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045076702000000003 2024-09-15T00:12:27.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654281+0000", 2024-09-15T00:12:27.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9798e-05 2024-09-15T00:12:27.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654338+0000", 2024-09-15T00:12:27.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6498e-05 2024-09-15T00:12:27.110 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.110 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.110 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.110 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48962 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 13631488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.110 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:27.110 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.470105631999999, 2024-09-15T00:12:27.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045312458, 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48962 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:27.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609162+0000", 2024-09-15T00:12:27.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:27.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609164+0000", 2024-09-15T00:12:27.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0099999999999998e-06 2024-09-15T00:12:27.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609171+0000", 2024-09-15T00:12:27.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7530000000000001e-06 2024-09-15T00:12:27.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609178+0000", 2024-09-15T00:12:27.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1709999999999999e-06 2024-09-15T00:12:27.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654365+0000", 2024-09-15T00:12:27.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045187867999999999 2024-09-15T00:12:27.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654420+0000", 2024-09-15T00:12:27.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4376000000000003e-05 2024-09-15T00:12:27.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654474+0000", 2024-09-15T00:12:27.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3879999999999999e-05 2024-09-15T00:12:27.114 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.114 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.114 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.115 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48971 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.115 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:27.115 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.470074638, 2024-09-15T00:12:27.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045896151000000003, 2024-09-15T00:12:27.115 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.115 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.115 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.115 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.115 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.115 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.115 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48971 2024-09-15T00:12:27.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.116 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:27.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:27.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609193+0000", 2024-09-15T00:12:27.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7599999999999997e-07 2024-09-15T00:12:27.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609195+0000", 2024-09-15T00:12:27.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:12:27.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609210+0000", 2024-09-15T00:12:27.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4956999999999999e-05 2024-09-15T00:12:27.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609505+0000", 2024-09-15T00:12:27.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000295078 2024-09-15T00:12:27.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654498+0000", 2024-09-15T00:12:27.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044992277999999997 2024-09-15T00:12:27.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654541+0000", 2024-09-15T00:12:27.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3525000000000001e-05 2024-09-15T00:12:27.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655088+0000", 2024-09-15T00:12:27.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054726799999999995 2024-09-15T00:12:27.119 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.119 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.119 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.119 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48972 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 2097152~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.119 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:27.120 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.469748064000001, 2024-09-15T00:12:27.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046150885000000003, 2024-09-15T00:12:27.120 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.120 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.120 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.120 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.120 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.120 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.120 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48972 2024-09-15T00:12:27.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.120 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:27.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:27.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:27.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6400000000000002e-07 2024-09-15T00:12:27.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609523+0000", 2024-09-15T00:12:27.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5300000000000001e-06 2024-09-15T00:12:27.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609533+0000", 2024-09-15T00:12:27.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0023999999999999e-05 2024-09-15T00:12:27.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609539+0000", 2024-09-15T00:12:27.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3119999999999999e-06 2024-09-15T00:12:27.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655115+0000", 2024-09-15T00:12:27.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045576094999999997 2024-09-15T00:12:27.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655197+0000", 2024-09-15T00:12:27.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1129000000000002e-05 2024-09-15T00:12:27.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655670+0000", 2024-09-15T00:12:27.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00047313100000000001 2024-09-15T00:12:27.125 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.125 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.125 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.125 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.125 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:27.125 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.469719802, 2024-09-15T00:12:27.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:12:27.125 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.125 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.125 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.125 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.126 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.126 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.126 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:12:27.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.126 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:27.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:27.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:27.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:12:27.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:12:27.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:12:27.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:12:27.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:12:27.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:12:27.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:12:27.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:12:27.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:12:27.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:12:27.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:12:27.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:12:27.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:12:27.129 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.129 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.129 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.129 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.129 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:27.129 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.469698573000002, 2024-09-15T00:12:27.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:12:27.130 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.130 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.130 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.130 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.130 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.130 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.130 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:12:27.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.130 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:27.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:27.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:12:27.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:12:27.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:12:27.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:12:27.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:12:27.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:12:27.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:12:27.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:12:27.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:12:27.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:12:27.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:12:27.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:12:27.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:12:27.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:12:27.133 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.133 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.133 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.134 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.134 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:27.134 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.469673032000003, 2024-09-15T00:12:27.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:12:27.134 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.134 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.134 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.134 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.134 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.134 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.134 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:12:27.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.135 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:27.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:27.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:27.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:12:27.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:12:27.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:12:27.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:12:27.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:12:27.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:12:27.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:12:27.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:12:27.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:12:27.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:12:27.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:12:27.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:12:27.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:12:27.138 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.138 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.138 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.138 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.138 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:27.138 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.469650557000001, 2024-09-15T00:12:27.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:12:27.138 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.138 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.138 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.139 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.139 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.139 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.139 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:12:27.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.139 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:27.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:27.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:12:27.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:12:27.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:12:27.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:12:27.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:12:27.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:12:27.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:12:27.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:12:27.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:12:27.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:12:27.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:12:27.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:12:27.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:12:27.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:12:27.142 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.142 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.142 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.142 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.142 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:27.143 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.469630047000003, 2024-09-15T00:12:27.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:12:27.143 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.143 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.143 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.143 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.143 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.143 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.143 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:12:27.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.143 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:27.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:27.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:27.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:12:27.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:12:27.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:12:27.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:12:27.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:12:27.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:12:27.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:12:27.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:12:27.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:12:27.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:12:27.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:12:27.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:12:27.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:12:27.146 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.146 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.147 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.147 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.147 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:27.147 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.469609732000002, 2024-09-15T00:12:27.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:12:27.147 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.147 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.147 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.147 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.147 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.147 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.147 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:12:27.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.148 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:27.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:27.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:12:27.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:12:27.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:12:27.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:12:27.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:12:27.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:12:27.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:12:27.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:12:27.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:12:27.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:12:27.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:12:27.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:12:27.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:12:27.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:12:27.151 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.151 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.151 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.151 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.151 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:27.151 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.469584955999998, 2024-09-15T00:12:27.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:12:27.151 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.152 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.152 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.152 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.152 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.152 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.152 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:12:27.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.152 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:27.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:27.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:27.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:12:27.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:12:27.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:12:27.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:12:27.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:12:27.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:12:27.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:12:27.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:12:27.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:12:27.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:12:27.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:12:27.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:12:27.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:12:27.155 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.155 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.156 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.156 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.156 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:27.156 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.290243568999998, 2024-09-15T00:12:27.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:12:27.156 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.156 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.156 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:27.156 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.156 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.156 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.157 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:12:27.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.157 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:27.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:27.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:12:27.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:12:27.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:12:27.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:12:27.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:12:27.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:12:27.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:12:27.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:12:27.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:12:27.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:12:27.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:12:27.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:12:27.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:27.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:12:27.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:12:27.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:27.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:12:27.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:12:27.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:27.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:12:27.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:12:27.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:27.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:12:27.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:12:27.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:12:27.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:12:27.161 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.161 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.162 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.162 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:12:27.162 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:27.162 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.273290117000002, 2024-09-15T00:12:27.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:12:27.162 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.162 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.162 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.162 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:27.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:27.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:12:27.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:12:27.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:12:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:12:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:12:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:12:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:12:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:12:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:12:27.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:12:27.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:12:27.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:12:27.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:12:27.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:12:27.165 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.166 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.166 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.166 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:12:27.166 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:27.166 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.97885921, 2024-09-15T00:12:27.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:12:27.166 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.166 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.166 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:27.166 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.166 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:12:27.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:12:27.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:12:27.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:12:27.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:12:27.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:12:27.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:12:27.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:12:27.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:12:27.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:12:27.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:12:27.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:12:27.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:12:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:12:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:12:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:12:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:12:27.172 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:27.172 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.966403396, 2024-09-15T00:12:27.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:12:27.172 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.172 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.172 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.172 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:27.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:27.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:12:27.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:12:27.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:12:27.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:12:27.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:12:27.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:12:27.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:12:27.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:12:27.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:12:27.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:12:27.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:12:27.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:12:27.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:12:27.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:12:27.175 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.175 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.175 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.175 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.176 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.176 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:27.177 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is unset 2024-09-15T00:12:27.178 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:27.178 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:27.174+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:27.278 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:27.275+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:27.322 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:27.322 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:27.322 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:27.323 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:27.324 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-15T00:12:27.331 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:27.379 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:27.377+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:27.435 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:27.436 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:27.436 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:27.436 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:27.436 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:27.444 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:27.480 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:27.477+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:27.560 INFO:teuthology.orchestra.run.smithi171.stdout:4192 write 0x4485b7c thru 0x4491fcb (0xc450 bytes) 2024-09-15T00:12:27.569 INFO:teuthology.orchestra.run.smithi171.stdout:4193 write 0x8c67c thru 0x9ba95 (0xf41a bytes) 2024-09-15T00:12:27.575 INFO:teuthology.orchestra.run.smithi171.stdout:4195 read 0x837fce2 thru 0x838af98 (0xb2b7 bytes) 2024-09-15T00:12:27.576 INFO:teuthology.orchestra.run.smithi171.stdout:4196 write 0x1ae86aa thru 0x1aef42b (0x6d82 bytes) 2024-09-15T00:12:27.581 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:27.581 INFO:teuthology.orchestra.run.smithi171.stdout:4198 read 0x9bc255f thru 0x9bc43f7 (0x1e99 bytes) 2024-09-15T00:12:27.581 INFO:teuthology.orchestra.run.smithi171.stdout:4199 punch from 0x6c8df9a to 0x6c91c94, (0x3cfa bytes) 2024-09-15T00:12:27.581 INFO:teuthology.orchestra.run.smithi171.stdout:4200 read 0x390ac6f thru 0x3917fcf (0xd361 bytes) 2024-09-15T00:12:27.581 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:27.579+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:27.588 INFO:teuthology.orchestra.run.smithi171.stdout:4201 trunc from 0xc0dc44b to 0x5a3cf7d 2024-09-15T00:12:27.591 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:19308 3.10 3:0ee1524d:::rbd_data.11398e1a091a.0000000000000017:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-15T00:12:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 48.602891724999999, 2024-09-15T00:12:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02547147, 2024-09-15T00:12:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 19308 2024-09-15T00:12:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985182+0000", 2024-09-15T00:12:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6500000000000001e-06 2024-09-15T00:12:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985189+0000", 2024-09-15T00:12:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9949999999999999e-06 2024-09-15T00:12:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985205+0000", 2024-09-15T00:12:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6055000000000001e-05 2024-09-15T00:12:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985212+0000", 2024-09-15T00:12:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:12:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985261+0000", 2024-09-15T00:12:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8380000000000001e-05 2024-09-15T00:12:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985311+0000", 2024-09-15T00:12:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0763999999999997e-05 2024-09-15T00:12:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.986489+0000", 2024-09-15T00:12:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0011773420000000001 2024-09-15T00:12:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010514+0000", 2024-09-15T00:12:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024025392999999999 2024-09-15T00:12:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010535+0000", 2024-09-15T00:12:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0652000000000001e-05 2024-09-15T00:12:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010597+0000", 2024-09-15T00:12:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2411999999999998e-05 2024-09-15T00:12:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010652+0000", 2024-09-15T00:12:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5074999999999997e-05 2024-09-15T00:12:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.920714756999999, 2024-09-15T00:12:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:12:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:12:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:12:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:12:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:12:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:12:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:12:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:12:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:12:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:12:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:12:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:12:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:12:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:12:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:12:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:12:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:12:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:12:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:12:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:12:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:12:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:12:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:12:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:12:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65520 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.705659701999998, 2024-09-15T00:12:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024513282000000001, 2024-09-15T00:12:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65520 2024-09-15T00:12:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.39e-07 2024-09-15T00:12:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882415+0000", 2024-09-15T00:12:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8020000000000001e-06 2024-09-15T00:12:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882421+0000", 2024-09-15T00:12:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7119999999999997e-06 2024-09-15T00:12:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882425+0000", 2024-09-15T00:12:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2660000000000003e-06 2024-09-15T00:12:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906919+0000", 2024-09-15T00:12:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024494438 2024-09-15T00:12:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906926+0000", 2024-09-15T00:12:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.725e-06 2024-09-15T00:12:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65525 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.705544834000001, 2024-09-15T00:12:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024517753, 2024-09-15T00:12:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65525 2024-09-15T00:12:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:12:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882530+0000", 2024-09-15T00:12:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.804e-06 2024-09-15T00:12:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882535+0000", 2024-09-15T00:12:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5500000000000002e-06 2024-09-15T00:12:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882540+0000", 2024-09-15T00:12:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6909999999999996e-06 2024-09-15T00:12:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907039+0000", 2024-09-15T00:12:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024498809 2024-09-15T00:12:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907045+0000", 2024-09-15T00:12:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5459999999999997e-06 2024-09-15T00:12:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66541 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1449984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:12:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.673937244999998, 2024-09-15T00:12:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024574262999999999, 2024-09-15T00:12:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66541 2024-09-15T00:12:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:12:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:12:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914136+0000", 2024-09-15T00:12:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3599999999999999e-07 2024-09-15T00:12:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914137+0000", 2024-09-15T00:12:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9309999999999998e-06 2024-09-15T00:12:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914143+0000", 2024-09-15T00:12:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0120000000000002e-06 2024-09-15T00:12:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914148+0000", 2024-09-15T00:12:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.707e-06 2024-09-15T00:12:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938536+0000", 2024-09-15T00:12:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024387671999999999 2024-09-15T00:12:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938618+0000", 2024-09-15T00:12:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1811999999999995e-05 2024-09-15T00:12:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938709+0000", 2024-09-15T00:12:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1792999999999993e-05 2024-09-15T00:12:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66542 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1560576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.673918104000002, 2024-09-15T00:12:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024740410000000001, 2024-09-15T00:12:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66542 2024-09-15T00:12:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914155+0000", 2024-09-15T00:12:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:12:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914157+0000", 2024-09-15T00:12:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:12:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914162+0000", 2024-09-15T00:12:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914167+0000", 2024-09-15T00:12:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3789999999999999e-06 2024-09-15T00:12:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938730+0000", 2024-09-15T00:12:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024562694 2024-09-15T00:12:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938803+0000", 2024-09-15T00:12:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3325999999999996e-05 2024-09-15T00:12:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938895+0000", 2024-09-15T00:12:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1884000000000003e-05 2024-09-15T00:12:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66543 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1572864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.673887845000003, 2024-09-15T00:12:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024893976000000002, 2024-09-15T00:12:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66543 2024-09-15T00:12:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914188+0000", 2024-09-15T00:12:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:12:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914193+0000", 2024-09-15T00:12:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4129999999999997e-06 2024-09-15T00:12:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914198+0000", 2024-09-15T00:12:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9309999999999999e-06 2024-09-15T00:12:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938914+0000", 2024-09-15T00:12:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024716380999999999 2024-09-15T00:12:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938993+0000", 2024-09-15T00:12:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8763000000000005e-05 2024-09-15T00:12:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939079+0000", 2024-09-15T00:12:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5372999999999997e-05 2024-09-15T00:12:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66544 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1683456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.673865442, 2024-09-15T00:12:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025051225, 2024-09-15T00:12:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66544 2024-09-15T00:12:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:12:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914210+0000", 2024-09-15T00:12:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2450000000000001e-06 2024-09-15T00:12:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914215+0000", 2024-09-15T00:12:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7869999999999998e-06 2024-09-15T00:12:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914220+0000", 2024-09-15T00:12:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7899999999999999e-06 2024-09-15T00:12:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939105+0000", 2024-09-15T00:12:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024884381000000001 2024-09-15T00:12:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939177+0000", 2024-09-15T00:12:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2138999999999995e-05 2024-09-15T00:12:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939258+0000", 2024-09-15T00:12:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1462e-05 2024-09-15T00:12:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66545 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1695744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.673843167999998, 2024-09-15T00:12:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025209788, 2024-09-15T00:12:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66545 2024-09-15T00:12:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914230+0000", 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1099999999999996e-07 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914232+0000", 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3080000000000002e-06 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914238+0000", 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.02e-06 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914243+0000", 2024-09-15T00:12:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1200000000000001e-06 2024-09-15T00:12:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939285+0000", 2024-09-15T00:12:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025042212000000001 2024-09-15T00:12:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939361+0000", 2024-09-15T00:12:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5558999999999999e-05 2024-09-15T00:12:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939439+0000", 2024-09-15T00:12:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8058000000000004e-05 2024-09-15T00:12:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66546 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.673823192999997, 2024-09-15T00:12:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025375960999999999, 2024-09-15T00:12:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66546 2024-09-15T00:12:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914250+0000", 2024-09-15T00:12:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:12:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914252+0000", 2024-09-15T00:12:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.052e-06 2024-09-15T00:12:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914258+0000", 2024-09-15T00:12:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9039999999999997e-06 2024-09-15T00:12:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914262+0000", 2024-09-15T00:12:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4719999999999997e-06 2024-09-15T00:12:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939466+0000", 2024-09-15T00:12:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025203550000000002 2024-09-15T00:12:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939537+0000", 2024-09-15T00:12:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1780999999999996e-05 2024-09-15T00:12:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939625+0000", 2024-09-15T00:12:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7828e-05 2024-09-15T00:12:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66547 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1818624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.67380352, 2024-09-15T00:12:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025552638999999999, 2024-09-15T00:12:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.643 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.643 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.643 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66547 2024-09-15T00:12:27.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.643 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:27.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:27.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:27.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:27.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914271+0000", 2024-09-15T00:12:27.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8920000000000001e-06 2024-09-15T00:12:27.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914277+0000", 2024-09-15T00:12:27.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:27.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914282+0000", 2024-09-15T00:12:27.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7190000000000001e-06 2024-09-15T00:12:27.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939654+0000", 2024-09-15T00:12:27.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025372372000000001 2024-09-15T00:12:27.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939734+0000", 2024-09-15T00:12:27.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9946999999999999e-05 2024-09-15T00:12:27.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939822+0000", 2024-09-15T00:12:27.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7680000000000001e-05 2024-09-15T00:12:27.646 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.646 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.646 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.646 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66548 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:27.646 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:27.647 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.673773132999997, 2024-09-15T00:12:27.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025712727000000001, 2024-09-15T00:12:27.647 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.647 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.647 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.647 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.647 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.647 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.647 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66548 2024-09-15T00:12:27.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.647 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:27.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:27.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914300+0000", 2024-09-15T00:12:27.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3599999999999999e-07 2024-09-15T00:12:27.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914302+0000", 2024-09-15T00:12:27.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3369999999999998e-06 2024-09-15T00:12:27.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914313+0000", 2024-09-15T00:12:27.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1399999999999999e-05 2024-09-15T00:12:27.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914319+0000", 2024-09-15T00:12:27.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9100000000000002e-06 2024-09-15T00:12:27.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939850+0000", 2024-09-15T00:12:27.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025531087000000001 2024-09-15T00:12:27.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939932+0000", 2024-09-15T00:12:27.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1534999999999998e-05 2024-09-15T00:12:27.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940012+0000", 2024-09-15T00:12:27.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0022000000000001e-05 2024-09-15T00:12:27.651 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.651 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.651 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.651 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66549 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1941504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:27.651 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:27.651 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.673708042000001, 2024-09-15T00:12:27.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025813836999999999, 2024-09-15T00:12:27.652 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.652 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.652 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.652 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66549 2024-09-15T00:12:27.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.652 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.652 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:27.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:27.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914365+0000", 2024-09-15T00:12:27.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2799999999999995e-07 2024-09-15T00:12:27.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914368+0000", 2024-09-15T00:12:27.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.92e-06 2024-09-15T00:12:27.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914376+0000", 2024-09-15T00:12:27.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8110000000000003e-06 2024-09-15T00:12:27.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914381+0000", 2024-09-15T00:12:27.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5119999999999999e-06 2024-09-15T00:12:27.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940040+0000", 2024-09-15T00:12:27.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025658271999999999 2024-09-15T00:12:27.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940103+0000", 2024-09-15T00:12:27.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2891000000000005e-05 2024-09-15T00:12:27.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940178+0000", 2024-09-15T00:12:27.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5703000000000003e-05 2024-09-15T00:12:27.655 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.656 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.656 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.656 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66550 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:27.656 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:27.656 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.673564462000002, 2024-09-15T00:12:27.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025820701000000001, 2024-09-15T00:12:27.656 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.656 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.656 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.656 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.656 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.657 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.657 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66550 2024-09-15T00:12:27.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.657 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:27.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:27.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914509+0000", 2024-09-15T00:12:27.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1999999999999999e-07 2024-09-15T00:12:27.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914513+0000", 2024-09-15T00:12:27.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1640000000000003e-06 2024-09-15T00:12:27.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914522+0000", 2024-09-15T00:12:27.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7639999999999994e-06 2024-09-15T00:12:27.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914527+0000", 2024-09-15T00:12:27.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9170000000000003e-06 2024-09-15T00:12:27.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940199+0000", 2024-09-15T00:12:27.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025672037000000002 2024-09-15T00:12:27.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940269+0000", 2024-09-15T00:12:27.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9208999999999994e-05 2024-09-15T00:12:27.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940329+0000", 2024-09-15T00:12:27.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.999e-05 2024-09-15T00:12:27.660 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.660 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.660 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.660 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66551 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:27.660 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:27.661 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.673507715, 2024-09-15T00:12:27.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025917345000000001, 2024-09-15T00:12:27.661 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.661 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.661 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.661 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.661 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.661 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.661 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66551 2024-09-15T00:12:27.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.661 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:27.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:27.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:27.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1699999999999997e-07 2024-09-15T00:12:27.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914569+0000", 2024-09-15T00:12:27.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.225e-06 2024-09-15T00:12:27.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914582+0000", 2024-09-15T00:12:27.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3648999999999999e-05 2024-09-15T00:12:27.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914588+0000", 2024-09-15T00:12:27.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8449999999999999e-06 2024-09-15T00:12:27.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940355+0000", 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025767110999999999 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940423+0000", 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7574e-05 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940482+0000", 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9224000000000001e-05 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.664 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.665 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:88237 3.a 3:52924f23:::rbd_data.11396cb1f0e4.0000000000000001:16 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-15T00:12:27.665 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:27.665 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 25.310342752, 2024-09-15T00:12:27.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024612426999999999, 2024-09-15T00:12:27.665 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.665 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.665 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.665 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.665 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.665 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.665 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88237 2024-09-15T00:12:27.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.665 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:27.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:27.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:27.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7899999999999999e-07 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277732+0000", 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1679999999999998e-06 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277738+0000", 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.207e-06 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277743+0000", 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.639e-06 2024-09-15T00:12:27.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.301683+0000", 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.023939750999999999 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.301765+0000", 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2566999999999997e-05 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302342+0000", 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00057671600000000001 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.668 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.669 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.669 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:88238 3.a 3:52924f23:::rbd_data.11396cb1f0e4.0000000000000001:16 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-15T00:12:27.669 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:27.669 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 25.310323417999999, 2024-09-15T00:12:27.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024851132000000001, 2024-09-15T00:12:27.669 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.669 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.669 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:27.669 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.669 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.669 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.669 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88238 2024-09-15T00:12:27.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.670 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:27.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:27.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277750+0000", 2024-09-15T00:12:27.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3300000000000002e-07 2024-09-15T00:12:27.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277751+0000", 2024-09-15T00:12:27.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:12:27.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277757+0000", 2024-09-15T00:12:27.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.767e-06 2024-09-15T00:12:27.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277769+0000", 2024-09-15T00:12:27.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1874e-05 2024-09-15T00:12:27.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302374+0000", 2024-09-15T00:12:27.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024604669999999999 2024-09-15T00:12:27.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302460+0000", 2024-09-15T00:12:27.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6489999999999994e-05 2024-09-15T00:12:27.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302600+0000", 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013985700000000001 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125618 3.10 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.749575675, 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030860942999999998, 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.673 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.674 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.674 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125618 2024-09-15T00:12:27.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.674 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.674 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.674 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:12:27.674 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.674 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.674 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:12:27.674 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838499+0000", 2024-09-15T00:12:27.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6189999999999998e-06 2024-09-15T00:12:27.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839599+0000", 2024-09-15T00:12:27.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001099764 2024-09-15T00:12:27.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839620+0000", 2024-09-15T00:12:27.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0764e-05 2024-09-15T00:12:27.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839631+0000", 2024-09-15T00:12:27.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1335000000000001e-05 2024-09-15T00:12:27.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839725+0000", 2024-09-15T00:12:27.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4217999999999998e-05 2024-09-15T00:12:27.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839910+0000", 2024-09-15T00:12:27.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000184298 2024-09-15T00:12:27.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843171+0000", 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032613540000000002 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869187+0000", 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026015467 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869240+0000", 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3733000000000002e-05 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869309+0000", 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8381999999999997e-05 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869358+0000", 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125619 3.a 3:507b29bf:::rbd_data.11396481e1ec.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.7484135460000001, 2024-09-15T00:12:27.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045768870000000003, 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125619 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:12:27.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839660+0000", 2024-09-15T00:12:27.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.22e-06 2024-09-15T00:12:27.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840511+0000", 2024-09-15T00:12:27.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000851284 2024-09-15T00:12:27.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840522+0000", 2024-09-15T00:12:27.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0569e-05 2024-09-15T00:12:27.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840527+0000", 2024-09-15T00:12:27.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.412e-06 2024-09-15T00:12:27.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840573+0000", 2024-09-15T00:12:27.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5247000000000002e-05 2024-09-15T00:12:27.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840682+0000", 2024-09-15T00:12:27.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109368 2024-09-15T00:12:27.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:27.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843976+0000", 2024-09-15T00:12:27.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294445 2024-09-15T00:12:27.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:27.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870833+0000", 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026856944000000001 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885344+0000", 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014510744000000001 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885382+0000", 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7821000000000003e-05 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885428+0000", 2024-09-15T00:12:27.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5816000000000003e-05 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125621 3.a 3:53594db9:::rbd_data.11396481e1ec.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.7348901669999996, 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032481797, 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:27.684 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125621 2024-09-15T00:12:27.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.685 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:27.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:27.685 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:12:27.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:27.685 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:12:27.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:27.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:27.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853185+0000", 2024-09-15T00:12:27.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.43e-06 2024-09-15T00:12:27.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:27.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854125+0000", 2024-09-15T00:12:27.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000940539 2024-09-15T00:12:27.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:27.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854141+0000", 2024-09-15T00:12:27.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5568000000000001e-05 2024-09-15T00:12:27.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:27.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854148+0000", 2024-09-15T00:12:27.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6529999999999994e-06 2024-09-15T00:12:27.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:27.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870872+0000", 2024-09-15T00:12:27.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016723135 2024-09-15T00:12:27.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:27.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870975+0000", 2024-09-15T00:12:27.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010314099999999999 2024-09-15T00:12:27.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.688 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:27.688 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.873082+0000", 2024-09-15T00:12:27.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021069859999999999 2024-09-15T00:12:27.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.688 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:27.688 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885439+0000", 2024-09-15T00:12:27.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012357343999999999 2024-09-15T00:12:27.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.688 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:27.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885569+0000", 2024-09-15T00:12:27.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012995199999999999 2024-09-15T00:12:27.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.689 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:27.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885624+0000", 2024-09-15T00:12:27.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5281999999999997e-05 2024-09-15T00:12:27.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:27.689 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:27.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:27.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885664+0000", 2024-09-15T00:12:27.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9767000000000001e-05 2024-09-15T00:12:27.689 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.689 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.690 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.690 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:27.690 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:27.690 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:27.691 INFO:teuthology.orchestra.run.smithi171.stdout:4202 punch from 0x39df686 to 0x39e296c, (0x32e6 bytes) 2024-09-15T00:12:27.691 INFO:teuthology.orchestra.run.smithi171.stdout:4206 clone 33 order 22 su 65536 sc 6 2024-09-15T00:12:27.691 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:27.689+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:27.692 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:27.705 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:12:27.792 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:27.790+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:27.837 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:27.837 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:27.846 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:27.893 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:27.892+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:27.950 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:27.951 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:27.959 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:27.994 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:27.993+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:28.006 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:12:28.005+0000 7f6214cef640 0 --2- 172.21.15.171:0/1428602130 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62101444d0 0x7f6210144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T00:12:28.027 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:28.027 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:28.043 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:28.043 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:28.044 INFO:teuthology.orchestra.run.smithi171.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T00:12:28.059 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:28.060 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:28.076 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:28.076 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:28.092 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:28.092 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:28.096 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:28.094+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:28.105 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:28.105 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:28.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:28.105 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:28.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.105 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:28.106 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:28.106 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 62.024336992000002, 2024-09-15T00:12:28.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:12:28.106 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.106 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.106 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:28.106 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.106 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.106 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.107 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:12:28.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.107 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:28.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:28.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:12:28.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:12:28.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:12:28.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:12:28.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:12:28.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:12:28.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:12:28.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:12:28.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:12:28.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:12:28.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:12:28.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:12:28.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:28.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:12:28.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:12:28.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:28.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:12:28.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:12:28.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:28.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:12:28.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:12:28.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:28.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:12:28.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:12:28.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:12:28.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:12:28.111 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.111 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.111 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.111 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:12:28.112 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:28.112 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 62.011596449999999, 2024-09-15T00:12:28.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:12:28.112 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.112 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.112 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:28.112 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:28.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:28.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:12:28.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:12:28.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:12:28.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:12:28.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:12:28.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:12:28.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:12:28.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:12:28.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:12:28.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:12:28.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:12:28.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:12:28.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:12:28.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:12:28.115 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.115 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.115 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.116 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:28.116 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:28.116 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.879240691, 2024-09-15T00:12:28.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:12:28.116 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.116 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.116 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:28.116 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.117 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.117 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.117 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:12:28.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.117 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:28.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:28.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:12:28.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:12:28.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:12:28.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:12:28.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:12:28.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:12:28.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:12:28.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:12:28.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:12:28.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:12:28.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:12:28.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:12:28.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:28.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:12:28.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:12:28.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:28.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:12:28.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:12:28.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:28.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:12:28.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:12:28.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:28.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:12:28.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:12:28.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:12:28.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:12:28.122 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.122 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.122 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.122 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:12:28.123 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:28.123 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.865779046, 2024-09-15T00:12:28.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:12:28.123 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.123 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.123 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:28.123 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:28.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:28.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:12:28.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:12:28.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:12:28.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:12:28.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:12:28.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:12:28.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:12:28.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:12:28.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:12:28.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:12:28.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:12:28.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:12:28.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:12:28.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:12:28.127 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.127 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.127 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.127 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:28.127 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:28.127 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.680777562999999, 2024-09-15T00:12:28.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:12:28.127 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.127 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.127 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:28.127 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.128 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.128 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.128 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:12:28.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.128 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:28.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:28.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:12:28.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:12:28.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:12:28.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:12:28.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:12:28.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:12:28.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:12:28.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:12:28.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:12:28.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:12:28.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:12:28.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:12:28.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:28.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:12:28.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:12:28.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:28.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:12:28.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:12:28.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:28.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:12:28.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:12:28.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:28.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:12:28.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:12:28.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:12:28.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:12:28.133 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.133 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.133 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.133 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:12:28.133 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:28.133 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.667901295999997, 2024-09-15T00:12:28.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:12:28.133 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.133 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.134 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:28.134 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:28.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:28.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:12:28.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:12:28.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:12:28.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:12:28.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:12:28.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:12:28.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:12:28.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:12:28.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:12:28.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:12:28.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:12:28.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:12:28.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:12:28.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:12:28.137 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.137 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.137 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.137 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:12:28.137 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:28.137 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 47.209386152, 2024-09-15T00:12:28.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:12:28.137 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.137 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.137 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:28.137 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.137 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.138 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.138 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:12:28.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.138 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:28.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:28.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:12:28.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:28.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:12:28.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:12:28.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:12:28.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:12:28.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:12:28.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:12:28.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:12:28.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:12:28.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:12:28.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:12:28.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:28.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:12:28.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:12:28.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:28.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:12:28.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:12:28.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:28.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:12:28.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:12:28.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:28.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:12:28.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:12:28.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:12:28.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:12:28.144 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.144 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.144 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.144 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:28.144 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:28.144 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 43.491875592, 2024-09-15T00:12:28.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:12:28.145 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.145 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.145 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:28.145 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.145 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.145 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.145 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:12:28.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.145 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:28.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:28.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:12:28.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:12:28.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:12:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:12:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:12:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:12:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:12:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:12:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:12:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:12:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:12:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:12:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:12:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:12:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 43.491856982999998, 2024-09-15T00:12:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:12:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:12:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:28.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:28.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:12:28.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:12:28.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:12:28.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:12:28.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:12:28.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:12:28.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:12:28.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:12:28.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:12:28.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:12:28.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:12:28.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:12:28.153 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.153 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.153 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.153 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:28.153 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:28.154 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 43.491831658999999, 2024-09-15T00:12:28.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:12:28.154 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.154 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.154 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:28.154 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.154 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.154 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.154 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:12:28.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.155 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:28.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:28.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:28.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:12:28.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:12:28.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:12:28.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:12:28.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:12:28.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:12:28.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:12:28.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:12:28.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:12:28.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:12:28.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:12:28.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:12:28.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:12:28.159 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.159 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.159 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.159 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:28.159 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:28.159 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 43.491813335000003, 2024-09-15T00:12:28.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:12:28.160 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.160 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.160 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:28.160 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.160 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.160 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.160 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:12:28.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.160 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:28.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:28.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:12:28.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:12:28.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:12:28.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:12:28.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:12:28.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:28.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:12:28.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:12:28.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:12:28.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:12:28.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:12:28.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:12:28.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:12:28.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:12:28.163 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.164 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.164 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.164 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:28.164 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:28.164 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 43.491630762, 2024-09-15T00:12:28.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:12:28.164 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.164 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.164 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:28.164 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.164 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.164 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.165 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:12:28.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.165 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:28.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:28.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:28.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:12:28.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:12:28.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:12:28.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:12:28.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:12:28.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:12:28.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:28.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:12:28.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:12:28.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:12:28.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:12:28.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:12:28.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:12:28.168 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.168 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.168 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.169 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:28.169 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:28.169 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 43.491601920999997, 2024-09-15T00:12:28.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:12:28.169 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.169 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.169 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:28.169 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.169 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.169 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.169 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:12:28.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.169 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:28.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:28.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:28.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:12:28.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:12:28.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:12:28.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:12:28.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:12:28.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:12:28.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:12:28.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:12:28.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:12:28.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:12:28.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:12:28.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:12:28.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:12:28.173 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.173 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.173 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.173 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:28.173 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:28.173 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 43.491579710000003, 2024-09-15T00:12:28.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:12:28.173 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.173 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.174 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:28.174 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.174 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.174 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.174 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:12:28.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.174 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:28.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:28.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:28.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:12:28.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:12:28.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:12:28.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:12:28.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:12:28.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:12:28.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:28.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:12:28.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:12:28.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:12:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:12:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:12:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:12:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:28.178 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:28.178 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 43.491558550999997, 2024-09-15T00:12:28.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:12:28.178 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.178 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.178 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:28.178 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.178 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.178 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.178 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:12:28.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.178 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:28.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:28.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:28.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:28.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:12:28.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:12:28.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:12:28.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:12:28.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:12:28.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:12:28.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:12:28.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:12:28.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:12:28.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:12:28.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:12:28.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:12:28.182 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.182 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.182 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.182 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:28.182 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:28.182 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 43.491528602999999, 2024-09-15T00:12:28.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:12:28.182 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.182 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.182 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:28.182 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.183 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.183 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.183 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:12:28.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.183 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.183 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:28.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.183 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:28.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:28.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:12:28.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:12:28.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:12:28.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:12:28.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:12:28.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:12:28.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:12:28.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:12:28.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:12:28.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:12:28.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:12:28.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:12:28.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:12:28.186 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.186 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 43.491506639000001, 2024-09-15T00:12:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:12:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:12:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:12:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:12:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:12:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:12:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:12:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:12:28.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:12:28.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:12:28.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:12:28.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:12:28.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:12:28.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:12:28.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:12:28.191 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.191 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.191 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.191 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:28.191 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:28.191 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 43.491489211999998, 2024-09-15T00:12:28.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:12:28.191 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.191 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.192 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:28.192 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.192 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.192 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.192 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:12:28.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.192 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.192 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.192 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:28.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.192 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:28.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.193 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:12:28.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:12:28.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.193 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:12:28.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:12:28.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:12:28.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:28.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:12:28.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:12:28.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:12:28.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:12:28.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:12:28.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:12:28.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:12:28.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:12:28.195 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.195 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.195 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.196 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:28.196 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:28.196 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 43.491467413000002, 2024-09-15T00:12:28.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:12:28.196 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.196 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.196 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:28.196 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.196 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.196 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.196 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:12:28.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.196 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:28.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:28.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:12:28.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:12:28.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:12:28.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:12:28.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:12:28.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:12:28.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:12:28.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:12:28.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:12:28.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:12:28.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:12:28.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:12:28.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:12:28.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:12:28.200 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.200 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.200 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.200 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:28.201 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:28.201 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 43.442239053999998, 2024-09-15T00:12:28.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:12:28.201 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:28.201 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:28.201 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:28.201 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:28.202 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:28.202 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:28.202 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:12:28.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.202 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:28.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:28.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:28.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:28.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:28.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:28.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:28.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:12:28.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:12:28.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:28.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:12:28.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:12:28.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:28.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:12:28.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:12:28.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:28.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:12:28.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:12:28.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:28.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:12:28.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:12:28.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:28.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:12:28.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:12:28.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:28.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:12:28.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:12:28.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:28.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:12:28.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:12:28.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:28.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:12:28.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:12:28.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:28.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:12:28.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:12:28.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:28.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:28.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:28.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:12:28.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:12:28.207 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.207 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.207 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.207 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:28.207 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:28.207 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:28.208 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:28.209 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:28.209 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:28.210 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:28.207+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:28.210 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:28.211 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:12:28.212 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:28.212 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:12:28.212 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone32 from 0x5a3cf7d (overlap 0x5a3cf7d) to 0x3f2aad7 2024-09-15T00:12:28.309 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:28.307+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:28.314 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-15T00:12:28.371 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:28.371 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:28.384 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:28.410 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:28.408+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:28.494 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:28.494 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:28.504 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:28.535 INFO:teuthology.orchestra.run.smithi171.stdout:4207 writesame 0x1b148d4 thru 0x1b1d053 data_size 0x40(0x8780 bytes) 2024-09-15T00:12:28.536 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:28.533+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:28.545 INFO:teuthology.orchestra.run.smithi171.stdout:4208 read 0x5ea998 thru 0x5f2b9e (0x8207 bytes) 2024-09-15T00:12:28.553 INFO:teuthology.orchestra.run.smithi171.stdout:4209 trunc from 0x5a3cf7d to 0xbee3916 2024-09-15T00:12:28.611 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:28.610+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:28.613 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:28.613 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:28.622 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:28.713 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:28.711+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:28.766 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:28.766 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:28.766 INFO:teuthology.orchestra.run.smithi171.stdout:4210 write 0xd9c323d thru 0xd9cba42 (0x8806 bytes) 2024-09-15T00:12:28.778 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:28.784 INFO:teuthology.orchestra.run.smithi171.stdout:4211 trunc from 0xd9cba43 to 0x184a70f 2024-09-15T00:12:28.808 INFO:teuthology.orchestra.run.smithi171.stdout:4213 punch from 0x925bc2 to 0x929300, (0x373e bytes) 2024-09-15T00:12:28.809 INFO:teuthology.orchestra.run.smithi171.stdout:4214 flatten 2024-09-15T00:12:28.813 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:28.811+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:28.873 INFO:teuthology.orchestra.run.smithi171.stdout:4215 read 0xdb2018 thru 0xdb8507 (0x64f0 bytes) 2024-09-15T00:12:28.873 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:28.883 INFO:teuthology.orchestra.run.smithi171.stdout:4219 write 0x4d60f22 thru 0x4d703c6 (0xf4a5 bytes) 2024-09-15T00:12:28.901 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:28.901 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:28.902 INFO:teuthology.orchestra.run.smithi171.stdout:4220 writesame 0xb3c2f3b thru 0xb3cf3ce data_size 0x2e(0xc494 bytes) 2024-09-15T00:12:28.911 INFO:teuthology.orchestra.run.smithi171.stdout:4221 trunc from 0xb3cf3cf to 0x3c91328 2024-09-15T00:12:28.915 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:28.913+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:28.915 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:28.921 INFO:teuthology.orchestra.run.smithi171.stdout:4222 clone 34 order 21 su 131072 sc 12 2024-09-15T00:12:29.015 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:29.013+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:29.024 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:29.024 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:29.033 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:29.116 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:29.115+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:29.179 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:12:29.183 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:29.184 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:29.194 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:29.235 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:29.233+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:29.272 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-15T00:12:29.318 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:29.316+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:29.363 INFO:teuthology.orchestra.run.smithi171.stdout:4223 write 0x20cdbb1 thru 0x20ce669 (0xab9 bytes) 2024-09-15T00:12:29.371 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:29.371 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:29.373 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:29.381 INFO:teuthology.orchestra.run.smithi171.stdout:4227 write 0x5cd8295 thru 0x5cd8de8 (0xb54 bytes) 2024-09-15T00:12:29.383 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:29.385 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:29.391 INFO:teuthology.orchestra.run.smithi171.stdout:4230 trunc from 0x5cd8de9 to 0xb14deae 2024-09-15T00:12:29.397 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:29.397 INFO:teuthology.orchestra.run.smithi171.stdout:4232 read 0xae834ab thru 0xae864c8 (0x301e bytes) 2024-09-15T00:12:29.397 INFO:teuthology.orchestra.run.smithi171.stdout:4234 write 0x269bff6 thru 0x26a2cfc (0x6d07 bytes) 2024-09-15T00:12:29.414 INFO:teuthology.orchestra.run.smithi171.stdout:4235 writesame 0xbfcd016 thru 0xbfda428 data_size 0x1(0xd413 bytes) 2024-09-15T00:12:29.414 INFO:teuthology.orchestra.run.smithi171.stdout:4236 write 0x40bac42 thru 0x40c096b (0x5d2a bytes) 2024-09-15T00:12:29.419 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:29.417+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:29.428 INFO:teuthology.orchestra.run.smithi171.stdout:4237 write 0xd953c22 thru 0xd9573f5 (0x37d4 bytes) 2024-09-15T00:12:29.431 INFO:teuthology.orchestra.run.smithi171.stdout:4239 punch from 0xa065ca3 to 0xa073fd0, (0xe32d bytes) 2024-09-15T00:12:29.431 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:29.447 INFO:teuthology.orchestra.run.smithi171.stdout:4242 trunc from 0xd9573f6 to 0x2a553e2 2024-09-15T00:12:29.494 INFO:teuthology.orchestra.run.smithi171.stdout:4243 writesame 0x4d4d127 thru 0x4d4fabd data_size 0x3f(0x2997 bytes) 2024-09-15T00:12:29.494 INFO:teuthology.orchestra.run.smithi171.stdout:4244 clone 35 order 22 su 32768 sc 2 2024-09-15T00:12:29.515 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:29.515 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:29.519 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:29.518+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:29.524 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:29.620 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:29.618+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:29.628 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:29.628 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:29.637 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:29.721 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:29.719+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:29.781 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:29.781 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:29.790 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:29.834 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:29.832+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:29.893 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:29.893 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:29.901 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:29.943 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:29.941+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:30.023 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:30.021+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:30.045 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:30.045 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:30.054 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:30.137 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:30.136+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:30.157 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is set 2024-09-15T00:12:30.204 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:30.204 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:30.213 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:30.214 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:12:30.225 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone34 from 0x4d4fabe (overlap 0x2a553e2) to 0x1da211e 2024-09-15T00:12:30.255 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:30.253+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:30.296 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-15T00:12:30.325 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:30.324+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:30.351 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:30.352 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:30.363 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:30.363 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:30.369 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:30.370 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:30.370 INFO:teuthology.orchestra.run.smithi171.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T00:12:30.372 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:30.387 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:30.387 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:30.390 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:30.390 INFO:teuthology.orchestra.run.smithi171.stdout:4246 read 0x3dc85e4 thru 0x3dd70b8 (0xead5 bytes) 2024-09-15T00:12:30.405 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:30.405 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:30.407 INFO:teuthology.orchestra.run.smithi171.stdout:4247 writesame 0xaedbf6e thru 0xaeeab90 data_size 0x1(0xec23 bytes) 2024-09-15T00:12:30.407 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:30.407 INFO:teuthology.orchestra.run.smithi171.stdout:4249 writesame 0x20aee22 thru 0x20b7c47 data_size 0xf(0x8e26 bytes) 2024-09-15T00:12:30.426 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:30.425+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:30.428 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:30.429 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:30.446 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:30.446 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:30.451 INFO:teuthology.orchestra.run.smithi171.stdout:4250 punch from 0x24b15d2 to 0x24b473e, (0x316c bytes) 2024-09-15T00:12:30.451 INFO:teuthology.orchestra.run.smithi171.stdout:4251 read 0x6bcff89 thru 0x6bdb535 (0xb5ad bytes) 2024-09-15T00:12:30.451 INFO:teuthology.orchestra.run.smithi171.stdout:4252 read 0x14eb2cc thru 0x14eec58 (0x398d bytes) 2024-09-15T00:12:30.451 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:30.451 INFO:teuthology.orchestra.run.smithi171.stdout:4254 read 0x84f0938 thru 0x84f8576 (0x7c3f bytes) 2024-09-15T00:12:30.451 INFO:teuthology.orchestra.run.smithi171.stdout:4255 writesame 0x5ed5450 thru 0x5eda8ee data_size 0x1(0x549f bytes) 2024-09-15T00:12:30.465 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:30.466 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:30.469 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:12:30.469 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:12:30.472 INFO:teuthology.orchestra.run.smithi171.stdout:4256 write 0xacf0c11 thru 0xad009f2 (0xfde2 bytes) 2024-09-15T00:12:30.477 INFO:teuthology.orchestra.run.smithi171.stdout:4257 write 0x85b8956 thru 0x85c32c6 (0xa971 bytes) 2024-09-15T00:12:30.482 INFO:teuthology.orchestra.run.smithi171.stdout:4258 writesame 0x44aff3d thru 0x44b4708 data_size 0xa(0x47cc bytes) 2024-09-15T00:12:30.487 INFO:teuthology.orchestra.run.smithi171.stdout:4259 read 0x81ea1c thru 0x82d61e (0xec03 bytes) 2024-09-15T00:12:30.487 INFO:teuthology.orchestra.run.smithi171.stdout:4263 read 0x6eca318 thru 0x6ed3d53 (0x9a3c bytes) 2024-09-15T00:12:30.487 INFO:teuthology.orchestra.run.smithi171.stdout:4264 write 0x2fd2a6f thru 0x2fdb1a9 (0x873b bytes) 2024-09-15T00:12:30.496 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:30.496 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:30.496 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:30.496 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:30.505 INFO:teuthology.orchestra.run.smithi171.stdout:4265 trunc from 0xaeeab91 to 0xd95c938 2024-09-15T00:12:30.506 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:30.511 INFO:teuthology.orchestra.run.smithi171.stdout:4266 punch from 0x310109f to 0x311016a, (0xf0cb bytes) 2024-09-15T00:12:30.511 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:30.511 INFO:teuthology.orchestra.run.smithi171.stdout:4268 read 0x6f84dff thru 0x6f9432d (0xf52f bytes) 2024-09-15T00:12:30.511 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:30.511 INFO:teuthology.orchestra.run.smithi171.stdout:4271 trunc from 0xd95c938 to 0xcfa83cb 2024-09-15T00:12:30.517 INFO:teuthology.orchestra.run.smithi171.stdout:4272 flatten 2024-09-15T00:12:30.547 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:30.545+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:30.628 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:30.626+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:30.636 INFO:teuthology.orchestra.run.smithi171.stdout:4274 write 0x76b7886 thru 0x76ba3ca (0x2b45 bytes) 2024-09-15T00:12:30.641 INFO:teuthology.orchestra.run.smithi171.stdout:4275 write 0x6db53f1 thru 0x6dc06b4 (0xb2c4 bytes) 2024-09-15T00:12:30.646 INFO:teuthology.orchestra.run.smithi171.stdout:4276 write 0x5c85bca thru 0x5c85eb8 (0x2ef bytes) 2024-09-15T00:12:30.662 INFO:teuthology.orchestra.run.smithi171.stdout:4277 trunc from 0xcfa83cb to 0xa9ece1 2024-09-15T00:12:30.662 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:30.662 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:30.662 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:30.663 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:30.663 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:30.673 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:30.705 INFO:teuthology.orchestra.run.smithi171.stdout:4278 write 0xd9024a6 thru 0xd90e98c (0xc4e7 bytes) 2024-09-15T00:12:30.713 INFO:teuthology.orchestra.run.smithi171.stdout:4279 punch from 0x2ef07be to 0x2efd91e, (0xd160 bytes) 2024-09-15T00:12:30.713 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:30.713 INFO:teuthology.orchestra.run.smithi171.stdout:4281 writesame 0x1971753 thru 0x1971dea data_size 0x4(0x698 bytes) 2024-09-15T00:12:30.716 INFO:teuthology.orchestra.run.smithi171.stdout:4282 write 0x220ce54 thru 0x2219f13 (0xd0c0 bytes) 2024-09-15T00:12:30.720 INFO:teuthology.orchestra.run.smithi171.stdout:4283 read 0xb1650c0 thru 0xb16f666 (0xa5a7 bytes) 2024-09-15T00:12:30.721 INFO:teuthology.orchestra.run.smithi171.stdout:4285 read 0x43bc3ed thru 0x43c7214 (0xae28 bytes) 2024-09-15T00:12:30.721 INFO:teuthology.orchestra.run.smithi171.stdout:4286 write 0x3fc4258 thru 0x3fcb90d (0x76b6 bytes) 2024-09-15T00:12:30.724 INFO:teuthology.orchestra.run.smithi171.stdout:4288 write 0x5b37670 thru 0x5b3bd16 (0x46a7 bytes) 2024-09-15T00:12:30.728 INFO:teuthology.orchestra.run.smithi171.stdout:4289 read 0x593ef75 thru 0x594acd2 (0xbd5e bytes) 2024-09-15T00:12:30.728 INFO:teuthology.orchestra.run.smithi171.stdout:4290 punch from 0x3e9ac0f to 0x3ea236a, (0x775b bytes) 2024-09-15T00:12:30.729 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:30.729 INFO:teuthology.orchestra.run.smithi171.stdout:4293 read 0x939c7fd thru 0x93a6f8a (0xa78e bytes) 2024-09-15T00:12:30.729 INFO:teuthology.orchestra.run.smithi171.stdout:4294 read 0x5dab5e4 thru 0x5dbaaa3 (0xf4c0 bytes) 2024-09-15T00:12:30.729 INFO:teuthology.orchestra.run.smithi171.stdout:4295 write 0x43533c8 thru 0x43561f5 (0x2e2e bytes) 2024-09-15T00:12:30.744 INFO:teuthology.orchestra.run.smithi171.stdout:4296 trunc from 0xd90e98d to 0x314c567 2024-09-15T00:12:30.754 INFO:teuthology.orchestra.run.smithi171.stdout:4298 read 0x3055659 thru 0x305a0ed (0x4a95 bytes) 2024-09-15T00:12:30.756 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:30.755+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:30.765 INFO:teuthology.orchestra.run.smithi171.stdout:4299 write 0x69efec0 thru 0x69f1910 (0x1a51 bytes) 2024-09-15T00:12:30.772 INFO:teuthology.orchestra.run.smithi171.stdout:4300 trunc from 0x69f1911 to 0x3c4c5a1 2024-09-15T00:12:30.800 INFO:teuthology.orchestra.run.smithi171.stdout:4301 writesame 0xd3b4b59 thru 0xd3c1561 data_size 0x1(0xca09 bytes) 2024-09-15T00:12:30.800 INFO:teuthology.orchestra.run.smithi171.stdout:4302 read 0x42e328f thru 0x42ee686 (0xb3f8 bytes) 2024-09-15T00:12:30.800 INFO:teuthology.orchestra.run.smithi171.stdout:4303 write 0xcb44107 thru 0xcb4a3d4 (0x62ce bytes) 2024-09-15T00:12:30.807 INFO:teuthology.orchestra.run.smithi171.stdout:4305 read 0x65f2738 thru 0x65f3bea (0x14b3 bytes) 2024-09-15T00:12:30.807 INFO:teuthology.orchestra.run.smithi171.stdout:4306 write 0x8c80f3b thru 0x8c8919f (0x8265 bytes) 2024-09-15T00:12:30.812 INFO:teuthology.orchestra.run.smithi171.stdout:4307 writesame 0x9fdc88 thru 0xa0bbe1 data_size 0xb(0xdf5a bytes) 2024-09-15T00:12:30.816 INFO:teuthology.orchestra.run.smithi171.stdout:4309 write 0x637b493 thru 0x6385f76 (0xaae4 bytes) 2024-09-15T00:12:30.820 INFO:teuthology.orchestra.run.smithi171.stdout:4310 write 0x95de184 thru 0x95e552c (0x73a9 bytes) 2024-09-15T00:12:30.823 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:30.823 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:30.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:30.823 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:30.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.823 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:19308 3.10 3:0ee1524d:::rbd_data.11398e1a091a.0000000000000017:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-15T00:12:30.823 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:30.823 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 51.835015200999997, 2024-09-15T00:12:30.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02547147, 2024-09-15T00:12:30.824 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.824 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.824 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:30.824 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.824 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.824 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.824 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 19308 2024-09-15T00:12:30.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.824 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:30.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:30.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985182+0000", 2024-09-15T00:12:30.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6500000000000001e-06 2024-09-15T00:12:30.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985189+0000", 2024-09-15T00:12:30.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9949999999999999e-06 2024-09-15T00:12:30.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985205+0000", 2024-09-15T00:12:30.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6055000000000001e-05 2024-09-15T00:12:30.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985212+0000", 2024-09-15T00:12:30.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:12:30.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985261+0000", 2024-09-15T00:12:30.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8380000000000001e-05 2024-09-15T00:12:30.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985311+0000", 2024-09-15T00:12:30.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0763999999999997e-05 2024-09-15T00:12:30.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:30.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.986489+0000", 2024-09-15T00:12:30.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0011773420000000001 2024-09-15T00:12:30.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:30.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010514+0000", 2024-09-15T00:12:30.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024025392999999999 2024-09-15T00:12:30.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:30.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010535+0000", 2024-09-15T00:12:30.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0652000000000001e-05 2024-09-15T00:12:30.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:30.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010597+0000", 2024-09-15T00:12:30.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2411999999999998e-05 2024-09-15T00:12:30.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010652+0000", 2024-09-15T00:12:30.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5074999999999997e-05 2024-09-15T00:12:30.829 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.829 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.829 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.829 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:30.829 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:30.829 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.152838232999997, 2024-09-15T00:12:30.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:12:30.830 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.830 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.830 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:30.830 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.830 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.830 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.830 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:12:30.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.830 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:30.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:30.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:12:30.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:12:30.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:12:30.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:12:30.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:12:30.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:12:30.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:12:30.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:12:30.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:12:30.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:12:30.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:12:30.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:12:30.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:30.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:12:30.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:12:30.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:30.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:12:30.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:12:30.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:30.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:12:30.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:12:30.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:30.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:12:30.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:12:30.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:12:30.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:12:30.835 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.835 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.835 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.835 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65520 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:30.835 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:30.835 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.937783177999997, 2024-09-15T00:12:30.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024513282000000001, 2024-09-15T00:12:30.836 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.836 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.836 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:30.836 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.836 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.836 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.836 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65520 2024-09-15T00:12:30.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.836 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:30.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:30.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:30.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.39e-07 2024-09-15T00:12:30.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882415+0000", 2024-09-15T00:12:30.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8020000000000001e-06 2024-09-15T00:12:30.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882421+0000", 2024-09-15T00:12:30.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7119999999999997e-06 2024-09-15T00:12:30.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882425+0000", 2024-09-15T00:12:30.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2660000000000003e-06 2024-09-15T00:12:30.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906919+0000", 2024-09-15T00:12:30.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024494438 2024-09-15T00:12:30.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906926+0000", 2024-09-15T00:12:30.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.725e-06 2024-09-15T00:12:30.839 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.839 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.839 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.839 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65525 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:30.839 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:30.840 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.937668309999999, 2024-09-15T00:12:30.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024517753, 2024-09-15T00:12:30.840 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.840 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.840 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:30.840 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.840 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.840 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.840 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65525 2024-09-15T00:12:30.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.840 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:30.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:30.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:30.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:12:30.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882530+0000", 2024-09-15T00:12:30.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.804e-06 2024-09-15T00:12:30.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882535+0000", 2024-09-15T00:12:30.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5500000000000002e-06 2024-09-15T00:12:30.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882540+0000", 2024-09-15T00:12:30.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6909999999999996e-06 2024-09-15T00:12:30.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907039+0000", 2024-09-15T00:12:30.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024498809 2024-09-15T00:12:30.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907045+0000", 2024-09-15T00:12:30.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5459999999999997e-06 2024-09-15T00:12:30.843 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.843 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.843 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.843 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66541 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1449984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:30.844 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:12:30.844 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.906060721000003, 2024-09-15T00:12:30.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024574262999999999, 2024-09-15T00:12:30.844 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.844 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.844 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:30.844 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.844 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.844 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.844 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66541 2024-09-15T00:12:30.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.844 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:12:30.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:12:30.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914136+0000", 2024-09-15T00:12:30.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3599999999999999e-07 2024-09-15T00:12:30.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914137+0000", 2024-09-15T00:12:30.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9309999999999998e-06 2024-09-15T00:12:30.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914143+0000", 2024-09-15T00:12:30.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0120000000000002e-06 2024-09-15T00:12:30.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914148+0000", 2024-09-15T00:12:30.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.707e-06 2024-09-15T00:12:30.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938536+0000", 2024-09-15T00:12:30.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024387671999999999 2024-09-15T00:12:30.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938618+0000", 2024-09-15T00:12:30.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1811999999999995e-05 2024-09-15T00:12:30.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938709+0000", 2024-09-15T00:12:30.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1792999999999993e-05 2024-09-15T00:12:30.848 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.848 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.848 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.848 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66542 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1560576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:30.848 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:30.848 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.90604158, 2024-09-15T00:12:30.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024740410000000001, 2024-09-15T00:12:30.848 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.848 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.848 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:30.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.849 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66542 2024-09-15T00:12:30.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.849 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:30.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:30.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914155+0000", 2024-09-15T00:12:30.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:12:30.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914157+0000", 2024-09-15T00:12:30.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:12:30.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914162+0000", 2024-09-15T00:12:30.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:30.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914167+0000", 2024-09-15T00:12:30.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3789999999999999e-06 2024-09-15T00:12:30.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938730+0000", 2024-09-15T00:12:30.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024562694 2024-09-15T00:12:30.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938803+0000", 2024-09-15T00:12:30.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3325999999999996e-05 2024-09-15T00:12:30.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938895+0000", 2024-09-15T00:12:30.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1884000000000003e-05 2024-09-15T00:12:30.852 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.852 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.852 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.852 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66543 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1572864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:30.852 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:30.853 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.906011321000001, 2024-09-15T00:12:30.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024893976000000002, 2024-09-15T00:12:30.853 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.853 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.853 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:30.853 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.853 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.853 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.853 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66543 2024-09-15T00:12:30.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.853 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:30.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:30.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:30.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:30.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914188+0000", 2024-09-15T00:12:30.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:12:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914193+0000", 2024-09-15T00:12:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4129999999999997e-06 2024-09-15T00:12:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914198+0000", 2024-09-15T00:12:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9309999999999999e-06 2024-09-15T00:12:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938914+0000", 2024-09-15T00:12:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024716380999999999 2024-09-15T00:12:30.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938993+0000", 2024-09-15T00:12:30.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8763000000000005e-05 2024-09-15T00:12:30.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939079+0000", 2024-09-15T00:12:30.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5372999999999997e-05 2024-09-15T00:12:30.856 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.856 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.856 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.857 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66544 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1683456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:30.857 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:30.857 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.905988917999998, 2024-09-15T00:12:30.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025051225, 2024-09-15T00:12:30.857 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.857 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.857 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:30.857 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.857 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.857 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.857 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66544 2024-09-15T00:12:30.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.857 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:30.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:30.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:30.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:12:30.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914210+0000", 2024-09-15T00:12:30.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2450000000000001e-06 2024-09-15T00:12:30.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914215+0000", 2024-09-15T00:12:30.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7869999999999998e-06 2024-09-15T00:12:30.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914220+0000", 2024-09-15T00:12:30.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7899999999999999e-06 2024-09-15T00:12:30.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939105+0000", 2024-09-15T00:12:30.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024884381000000001 2024-09-15T00:12:30.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939177+0000", 2024-09-15T00:12:30.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2138999999999995e-05 2024-09-15T00:12:30.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939258+0000", 2024-09-15T00:12:30.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1462e-05 2024-09-15T00:12:30.861 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.861 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.861 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.861 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66545 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1695744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:30.861 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:30.861 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.905966644000003, 2024-09-15T00:12:30.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025209788, 2024-09-15T00:12:30.861 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.861 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.862 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:30.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.862 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66545 2024-09-15T00:12:30.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.862 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:30.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:30.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914230+0000", 2024-09-15T00:12:30.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1099999999999996e-07 2024-09-15T00:12:30.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914232+0000", 2024-09-15T00:12:30.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3080000000000002e-06 2024-09-15T00:12:30.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914238+0000", 2024-09-15T00:12:30.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.02e-06 2024-09-15T00:12:30.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914243+0000", 2024-09-15T00:12:30.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1200000000000001e-06 2024-09-15T00:12:30.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939285+0000", 2024-09-15T00:12:30.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025042212000000001 2024-09-15T00:12:30.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939361+0000", 2024-09-15T00:12:30.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5558999999999999e-05 2024-09-15T00:12:30.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939439+0000", 2024-09-15T00:12:30.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8058000000000004e-05 2024-09-15T00:12:30.865 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.865 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.865 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.866 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66546 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:30.866 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:30.866 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.905946669000002, 2024-09-15T00:12:30.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025375960999999999, 2024-09-15T00:12:30.866 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.866 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.866 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:30.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.866 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66546 2024-09-15T00:12:30.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914250+0000", 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:12:30.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914252+0000", 2024-09-15T00:12:30.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.052e-06 2024-09-15T00:12:30.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914258+0000", 2024-09-15T00:12:30.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9039999999999997e-06 2024-09-15T00:12:30.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914262+0000", 2024-09-15T00:12:30.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4719999999999997e-06 2024-09-15T00:12:30.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939466+0000", 2024-09-15T00:12:30.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025203550000000002 2024-09-15T00:12:30.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939537+0000", 2024-09-15T00:12:30.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1780999999999996e-05 2024-09-15T00:12:30.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939625+0000", 2024-09-15T00:12:30.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7828e-05 2024-09-15T00:12:30.869 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66547 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1818624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.905926995999998, 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025552638999999999, 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66547 2024-09-15T00:12:30.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:30.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914271+0000", 2024-09-15T00:12:30.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8920000000000001e-06 2024-09-15T00:12:30.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914277+0000", 2024-09-15T00:12:30.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:30.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914282+0000", 2024-09-15T00:12:30.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7190000000000001e-06 2024-09-15T00:12:30.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939654+0000", 2024-09-15T00:12:30.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025372372000000001 2024-09-15T00:12:30.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939734+0000", 2024-09-15T00:12:30.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9946999999999999e-05 2024-09-15T00:12:30.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939822+0000", 2024-09-15T00:12:30.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7680000000000001e-05 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66548 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.905896609000003, 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025712727000000001, 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.874 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66548 2024-09-15T00:12:30.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.875 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:30.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:30.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914300+0000", 2024-09-15T00:12:30.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3599999999999999e-07 2024-09-15T00:12:30.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914302+0000", 2024-09-15T00:12:30.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3369999999999998e-06 2024-09-15T00:12:30.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914313+0000", 2024-09-15T00:12:30.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1399999999999999e-05 2024-09-15T00:12:30.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914319+0000", 2024-09-15T00:12:30.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9100000000000002e-06 2024-09-15T00:12:30.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939850+0000", 2024-09-15T00:12:30.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025531087000000001 2024-09-15T00:12:30.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939932+0000", 2024-09-15T00:12:30.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1534999999999998e-05 2024-09-15T00:12:30.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940012+0000", 2024-09-15T00:12:30.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0022000000000001e-05 2024-09-15T00:12:30.878 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.878 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.878 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.878 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66549 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1941504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:30.878 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:30.878 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.905831517999999, 2024-09-15T00:12:30.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025813836999999999, 2024-09-15T00:12:30.878 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.878 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.879 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:30.879 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.879 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.879 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.879 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66549 2024-09-15T00:12:30.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.879 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:30.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:30.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914365+0000", 2024-09-15T00:12:30.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2799999999999995e-07 2024-09-15T00:12:30.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914368+0000", 2024-09-15T00:12:30.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.92e-06 2024-09-15T00:12:30.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914376+0000", 2024-09-15T00:12:30.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8110000000000003e-06 2024-09-15T00:12:30.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914381+0000", 2024-09-15T00:12:30.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5119999999999999e-06 2024-09-15T00:12:30.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940040+0000", 2024-09-15T00:12:30.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025658271999999999 2024-09-15T00:12:30.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940103+0000", 2024-09-15T00:12:30.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2891000000000005e-05 2024-09-15T00:12:30.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940178+0000", 2024-09-15T00:12:30.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5703000000000003e-05 2024-09-15T00:12:30.882 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.882 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.882 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.883 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66550 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:30.883 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:30.883 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.905687938, 2024-09-15T00:12:30.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025820701000000001, 2024-09-15T00:12:30.883 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.883 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.883 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:30.883 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.883 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.883 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.883 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66550 2024-09-15T00:12:30.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.884 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:30.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:30.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914509+0000", 2024-09-15T00:12:30.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1999999999999999e-07 2024-09-15T00:12:30.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914513+0000", 2024-09-15T00:12:30.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1640000000000003e-06 2024-09-15T00:12:30.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914522+0000", 2024-09-15T00:12:30.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7639999999999994e-06 2024-09-15T00:12:30.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914527+0000", 2024-09-15T00:12:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9170000000000003e-06 2024-09-15T00:12:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940199+0000", 2024-09-15T00:12:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025672037000000002 2024-09-15T00:12:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940269+0000", 2024-09-15T00:12:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9208999999999994e-05 2024-09-15T00:12:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940329+0000", 2024-09-15T00:12:30.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.999e-05 2024-09-15T00:12:30.887 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.887 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.887 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.887 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66551 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:30.887 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:30.887 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.905631190999998, 2024-09-15T00:12:30.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025917345000000001, 2024-09-15T00:12:30.887 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.887 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.887 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:30.888 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.888 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.888 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.888 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66551 2024-09-15T00:12:30.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.888 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:30.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:30.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:30.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1699999999999997e-07 2024-09-15T00:12:30.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914569+0000", 2024-09-15T00:12:30.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.225e-06 2024-09-15T00:12:30.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914582+0000", 2024-09-15T00:12:30.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3648999999999999e-05 2024-09-15T00:12:30.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914588+0000", 2024-09-15T00:12:30.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8449999999999999e-06 2024-09-15T00:12:30.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940355+0000", 2024-09-15T00:12:30.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025767110999999999 2024-09-15T00:12:30.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940423+0000", 2024-09-15T00:12:30.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7574e-05 2024-09-15T00:12:30.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940482+0000", 2024-09-15T00:12:30.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9224000000000001e-05 2024-09-15T00:12:30.891 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.891 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.891 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.891 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:88237 3.a 3:52924f23:::rbd_data.11396cb1f0e4.0000000000000001:16 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-15T00:12:30.892 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:30.892 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.542466227999999, 2024-09-15T00:12:30.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024612426999999999, 2024-09-15T00:12:30.892 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.892 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.892 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:30.892 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.892 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.892 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.892 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88237 2024-09-15T00:12:30.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.892 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:30.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:30.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:30.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7899999999999999e-07 2024-09-15T00:12:30.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277732+0000", 2024-09-15T00:12:30.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1679999999999998e-06 2024-09-15T00:12:30.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277738+0000", 2024-09-15T00:12:30.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.207e-06 2024-09-15T00:12:30.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277743+0000", 2024-09-15T00:12:30.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.639e-06 2024-09-15T00:12:30.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.301683+0000", 2024-09-15T00:12:30.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.023939750999999999 2024-09-15T00:12:30.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.301765+0000", 2024-09-15T00:12:30.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2566999999999997e-05 2024-09-15T00:12:30.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302342+0000", 2024-09-15T00:12:30.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00057671600000000001 2024-09-15T00:12:30.896 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.896 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.896 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.896 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:88238 3.a 3:52924f23:::rbd_data.11396cb1f0e4.0000000000000001:16 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-15T00:12:30.896 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:30.896 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.542446894000001, 2024-09-15T00:12:30.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024851132000000001, 2024-09-15T00:12:30.896 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.896 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.896 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:30.896 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.896 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.897 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.897 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88238 2024-09-15T00:12:30.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.897 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:30.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:30.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277750+0000", 2024-09-15T00:12:30.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3300000000000002e-07 2024-09-15T00:12:30.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277751+0000", 2024-09-15T00:12:30.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:12:30.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277757+0000", 2024-09-15T00:12:30.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.767e-06 2024-09-15T00:12:30.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277769+0000", 2024-09-15T00:12:30.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1874e-05 2024-09-15T00:12:30.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302374+0000", 2024-09-15T00:12:30.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024604669999999999 2024-09-15T00:12:30.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302460+0000", 2024-09-15T00:12:30.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6489999999999994e-05 2024-09-15T00:12:30.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302600+0000", 2024-09-15T00:12:30.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013985700000000001 2024-09-15T00:12:30.900 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.900 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.900 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.900 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125618 3.10 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:30.901 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:12:30.901 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9816991510000008, 2024-09-15T00:12:30.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030860942999999998, 2024-09-15T00:12:30.901 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.901 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.901 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:30.901 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.901 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.901 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.901 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125618 2024-09-15T00:12:30.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.901 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:12:30.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:12:30.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838499+0000", 2024-09-15T00:12:30.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6189999999999998e-06 2024-09-15T00:12:30.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839599+0000", 2024-09-15T00:12:30.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001099764 2024-09-15T00:12:30.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839620+0000", 2024-09-15T00:12:30.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0764e-05 2024-09-15T00:12:30.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839631+0000", 2024-09-15T00:12:30.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1335000000000001e-05 2024-09-15T00:12:30.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839725+0000", 2024-09-15T00:12:30.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4217999999999998e-05 2024-09-15T00:12:30.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839910+0000", 2024-09-15T00:12:30.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000184298 2024-09-15T00:12:30.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:30.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843171+0000", 2024-09-15T00:12:30.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032613540000000002 2024-09-15T00:12:30.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:30.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869187+0000", 2024-09-15T00:12:30.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026015467 2024-09-15T00:12:30.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:30.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869240+0000", 2024-09-15T00:12:30.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3733000000000002e-05 2024-09-15T00:12:30.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:30.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869309+0000", 2024-09-15T00:12:30.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8381999999999997e-05 2024-09-15T00:12:30.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869358+0000", 2024-09-15T00:12:30.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-15T00:12:30.906 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.906 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.906 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.906 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125619 3.a 3:507b29bf:::rbd_data.11396481e1ec.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:30.906 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:12:30.907 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.980537022, 2024-09-15T00:12:30.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045768870000000003, 2024-09-15T00:12:30.907 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.907 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.907 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:30.907 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.907 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.907 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.907 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125619 2024-09-15T00:12:30.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.907 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:12:30.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:12:30.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839660+0000", 2024-09-15T00:12:30.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.22e-06 2024-09-15T00:12:30.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840511+0000", 2024-09-15T00:12:30.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000851284 2024-09-15T00:12:30.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840522+0000", 2024-09-15T00:12:30.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0569e-05 2024-09-15T00:12:30.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840527+0000", 2024-09-15T00:12:30.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.412e-06 2024-09-15T00:12:30.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840573+0000", 2024-09-15T00:12:30.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5247000000000002e-05 2024-09-15T00:12:30.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840682+0000", 2024-09-15T00:12:30.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109368 2024-09-15T00:12:30.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:30.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843976+0000", 2024-09-15T00:12:30.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294445 2024-09-15T00:12:30.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:30.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870833+0000", 2024-09-15T00:12:30.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026856944000000001 2024-09-15T00:12:30.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:30.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885344+0000", 2024-09-15T00:12:30.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014510744000000001 2024-09-15T00:12:30.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:30.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885382+0000", 2024-09-15T00:12:30.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7821000000000003e-05 2024-09-15T00:12:30.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885428+0000", 2024-09-15T00:12:30.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5816000000000003e-05 2024-09-15T00:12:30.912 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.912 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.912 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.912 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125621 3.a 3:53594db9:::rbd_data.11396481e1ec.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:30.912 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:12:30.912 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9670136429999996, 2024-09-15T00:12:30.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032481797, 2024-09-15T00:12:30.913 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:30.913 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:30.913 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:30.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:30.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:30.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:30.913 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125621 2024-09-15T00:12:30.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.913 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:30.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:30.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:12:30.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:30.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:12:30.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:30.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:30.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853185+0000", 2024-09-15T00:12:30.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.43e-06 2024-09-15T00:12:30.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:30.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854125+0000", 2024-09-15T00:12:30.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000940539 2024-09-15T00:12:30.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:30.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854141+0000", 2024-09-15T00:12:30.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5568000000000001e-05 2024-09-15T00:12:30.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:30.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854148+0000", 2024-09-15T00:12:30.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6529999999999994e-06 2024-09-15T00:12:30.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:30.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870872+0000", 2024-09-15T00:12:30.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016723135 2024-09-15T00:12:30.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:30.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870975+0000", 2024-09-15T00:12:30.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010314099999999999 2024-09-15T00:12:30.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:30.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.873082+0000", 2024-09-15T00:12:30.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021069859999999999 2024-09-15T00:12:30.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:30.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885439+0000", 2024-09-15T00:12:30.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012357343999999999 2024-09-15T00:12:30.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:30.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885569+0000", 2024-09-15T00:12:30.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012995199999999999 2024-09-15T00:12:30.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:30.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885624+0000", 2024-09-15T00:12:30.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5281999999999997e-05 2024-09-15T00:12:30.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:30.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:30.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:30.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885664+0000", 2024-09-15T00:12:30.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9767000000000001e-05 2024-09-15T00:12:30.918 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.918 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.918 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.918 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:30.918 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:30.918 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:30.920 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:30.920 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:30.918+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:30.921 INFO:teuthology.orchestra.run.smithi171.stdout:4312 writesame 0x6b7a3 thru 0x775de data_size 0x19(0xbe3c bytes) 2024-09-15T00:12:30.921 INFO:teuthology.orchestra.run.smithi171.stdout:4313 trunc from 0xd3c1562 to 0xa782f3f 2024-09-15T00:12:30.921 INFO:teuthology.orchestra.run.smithi171.stdout:4314 write 0x90bf4df thru 0x90c86b9 (0x91db bytes) 2024-09-15T00:12:30.921 INFO:teuthology.orchestra.run.smithi171.stdout:4316 writesame 0x827a9b2 thru 0x8287e22 data_size 0x49(0xd471 bytes) 2024-09-15T00:12:30.922 INFO:teuthology.orchestra.run.smithi171.stdout:4320 write 0xa4f8224 thru 0xa4fe0e8 (0x5ec5 bytes) 2024-09-15T00:12:30.922 INFO:teuthology.orchestra.run.smithi171.stdout:4321 read 0x98e9c9e thru 0x98ee191 (0x44f4 bytes) 2024-09-15T00:12:30.922 INFO:teuthology.orchestra.run.smithi171.stdout:4322 read 0x5a82c89 thru 0x5a92205 (0xf57d bytes) 2024-09-15T00:12:30.922 INFO:teuthology.orchestra.run.smithi171.stdout:4323 write 0xc90bdf7 thru 0xc91ad6c (0xef76 bytes) 2024-09-15T00:12:30.922 INFO:teuthology.orchestra.run.smithi171.stdout:4324 read 0x783276f thru 0x7841e55 (0xf6e7 bytes) 2024-09-15T00:12:30.922 INFO:teuthology.orchestra.run.smithi171.stdout:4327 read 0x7e7e8fd thru 0x7e84a99 (0x619d bytes) 2024-09-15T00:12:30.922 INFO:teuthology.orchestra.run.smithi171.stdout:4328 punch from 0x7b8b8c6 to 0x7b968a2, (0xafdc bytes) 2024-09-15T00:12:30.922 INFO:teuthology.orchestra.run.smithi171.stdout:4330 write 0x10bb16 thru 0x116270 (0xa75b bytes) 2024-09-15T00:12:30.922 INFO:teuthology.orchestra.run.smithi171.stdout:4331 clone 36 order 23 su 2097152 sc 15 2024-09-15T00:12:31.020 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:31.019+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:31.063 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:31.063 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:31.064 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:31.064 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:31.072 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:31.156 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:31.154+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:31.219 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:31.219 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:31.219 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:31.220 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:31.220 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:31.222 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:31.220+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:31.229 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:31.323 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:31.322+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:31.365 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-15T00:12:31.379 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:31.379 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:31.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:31.379 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:31.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.379 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:31.379 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:31.379 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.297409200000004, 2024-09-15T00:12:31.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:12:31.379 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.379 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.379 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:31.380 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.380 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.380 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.380 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:12:31.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.380 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:31.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:31.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:12:31.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:12:31.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:12:31.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:12:31.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:12:31.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:12:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:12:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:12:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:12:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:12:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:12:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:12:31.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:31.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:12:31.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:12:31.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:12:31.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:12:31.384 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.384 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.384 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.284668658000001, 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:31.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:12:31.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:12:31.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:12:31.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:12:31.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:12:31.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:12:31.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:12:31.388 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.388 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.388 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.388 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:31.388 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:31.388 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.152312898999995, 2024-09-15T00:12:31.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:12:31.388 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.388 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.389 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:31.389 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.389 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.389 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.389 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:12:31.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.389 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:31.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:12:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:12:31.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:31.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:12:31.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:12:31.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:31.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:12:31.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:12:31.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:31.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:12:31.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:12:31.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:31.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:12:31.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:12:31.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:12:31.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:12:31.393 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.393 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.393 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.393 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.138851254000002, 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:31.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:12:31.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:12:31.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:12:31.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:12:31.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:12:31.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:12:31.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:12:31.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:12:31.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:12:31.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:12:31.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:12:31.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:12:31.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:12:31.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:12:31.398 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.398 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.398 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.398 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:31.398 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:31.398 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.953849771000002, 2024-09-15T00:12:31.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:12:31.398 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.399 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.399 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:31.399 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.399 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.399 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.399 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:12:31.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.399 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:31.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:31.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:12:31.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:12:31.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:12:31.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:12:31.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:12:31.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:12:31.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:12:31.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:12:31.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:12:31.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:12:31.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:12:31.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:12:31.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:31.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:12:31.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:12:31.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:31.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:12:31.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:12:31.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:31.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:12:31.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:12:31.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:31.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:12:31.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:12:31.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:12:31.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:12:31.404 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.404 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.404 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.404 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:12:31.404 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:31.404 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.940973503999999, 2024-09-15T00:12:31.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:12:31.405 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.405 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.405 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:31.405 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:31.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:31.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:12:31.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:12:31.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:12:31.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:12:31.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:12:31.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:12:31.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:12:31.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:12:31.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:12:31.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:12:31.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:12:31.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:12:31.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:12:31.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:12:31.408 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.408 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.408 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.409 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:12:31.409 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:31.409 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 50.482458360000003, 2024-09-15T00:12:31.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:12:31.409 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.409 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.409 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:31.409 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.409 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.409 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.409 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:12:31.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.410 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:31.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:31.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:12:31.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:31.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:12:31.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:12:31.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:12:31.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:12:31.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:12:31.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:12:31.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:12:31.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:12:31.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:12:31.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:12:31.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:31.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:12:31.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:12:31.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:31.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:12:31.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:12:31.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:31.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:12:31.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:12:31.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:31.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:12:31.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:12:31.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:12:31.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:12:31.414 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.414 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.414 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.415 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:31.415 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:31.415 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.764947800000002, 2024-09-15T00:12:31.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:12:31.415 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.415 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.415 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:31.415 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.415 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.415 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.415 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:12:31.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.415 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:31.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:31.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:12:31.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:12:31.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:12:31.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:12:31.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:12:31.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:12:31.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:12:31.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:12:31.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:12:31.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:12:31.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:12:31.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:12:31.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:12:31.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:12:31.419 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.419 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.419 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.419 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:31.419 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:31.419 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.764929191, 2024-09-15T00:12:31.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:12:31.419 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.419 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.419 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:31.420 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.420 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.420 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.420 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:12:31.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.420 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:31.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:31.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:31.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:31.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:12:31.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:12:31.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:12:31.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:12:31.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:12:31.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:12:31.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:12:31.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:12:31.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:12:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:12:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:12:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:12:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:31.423 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:31.424 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.764903867000001, 2024-09-15T00:12:31.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:12:31.424 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.424 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.424 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:31.424 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.424 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.424 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.424 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:12:31.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.424 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:31.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:31.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:31.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:12:31.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:12:31.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:12:31.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:12:31.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:12:31.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:12:31.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:12:31.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:12:31.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:12:31.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:12:31.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:12:31.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:12:31.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:12:31.427 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.428 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.428 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.428 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:31.428 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:31.428 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.764885542999998, 2024-09-15T00:12:31.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:12:31.428 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.428 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.428 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:31.429 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.429 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.429 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.429 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:12:31.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.429 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:31.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:31.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:12:31.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:12:31.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:12:31.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:12:31.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:12:31.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:31.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:12:31.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:12:31.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:12:31.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:12:31.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:12:31.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:12:31.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:12:31.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:12:31.432 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.433 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.433 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.433 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:31.433 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:31.433 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.764702970000002, 2024-09-15T00:12:31.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:12:31.433 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.433 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.433 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:31.433 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.433 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.434 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:12:31.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.434 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:31.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:31.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:31.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:12:31.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:12:31.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:12:31.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:12:31.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:12:31.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:12:31.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:31.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:12:31.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:12:31.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:12:31.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:12:31.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:12:31.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:12:31.437 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.437 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.437 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.438 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:31.438 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:31.438 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.764674128999999, 2024-09-15T00:12:31.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:12:31.438 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.438 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.438 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:31.438 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.438 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.438 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.438 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:12:31.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.439 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:31.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:31.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:31.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:12:31.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:12:31.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:12:31.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:12:31.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:12:31.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:12:31.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:12:31.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:12:31.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:12:31.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:12:31.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:12:31.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:12:31.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:12:31.442 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.442 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.442 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.442 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:31.442 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:31.442 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.764651917999998, 2024-09-15T00:12:31.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:12:31.442 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.443 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.443 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:31.443 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.443 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.443 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.443 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:12:31.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.443 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:31.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:31.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:31.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:12:31.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:12:31.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:12:31.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:12:31.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:12:31.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:12:31.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:31.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:12:31.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:12:31.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:12:31.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:12:31.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:12:31.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:12:31.447 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.447 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.447 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.447 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:31.447 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:31.447 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.764630758999999, 2024-09-15T00:12:31.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:12:31.447 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.447 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.447 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:31.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.448 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:12:31.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.448 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:31.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:31.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:31.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:31.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:12:31.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:12:31.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:12:31.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:12:31.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:12:31.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:12:31.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:12:31.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:12:31.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:12:31.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:12:31.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:12:31.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:12:31.451 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.451 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.451 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.452 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:31.452 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:31.452 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.764600811000001, 2024-09-15T00:12:31.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:12:31.452 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.452 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.452 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:31.452 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.452 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.452 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.452 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:12:31.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.452 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:31.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:31.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:31.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:12:31.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:12:31.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:12:31.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:12:31.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:12:31.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:12:31.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:12:31.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:12:31.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:12:31.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:12:31.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:12:31.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:12:31.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:12:31.456 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.456 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.456 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.456 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:31.456 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:31.456 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.764578847000003, 2024-09-15T00:12:31.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:12:31.456 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.456 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:31.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:31.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:12:31.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:12:31.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:12:31.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:12:31.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:12:31.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:12:31.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:12:31.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:12:31.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:12:31.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:12:31.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:12:31.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:12:31.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:12:31.460 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.460 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.460 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.461 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:31.461 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:31.461 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.76456142, 2024-09-15T00:12:31.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:12:31.461 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.461 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.461 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:31.461 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.461 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.461 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.461 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:12:31.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.462 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:31.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:31.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:12:31.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:12:31.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:12:31.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:12:31.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:12:31.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:31.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:12:31.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:12:31.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:12:31.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:12:31.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:12:31.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:12:31.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:12:31.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:12:31.465 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.465 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.465 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.465 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:31.465 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:31.466 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.764539620999997, 2024-09-15T00:12:31.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:12:31.466 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.466 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.466 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:31.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.466 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:12:31.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.466 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:31.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:31.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:12:31.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:12:31.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:12:31.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:12:31.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:12:31.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:12:31.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:12:31.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:12:31.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:12:31.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:12:31.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:12:31.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:12:31.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:12:31.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:12:31.469 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.469 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.470 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.470 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:31.470 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:31.470 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.715311262, 2024-09-15T00:12:31.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:12:31.470 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:31.470 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:31.470 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:31.470 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:31.470 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:31.470 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:31.470 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:12:31.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:12:31.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:31.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:12:31.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:12:31.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:31.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:12:31.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:12:31.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:12:31.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:12:31.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:31.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:12:31.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:12:31.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:31.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:12:31.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:12:31.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:31.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:12:31.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:12:31.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:31.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:12:31.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:12:31.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:31.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:12:31.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:12:31.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:31.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:12:31.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:12:31.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:31.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:31.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:31.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:12:31.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:12:31.475 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.475 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.475 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.475 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:31.475 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:31.475 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:31.477 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:31.475+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:31.478 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:31.478 INFO:teuthology.orchestra.run.smithi171.stdout:4332 write 0x12cd3d3 thru 0x12d3949 (0x6577 bytes) 2024-09-15T00:12:31.478 INFO:teuthology.orchestra.run.smithi171.stdout:4334 read 0x867d3db thru 0x8688bda (0xb800 bytes) 2024-09-15T00:12:31.478 INFO:teuthology.orchestra.run.smithi171.stdout:4335 writesame 0xb48d3ba thru 0xb4996b3 data_size 0x3b(0xc2fa bytes) 2024-09-15T00:12:31.478 INFO:teuthology.orchestra.run.smithi171.stdout:4336 write 0x244dde4 thru 0x245644c (0x8669 bytes) 2024-09-15T00:12:31.478 INFO:teuthology.orchestra.run.smithi171.stdout:4338 read 0x2950d8f thru 0x2956779 (0x59eb bytes) 2024-09-15T00:12:31.479 INFO:teuthology.orchestra.run.smithi171.stdout:4339 read 0x1731782 thru 0x173f4ed (0xdd6c bytes) 2024-09-15T00:12:31.479 INFO:teuthology.orchestra.run.smithi171.stdout:4343 writesame 0xdcabfa5 thru 0xdcaca07 data_size 0x1(0xa63 bytes) 2024-09-15T00:12:31.479 INFO:teuthology.orchestra.run.smithi171.stdout:4345 trunc from 0xdcaca08 to 0x9722b0 2024-09-15T00:12:31.487 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 0] out_osds: [2, 7] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-09-15T00:12:31.487 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:12:31.487 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-15T00:12:31.488 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T00:12:31.488 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T00:12:31.488 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-15T00:12:31.546 INFO:teuthology.orchestra.run.smithi171.stdout:4347 write 0xc8535f8 thru 0xc8588d2 (0x52db bytes) 2024-09-15T00:12:31.549 INFO:teuthology.orchestra.run.smithi171.stdout:4348 write 0x7f1055a thru 0x7f1a8cf (0xa376 bytes) 2024-09-15T00:12:31.555 INFO:teuthology.orchestra.run.smithi171.stdout:4349 trunc from 0xc8588d3 to 0xdddff2a 2024-09-15T00:12:31.562 INFO:teuthology.orchestra.run.smithi171.stdout:4352 read 0x3c5e588 thru 0x3c60ea8 (0x2921 bytes) 2024-09-15T00:12:31.569 INFO:teuthology.orchestra.run.smithi171.stdout:4353 writesame 0xee3671b thru 0xee42319 data_size 0x13(0xbbff bytes) 2024-09-15T00:12:31.569 INFO:teuthology.orchestra.run.smithi171.stdout:4356 read 0x862a314 thru 0x862e468 (0x4155 bytes) 2024-09-15T00:12:31.577 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:31.576+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:31.582 INFO:teuthology.orchestra.run.smithi171.stdout:4357 trunc from 0xee4231a to 0x3d1e3c5 2024-09-15T00:12:31.597 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:31.598 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:31.598 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:31.598 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:31.607 INFO:teuthology.orchestra.run.smithi171.stdout:4362 trunc from 0x3d1e3c5 to 0xed25df6 2024-09-15T00:12:31.612 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:31.623 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:31.623 INFO:teuthology.orchestra.run.smithi184.stdout: "success": "" 2024-09-15T00:12:31.624 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:31.627 INFO:teuthology.orchestra.run.smithi171.stdout:4363 trunc from 0xed25df6 to 0x3ad7ce8 2024-09-15T00:12:31.649 INFO:teuthology.orchestra.run.smithi171.stdout:4364 writesame 0xb38d6df thru 0xb39b514 data_size 0x39(0xde36 bytes) 2024-09-15T00:12:31.649 INFO:teuthology.orchestra.run.smithi171.stdout:4365 write 0xa456b62 thru 0xa45d739 (0x6bd8 bytes) 2024-09-15T00:12:31.653 INFO:teuthology.orchestra.run.smithi171.stdout:4367 read 0x4b4c34e thru 0x4b51034 (0x4ce7 bytes) 2024-09-15T00:12:31.653 INFO:teuthology.orchestra.run.smithi171.stdout:4368 punch from 0xa1ec19 to 0xa1eeec, (0x2d3 bytes) 2024-09-15T00:12:31.653 INFO:teuthology.orchestra.run.smithi171.stdout:4369 write 0x35ae2ee thru 0x35b0018 (0x1d2b bytes) 2024-09-15T00:12:31.666 INFO:teuthology.orchestra.run.smithi171.stdout:4371 writesame 0xe6f5a8a thru 0xe701032 data_size 0x47(0xb5a9 bytes) 2024-09-15T00:12:31.666 INFO:teuthology.orchestra.run.smithi171.stdout:4372 punch from 0x849b36a to 0x84a2f1b, (0x7bb1 bytes) 2024-09-15T00:12:31.666 INFO:teuthology.orchestra.run.smithi171.stdout:4373 writesame 0x2c93b87 thru 0x2c944d0 data_size 0x1d(0x94a bytes) 2024-09-15T00:12:31.669 INFO:teuthology.orchestra.run.smithi171.stdout:4374 punch from 0xd6a883 to 0xd779ac, (0xd129 bytes) 2024-09-15T00:12:31.671 INFO:teuthology.orchestra.run.smithi171.stdout:4375 trunc from 0xe701033 to 0xcba309b 2024-09-15T00:12:31.678 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:31.676+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:31.678 INFO:teuthology.orchestra.run.smithi171.stdout:4376 writesame 0x260bfac thru 0x2613db5 data_size 0x49(0x7e0a bytes) 2024-09-15T00:12:31.680 INFO:teuthology.orchestra.run.smithi171.stdout:4377 write 0x708f320 thru 0x709c0dd (0xcdbe bytes) 2024-09-15T00:12:31.684 INFO:teuthology.orchestra.run.smithi171.stdout:4378 read 0xc42a852 thru 0xc438fc0 (0xe76f bytes) 2024-09-15T00:12:31.685 INFO:teuthology.orchestra.run.smithi171.stdout:4380 write 0x87b9ec8 thru 0x87c9129 (0xf262 bytes) 2024-09-15T00:12:31.691 INFO:teuthology.orchestra.run.smithi171.stdout:4381 writesame 0x9253f6f thru 0x925bc11 data_size 0x1(0x7ca3 bytes) 2024-09-15T00:12:31.743 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:31.744 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:31.750 INFO:teuthology.orchestra.run.smithi171.stdout:4383 writesame 0x4f2b17a thru 0x4f311cd data_size 0x24(0x6054 bytes) 2024-09-15T00:12:31.754 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:31.754 INFO:teuthology.orchestra.run.smithi171.stdout:4385 read 0x6c98689 thru 0x6c99982 (0x12fa bytes) 2024-09-15T00:12:31.754 INFO:teuthology.orchestra.run.smithi171.stdout:4386 writesame 0x1838305 thru 0x183fc3f data_size 0x3(0x793b bytes) 2024-09-15T00:12:31.774 INFO:teuthology.orchestra.run.smithi171.stdout:4387 write 0x89894e1 thru 0x89947a6 (0xb2c6 bytes) 2024-09-15T00:12:31.778 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:31.777+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:31.780 INFO:teuthology.orchestra.run.smithi171.stdout:4388 clone 37 order 20 su 8192 sc 6 2024-09-15T00:12:31.859 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:31.859 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:31.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:31.859 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:31.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.859 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-15T00:12:31.860 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:31.860 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 85.331939917, 2024-09-15T00:12:31.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077021529999999998, 2024-09-15T00:12:31.860 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.860 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.860 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:31.860 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:31.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524698+0000", 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6300000000000001e-07 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524778+0000", 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0598000000000003e-05 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524785+0000", 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6359999999999997e-06 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524788+0000", 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2540000000000001e-06 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524817+0000", 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8784000000000001e-05 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524828+0000", 2024-09-15T00:12:31.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-15T00:12:31.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:31.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532360+0000", 2024-09-15T00:12:31.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007531757 2024-09-15T00:12:31.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532400+0000", 2024-09-15T00:12:31.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9835000000000001e-05 2024-09-15T00:12:31.863 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.863 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.863 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.864 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=16'106)", 2024-09-15T00:12:31.864 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:31.864 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 85.330997764000003, 2024-09-15T00:12:31.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067923050000000002, 2024-09-15T00:12:31.864 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.864 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.864 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:31.864 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:31.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:31.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:31.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-15T00:12:31.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525644+0000", 2024-09-15T00:12:31.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.709e-06 2024-09-15T00:12:31.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525651+0000", 2024-09-15T00:12:31.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7970000000000003e-06 2024-09-15T00:12:31.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525654+0000", 2024-09-15T00:12:31.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.433e-06 2024-09-15T00:12:31.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525685+0000", 2024-09-15T00:12:31.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0970000000000003e-05 2024-09-15T00:12:31.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525697+0000", 2024-09-15T00:12:31.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1759e-05 2024-09-15T00:12:31.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:31.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532401+0000", 2024-09-15T00:12:31.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006704425 2024-09-15T00:12:31.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532432+0000", 2024-09-15T00:12:31.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0709e-05 2024-09-15T00:12:31.868 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.868 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.868 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.868 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'106)", 2024-09-15T00:12:31.868 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:31.868 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 85.330535510000004, 2024-09-15T00:12:31.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063539190000000004, 2024-09-15T00:12:31.868 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.869 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.869 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:31.869 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:31.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:31.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:31.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:12:31.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526107+0000", 2024-09-15T00:12:31.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:12:31.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526117+0000", 2024-09-15T00:12:31.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7750000000000004e-06 2024-09-15T00:12:31.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526121+0000", 2024-09-15T00:12:31.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:12:31.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526163+0000", 2024-09-15T00:12:31.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1360999999999999e-05 2024-09-15T00:12:31.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526175+0000", 2024-09-15T00:12:31.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2729e-05 2024-09-15T00:12:31.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:31.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532433+0000", 2024-09-15T00:12:31.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062576999999999997 2024-09-15T00:12:31.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532456+0000", 2024-09-15T00:12:31.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2773e-05 2024-09-15T00:12:31.872 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.872 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.872 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.873 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:31.873 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.504060+0000", 2024-09-15T00:12:31.873 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.352577071999999, 2024-09-15T00:12:31.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060970119999999997, 2024-09-15T00:12:31.873 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.873 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.873 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:31.873 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.504060+0000", 2024-09-15T00:12:31.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.504060+0000", 2024-09-15T00:12:31.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.504061+0000", 2024-09-15T00:12:31.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3200000000000005e-07 2024-09-15T00:12:31.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505702+0000", 2024-09-15T00:12:31.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016412149999999999 2024-09-15T00:12:31.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505724+0000", 2024-09-15T00:12:31.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1900999999999999e-05 2024-09-15T00:12:31.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505743+0000", 2024-09-15T00:12:31.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8529000000000002e-05 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505840+0000", 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7533000000000003e-05 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505854+0000", 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4070000000000001e-05 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.510157+0000", 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043031320000000003 2024-09-15T00:12:31.876 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.877 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.877 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.877 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [2097152~420557,2629632~4096,2650112~4096,2670592~4096,2691072~4096,2711552~4096,2732032~4096,2756608~12288,2777088~12288,2797568~12288,2818048~12288,2838528~12288,2859008~12288,2879488~529715], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3409203, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:31.877 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:31.877 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.343051033999998, 2024-09-15T00:12:31.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0071155660000000003, 2024-09-15T00:12:31.877 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.877 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.877 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:31.877 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:31.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:31.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513587+0000", 2024-09-15T00:12:31.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4899999999999995e-07 2024-09-15T00:12:31.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514349+0000", 2024-09-15T00:12:31.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00076184799999999997 2024-09-15T00:12:31.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514368+0000", 2024-09-15T00:12:31.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9507000000000001e-05 2024-09-15T00:12:31.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514391+0000", 2024-09-15T00:12:31.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2497000000000002e-05 2024-09-15T00:12:31.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514491+0000", 2024-09-15T00:12:31.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9838000000000002e-05 2024-09-15T00:12:31.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514500+0000", 2024-09-15T00:12:31.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4860000000000003e-06 2024-09-15T00:12:31.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.520702+0000", 2024-09-15T00:12:31.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062017410000000002 2024-09-15T00:12:31.880 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.880 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [3409203~654029,4194304~266240,4476928~4096,4497408~4096,4517888~4096,4538368~4096,4558848~4096,4579328~4096,4587520~103731], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4691251, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3409203, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.332192957, 2024-09-15T00:12:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079084649999999999, 2024-09-15T00:12:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:31.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:31.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524446+0000", 2024-09-15T00:12:31.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.243e-06 2024-09-15T00:12:31.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525115+0000", 2024-09-15T00:12:31.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00066896999999999996 2024-09-15T00:12:31.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525135+0000", 2024-09-15T00:12:31.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0137999999999999e-05 2024-09-15T00:12:31.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525148+0000", 2024-09-15T00:12:31.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3591000000000001e-05 2024-09-15T00:12:31.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525242+0000", 2024-09-15T00:12:31.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3603999999999994e-05 2024-09-15T00:12:31.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525258+0000", 2024-09-15T00:12:31.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6150999999999998e-05 2024-09-15T00:12:31.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.532353+0000", 2024-09-15T00:12:31.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0070947680000000004 2024-09-15T00:12:31.884 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.884 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.885 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.885 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [4691251~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5739827, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4691251, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:31.885 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:31.885 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.318509396, 2024-09-15T00:12:31.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0085554080000000005, 2024-09-15T00:12:31.885 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.885 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.885 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:31.885 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:31.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:31.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538129+0000", 2024-09-15T00:12:31.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:12:31.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539029+0000", 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090036599999999999 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539047+0000", 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7776999999999999e-05 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539058+0000", 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1070000000000001e-05 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539128+0000", 2024-09-15T00:12:31.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0531999999999995e-05 2024-09-15T00:12:31.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539138+0000", 2024-09-15T00:12:31.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3910000000000004e-06 2024-09-15T00:12:31.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.546683+0000", 2024-09-15T00:12:31.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075455519999999996 2024-09-15T00:12:31.888 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.888 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.888 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.889 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:6618a9b3:::rbd_data.113947ee3a89.0000000000000009:15, version: 59'176, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:6618a9b3:::rbd_data.113947ee3a89.0000000000000009:15@59'176, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:31.889 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:31.889 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.2491916, 2024-09-15T00:12:31.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073968319999999999, 2024-09-15T00:12:31.889 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.889 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.889 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:31.889 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:31.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:31.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607447+0000", 2024-09-15T00:12:31.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2100000000000001e-06 2024-09-15T00:12:31.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607778+0000", 2024-09-15T00:12:31.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033101199999999999 2024-09-15T00:12:31.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607800+0000", 2024-09-15T00:12:31.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2323e-05 2024-09-15T00:12:31.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607815+0000", 2024-09-15T00:12:31.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5073e-05 2024-09-15T00:12:31.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607926+0000", 2024-09-15T00:12:31.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110538 2024-09-15T00:12:31.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607942+0000", 2024-09-15T00:12:31.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5662999999999999e-05 2024-09-15T00:12:31.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.614843+0000", 2024-09-15T00:12:31.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0069010130000000001 2024-09-15T00:12:31.892 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.892 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.892 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.893 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:62885eaf:::rbd_data.1139e6d65aff.0000000000000005:head, version: 62'226, data_included: [0~262144,393216~131072,655360~655360], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:62885eaf:::rbd_data.1139e6d65aff.0000000000000005:head@62'226, size: 1572864, copy_subset: [(0, 1572864)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1310720, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:31.893 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.639003+0000", 2024-09-15T00:12:31.893 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.217634038, 2024-09-15T00:12:31.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0070761069999999999, 2024-09-15T00:12:31.893 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.893 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.893 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:31.893 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639003+0000", 2024-09-15T00:12:31.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639003+0000", 2024-09-15T00:12:31.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639004+0000", 2024-09-15T00:12:31.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:12:31.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639567+0000", 2024-09-15T00:12:31.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00056333800000000003 2024-09-15T00:12:31.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639583+0000", 2024-09-15T00:12:31.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5906999999999999e-05 2024-09-15T00:12:31.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639594+0000", 2024-09-15T00:12:31.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1093e-05 2024-09-15T00:12:31.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639667+0000", 2024-09-15T00:12:31.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3089000000000004e-05 2024-09-15T00:12:31.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639680+0000", 2024-09-15T00:12:31.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2689e-05 2024-09-15T00:12:31.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.646079+0000", 2024-09-15T00:12:31.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063994990000000003 2024-09-15T00:12:31.896 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.896 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.897 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.897 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1 66/65 [PushOp(3:8182271c:::rbd_data.1139e2e18583.000000000000000a:9, version: 37'34, data_included: [1048576~499712,1744896~180224], data_size: 679936, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:8182271c:::rbd_data.1139e2e18583.000000000000000a:9@37'34, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1925120, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:31.897 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.698673+0000", 2024-09-15T00:12:31.897 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.157964571000001, 2024-09-15T00:12:31.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060406629999999999, 2024-09-15T00:12:31.897 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.897 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.897 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:31.897 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.698673+0000", 2024-09-15T00:12:31.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.698673+0000", 2024-09-15T00:12:31.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.698674+0000", 2024-09-15T00:12:31.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5600000000000004e-07 2024-09-15T00:12:31.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699307+0000", 2024-09-15T00:12:31.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00063296200000000004 2024-09-15T00:12:31.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699324+0000", 2024-09-15T00:12:31.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7115000000000001e-05 2024-09-15T00:12:31.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699336+0000", 2024-09-15T00:12:31.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2047e-05 2024-09-15T00:12:31.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699428+0000", 2024-09-15T00:12:31.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2305000000000005e-05 2024-09-15T00:12:31.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699443+0000", 2024-09-15T00:12:31.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4386000000000001e-05 2024-09-15T00:12:31.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.704713+0000", 2024-09-15T00:12:31.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052708920000000001 2024-09-15T00:12:31.901 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.901 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.901 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.901 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1 66/65 [PushOp(3:83414961:::rbd_data.11392b39946e.0000000000000005:18, version: 62'186, data_included: [0~655360,5144177~35006], data_size: 690366, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:83414961:::rbd_data.11392b39946e.0000000000000005:18@62'186, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:31.901 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.781708+0000", 2024-09-15T00:12:31.901 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.074929078, 2024-09-15T00:12:31.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0064762520000000001, 2024-09-15T00:12:31.901 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.901 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.901 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:31.901 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.781708+0000", 2024-09-15T00:12:31.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.781708+0000", 2024-09-15T00:12:31.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.781709+0000", 2024-09-15T00:12:31.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7000000000000004e-07 2024-09-15T00:12:31.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782362+0000", 2024-09-15T00:12:31.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00065286900000000004 2024-09-15T00:12:31.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782394+0000", 2024-09-15T00:12:31.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1767999999999998e-05 2024-09-15T00:12:31.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782416+0000", 2024-09-15T00:12:31.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2833e-05 2024-09-15T00:12:31.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782497+0000", 2024-09-15T00:12:31.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0706999999999998e-05 2024-09-15T00:12:31.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782507+0000", 2024-09-15T00:12:31.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3540000000000006e-06 2024-09-15T00:12:31.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.788185+0000", 2024-09-15T00:12:31.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056779509999999997 2024-09-15T00:12:31.905 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.905 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.905 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.905 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1 66/65 [PushOp(3:84afde92:::rbd_data.11392b39946e.0000000000000007:18, version: 62'188, data_included: [0~786432,4718224~368,6422528~38621], data_size: 825421, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:84afde92:::rbd_data.11392b39946e.0000000000000007:18@62'188, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:31.905 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.791890+0000", 2024-09-15T00:12:31.905 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.064747014999998, 2024-09-15T00:12:31.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006572825, 2024-09-15T00:12:31.905 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.905 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.906 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:31.906 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.791890+0000", 2024-09-15T00:12:31.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.791890+0000", 2024-09-15T00:12:31.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.791892+0000", 2024-09-15T00:12:31.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9630000000000001e-06 2024-09-15T00:12:31.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792684+0000", 2024-09-15T00:12:31.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00079155399999999998 2024-09-15T00:12:31.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792736+0000", 2024-09-15T00:12:31.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2490000000000001e-05 2024-09-15T00:12:31.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792774+0000", 2024-09-15T00:12:31.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7358999999999999e-05 2024-09-15T00:12:31.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792889+0000", 2024-09-15T00:12:31.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114987 2024-09-15T00:12:31.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792902+0000", 2024-09-15T00:12:31.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3684e-05 2024-09-15T00:12:31.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.798463+0000", 2024-09-15T00:12:31.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055607879999999997 2024-09-15T00:12:31.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.909 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.909 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.9 77/76 [PushOp(3:9229c9d9:::rbd_data.1139766160da.0000000000000000:13, version: 57'689, data_included: [0~65536,229376~65536,385024~5116,458752~32768,589824~131072,851968~131072,1081344~32768,1146880~65536,1310720~32768,1409024~32768,1507328~49152,1572864~32768,1638400~32768,1802240~32768], data_size: 742396, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:9229c9d9:::rbd_data.1139766160da.0000000000000000:13@57'689, size: 1835008, copy_subset: [(0, 1835008)], clone_subset: {}, snapset: 13=[]:{13: [13]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1835008, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:31.909 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.386155+0000", 2024-09-15T00:12:31.909 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.470482789, 2024-09-15T00:12:31.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062156169999999997, 2024-09-15T00:12:31.910 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.910 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.910 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:31.910 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386155+0000", 2024-09-15T00:12:31.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386155+0000", 2024-09-15T00:12:31.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386155+0000", 2024-09-15T00:12:31.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0999999999999997e-07 2024-09-15T00:12:31.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386629+0000", 2024-09-15T00:12:31.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047386200000000002 2024-09-15T00:12:31.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386660+0000", 2024-09-15T00:12:31.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0834999999999999e-05 2024-09-15T00:12:31.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386686+0000", 2024-09-15T00:12:31.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5488999999999999e-05 2024-09-15T00:12:31.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386804+0000", 2024-09-15T00:12:31.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011882700000000001 2024-09-15T00:12:31.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386820+0000", 2024-09-15T00:12:31.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5330000000000001e-05 2024-09-15T00:12:31.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.392370+0000", 2024-09-15T00:12:31.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055503640000000003 2024-09-15T00:12:31.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.913 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.913 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.9 77/76 [PushOp(3:95907793:::rbd_data.11396cb1f0e4.0000000000000007:head, version: 59'729, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:95907793:::rbd_data.11396cb1f0e4.0000000000000007:head@59'729, size: 5701632, copy_subset: [(0, 5701632)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:31.914 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.428602+0000", 2024-09-15T00:12:31.914 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.428035770999999, 2024-09-15T00:12:31.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006605606, 2024-09-15T00:12:31.914 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.914 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.914 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:31.914 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.428602+0000", 2024-09-15T00:12:31.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.428602+0000", 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.428603+0000", 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1000000000000001e-06 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.429252+0000", 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064978799999999999 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.429282+0000", 2024-09-15T00:12:31.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9352e-05 2024-09-15T00:12:31.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.429301+0000", 2024-09-15T00:12:31.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9436000000000002e-05 2024-09-15T00:12:31.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.429400+0000", 2024-09-15T00:12:31.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8862000000000003e-05 2024-09-15T00:12:31.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.429415+0000", 2024-09-15T00:12:31.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5145e-05 2024-09-15T00:12:31.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.435207+0000", 2024-09-15T00:12:31.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005791923 2024-09-15T00:12:31.917 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.917 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.917 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.917 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:125615 3.f e77/68 3:f382b49c:::rbd_data.11396481e1ec.0000000000000004:head v 77'241, mlcod=74'239)", 2024-09-15T00:12:31.917 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.839171+0000", 2024-09-15T00:12:31.918 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.017466763, 2024-09-15T00:12:31.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0072384550000000004, 2024-09-15T00:12:31.918 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.918 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.918 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:31.918 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.839171+0000", 2024-09-15T00:12:31.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.839171+0000", 2024-09-15T00:12:31.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.839172+0000", 2024-09-15T00:12:31.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3769999999999999e-06 2024-09-15T00:12:31.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840892+0000", 2024-09-15T00:12:31.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001720271 2024-09-15T00:12:31.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840905+0000", 2024-09-15T00:12:31.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2461e-05 2024-09-15T00:12:31.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840913+0000", 2024-09-15T00:12:31.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1990000000000006e-06 2024-09-15T00:12:31.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840962+0000", 2024-09-15T00:12:31.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8760000000000001e-05 2024-09-15T00:12:31.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840979+0000", 2024-09-15T00:12:31.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7280999999999999e-05 2024-09-15T00:12:31.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:31.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.846347+0000", 2024-09-15T00:12:31.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005367925 2024-09-15T00:12:31.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.846409+0000", 2024-09-15T00:12:31.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2181000000000007e-05 2024-09-15T00:12:31.921 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.921 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.922 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.922 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:125618 3.10 e77/68 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head v 77'459, mlcod=74'457)", 2024-09-15T00:12:31.922 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.844048+0000", 2024-09-15T00:12:31.922 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.012589486, 2024-09-15T00:12:31.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0064617310000000001, 2024-09-15T00:12:31.922 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.922 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.922 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:31.922 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.844048+0000", 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.844048+0000", 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.844048+0000", 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.34e-07 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.845223+0000", 2024-09-15T00:12:31.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011745569999999999 2024-09-15T00:12:31.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.845235+0000", 2024-09-15T00:12:31.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2602999999999999e-05 2024-09-15T00:12:31.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.845243+0000", 2024-09-15T00:12:31.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9990000000000008e-06 2024-09-15T00:12:31.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.845315+0000", 2024-09-15T00:12:31.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2059999999999998e-05 2024-09-15T00:12:31.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.845341+0000", 2024-09-15T00:12:31.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5933999999999999e-05 2024-09-15T00:12:31.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:31.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.850456+0000", 2024-09-15T00:12:31.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051145269999999998 2024-09-15T00:12:31.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.850510+0000", 2024-09-15T00:12:31.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3616999999999998e-05 2024-09-15T00:12:31.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.926 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.926 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:127612 3.1b e78/69 3:d99663fc:::rbd_data.11396481e1ec.0000000000000000:head v 78'323, mlcod=73'321)", 2024-09-15T00:12:31.926 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.120437+0000", 2024-09-15T00:12:31.926 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.7361999669999992, 2024-09-15T00:12:31.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0064173049999999999, 2024-09-15T00:12:31.926 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.926 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.926 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:31.926 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.120437+0000", 2024-09-15T00:12:31.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.120437+0000", 2024-09-15T00:12:31.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.120438+0000", 2024-09-15T00:12:31.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-07 2024-09-15T00:12:31.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.121533+0000", 2024-09-15T00:12:31.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001095897 2024-09-15T00:12:31.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.121548+0000", 2024-09-15T00:12:31.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4888e-05 2024-09-15T00:12:31.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.121558+0000", 2024-09-15T00:12:31.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3260000000000002e-06 2024-09-15T00:12:31.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.121608+0000", 2024-09-15T00:12:31.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9948999999999999e-05 2024-09-15T00:12:31.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.121622+0000", 2024-09-15T00:12:31.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4898e-05 2024-09-15T00:12:31.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.126807+0000", 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005184676 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.126855+0000", 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497999999999998e-05 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:127618 3.1 e78/67 3:84066653:::rbd_data.11396481e1ec.0000000000000003:head v 78'284, mlcod=77'282)", 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.124454+0000", 2024-09-15T00:12:31.930 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.7321836360000002, 2024-09-15T00:12:31.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0064568020000000002, 2024-09-15T00:12:31.931 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.931 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.931 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:31.931 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.124454+0000", 2024-09-15T00:12:31.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.124454+0000", 2024-09-15T00:12:31.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.124455+0000", 2024-09-15T00:12:31.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2139999999999999e-06 2024-09-15T00:12:31.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.125404+0000", 2024-09-15T00:12:31.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00094884799999999997 2024-09-15T00:12:31.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.125422+0000", 2024-09-15T00:12:31.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7829000000000002e-05 2024-09-15T00:12:31.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.125431+0000", 2024-09-15T00:12:31.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5670000000000001e-06 2024-09-15T00:12:31.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.125497+0000", 2024-09-15T00:12:31.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5419999999999994e-05 2024-09-15T00:12:31.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.125519+0000", 2024-09-15T00:12:31.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2297e-05 2024-09-15T00:12:31.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:31.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.130879+0000", 2024-09-15T00:12:31.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053604350000000002 2024-09-15T00:12:31.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.130910+0000", 2024-09-15T00:12:31.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1192000000000002e-05 2024-09-15T00:12:31.934 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.934 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.934 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.935 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133834 3.1b e85/69 3:da9a78a3:::rbd_data.11395dfcf2b8.0000000000000000:head v 85'356, mlcod=85'354)", 2024-09-15T00:12:31.935 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:31.935 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.2933010300000001, 2024-09-15T00:12:31.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026079054000000001, 2024-09-15T00:12:31.935 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.935 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.935 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:31.935 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:31.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563338+0000", 2024-09-15T00:12:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.418e-06 2024-09-15T00:12:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569385+0000", 2024-09-15T00:12:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006047335 2024-09-15T00:12:31.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569418+0000", 2024-09-15T00:12:31.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2957999999999999e-05 2024-09-15T00:12:31.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569432+0000", 2024-09-15T00:12:31.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3718e-05 2024-09-15T00:12:31.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569519+0000", 2024-09-15T00:12:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7022999999999997e-05 2024-09-15T00:12:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569540+0000", 2024-09-15T00:12:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1426000000000001e-05 2024-09-15T00:12:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589362+0000", 2024-09-15T00:12:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019821398 2024-09-15T00:12:31.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589415+0000", 2024-09-15T00:12:31.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3777999999999999e-05 2024-09-15T00:12:31.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.939 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.939 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133836 3.1 e85/67 3:849f9a98:::rbd_data.11395dfcf2b8.0000000000000002:head v 85'304, mlcod=84'301)", 2024-09-15T00:12:31.939 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:31.939 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.28560612, 2024-09-15T00:12:31.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023231600000000002, 2024-09-15T00:12:31.939 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:31.939 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:31.940 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:31.940 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:31.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:31.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:31.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:31.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:31.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:31.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:31.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571032+0000", 2024-09-15T00:12:31.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-15T00:12:31.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:31.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576635+0000", 2024-09-15T00:12:31.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056028700000000002 2024-09-15T00:12:31.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:31.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576650+0000", 2024-09-15T00:12:31.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.505e-05 2024-09-15T00:12:31.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:31.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576658+0000", 2024-09-15T00:12:31.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6879999999999999e-06 2024-09-15T00:12:31.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:31.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576717+0000", 2024-09-15T00:12:31.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175000000000002e-05 2024-09-15T00:12:31.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:31.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576742+0000", 2024-09-15T00:12:31.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4734000000000001e-05 2024-09-15T00:12:31.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:31.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594234+0000", 2024-09-15T00:12:31.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017492527000000001 2024-09-15T00:12:31.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:31.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:31.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:31.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594263+0000", 2024-09-15T00:12:31.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8629e-05 2024-09-15T00:12:31.943 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.943 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.943 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.943 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:31.943 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:31.944 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:31.945 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:31.945 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:31.943+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:32.045 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:32.044+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:32.088 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:32.089 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:32.089 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:32.089 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:32.097 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:32.147 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:32.145+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:32.173 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:12:32.207 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:32.207 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:32.207 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:32.207 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:32.207 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:32.217 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:32.232 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone36 from 0xcba309b (overlap 0x9722b0) to 0x88059e 2024-09-15T00:12:32.275 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:32.274+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:32.314 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-15T00:12:32.349 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:32.347+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:32.355 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:32.355 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:32.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:32.356 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:32.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.356 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:32.356 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:32.356 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 66.172306433000003, 2024-09-15T00:12:32.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:12:32.356 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.356 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.356 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:32.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.357 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:12:32.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.357 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:32.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:32.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:12:32.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:12:32.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:12:32.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:12:32.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:12:32.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:12:32.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:12:32.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:12:32.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:12:32.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:12:32.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:32.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:12:32.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:12:32.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:32.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:12:32.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:12:32.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:32.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:12:32.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:12:32.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:32.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:12:32.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:12:32.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:32.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:12:32.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:12:32.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:12:32.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:12:32.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.362 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.363 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:32.363 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:32.363 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.442465253999998, 2024-09-15T00:12:32.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:12:32.363 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.363 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.363 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:32.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.363 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:12:32.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.364 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:32.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:32.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:12:32.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:12:32.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:12:32.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:12:32.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:12:32.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:12:32.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:12:32.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:12:32.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:12:32.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:12:32.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:32.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:12:32.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:12:32.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:32.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:12:32.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:12:32.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:32.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:12:32.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:12:32.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:32.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:12:32.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:12:32.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:32.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:12:32.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:12:32.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:12:32.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:12:32.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.369 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.369 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:12:32.369 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:32.369 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.435506891000003, 2024-09-15T00:12:32.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:12:32.370 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.370 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.370 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:32.370 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:32.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:32.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:12:32.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:12:32.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:12:32.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:12:32.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:12:32.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:12:32.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:12:32.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:12:32.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:12:32.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:12:32.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:32.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:12:32.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:12:32.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:12:32.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:12:32.373 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.373 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.374 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:32.374 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:32.374 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.688318054, 2024-09-15T00:12:32.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:12:32.374 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.374 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.374 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:32.374 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.375 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:12:32.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.375 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:32.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:32.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:12:32.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:12:32.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:12:32.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:12:32.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:12:32.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:12:32.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:12:32.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:12:32.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:12:32.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:12:32.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:32.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:12:32.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:12:32.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:32.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:12:32.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:12:32.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:32.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:12:32.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:12:32.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:32.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:12:32.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:12:32.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:32.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:12:32.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:12:32.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:12:32.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:12:32.386 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.386 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.386 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.387 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97493 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:32.387 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:32.387 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.498735333999999, 2024-09-15T00:12:32.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033259141999999998, 2024-09-15T00:12:32.388 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.388 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.388 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:32.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.390 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97493 2024-09-15T00:12:32.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.390 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:32.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:32.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:32.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-15T00:12:32.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853401+0000", 2024-09-15T00:12:32.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:12:32.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853407+0000", 2024-09-15T00:12:32.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-15T00:12:32.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853411+0000", 2024-09-15T00:12:32.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.481e-06 2024-09-15T00:12:32.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886544+0000", 2024-09-15T00:12:32.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033132861999999999 2024-09-15T00:12:32.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:32.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886587+0000", 2024-09-15T00:12:32.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2942000000000003e-05 2024-09-15T00:12:32.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886655+0000", 2024-09-15T00:12:32.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7711000000000003e-05 2024-09-15T00:12:32.399 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.399 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.399 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.399 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97494 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:32.400 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:32.400 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.498714210999999, 2024-09-15T00:12:32.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033392390000000001, 2024-09-15T00:12:32.400 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.400 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.401 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:32.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.401 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97494 2024-09-15T00:12:32.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.402 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:32.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:32.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:32.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-15T00:12:32.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853419+0000", 2024-09-15T00:12:32.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.917e-06 2024-09-15T00:12:32.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853425+0000", 2024-09-15T00:12:32.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:12:32.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853433+0000", 2024-09-15T00:12:32.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8120000000000008e-06 2024-09-15T00:12:32.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886680+0000", 2024-09-15T00:12:32.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033247207000000001 2024-09-15T00:12:32.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:32.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886726+0000", 2024-09-15T00:12:32.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5902000000000002e-05 2024-09-15T00:12:32.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886809+0000", 2024-09-15T00:12:32.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3475999999999998e-05 2024-09-15T00:12:32.407 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.407 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.407 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.407 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97495 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:32.407 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:32.408 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.498693257999999, 2024-09-15T00:12:32.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033640503000000002, 2024-09-15T00:12:32.408 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.408 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.408 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:32.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.408 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97495 2024-09-15T00:12:32.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.408 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:32.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:32.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:32.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:12:32.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853440+0000", 2024-09-15T00:12:32.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.804e-06 2024-09-15T00:12:32.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853446+0000", 2024-09-15T00:12:32.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5559999999999998e-06 2024-09-15T00:12:32.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853450+0000", 2024-09-15T00:12:32.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9519999999999996e-06 2024-09-15T00:12:32.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886904+0000", 2024-09-15T00:12:32.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033454207999999999 2024-09-15T00:12:32.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:32.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887007+0000", 2024-09-15T00:12:32.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102904 2024-09-15T00:12:32.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887078+0000", 2024-09-15T00:12:32.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1719999999999998e-05 2024-09-15T00:12:32.412 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.412 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.412 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.412 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97496 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:32.412 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:32.412 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.498676159999999, 2024-09-15T00:12:32.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033802431000000001, 2024-09-15T00:12:32.413 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.413 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.413 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:32.413 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.413 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.413 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.413 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97496 2024-09-15T00:12:32.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.413 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:32.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:32.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:32.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.53e-07 2024-09-15T00:12:32.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853462+0000", 2024-09-15T00:12:32.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-15T00:12:32.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853468+0000", 2024-09-15T00:12:32.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3609999999999997e-06 2024-09-15T00:12:32.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853473+0000", 2024-09-15T00:12:32.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-15T00:12:32.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887111+0000", 2024-09-15T00:12:32.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033638376999999997 2024-09-15T00:12:32.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:32.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887178+0000", 2024-09-15T00:12:32.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6877000000000004e-05 2024-09-15T00:12:32.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887257+0000", 2024-09-15T00:12:32.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9616999999999993e-05 2024-09-15T00:12:32.417 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.417 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.417 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.417 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97587 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:32.417 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:32.417 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.498652925999998, 2024-09-15T00:12:32.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033973642999999998, 2024-09-15T00:12:32.417 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.417 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.417 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:32.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.418 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.418 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97587 2024-09-15T00:12:32.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.418 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:32.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:32.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853479+0000", 2024-09-15T00:12:32.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:12:32.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853481+0000", 2024-09-15T00:12:32.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.136e-06 2024-09-15T00:12:32.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853487+0000", 2024-09-15T00:12:32.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2439999999999998e-06 2024-09-15T00:12:32.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853492+0000", 2024-09-15T00:12:32.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:12:32.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887284+0000", 2024-09-15T00:12:32.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033791827000000003 2024-09-15T00:12:32.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:32.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887361+0000", 2024-09-15T00:12:32.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6983000000000005e-05 2024-09-15T00:12:32.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887452+0000", 2024-09-15T00:12:32.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1236999999999994e-05 2024-09-15T00:12:32.423 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.423 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.423 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.423 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97588 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:32.423 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:32.423 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.498632769, 2024-09-15T00:12:32.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034134617999999999, 2024-09-15T00:12:32.424 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.424 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.424 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:32.424 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.424 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.424 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.424 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97588 2024-09-15T00:12:32.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.425 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:32.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:32.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853499+0000", 2024-09-15T00:12:32.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:12:32.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853501+0000", 2024-09-15T00:12:32.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.26e-06 2024-09-15T00:12:32.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853507+0000", 2024-09-15T00:12:32.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2899999999999999e-06 2024-09-15T00:12:32.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853512+0000", 2024-09-15T00:12:32.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8330000000000001e-06 2024-09-15T00:12:32.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887495+0000", 2024-09-15T00:12:32.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033982529999999997 2024-09-15T00:12:32.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:32.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887548+0000", 2024-09-15T00:12:32.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3445000000000001e-05 2024-09-15T00:12:32.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887633+0000", 2024-09-15T00:12:32.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4890999999999997e-05 2024-09-15T00:12:32.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.430 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.431 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126866 3.6 3.973c9fa6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:32.431 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:32.431 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.395310823999999, 2024-09-15T00:12:32.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153301000000003, 2024-09-15T00:12:32.431 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.431 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.432 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:32.432 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.432 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.432 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.432 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126866 2024-09-15T00:12:32.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.432 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:32.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:32.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956821+0000", 2024-09-15T00:12:32.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.91e-07 2024-09-15T00:12:32.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956822+0000", 2024-09-15T00:12:32.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.705e-06 2024-09-15T00:12:32.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956828+0000", 2024-09-15T00:12:32.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6760000000000004e-06 2024-09-15T00:12:32.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956833+0000", 2024-09-15T00:12:32.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5379999999999996e-06 2024-09-15T00:12:32.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989970+0000", 2024-09-15T00:12:32.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033137560000000003 2024-09-15T00:12:32.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989974+0000", 2024-09-15T00:12:32.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-15T00:12:32.436 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.436 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.436 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.437 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126951 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:32.437 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:32.437 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.395185141000001, 2024-09-15T00:12:32.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033152585999999998, 2024-09-15T00:12:32.437 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.437 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.437 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:32.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.438 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.438 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126951 2024-09-15T00:12:32.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.438 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:32.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:32.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:32.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-07 2024-09-15T00:12:32.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956953+0000", 2024-09-15T00:12:32.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9310000000000002e-06 2024-09-15T00:12:32.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956957+0000", 2024-09-15T00:12:32.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5379999999999998e-06 2024-09-15T00:12:32.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956959+0000", 2024-09-15T00:12:32.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6599999999999999e-06 2024-09-15T00:12:32.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990095+0000", 2024-09-15T00:12:32.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033135855999999998 2024-09-15T00:12:32.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990099+0000", 2024-09-15T00:12:32.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4010000000000001e-06 2024-09-15T00:12:32.441 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.441 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.441 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.441 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126952 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:32.441 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:32.441 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.395168152, 2024-09-15T00:12:32.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033156066999999997, 2024-09-15T00:12:32.442 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.442 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.442 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:32.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.442 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126952 2024-09-15T00:12:32.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.442 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:32.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:32.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:32.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:12:32.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956965+0000", 2024-09-15T00:12:32.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3379999999999999e-06 2024-09-15T00:12:32.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956968+0000", 2024-09-15T00:12:32.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0419999999999998e-06 2024-09-15T00:12:32.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956970+0000", 2024-09-15T00:12:32.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4899999999999999e-06 2024-09-15T00:12:32.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990113+0000", 2024-09-15T00:12:32.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033142642 2024-09-15T00:12:32.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990119+0000", 2024-09-15T00:12:32.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2979999999999997e-06 2024-09-15T00:12:32.445 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.445 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.445 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.445 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126954 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:32.446 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:32.446 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.395122657, 2024-09-15T00:12:32.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033158449, 2024-09-15T00:12:32.446 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.446 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.446 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:32.446 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.446 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.446 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.446 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126954 2024-09-15T00:12:32.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.446 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:32.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:32.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:32.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-15T00:12:32.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957010+0000", 2024-09-15T00:12:32.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3659999999999999e-06 2024-09-15T00:12:32.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957013+0000", 2024-09-15T00:12:32.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1140000000000002e-06 2024-09-15T00:12:32.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957016+0000", 2024-09-15T00:12:32.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1209999999999998e-06 2024-09-15T00:12:32.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990163+0000", 2024-09-15T00:12:32.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033146451 2024-09-15T00:12:32.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990167+0000", 2024-09-15T00:12:32.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0579999999999999e-06 2024-09-15T00:12:32.449 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.449 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.449 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.450 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126955 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:32.450 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:32.450 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.395111442999999, 2024-09-15T00:12:32.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033160617000000003, 2024-09-15T00:12:32.450 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.450 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.450 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:32.450 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.450 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.450 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.450 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126955 2024-09-15T00:12:32.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.451 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:32.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:32.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:32.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-15T00:12:32.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957021+0000", 2024-09-15T00:12:32.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-15T00:12:32.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957024+0000", 2024-09-15T00:12:32.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1599999999999998e-06 2024-09-15T00:12:32.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957027+0000", 2024-09-15T00:12:32.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000001e-06 2024-09-15T00:12:32.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990177+0000", 2024-09-15T00:12:32.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033149741000000003 2024-09-15T00:12:32.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990180+0000", 2024-09-15T00:12:32.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4910000000000001e-06 2024-09-15T00:12:32.453 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.453 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.454 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.454 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126956 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:32.454 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:32.454 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.395101322, 2024-09-15T00:12:32.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033163633999999997, 2024-09-15T00:12:32.454 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.454 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.454 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:32.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.454 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126956 2024-09-15T00:12:32.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.455 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:32.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:32.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:32.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:32.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957031+0000", 2024-09-15T00:12:32.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-06 2024-09-15T00:12:32.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957034+0000", 2024-09-15T00:12:32.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9960000000000002e-06 2024-09-15T00:12:32.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957037+0000", 2024-09-15T00:12:32.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6419999999999999e-06 2024-09-15T00:12:32.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990190+0000", 2024-09-15T00:12:32.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153152999999998 2024-09-15T00:12:32.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990193+0000", 2024-09-15T00:12:32.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6069999999999999e-06 2024-09-15T00:12:32.457 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.457 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126957 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.395091672, 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033166442999999997, 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126957 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957040+0000", 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957041+0000", 2024-09-15T00:12:32.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0440000000000001e-06 2024-09-15T00:12:32.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957044+0000", 2024-09-15T00:12:32.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0170000000000001e-06 2024-09-15T00:12:32.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957048+0000", 2024-09-15T00:12:32.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0319999999999997e-06 2024-09-15T00:12:32.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990202+0000", 2024-09-15T00:12:32.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033154640999999999 2024-09-15T00:12:32.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990206+0000", 2024-09-15T00:12:32.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.507e-06 2024-09-15T00:12:32.461 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.461 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.461 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.461 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126958 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:32.462 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:32.462 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.395080441999999, 2024-09-15T00:12:32.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033167613999999998, 2024-09-15T00:12:32.462 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.462 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.462 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:32.462 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.462 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.462 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.462 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126958 2024-09-15T00:12:32.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.462 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:32.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:32.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:32.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:12:32.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957053+0000", 2024-09-15T00:12:32.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.756e-06 2024-09-15T00:12:32.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957059+0000", 2024-09-15T00:12:32.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2770000000000002e-06 2024-09-15T00:12:32.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957063+0000", 2024-09-15T00:12:32.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2459999999999997e-06 2024-09-15T00:12:32.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990215+0000", 2024-09-15T00:12:32.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033151509000000003 2024-09-15T00:12:32.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990218+0000", 2024-09-15T00:12:32.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5099999999999999e-06 2024-09-15T00:12:32.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.465 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.465 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126959 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:32.466 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:32.466 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.395061438000001, 2024-09-15T00:12:32.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033161673000000003, 2024-09-15T00:12:32.466 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.466 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.466 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:32.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.466 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126959 2024-09-15T00:12:32.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.466 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:32.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:32.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:32.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:12:32.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957072+0000", 2024-09-15T00:12:32.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2239999999999998e-06 2024-09-15T00:12:32.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957079+0000", 2024-09-15T00:12:32.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.19e-06 2024-09-15T00:12:32.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957083+0000", 2024-09-15T00:12:32.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:12:32.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990228+0000", 2024-09-15T00:12:32.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033144818 2024-09-15T00:12:32.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990231+0000", 2024-09-15T00:12:32.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:12:32.469 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.469 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.469 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.469 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126960 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:32.470 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:32.470 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.395042631000001, 2024-09-15T00:12:32.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033155166, 2024-09-15T00:12:32.470 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:32.470 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:32.470 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:32.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:32.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:32.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:32.470 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126960 2024-09-15T00:12:32.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.470 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:32.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:32.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957091+0000", 2024-09-15T00:12:32.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:12:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957097+0000", 2024-09-15T00:12:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6050000000000003e-06 2024-09-15T00:12:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957102+0000", 2024-09-15T00:12:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5539999999999999e-06 2024-09-15T00:12:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990240+0000", 2024-09-15T00:12:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033138289000000001 2024-09-15T00:12:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990244+0000", 2024-09-15T00:12:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4740000000000001e-06 2024-09-15T00:12:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:32.473 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:32.474 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:12:32.475 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:32.473+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:32.476 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:32.548 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:32.549 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:32.549 INFO:teuthology.orchestra.run.smithi171.stdout:4392 write 0xa665573 thru 0xa6663b8 (0xe46 bytes) 2024-09-15T00:12:32.575 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:32.575 INFO:teuthology.orchestra.run.smithi171.stdout:4395 write 0x57e0ca7 thru 0x57eab9f (0x9ef9 bytes) 2024-09-15T00:12:32.575 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:32.573+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:32.607 INFO:teuthology.orchestra.run.smithi171.stdout:4399 read 0xb0c425f thru 0xb0ce6e4 (0xa486 bytes) 2024-09-15T00:12:32.608 INFO:teuthology.orchestra.run.smithi171.stdout:4400 trunc from 0xcba309b to 0xc83fc7f 2024-09-15T00:12:32.628 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:32.628 INFO:teuthology.orchestra.run.smithi171.stdout:4402 punch from 0xbcf1e5a to 0xbd01996, (0xfb3c bytes) 2024-09-15T00:12:32.631 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:32.631 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:32.631 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:32.631 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:32.636 INFO:teuthology.orchestra.run.smithi171.stdout:4404 trunc from 0xc83fc7f to 0x627a34b 2024-09-15T00:12:32.644 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:32.651 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:32.651 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:32.676 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:32.674+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:32.677 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:32.677 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:32.677 INFO:teuthology.orchestra.run.smithi171.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T00:12:32.696 INFO:teuthology.orchestra.run.smithi171.stdout:4405 read 0x53cc985 thru 0x53d843e (0xbaba bytes) 2024-09-15T00:12:32.697 INFO:teuthology.orchestra.run.smithi171.stdout:4406 read 0x1b372d2 thru 0x1b4597a (0xe6a9 bytes) 2024-09-15T00:12:32.697 INFO:teuthology.orchestra.run.smithi171.stdout:4407 read 0x76801e thru 0x773df3 (0xbdd6 bytes) 2024-09-15T00:12:32.698 INFO:teuthology.orchestra.run.smithi171.stdout:4408 read 0x332db48 thru 0x3332a05 (0x4ebe bytes) 2024-09-15T00:12:32.698 INFO:teuthology.orchestra.run.smithi171.stdout:4411 write 0xa05e00 thru 0xa143d4 (0xe5d5 bytes) 2024-09-15T00:12:32.699 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:32.700 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:32.725 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:32.725 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:32.751 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:32.751 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:32.762 INFO:teuthology.orchestra.run.smithi171.stdout:4412 write 0xebb9392 thru 0xebba5cd (0x123c bytes) 2024-09-15T00:12:32.769 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:32.770 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:32.770 INFO:teuthology.orchestra.run.smithi171.stdout:4413 writesame 0x344065b thru 0x34456c2 data_size 0x1f(0x5068 bytes) 2024-09-15T00:12:32.777 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:32.775+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:32.786 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:32.786 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:32.793 INFO:teuthology.orchestra.run.smithi171.stdout:4415 writesame 0x7fee0c1 thru 0x7ffa0d0 data_size 0x38(0xc010 bytes) 2024-09-15T00:12:32.796 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:32.797 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:32.797 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:32.799 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:12:32.800 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:12:32.802 INFO:teuthology.orchestra.run.smithi171.stdout:4416 write 0x6c4e3c3 thru 0x6c558ab (0x74e9 bytes) 2024-09-15T00:12:32.811 INFO:teuthology.orchestra.run.smithi171.stdout:4418 punch from 0x847b41d to 0x8487e01, (0xc9e4 bytes) 2024-09-15T00:12:32.811 INFO:teuthology.orchestra.run.smithi171.stdout:4419 read 0xd4c5270 thru 0xd4c9768 (0x44f9 bytes) 2024-09-15T00:12:32.811 INFO:teuthology.orchestra.run.smithi171.stdout:4421 write 0x16eca1f thru 0x16f03a3 (0x3985 bytes) 2024-09-15T00:12:32.836 INFO:teuthology.orchestra.run.smithi171.stdout:4422 write 0x2a6c59a thru 0x2a7b517 (0xef7e bytes) 2024-09-15T00:12:32.878 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:32.876+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:32.888 INFO:teuthology.orchestra.run.smithi171.stdout:4426 read 0x7b5da65 thru 0x7b67dcc (0xa368 bytes) 2024-09-15T00:12:32.888 INFO:teuthology.orchestra.run.smithi171.stdout:4428 write 0x617c9c0 thru 0x61882a1 (0xb8e2 bytes) 2024-09-15T00:12:32.898 INFO:teuthology.orchestra.run.smithi171.stdout:4429 punch from 0x68253d5 to 0x682d59f, (0x81ca bytes) 2024-09-15T00:12:32.898 INFO:teuthology.orchestra.run.smithi171.stdout:4430 read 0xd4490c2 thru 0xd453aab (0xa9ea bytes) 2024-09-15T00:12:32.898 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:32.898 INFO:teuthology.orchestra.run.smithi171.stdout:4432 write 0x87ae3f7 thru 0x87aef11 (0xb1b bytes) 2024-09-15T00:12:32.901 INFO:teuthology.orchestra.run.smithi171.stdout:4434 read 0xe5d0eb4 thru 0xe5da138 (0x9285 bytes) 2024-09-15T00:12:32.913 INFO:teuthology.orchestra.run.smithi171.stdout:4435 trunc from 0xebba5ce to 0x4fc9fdd 2024-09-15T00:12:32.925 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:32.925 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:32.936 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:32.980 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:32.978+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:32.985 INFO:teuthology.orchestra.run.smithi171.stdout:4436 writesame 0x68ddfe8 thru 0x68e2bcc data_size 0x1(0x4be5 bytes) 2024-09-15T00:12:32.999 INFO:teuthology.orchestra.run.smithi171.stdout:4440 write 0xd945003 thru 0xd954534 (0xf532 bytes) 2024-09-15T00:12:33.012 INFO:teuthology.orchestra.run.smithi171.stdout:4441 trunc from 0xd954535 to 0xd76f39a 2024-09-15T00:12:33.027 INFO:teuthology.orchestra.run.smithi171.stdout:4442 write 0xd8eb86a thru 0xd8f0c08 (0x539f bytes) 2024-09-15T00:12:33.037 INFO:teuthology.orchestra.run.smithi171.stdout:4444 punch from 0x98e7629 to 0x98edc34, (0x660b bytes) 2024-09-15T00:12:33.037 INFO:teuthology.orchestra.run.smithi171.stdout:4446 read 0x647a92b thru 0x6486f23 (0xc5f9 bytes) 2024-09-15T00:12:33.037 INFO:teuthology.orchestra.run.smithi171.stdout:4447 write 0x82f167a thru 0x82f28b2 (0x1239 bytes) 2024-09-15T00:12:33.045 INFO:teuthology.orchestra.run.smithi171.stdout:4448 write 0x6d656da thru 0x6d6bf97 (0x68be bytes) 2024-09-15T00:12:33.053 INFO:teuthology.orchestra.run.smithi171.stdout:4449 punch from 0xb40e4c0 to 0xb41e0b4, (0xfbf4 bytes) 2024-09-15T00:12:33.053 INFO:teuthology.orchestra.run.smithi171.stdout:4450 punch from 0xbaa6506 to 0xbab366a, (0xd164 bytes) 2024-09-15T00:12:33.053 INFO:teuthology.orchestra.run.smithi171.stdout:4451 write 0x70f6595 thru 0x70f9b63 (0x35cf bytes) 2024-09-15T00:12:33.056 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:33.056 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:33.065 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:33.068 INFO:teuthology.orchestra.run.smithi171.stdout:4452 trunc from 0xd8f0c09 to 0x27843e5 2024-09-15T00:12:33.108 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:33.106+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:33.139 INFO:teuthology.orchestra.run.smithi171.stdout:4453 read 0x9c7183 thru 0x9c9d7b (0x2bf9 bytes) 2024-09-15T00:12:33.155 INFO:teuthology.orchestra.run.smithi171.stdout:4454 write 0xafc2c64 thru 0xafcfe07 (0xd1a4 bytes) 2024-09-15T00:12:33.163 INFO:teuthology.orchestra.run.smithi171.stdout:4456 read 0x8738d30 thru 0x873c45d (0x372e bytes) 2024-09-15T00:12:33.172 INFO:teuthology.orchestra.run.smithi171.stdout:4457 write 0xe2aae6b thru 0xe2acdc0 (0x1f56 bytes) 2024-09-15T00:12:33.175 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is set 2024-09-15T00:12:33.178 INFO:teuthology.orchestra.run.smithi171.stdout:4459 write 0x33d8469 thru 0x33e5f4d (0xdae5 bytes) 2024-09-15T00:12:33.181 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:33.179+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:33.187 INFO:teuthology.orchestra.run.smithi171.stdout:4460 writesame 0xce4cd02 thru 0xce4ea4d data_size 0x2(0x1d4c bytes) 2024-09-15T00:12:33.191 INFO:teuthology.orchestra.run.smithi171.stdout:4463 writesame 0x4d08fd8 thru 0x4d11e43 data_size 0xa(0x8e6c bytes) 2024-09-15T00:12:33.197 INFO:teuthology.orchestra.run.smithi171.stdout:4464 write 0x837a50c thru 0x837c6d8 (0x21cd bytes) 2024-09-15T00:12:33.201 INFO:teuthology.orchestra.run.smithi171.stdout:4465 writesame 0xde12797 thru 0xde161a9 data_size 0x1(0x3a13 bytes) 2024-09-15T00:12:33.227 INFO:teuthology.orchestra.run.smithi171.stdout:4470 trunc from 0xe2acdc1 to 0x6178db0 2024-09-15T00:12:33.232 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:33.232 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:33.242 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:33.257 INFO:teuthology.orchestra.run.smithi171.stdout:4472 writesame 0xebcca41 thru 0xebdc920 data_size 0x10(0xfee0 bytes) 2024-09-15T00:12:33.258 INFO:teuthology.orchestra.run.smithi171.stdout:4474 write 0x1c916ca thru 0x1c93a28 (0x235f bytes) 2024-09-15T00:12:33.270 INFO:teuthology.orchestra.run.smithi171.stdout:4475 write 0xbdd6b9a thru 0xbdd8cdf (0x2146 bytes) 2024-09-15T00:12:33.276 INFO:teuthology.orchestra.run.smithi171.stdout:4476 punch from 0x1131612 to 0x113abbf, (0x95ad bytes) 2024-09-15T00:12:33.277 INFO:teuthology.orchestra.run.smithi171.stdout:4477 writesame 0x1e6f2a thru 0x1ededc data_size 0x7(0x6fb3 bytes) 2024-09-15T00:12:33.282 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:33.280+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:33.331 INFO:teuthology.orchestra.run.smithi171.stdout:4478 trunc from 0xebdc921 to 0x92786c7 2024-09-15T00:12:33.342 INFO:teuthology.orchestra.run.smithi171.stdout:4480 trunc from 0x92786c7 to 0x937e11f 2024-09-15T00:12:33.346 INFO:teuthology.orchestra.run.smithi171.stdout:4482 write 0x860b1e3 thru 0x860b4fe (0x31c bytes) 2024-09-15T00:12:33.349 INFO:teuthology.orchestra.run.smithi171.stdout:4483 writesame 0x7ca1733 thru 0x7ca60e6 data_size 0x4(0x49b4 bytes) 2024-09-15T00:12:33.356 INFO:teuthology.orchestra.run.smithi171.stdout:4484 read 0x6b7846e thru 0x6b788b7 (0x44a bytes) 2024-09-15T00:12:33.360 INFO:teuthology.orchestra.run.smithi171.stdout:4485 trunc from 0x937e11f to 0x634cb89 2024-09-15T00:12:33.373 INFO:teuthology.orchestra.run.smithi171.stdout:4486 punch from 0x5109a10 to 0x5113903, (0x9ef3 bytes) 2024-09-15T00:12:33.381 INFO:teuthology.orchestra.run.smithi171.stdout:4487 trunc from 0x634cb89 to 0xce41b96 2024-09-15T00:12:33.382 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:33.381+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:33.393 INFO:teuthology.orchestra.run.smithi171.stdout:4490 write 0xe9970de thru 0xe99908a (0x1fad bytes) 2024-09-15T00:12:33.397 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:33.398 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:33.401 INFO:teuthology.orchestra.run.smithi171.stdout:4491 trunc from 0xe99908b to 0xa682662 2024-09-15T00:12:33.407 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:33.412 INFO:teuthology.orchestra.run.smithi171.stdout:4492 write 0x9615ef9 thru 0x961cf68 (0x7070 bytes) 2024-09-15T00:12:33.422 INFO:teuthology.orchestra.run.smithi171.stdout:4494 punch from 0x16112ba to 0x161b869, (0xa5af bytes) 2024-09-15T00:12:33.422 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:33.423 INFO:teuthology.orchestra.run.smithi171.stdout:4496 write 0x489c37b thru 0x48a50a3 (0x8d29 bytes) 2024-09-15T00:12:33.426 INFO:teuthology.orchestra.run.smithi171.stdout:4497 read 0x11ba85 thru 0x11db3a (0x20b6 bytes) 2024-09-15T00:12:33.435 INFO:teuthology.orchestra.run.smithi171.stdout:4499 write 0xd170c4d thru 0xd173486 (0x283a bytes) 2024-09-15T00:12:33.439 INFO:teuthology.orchestra.run.smithi171.stdout:4501 write 0x312166 thru 0x316833 (0x46ce bytes) 2024-09-15T00:12:33.441 INFO:teuthology.orchestra.run.smithi171.stdout:4502 write 0xc3a9b49 thru 0xc3ab7d4 (0x1c8c bytes) 2024-09-15T00:12:33.445 INFO:teuthology.orchestra.run.smithi171.stdout:4503 writesame 0x1c71715 thru 0x1c7caf0 data_size 0x12(0xb3dc bytes) 2024-09-15T00:12:33.473 INFO:teuthology.orchestra.run.smithi171.stdout:4504 punch from 0xb678f19 to 0xb67abff, (0x1ce6 bytes) 2024-09-15T00:12:33.474 INFO:teuthology.orchestra.run.smithi171.stdout:4505 read 0x1c47043 thru 0x1c5431e (0xd2dc bytes) 2024-09-15T00:12:33.474 INFO:teuthology.orchestra.run.smithi171.stdout:4506 read 0xa20495b thru 0xa20a542 (0x5be8 bytes) 2024-09-15T00:12:33.483 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:33.482+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:33.484 INFO:teuthology.orchestra.run.smithi171.stdout:4507 trunc from 0xd173487 to 0x4e5e6d5 2024-09-15T00:12:33.496 INFO:teuthology.orchestra.run.smithi171.stdout:4508 trunc from 0x4e5e6d5 to 0x6739f3e 2024-09-15T00:12:33.502 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:33.502 INFO:teuthology.orchestra.run.smithi171.stdout:4510 read 0x6371e5c thru 0x637d37a (0xb51f bytes) 2024-09-15T00:12:33.502 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:33.502 INFO:teuthology.orchestra.run.smithi171.stdout:4513 writesame 0x260545e thru 0x26148b4 data_size 0x1(0xf457 bytes) 2024-09-15T00:12:33.530 INFO:teuthology.orchestra.run.smithi171.stdout:4515 punch from 0x24748e6 to 0x2479372, (0x4a8c bytes) 2024-09-15T00:12:33.530 INFO:teuthology.orchestra.run.smithi171.stdout:4518 read 0x1bd0a87 thru 0x1bd6b77 (0x60f1 bytes) 2024-09-15T00:12:33.531 INFO:teuthology.orchestra.run.smithi171.stdout:4520 write 0x59ab63b thru 0x59ae128 (0x2aee bytes) 2024-09-15T00:12:33.532 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:33.532 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:33.543 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:33.546 INFO:teuthology.orchestra.run.smithi171.stdout:4521 writesame 0xaaab24f thru 0xaaac5af data_size 0x1(0x1361 bytes) 2024-09-15T00:12:33.546 INFO:teuthology.orchestra.run.smithi171.stdout:4522 write 0x1741f5f thru 0x17493ed (0x748f bytes) 2024-09-15T00:12:33.564 INFO:teuthology.orchestra.run.smithi171.stdout:4523 punch from 0x8a6324b to 0x8a69e08, (0x6bbd bytes) 2024-09-15T00:12:33.565 INFO:teuthology.orchestra.run.smithi171.stdout:4524 read 0x20a4193 thru 0x20b0aaf (0xc91d bytes) 2024-09-15T00:12:33.566 INFO:teuthology.orchestra.run.smithi171.stdout:4525 write 0x7d78286 thru 0x7d8266e (0xa3e9 bytes) 2024-09-15T00:12:33.574 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:33.584 INFO:teuthology.orchestra.run.smithi171.stdout:4529 write 0xde48895 thru 0xde56986 (0xe0f2 bytes) 2024-09-15T00:12:33.584 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:33.582+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:33.606 INFO:teuthology.orchestra.run.smithi171.stdout:4530 trunc from 0xde56987 to 0x388713f 2024-09-15T00:12:33.627 INFO:teuthology.orchestra.run.smithi171.stdout:4531 read 0x305904d thru 0x3068214 (0xf1c8 bytes) 2024-09-15T00:12:33.628 INFO:teuthology.orchestra.run.smithi171.stdout:4534 punch from 0x11dca4c to 0x11ddac2, (0x1076 bytes) 2024-09-15T00:12:33.637 INFO:teuthology.orchestra.run.smithi171.stdout:4535 write 0x6e79ffa thru 0x6e7e5e7 (0x45ee bytes) 2024-09-15T00:12:33.648 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:33.654 INFO:teuthology.orchestra.run.smithi171.stdout:4539 trunc from 0x6e7e5e8 to 0xc24ee87 2024-09-15T00:12:33.660 INFO:teuthology.orchestra.run.smithi171.stdout:4540 write 0x19ff930 thru 0x1a0c45d (0xcb2e bytes) 2024-09-15T00:12:33.665 INFO:teuthology.orchestra.run.smithi171.stdout:4541 write 0x80f1fe thru 0x81003f (0xe42 bytes) 2024-09-15T00:12:33.666 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:33.666 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:33.677 INFO:teuthology.orchestra.run.smithi171.stdout:4545 trunc from 0xc24ee87 to 0x3684b9a 2024-09-15T00:12:33.685 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:33.683+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:33.687 INFO:teuthology.orchestra.run.smithi171.stdout:4547 read 0xcf80db thru 0xcf940d (0x1333 bytes) 2024-09-15T00:12:33.688 INFO:teuthology.orchestra.run.smithi171.stdout:4548 trunc from 0x3684b9a to 0x42a34d1 2024-09-15T00:12:33.693 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:33.693 INFO:teuthology.orchestra.run.smithi171.stdout:4550 punch from 0xa7d63d to 0xa89fe2, (0xc9a5 bytes) 2024-09-15T00:12:33.693 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:33.693 INFO:teuthology.orchestra.run.smithi171.stdout:4553 read 0x127b1b8 thru 0x12880a4 (0xceed bytes) 2024-09-15T00:12:33.694 INFO:teuthology.orchestra.run.smithi171.stdout:4554 write 0x28513d thru 0x293e2d (0xecf1 bytes) 2024-09-15T00:12:33.696 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:33.696 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:33.709 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:33.718 INFO:teuthology.orchestra.run.smithi171.stdout:4555 write 0x3ad3fe4 thru 0x3ad5904 (0x1921 bytes) 2024-09-15T00:12:33.732 INFO:teuthology.orchestra.run.smithi171.stdout:4556 writesame 0x7477626 thru 0x747b2a0 data_size 0xd(0x3c7b bytes) 2024-09-15T00:12:33.732 INFO:teuthology.orchestra.run.smithi171.stdout:4558 read 0x10ec61e thru 0x10ef660 (0x3043 bytes) 2024-09-15T00:12:33.740 INFO:teuthology.orchestra.run.smithi171.stdout:4559 trunc from 0x747b2a1 to 0xda5a455 2024-09-15T00:12:33.745 INFO:teuthology.orchestra.run.smithi171.stdout:4561 punch from 0x9384b13 to 0x93876e3, (0x2bd0 bytes) 2024-09-15T00:12:33.745 INFO:teuthology.orchestra.run.smithi171.stdout:4562 write 0x5a4b062 thru 0x5a55ff0 (0xaf8f bytes) 2024-09-15T00:12:33.753 INFO:teuthology.orchestra.run.smithi171.stdout:4563 write 0x34e13d1 thru 0x34ea3f3 (0x9023 bytes) 2024-09-15T00:12:33.758 INFO:teuthology.orchestra.run.smithi171.stdout:4564 write 0x1b4b421 thru 0x1b5259a (0x717a bytes) 2024-09-15T00:12:33.760 INFO:teuthology.orchestra.run.smithi171.stdout:4566 read 0x1aa0ca0 thru 0x1aa58e6 (0x4c47 bytes) 2024-09-15T00:12:33.761 INFO:teuthology.orchestra.run.smithi171.stdout:4567 writesame 0x19d0d60 thru 0x19d66ad data_size 0x47(0x594e bytes) 2024-09-15T00:12:33.763 INFO:teuthology.orchestra.run.smithi171.stdout:4568 writesame 0xaa1140 thru 0xaa2692 data_size 0x1(0x1553 bytes) 2024-09-15T00:12:33.766 INFO:teuthology.orchestra.run.smithi171.stdout:4569 punch from 0x26dce0e to 0x26e0274, (0x3466 bytes) 2024-09-15T00:12:33.766 INFO:teuthology.orchestra.run.smithi171.stdout:4570 write 0xb49b9ad thru 0xb49e2d3 (0x2927 bytes) 2024-09-15T00:12:33.771 INFO:teuthology.orchestra.run.smithi171.stdout:4573 read 0x58942a2 thru 0x589eb70 (0xa8cf bytes) 2024-09-15T00:12:33.774 INFO:teuthology.orchestra.run.smithi171.stdout:4574 trunc from 0xda5a455 to 0xb91100c 2024-09-15T00:12:33.785 INFO:teuthology.orchestra.run.smithi171.stdout:4575 trunc from 0xb91100c to 0x7db5e47 2024-09-15T00:12:33.785 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:33.783+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:33.795 INFO:teuthology.orchestra.run.smithi171.stdout:4576 read 0x6bea97c thru 0x6bf90a3 (0xe728 bytes) 2024-09-15T00:12:33.800 INFO:teuthology.orchestra.run.smithi171.stdout:4578 trunc from 0x7db5e47 to 0xc0520af 2024-09-15T00:12:33.805 INFO:teuthology.orchestra.run.smithi171.stdout:4579 writesame 0x5f84cfc thru 0x5f86e0f data_size 0x1d(0x2114 bytes) 2024-09-15T00:12:33.808 INFO:teuthology.orchestra.run.smithi171.stdout:4581 write 0x550c07a thru 0x551b924 (0xf8ab bytes) 2024-09-15T00:12:33.816 INFO:teuthology.orchestra.run.smithi171.stdout:4582 punch from 0x27768a6 to 0x277cf25, (0x667f bytes) 2024-09-15T00:12:33.816 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:33.821 INFO:teuthology.orchestra.run.smithi171.stdout:4585 write 0xc66858c thru 0xc6713a7 (0x8e1c bytes) 2024-09-15T00:12:33.827 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:33.828 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:33.829 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:33.829 INFO:teuthology.orchestra.run.smithi171.stdout:4589 punch from 0x1db0742 to 0x1dbc0af, (0xb96d bytes) 2024-09-15T00:12:33.837 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:33.837 INFO:teuthology.orchestra.run.smithi171.stdout:4590 trunc from 0xc6713a8 to 0x5083bdb 2024-09-15T00:12:33.850 INFO:teuthology.orchestra.run.smithi171.stdout:4591 write 0x4090dab thru 0x4097d40 (0x6f96 bytes) 2024-09-15T00:12:33.871 INFO:teuthology.orchestra.run.smithi171.stdout:4592 write 0xa5e5fbb thru 0xa5ed5f4 (0x763a bytes) 2024-09-15T00:12:33.880 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:33.886 INFO:teuthology.orchestra.run.smithi171.stdout:4594 trunc from 0xa5ed5f5 to 0x52abb0d 2024-09-15T00:12:33.913 INFO:teuthology.orchestra.run.smithi171.stdout:4596 write 0xc8cff04 thru 0xc8dcdd7 (0xced4 bytes) 2024-09-15T00:12:33.921 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:33.919+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:33.926 INFO:teuthology.orchestra.run.smithi171.stdout:4597 read 0x633017a thru 0x6334a45 (0x48cc bytes) 2024-09-15T00:12:33.932 INFO:teuthology.orchestra.run.smithi171.stdout:4599 trunc from 0xc8dcdd8 to 0x800fac1 2024-09-15T00:12:33.945 INFO:teuthology.orchestra.run.smithi171.stdout:4600 flatten 2024-09-15T00:12:33.987 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:33.985+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:34.002 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:34.002 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:34.011 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:34.013 INFO:teuthology.orchestra.run.smithi171.stdout:4601 write 0x2622007 thru 0x262a990 (0x898a bytes) 2024-09-15T00:12:34.016 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:34.016 INFO:teuthology.orchestra.run.smithi171.stdout:4603 write 0x412bf90 thru 0x413a6b5 (0xe726 bytes) 2024-09-15T00:12:34.025 INFO:teuthology.orchestra.run.smithi171.stdout:4605 read 0x5eef1e2 thru 0x5ef1436 (0x2255 bytes) 2024-09-15T00:12:34.025 INFO:teuthology.orchestra.run.smithi171.stdout:4606 read 0x5c19039 thru 0x5c1ac0d (0x1bd5 bytes) 2024-09-15T00:12:34.025 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:34.033 INFO:teuthology.orchestra.run.smithi171.stdout:4609 write 0xa1d04c9 thru 0xa1dfe3c (0xf974 bytes) 2024-09-15T00:12:34.044 INFO:teuthology.orchestra.run.smithi171.stdout:4610 read 0x99b8220 thru 0x99c6311 (0xe0f2 bytes) 2024-09-15T00:12:34.049 INFO:teuthology.orchestra.run.smithi171.stdout:4611 trunc from 0xa1dfe3d to 0x60d6cb7 2024-09-15T00:12:34.059 INFO:teuthology.orchestra.run.smithi171.stdout:4612 read 0x2727231 thru 0x272afcb (0x3d9b bytes) 2024-09-15T00:12:34.064 INFO:teuthology.orchestra.run.smithi171.stdout:4613 writesame 0x60ffdde thru 0x6100302 data_size 0x3(0x525 bytes) 2024-09-15T00:12:34.064 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:34.064 INFO:teuthology.orchestra.run.smithi171.stdout:4615 read 0x1770ef2 thru 0x177bc7b (0xad8a bytes) 2024-09-15T00:12:34.076 INFO:teuthology.orchestra.run.smithi171.stdout:4616 write 0xb004861 thru 0xb00a1f7 (0x5997 bytes) 2024-09-15T00:12:34.082 INFO:teuthology.orchestra.run.smithi171.stdout:4617 read 0x17e0971 thru 0x17e26c1 (0x1d51 bytes) 2024-09-15T00:12:34.083 INFO:teuthology.orchestra.run.smithi171.stdout:4618 read 0xcce606 thru 0xcdacac (0xc6a7 bytes) 2024-09-15T00:12:34.083 INFO:teuthology.orchestra.run.smithi171.stdout:4619 read 0x5f5b89b thru 0x5f6125a (0x59c0 bytes) 2024-09-15T00:12:34.083 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:34.083 INFO:teuthology.orchestra.run.smithi171.stdout:4621 read 0xa9171b thru 0xa91bcb (0x4b1 bytes) 2024-09-15T00:12:34.084 INFO:teuthology.orchestra.run.smithi171.stdout:4622 trunc from 0xb00a1f8 to 0x9ffa9aa 2024-09-15T00:12:34.087 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:34.086+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:34.094 INFO:teuthology.orchestra.run.smithi171.stdout:4624 read 0x1fcf681 thru 0x1fd4073 (0x49f3 bytes) 2024-09-15T00:12:34.094 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:34.099 INFO:teuthology.orchestra.run.smithi171.stdout:4626 write 0xa025793 thru 0xa027436 (0x1ca4 bytes) 2024-09-15T00:12:34.103 INFO:teuthology.orchestra.run.smithi171.stdout:4627 write 0x91083fa thru 0x9113ee5 (0xbaec bytes) 2024-09-15T00:12:34.114 INFO:teuthology.orchestra.run.smithi171.stdout:4629 trunc from 0xa027437 to 0x94b4a44 2024-09-15T00:12:34.131 INFO:teuthology.orchestra.run.smithi171.stdout:4630 read 0x5a2c894 thru 0x5a38511 (0xbc7e bytes) 2024-09-15T00:12:34.131 INFO:teuthology.orchestra.run.smithi171.stdout:4632 writesame 0x3c6e8e4 thru 0x3c7cf9b data_size 0x45(0xe6b8 bytes) 2024-09-15T00:12:34.134 INFO:teuthology.orchestra.run.smithi171.stdout:4633 write 0x7677cce thru 0x7677ed3 (0x206 bytes) 2024-09-15T00:12:34.137 INFO:teuthology.orchestra.run.smithi171.stdout:4634 write 0x58d57fc thru 0x58d8a63 (0x3268 bytes) 2024-09-15T00:12:34.142 INFO:teuthology.orchestra.run.smithi171.stdout:4635 write 0x9373c70 thru 0x9374270 (0x601 bytes) 2024-09-15T00:12:34.149 INFO:teuthology.orchestra.run.smithi171.stdout:4636 write 0x98416cb thru 0x98505d5 (0xef0b bytes) 2024-09-15T00:12:34.159 INFO:teuthology.orchestra.run.smithi171.stdout:4639 read 0x77cc9bd thru 0x77ce9ba (0x1ffe bytes) 2024-09-15T00:12:34.165 INFO:teuthology.orchestra.run.smithi171.stdout:4641 trunc from 0x98505d6 to 0x3d5fc9b 2024-09-15T00:12:34.168 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:34.168 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:34.179 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:34.186 INFO:teuthology.orchestra.run.smithi171.stdout:4642 writesame 0x2a0978f thru 0x2a0a108 data_size 0x1(0x97a bytes) 2024-09-15T00:12:34.197 INFO:teuthology.orchestra.run.smithi171.stdout:4643 trunc from 0x3d5fc9b to 0x89ed9ec 2024-09-15T00:12:34.204 INFO:teuthology.orchestra.run.smithi171.stdout:4644 trunc from 0x89ed9ec to 0x6b3ba4e 2024-09-15T00:12:34.212 INFO:teuthology.orchestra.run.smithi171.stdout:4649 write 0x23a85b0 thru 0x23aadaf (0x2800 bytes) 2024-09-15T00:12:34.222 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:34.221+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:34.224 INFO:teuthology.orchestra.run.smithi171.stdout:4650 trunc from 0x6b3ba4e to 0xb54f520 2024-09-15T00:12:34.230 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:34.230 INFO:teuthology.orchestra.run.smithi171.stdout:4652 write 0xb34e2c3 thru 0xb353ff8 (0x5d36 bytes) 2024-09-15T00:12:34.238 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:34.238 INFO:teuthology.orchestra.run.smithi171.stdout:4656 write 0x1675ad9 thru 0x167f39d (0x98c5 bytes) 2024-09-15T00:12:34.242 INFO:teuthology.orchestra.run.smithi171.stdout:4657 punch from 0xd0da27 to 0xd0ee17, (0x13f0 bytes) 2024-09-15T00:12:34.242 INFO:teuthology.orchestra.run.smithi171.stdout:4658 write 0x20d90f9 thru 0x20e1d81 (0x8c89 bytes) 2024-09-15T00:12:34.254 INFO:teuthology.orchestra.run.smithi171.stdout:4660 write 0x9d02010 thru 0x9d06462 (0x4453 bytes) 2024-09-15T00:12:34.267 INFO:teuthology.orchestra.run.smithi171.stdout:4662 trunc from 0xb54f520 to 0xea85ac3 2024-09-15T00:12:34.272 INFO:teuthology.orchestra.run.smithi171.stdout:4663 write 0xaa4946c thru 0xaa4d29d (0x3e32 bytes) 2024-09-15T00:12:34.278 INFO:teuthology.orchestra.run.smithi171.stdout:4668 punch from 0x4b6b16e to 0x4b785f0, (0xd482 bytes) 2024-09-15T00:12:34.278 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:34.278 INFO:teuthology.orchestra.run.smithi171.stdout:4670 write 0xe2b8a60 thru 0xe2c4190 (0xb731 bytes) 2024-09-15T00:12:34.288 INFO:teuthology.orchestra.run.smithi171.stdout:4671 write 0x4f32e2b thru 0x4f3d49c (0xa672 bytes) 2024-09-15T00:12:34.288 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:34.287+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:34.295 INFO:teuthology.orchestra.run.smithi171.stdout:4672 writesame 0xdb72ece thru 0xdb7b27f data_size 0x9(0x83b2 bytes) 2024-09-15T00:12:34.298 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:34.299 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:34.302 INFO:teuthology.orchestra.run.smithi171.stdout:4673 write 0xc4b8a3a thru 0xc4b8f4f (0x516 bytes) 2024-09-15T00:12:34.306 INFO:teuthology.orchestra.run.smithi171.stdout:4674 write 0xc7b5679 thru 0xc7c3bcf (0xe557 bytes) 2024-09-15T00:12:34.310 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:34.315 INFO:teuthology.orchestra.run.smithi171.stdout:4676 writesame 0x8945008 thru 0x894afac data_size 0x3b(0x5fa5 bytes) 2024-09-15T00:12:34.321 INFO:teuthology.orchestra.run.smithi171.stdout:4680 write 0x8c7c126 thru 0x8c804e2 (0x43bd bytes) 2024-09-15T00:12:34.329 INFO:teuthology.orchestra.run.smithi171.stdout:4681 read 0x1411da5 thru 0x141b50d (0x9769 bytes) 2024-09-15T00:12:34.330 INFO:teuthology.orchestra.run.smithi171.stdout:4683 write 0xd3502c7 thru 0xd357896 (0x75d0 bytes) 2024-09-15T00:12:34.341 INFO:teuthology.orchestra.run.smithi171.stdout:4684 punch from 0x1cd8b2b to 0x1cda42f, (0x1904 bytes) 2024-09-15T00:12:34.341 INFO:teuthology.orchestra.run.smithi171.stdout:4686 write 0x668870b thru 0x66983c9 (0xfcbf bytes) 2024-09-15T00:12:34.353 INFO:teuthology.orchestra.run.smithi171.stdout:4688 trunc from 0xea85ac3 to 0xe7813ee 2024-09-15T00:12:34.358 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:34.358 INFO:teuthology.orchestra.run.smithi171.stdout:4691 read 0x348f0af thru 0x34972d1 (0x8223 bytes) 2024-09-15T00:12:34.359 INFO:teuthology.orchestra.run.smithi171.stdout:4692 punch from 0xae2a7e3 to 0xae2d7e7, (0x3004 bytes) 2024-09-15T00:12:34.359 INFO:teuthology.orchestra.run.smithi171.stdout:4693 read 0xe0cf112 thru 0xe0d1b23 (0x2a12 bytes) 2024-09-15T00:12:34.359 INFO:teuthology.orchestra.run.smithi171.stdout:4694 write 0x767233c thru 0x7679f5b (0x7c20 bytes) 2024-09-15T00:12:34.371 INFO:teuthology.orchestra.run.smithi171.stdout:4695 writesame 0x89ed624 thru 0x89ee5ed data_size 0x2(0xfca bytes) 2024-09-15T00:12:34.373 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:34.373 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:34.373 INFO:teuthology.orchestra.run.smithi171.stdout:4698 read 0x868d007 thru 0x8695071 (0x806b bytes) 2024-09-15T00:12:34.374 INFO:teuthology.orchestra.run.smithi171.stdout:4700 clone 38 order 19 su 8192 sc 4 2024-09-15T00:12:34.393 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:34.392+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:34.454 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:34.454 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:34.462 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:34.490 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:34.488+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:34.593 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:34.591+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:34.605 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:34.605 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:34.614 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:34.698 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:34.697+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:34.758 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:34.759 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:34.767 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:34.801 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:12:34.811 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:34.810+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:34.885 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:34.885 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:34.894 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:34.896 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:34.894+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:34.940 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:34.941 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:34.959 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:34.959 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:34.960 INFO:teuthology.orchestra.run.smithi171.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T00:12:34.976 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:34.976 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:34.993 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:34.993 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:34.996 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:34.995+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:35.006 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:35.006 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:35.006 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:35.006 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:35.006 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:35.010 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:35.010 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:35.013 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:35.028 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:35.028 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:35.048 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:35.049 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:35.052 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:12:35.052 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:12:35.098 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:35.096+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:35.122 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-15T00:12:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 88.595469425000005, 2024-09-15T00:12:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077021529999999998, 2024-09-15T00:12:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:35.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:35.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524698+0000", 2024-09-15T00:12:35.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6300000000000001e-07 2024-09-15T00:12:35.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524778+0000", 2024-09-15T00:12:35.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0598000000000003e-05 2024-09-15T00:12:35.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524785+0000", 2024-09-15T00:12:35.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6359999999999997e-06 2024-09-15T00:12:35.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524788+0000", 2024-09-15T00:12:35.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2540000000000001e-06 2024-09-15T00:12:35.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524817+0000", 2024-09-15T00:12:35.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8784000000000001e-05 2024-09-15T00:12:35.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524828+0000", 2024-09-15T00:12:35.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-15T00:12:35.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:35.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532360+0000", 2024-09-15T00:12:35.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007531757 2024-09-15T00:12:35.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532400+0000", 2024-09-15T00:12:35.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9835000000000001e-05 2024-09-15T00:12:35.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.127 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.127 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=16'106)", 2024-09-15T00:12:35.127 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:35.128 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 88.594527271999993, 2024-09-15T00:12:35.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067923050000000002, 2024-09-15T00:12:35.128 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.128 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.128 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:35.128 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:35.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:35.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:35.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-15T00:12:35.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525644+0000", 2024-09-15T00:12:35.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.709e-06 2024-09-15T00:12:35.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525651+0000", 2024-09-15T00:12:35.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7970000000000003e-06 2024-09-15T00:12:35.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525654+0000", 2024-09-15T00:12:35.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.433e-06 2024-09-15T00:12:35.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525685+0000", 2024-09-15T00:12:35.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0970000000000003e-05 2024-09-15T00:12:35.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525697+0000", 2024-09-15T00:12:35.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1759e-05 2024-09-15T00:12:35.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:35.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532401+0000", 2024-09-15T00:12:35.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006704425 2024-09-15T00:12:35.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532432+0000", 2024-09-15T00:12:35.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0709e-05 2024-09-15T00:12:35.132 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.132 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.132 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.132 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'106)", 2024-09-15T00:12:35.132 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:35.132 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 88.594065017999995, 2024-09-15T00:12:35.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063539190000000004, 2024-09-15T00:12:35.132 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.132 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.133 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:35.133 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:35.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:35.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526102+0000", 2024-09-15T00:12:35.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:12:35.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526107+0000", 2024-09-15T00:12:35.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:12:35.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526117+0000", 2024-09-15T00:12:35.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7750000000000004e-06 2024-09-15T00:12:35.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526121+0000", 2024-09-15T00:12:35.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4959999999999998e-06 2024-09-15T00:12:35.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526163+0000", 2024-09-15T00:12:35.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1360999999999999e-05 2024-09-15T00:12:35.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.526175+0000", 2024-09-15T00:12:35.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2729e-05 2024-09-15T00:12:35.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:35.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532433+0000", 2024-09-15T00:12:35.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062576999999999997 2024-09-15T00:12:35.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532456+0000", 2024-09-15T00:12:35.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2773e-05 2024-09-15T00:12:35.136 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.136 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.136 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.137 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:35.137 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.504060+0000", 2024-09-15T00:12:35.137 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.61610658, 2024-09-15T00:12:35.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060970119999999997, 2024-09-15T00:12:35.137 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.137 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.137 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.137 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.504060+0000", 2024-09-15T00:12:35.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.504060+0000", 2024-09-15T00:12:35.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.504061+0000", 2024-09-15T00:12:35.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3200000000000005e-07 2024-09-15T00:12:35.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505702+0000", 2024-09-15T00:12:35.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016412149999999999 2024-09-15T00:12:35.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505724+0000", 2024-09-15T00:12:35.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1900999999999999e-05 2024-09-15T00:12:35.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505743+0000", 2024-09-15T00:12:35.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8529000000000002e-05 2024-09-15T00:12:35.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505840+0000", 2024-09-15T00:12:35.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7533000000000003e-05 2024-09-15T00:12:35.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.505854+0000", 2024-09-15T00:12:35.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4070000000000001e-05 2024-09-15T00:12:35.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.510157+0000", 2024-09-15T00:12:35.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043031320000000003 2024-09-15T00:12:35.140 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.140 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.141 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.141 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [2097152~420557,2629632~4096,2650112~4096,2670592~4096,2691072~4096,2711552~4096,2732032~4096,2756608~12288,2777088~12288,2797568~12288,2818048~12288,2838528~12288,2859008~12288,2879488~529715], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3409203, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:35.141 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:35.141 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.606580542, 2024-09-15T00:12:35.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0071155660000000003, 2024-09-15T00:12:35.141 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.141 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.141 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.141 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513587+0000", 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4899999999999995e-07 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514349+0000", 2024-09-15T00:12:35.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00076184799999999997 2024-09-15T00:12:35.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514368+0000", 2024-09-15T00:12:35.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9507000000000001e-05 2024-09-15T00:12:35.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514391+0000", 2024-09-15T00:12:35.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2497000000000002e-05 2024-09-15T00:12:35.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514491+0000", 2024-09-15T00:12:35.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9838000000000002e-05 2024-09-15T00:12:35.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514500+0000", 2024-09-15T00:12:35.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4860000000000003e-06 2024-09-15T00:12:35.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.520702+0000", 2024-09-15T00:12:35.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062017410000000002 2024-09-15T00:12:35.145 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.145 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.145 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.145 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [3409203~654029,4194304~266240,4476928~4096,4497408~4096,4517888~4096,4538368~4096,4558848~4096,4579328~4096,4587520~103731], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4691251, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3409203, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:35.145 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:35.145 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.595722465000001, 2024-09-15T00:12:35.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079084649999999999, 2024-09-15T00:12:35.145 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.145 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.145 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.145 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:35.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:35.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524446+0000", 2024-09-15T00:12:35.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.243e-06 2024-09-15T00:12:35.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525115+0000", 2024-09-15T00:12:35.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00066896999999999996 2024-09-15T00:12:35.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525135+0000", 2024-09-15T00:12:35.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0137999999999999e-05 2024-09-15T00:12:35.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525148+0000", 2024-09-15T00:12:35.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3591000000000001e-05 2024-09-15T00:12:35.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525242+0000", 2024-09-15T00:12:35.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3603999999999994e-05 2024-09-15T00:12:35.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525258+0000", 2024-09-15T00:12:35.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6150999999999998e-05 2024-09-15T00:12:35.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.532353+0000", 2024-09-15T00:12:35.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0070947680000000004 2024-09-15T00:12:35.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.149 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.149 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [4691251~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5739827, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4691251, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:35.149 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:35.149 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.582038904000001, 2024-09-15T00:12:35.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0085554080000000005, 2024-09-15T00:12:35.149 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.149 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.149 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.150 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:35.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:35.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538129+0000", 2024-09-15T00:12:35.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:12:35.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539029+0000", 2024-09-15T00:12:35.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090036599999999999 2024-09-15T00:12:35.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539047+0000", 2024-09-15T00:12:35.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7776999999999999e-05 2024-09-15T00:12:35.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539058+0000", 2024-09-15T00:12:35.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1070000000000001e-05 2024-09-15T00:12:35.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539128+0000", 2024-09-15T00:12:35.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0531999999999995e-05 2024-09-15T00:12:35.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539138+0000", 2024-09-15T00:12:35.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3910000000000004e-06 2024-09-15T00:12:35.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.546683+0000", 2024-09-15T00:12:35.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075455519999999996 2024-09-15T00:12:35.153 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.153 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.153 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.153 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:6618a9b3:::rbd_data.113947ee3a89.0000000000000009:15, version: 59'176, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:6618a9b3:::rbd_data.113947ee3a89.0000000000000009:15@59'176, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:35.153 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:35.153 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.512721108000001, 2024-09-15T00:12:35.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073968319999999999, 2024-09-15T00:12:35.154 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.154 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.154 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.154 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:35.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:35.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607447+0000", 2024-09-15T00:12:35.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2100000000000001e-06 2024-09-15T00:12:35.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607778+0000", 2024-09-15T00:12:35.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033101199999999999 2024-09-15T00:12:35.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607800+0000", 2024-09-15T00:12:35.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2323e-05 2024-09-15T00:12:35.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607815+0000", 2024-09-15T00:12:35.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5073e-05 2024-09-15T00:12:35.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607926+0000", 2024-09-15T00:12:35.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110538 2024-09-15T00:12:35.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607942+0000", 2024-09-15T00:12:35.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5662999999999999e-05 2024-09-15T00:12:35.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.614843+0000", 2024-09-15T00:12:35.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0069010130000000001 2024-09-15T00:12:35.157 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.157 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.157 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.157 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:62885eaf:::rbd_data.1139e6d65aff.0000000000000005:head, version: 62'226, data_included: [0~262144,393216~131072,655360~655360], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:62885eaf:::rbd_data.1139e6d65aff.0000000000000005:head@62'226, size: 1572864, copy_subset: [(0, 1572864)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1310720, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:35.158 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.639003+0000", 2024-09-15T00:12:35.158 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.481163546000001, 2024-09-15T00:12:35.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0070761069999999999, 2024-09-15T00:12:35.158 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.158 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.158 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.158 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639003+0000", 2024-09-15T00:12:35.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639003+0000", 2024-09-15T00:12:35.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639004+0000", 2024-09-15T00:12:35.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:12:35.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639567+0000", 2024-09-15T00:12:35.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00056333800000000003 2024-09-15T00:12:35.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639583+0000", 2024-09-15T00:12:35.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5906999999999999e-05 2024-09-15T00:12:35.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639594+0000", 2024-09-15T00:12:35.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1093e-05 2024-09-15T00:12:35.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639667+0000", 2024-09-15T00:12:35.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3089000000000004e-05 2024-09-15T00:12:35.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639680+0000", 2024-09-15T00:12:35.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2689e-05 2024-09-15T00:12:35.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.646079+0000", 2024-09-15T00:12:35.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063994990000000003 2024-09-15T00:12:35.161 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.161 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.161 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.161 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1 66/65 [PushOp(3:8182271c:::rbd_data.1139e2e18583.000000000000000a:9, version: 37'34, data_included: [1048576~499712,1744896~180224], data_size: 679936, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:8182271c:::rbd_data.1139e2e18583.000000000000000a:9@37'34, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1925120, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:35.162 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.698673+0000", 2024-09-15T00:12:35.162 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.421494078999999, 2024-09-15T00:12:35.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060406629999999999, 2024-09-15T00:12:35.162 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.162 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.162 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.162 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.698673+0000", 2024-09-15T00:12:35.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.698673+0000", 2024-09-15T00:12:35.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.698674+0000", 2024-09-15T00:12:35.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5600000000000004e-07 2024-09-15T00:12:35.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699307+0000", 2024-09-15T00:12:35.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00063296200000000004 2024-09-15T00:12:35.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699324+0000", 2024-09-15T00:12:35.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7115000000000001e-05 2024-09-15T00:12:35.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699336+0000", 2024-09-15T00:12:35.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2047e-05 2024-09-15T00:12:35.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699428+0000", 2024-09-15T00:12:35.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2305000000000005e-05 2024-09-15T00:12:35.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.699443+0000", 2024-09-15T00:12:35.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4386000000000001e-05 2024-09-15T00:12:35.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.704713+0000", 2024-09-15T00:12:35.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052708920000000001 2024-09-15T00:12:35.165 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.165 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.165 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1 66/65 [PushOp(3:83414961:::rbd_data.11392b39946e.0000000000000005:18, version: 62'186, data_included: [0~655360,5144177~35006], data_size: 690366, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:83414961:::rbd_data.11392b39946e.0000000000000005:18@62'186, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.781708+0000", 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.338458586000002, 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0064762520000000001, 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.781708+0000", 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.781708+0000", 2024-09-15T00:12:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.781709+0000", 2024-09-15T00:12:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7000000000000004e-07 2024-09-15T00:12:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782362+0000", 2024-09-15T00:12:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00065286900000000004 2024-09-15T00:12:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782394+0000", 2024-09-15T00:12:35.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1767999999999998e-05 2024-09-15T00:12:35.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782416+0000", 2024-09-15T00:12:35.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2833e-05 2024-09-15T00:12:35.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782497+0000", 2024-09-15T00:12:35.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0706999999999998e-05 2024-09-15T00:12:35.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.782507+0000", 2024-09-15T00:12:35.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3540000000000006e-06 2024-09-15T00:12:35.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.788185+0000", 2024-09-15T00:12:35.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056779509999999997 2024-09-15T00:12:35.169 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.169 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.169 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.169 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1 66/65 [PushOp(3:84afde92:::rbd_data.11392b39946e.0000000000000007:18, version: 62'188, data_included: [0~786432,4718224~368,6422528~38621], data_size: 825421, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:84afde92:::rbd_data.11392b39946e.0000000000000007:18@62'188, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:35.170 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.791890+0000", 2024-09-15T00:12:35.170 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.328276523, 2024-09-15T00:12:35.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006572825, 2024-09-15T00:12:35.170 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.170 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.170 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.170 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.791890+0000", 2024-09-15T00:12:35.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.791890+0000", 2024-09-15T00:12:35.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.791892+0000", 2024-09-15T00:12:35.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9630000000000001e-06 2024-09-15T00:12:35.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792684+0000", 2024-09-15T00:12:35.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00079155399999999998 2024-09-15T00:12:35.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792736+0000", 2024-09-15T00:12:35.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2490000000000001e-05 2024-09-15T00:12:35.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792774+0000", 2024-09-15T00:12:35.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7358999999999999e-05 2024-09-15T00:12:35.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792889+0000", 2024-09-15T00:12:35.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114987 2024-09-15T00:12:35.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.792902+0000", 2024-09-15T00:12:35.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3684e-05 2024-09-15T00:12:35.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.798463+0000", 2024-09-15T00:12:35.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055607879999999997 2024-09-15T00:12:35.173 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.173 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.174 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.174 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.9 77/76 [PushOp(3:9229c9d9:::rbd_data.1139766160da.0000000000000000:13, version: 57'689, data_included: [0~65536,229376~65536,385024~5116,458752~32768,589824~131072,851968~131072,1081344~32768,1146880~65536,1310720~32768,1409024~32768,1507328~49152,1572864~32768,1638400~32768,1802240~32768], data_size: 742396, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:9229c9d9:::rbd_data.1139766160da.0000000000000000:13@57'689, size: 1835008, copy_subset: [(0, 1835008)], clone_subset: {}, snapset: 13=[]:{13: [13]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1835008, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:35.174 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.386155+0000", 2024-09-15T00:12:35.174 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.734012297, 2024-09-15T00:12:35.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062156169999999997, 2024-09-15T00:12:35.174 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.174 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.174 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.174 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386155+0000", 2024-09-15T00:12:35.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386155+0000", 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386155+0000", 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0999999999999997e-07 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386629+0000", 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047386200000000002 2024-09-15T00:12:35.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386660+0000", 2024-09-15T00:12:35.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0834999999999999e-05 2024-09-15T00:12:35.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386686+0000", 2024-09-15T00:12:35.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5488999999999999e-05 2024-09-15T00:12:35.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386804+0000", 2024-09-15T00:12:35.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011882700000000001 2024-09-15T00:12:35.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.386820+0000", 2024-09-15T00:12:35.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5330000000000001e-05 2024-09-15T00:12:35.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.392370+0000", 2024-09-15T00:12:35.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055503640000000003 2024-09-15T00:12:35.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.177 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.178 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.9 77/76 [PushOp(3:95907793:::rbd_data.11396cb1f0e4.0000000000000007:head, version: 59'729, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:95907793:::rbd_data.11396cb1f0e4.0000000000000007:head@59'729, size: 5701632, copy_subset: [(0, 5701632)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:35.178 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.428602+0000", 2024-09-15T00:12:35.178 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.691565279000001, 2024-09-15T00:12:35.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006605606, 2024-09-15T00:12:35.178 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.178 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.178 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.178 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.428602+0000", 2024-09-15T00:12:35.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.428602+0000", 2024-09-15T00:12:35.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.428603+0000", 2024-09-15T00:12:35.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1000000000000001e-06 2024-09-15T00:12:35.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.429252+0000", 2024-09-15T00:12:35.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00064978799999999999 2024-09-15T00:12:35.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.429282+0000", 2024-09-15T00:12:35.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9352e-05 2024-09-15T00:12:35.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.429301+0000", 2024-09-15T00:12:35.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9436000000000002e-05 2024-09-15T00:12:35.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.429400+0000", 2024-09-15T00:12:35.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8862000000000003e-05 2024-09-15T00:12:35.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.429415+0000", 2024-09-15T00:12:35.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5145e-05 2024-09-15T00:12:35.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.435207+0000", 2024-09-15T00:12:35.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005791923 2024-09-15T00:12:35.181 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.181 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.181 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.182 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:125615 3.f e77/68 3:f382b49c:::rbd_data.11396481e1ec.0000000000000004:head v 77'241, mlcod=74'239)", 2024-09-15T00:12:35.182 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.839171+0000", 2024-09-15T00:12:35.182 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.280996270999999, 2024-09-15T00:12:35.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0072384550000000004, 2024-09-15T00:12:35.182 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.182 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.182 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:35.182 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.839171+0000", 2024-09-15T00:12:35.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.839171+0000", 2024-09-15T00:12:35.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.839172+0000", 2024-09-15T00:12:35.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3769999999999999e-06 2024-09-15T00:12:35.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840892+0000", 2024-09-15T00:12:35.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001720271 2024-09-15T00:12:35.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840905+0000", 2024-09-15T00:12:35.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2461e-05 2024-09-15T00:12:35.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840913+0000", 2024-09-15T00:12:35.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1990000000000006e-06 2024-09-15T00:12:35.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840962+0000", 2024-09-15T00:12:35.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8760000000000001e-05 2024-09-15T00:12:35.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840979+0000", 2024-09-15T00:12:35.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7280999999999999e-05 2024-09-15T00:12:35.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:35.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.846347+0000", 2024-09-15T00:12:35.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005367925 2024-09-15T00:12:35.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.846409+0000", 2024-09-15T00:12:35.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2181000000000007e-05 2024-09-15T00:12:35.186 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.186 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.186 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.186 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:125618 3.10 e77/68 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head v 77'459, mlcod=74'457)", 2024-09-15T00:12:35.186 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.844048+0000", 2024-09-15T00:12:35.186 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.276118994000001, 2024-09-15T00:12:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0064617310000000001, 2024-09-15T00:12:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.844048+0000", 2024-09-15T00:12:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.844048+0000", 2024-09-15T00:12:35.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.844048+0000", 2024-09-15T00:12:35.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.34e-07 2024-09-15T00:12:35.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.845223+0000", 2024-09-15T00:12:35.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011745569999999999 2024-09-15T00:12:35.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.845235+0000", 2024-09-15T00:12:35.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2602999999999999e-05 2024-09-15T00:12:35.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.845243+0000", 2024-09-15T00:12:35.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9990000000000008e-06 2024-09-15T00:12:35.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.845315+0000", 2024-09-15T00:12:35.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2059999999999998e-05 2024-09-15T00:12:35.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.845341+0000", 2024-09-15T00:12:35.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5933999999999999e-05 2024-09-15T00:12:35.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:35.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.850456+0000", 2024-09-15T00:12:35.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051145269999999998 2024-09-15T00:12:35.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.850510+0000", 2024-09-15T00:12:35.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3616999999999998e-05 2024-09-15T00:12:35.191 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.191 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.191 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.191 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:127612 3.1b e78/69 3:d99663fc:::rbd_data.11396481e1ec.0000000000000000:head v 78'323, mlcod=73'321)", 2024-09-15T00:12:35.191 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.120437+0000", 2024-09-15T00:12:35.191 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.999729475000001, 2024-09-15T00:12:35.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0064173049999999999, 2024-09-15T00:12:35.191 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.191 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.191 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:35.191 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.120437+0000", 2024-09-15T00:12:35.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.120437+0000", 2024-09-15T00:12:35.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.120438+0000", 2024-09-15T00:12:35.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.73e-07 2024-09-15T00:12:35.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.121533+0000", 2024-09-15T00:12:35.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001095897 2024-09-15T00:12:35.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.121548+0000", 2024-09-15T00:12:35.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4888e-05 2024-09-15T00:12:35.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.121558+0000", 2024-09-15T00:12:35.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3260000000000002e-06 2024-09-15T00:12:35.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.121608+0000", 2024-09-15T00:12:35.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9948999999999999e-05 2024-09-15T00:12:35.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.121622+0000", 2024-09-15T00:12:35.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4898e-05 2024-09-15T00:12:35.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:35.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.126807+0000", 2024-09-15T00:12:35.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005184676 2024-09-15T00:12:35.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.126855+0000", 2024-09-15T00:12:35.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7497999999999998e-05 2024-09-15T00:12:35.195 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.195 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.195 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.195 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:127618 3.1 e78/67 3:84066653:::rbd_data.11396481e1ec.0000000000000003:head v 78'284, mlcod=77'282)", 2024-09-15T00:12:35.196 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.124454+0000", 2024-09-15T00:12:35.196 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.995713144, 2024-09-15T00:12:35.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0064568020000000002, 2024-09-15T00:12:35.196 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.196 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.196 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:35.196 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.124454+0000", 2024-09-15T00:12:35.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.124454+0000", 2024-09-15T00:12:35.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.124455+0000", 2024-09-15T00:12:35.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2139999999999999e-06 2024-09-15T00:12:35.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.125404+0000", 2024-09-15T00:12:35.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00094884799999999997 2024-09-15T00:12:35.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.125422+0000", 2024-09-15T00:12:35.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7829000000000002e-05 2024-09-15T00:12:35.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.125431+0000", 2024-09-15T00:12:35.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5670000000000001e-06 2024-09-15T00:12:35.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.125497+0000", 2024-09-15T00:12:35.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5419999999999994e-05 2024-09-15T00:12:35.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.125519+0000", 2024-09-15T00:12:35.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2297e-05 2024-09-15T00:12:35.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:35.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.130879+0000", 2024-09-15T00:12:35.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053604350000000002 2024-09-15T00:12:35.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.130910+0000", 2024-09-15T00:12:35.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1192000000000002e-05 2024-09-15T00:12:35.199 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.199 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.200 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.200 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133834 3.1b e85/69 3:da9a78a3:::rbd_data.11395dfcf2b8.0000000000000000:head v 85'356, mlcod=85'354)", 2024-09-15T00:12:35.200 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:35.200 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5568305379999998, 2024-09-15T00:12:35.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026079054000000001, 2024-09-15T00:12:35.200 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.200 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.200 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:35.200 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:35.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:35.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563338+0000", 2024-09-15T00:12:35.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.418e-06 2024-09-15T00:12:35.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569385+0000", 2024-09-15T00:12:35.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006047335 2024-09-15T00:12:35.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569418+0000", 2024-09-15T00:12:35.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2957999999999999e-05 2024-09-15T00:12:35.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569432+0000", 2024-09-15T00:12:35.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3718e-05 2024-09-15T00:12:35.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569519+0000", 2024-09-15T00:12:35.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7022999999999997e-05 2024-09-15T00:12:35.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569540+0000", 2024-09-15T00:12:35.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1426000000000001e-05 2024-09-15T00:12:35.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:35.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589362+0000", 2024-09-15T00:12:35.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019821398 2024-09-15T00:12:35.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589415+0000", 2024-09-15T00:12:35.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3777999999999999e-05 2024-09-15T00:12:35.204 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.204 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.204 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.204 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133836 3.1 e85/67 3:849f9a98:::rbd_data.11395dfcf2b8.0000000000000002:head v 85'304, mlcod=84'301)", 2024-09-15T00:12:35.204 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:35.204 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5491356280000002, 2024-09-15T00:12:35.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023231600000000002, 2024-09-15T00:12:35.205 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.205 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.205 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:35.205 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:35.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:35.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571032+0000", 2024-09-15T00:12:35.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-15T00:12:35.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576635+0000", 2024-09-15T00:12:35.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056028700000000002 2024-09-15T00:12:35.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576650+0000", 2024-09-15T00:12:35.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.505e-05 2024-09-15T00:12:35.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576658+0000", 2024-09-15T00:12:35.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6879999999999999e-06 2024-09-15T00:12:35.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576717+0000", 2024-09-15T00:12:35.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175000000000002e-05 2024-09-15T00:12:35.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576742+0000", 2024-09-15T00:12:35.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4734000000000001e-05 2024-09-15T00:12:35.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:35.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594234+0000", 2024-09-15T00:12:35.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017492527000000001 2024-09-15T00:12:35.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594263+0000", 2024-09-15T00:12:35.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8629e-05 2024-09-15T00:12:35.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.209 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.209 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.209 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:35.209 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:12:35.211 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:35.211 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:35.209+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:35.311 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:35.310+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:35.364 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:35.365 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:35.374 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:35.394 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-15T00:12:35.412 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:35.410+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:35.419 INFO:teuthology.orchestra.run.smithi171.stdout:4701 write 0x16ff301 thru 0x170a86e (0xb56e bytes) 2024-09-15T00:12:35.450 INFO:teuthology.orchestra.run.smithi171.stdout:4702 write 0xcba9d05 thru 0xcbb43dd (0xa6d9 bytes) 2024-09-15T00:12:35.483 INFO:teuthology.orchestra.run.smithi171.stdout:4704 trunc from 0xe7813ee to 0x24e0310 2024-09-15T00:12:35.513 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:35.512+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:35.516 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:35.516 INFO:teuthology.orchestra.run.smithi171.stdout:4707 punch from 0xe42f44 to 0xe4bfec, (0x90a8 bytes) 2024-09-15T00:12:35.521 INFO:teuthology.orchestra.run.smithi171.stdout:4708 trunc from 0x24e0310 to 0x6fb8cc3 2024-09-15T00:12:35.524 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:35.524 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:35.524 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:35.524 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:35.524 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:35.528 INFO:teuthology.orchestra.run.smithi171.stdout:4709 read 0x2bd2926 thru 0x2bde40a (0xbae5 bytes) 2024-09-15T00:12:35.528 INFO:teuthology.orchestra.run.smithi171.stdout:4710 punch from 0x1f7008e to 0x1f732fc, (0x326e bytes) 2024-09-15T00:12:35.528 INFO:teuthology.orchestra.run.smithi171.stdout:4712 trunc from 0x6fb8cc3 to 0x6e1dc1a 2024-09-15T00:12:35.534 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:35.545 INFO:teuthology.orchestra.run.smithi171.stdout:4713 writesame 0xbae9ed9 thru 0xbaee69b data_size 0x1(0x47c3 bytes) 2024-09-15T00:12:35.545 INFO:teuthology.orchestra.run.smithi171.stdout:4714 writesame 0x8e17a6 thru 0x8eaba4 data_size 0x3(0x93ff bytes) 2024-09-15T00:12:35.569 INFO:teuthology.orchestra.run.smithi171.stdout:4715 trunc from 0xbaee69c to 0xbb5bbd6 2024-09-15T00:12:35.575 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:35.575 INFO:teuthology.orchestra.run.smithi171.stdout:4718 punch from 0x97f4339 to 0x97f850d, (0x41d4 bytes) 2024-09-15T00:12:35.575 INFO:teuthology.orchestra.run.smithi171.stdout:4720 write 0x674734f thru 0x6755be6 (0xe898 bytes) 2024-09-15T00:12:35.590 INFO:teuthology.orchestra.run.smithi171.stdout:4722 writesame 0xd30bd56 thru 0xd315382 data_size 0x37(0x962d bytes) 2024-09-15T00:12:35.590 INFO:teuthology.orchestra.run.smithi171.stdout:4723 writesame 0x6d4a9dd thru 0x6d51069 data_size 0x3(0x668d bytes) 2024-09-15T00:12:35.597 INFO:teuthology.orchestra.run.smithi171.stdout:4725 punch from 0xc3f3e00 to 0xc3f87ce, (0x49ce bytes) 2024-09-15T00:12:35.597 INFO:teuthology.orchestra.run.smithi171.stdout:4727 punch from 0x5d6a773 to 0x5d72bbb, (0x8448 bytes) 2024-09-15T00:12:35.603 INFO:teuthology.orchestra.run.smithi171.stdout:4729 write 0xd9680d2 thru 0xd976cfe (0xec2d bytes) 2024-09-15T00:12:35.609 INFO:teuthology.orchestra.run.smithi171.stdout:4730 punch from 0x1cdb082 to 0x1ce7989, (0xc907 bytes) 2024-09-15T00:12:35.609 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:35.609 INFO:teuthology.orchestra.run.smithi171.stdout:4732 read 0x7417c34 thru 0x741b761 (0x3b2e bytes) 2024-09-15T00:12:35.609 INFO:teuthology.orchestra.run.smithi171.stdout:4734 write 0x38f2d24 thru 0x38f2d37 (0x14 bytes) 2024-09-15T00:12:35.611 INFO:teuthology.orchestra.run.smithi171.stdout:4735 read 0x3f8942f thru 0x3f90aeb (0x76bd bytes) 2024-09-15T00:12:35.611 INFO:teuthology.orchestra.run.smithi171.stdout:4736 punch from 0x526abff to 0x526b9ff, (0xe00 bytes) 2024-09-15T00:12:35.612 INFO:teuthology.orchestra.run.smithi171.stdout:4738 writesame 0x7b16364 thru 0x7b25d9c data_size 0x1(0xfa39 bytes) 2024-09-15T00:12:35.616 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:35.612+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:35.652 INFO:teuthology.orchestra.run.smithi171.stdout:4739 read 0xa8098c thru 0xa869e5 (0x605a bytes) 2024-09-15T00:12:35.672 INFO:teuthology.orchestra.run.smithi171.stdout:4740 trunc from 0xd976cff to 0xb86e7 2024-09-15T00:12:35.683 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:35.683 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:35.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:35.683 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:35.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.683 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:35.683 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.500560820999993, 2024-09-15T00:12:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:12:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:12:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:12:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:12:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:12:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:12:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:12:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:12:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:12:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:12:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:12:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:12:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:12:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:12:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:12:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:12:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:12:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:12:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:12:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:12:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:12:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:12:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:12:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:12:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.770719642000003, 2024-09-15T00:12:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:12:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:12:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:12:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:12:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:12:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:12:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:12:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:12:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:12:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:12:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:12:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:12:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:12:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:12:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:12:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:12:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:12:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:12:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:12:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:12:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:12:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:12:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:12:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:12:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:12:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.763761279000001, 2024-09-15T00:12:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:12:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:12:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:12:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:12:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:12:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:12:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:12:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:12:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:12:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:12:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:12:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:12:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:12:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:12:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:12:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.016572441999998, 2024-09-15T00:12:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:12:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:12:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:12:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:12:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:12:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:12:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:12:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:12:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:12:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:12:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:12:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:12:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:12:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:12:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:12:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:12:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:12:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:12:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:12:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:12:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:12:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:12:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:12:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:12:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97493 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.826989722, 2024-09-15T00:12:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033259141999999998, 2024-09-15T00:12:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97493 2024-09-15T00:12:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-15T00:12:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853401+0000", 2024-09-15T00:12:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:12:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853407+0000", 2024-09-15T00:12:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-15T00:12:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853411+0000", 2024-09-15T00:12:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.481e-06 2024-09-15T00:12:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886544+0000", 2024-09-15T00:12:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033132861999999999 2024-09-15T00:12:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886587+0000", 2024-09-15T00:12:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2942000000000003e-05 2024-09-15T00:12:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886655+0000", 2024-09-15T00:12:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7711000000000003e-05 2024-09-15T00:12:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97494 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.826968599000001, 2024-09-15T00:12:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033392390000000001, 2024-09-15T00:12:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97494 2024-09-15T00:12:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-15T00:12:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853419+0000", 2024-09-15T00:12:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.917e-06 2024-09-15T00:12:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853425+0000", 2024-09-15T00:12:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:12:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853433+0000", 2024-09-15T00:12:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8120000000000008e-06 2024-09-15T00:12:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886680+0000", 2024-09-15T00:12:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033247207000000001 2024-09-15T00:12:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886726+0000", 2024-09-15T00:12:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5902000000000002e-05 2024-09-15T00:12:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886809+0000", 2024-09-15T00:12:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3475999999999998e-05 2024-09-15T00:12:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97495 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.826947646000001, 2024-09-15T00:12:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033640503000000002, 2024-09-15T00:12:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97495 2024-09-15T00:12:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:12:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853440+0000", 2024-09-15T00:12:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.804e-06 2024-09-15T00:12:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853446+0000", 2024-09-15T00:12:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5559999999999998e-06 2024-09-15T00:12:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853450+0000", 2024-09-15T00:12:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9519999999999996e-06 2024-09-15T00:12:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886904+0000", 2024-09-15T00:12:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033454207999999999 2024-09-15T00:12:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887007+0000", 2024-09-15T00:12:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102904 2024-09-15T00:12:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887078+0000", 2024-09-15T00:12:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1719999999999998e-05 2024-09-15T00:12:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97496 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.826930548, 2024-09-15T00:12:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033802431000000001, 2024-09-15T00:12:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97496 2024-09-15T00:12:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.53e-07 2024-09-15T00:12:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853462+0000", 2024-09-15T00:12:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-15T00:12:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853468+0000", 2024-09-15T00:12:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3609999999999997e-06 2024-09-15T00:12:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853473+0000", 2024-09-15T00:12:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-15T00:12:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887111+0000", 2024-09-15T00:12:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033638376999999997 2024-09-15T00:12:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887178+0000", 2024-09-15T00:12:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6877000000000004e-05 2024-09-15T00:12:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887257+0000", 2024-09-15T00:12:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9616999999999993e-05 2024-09-15T00:12:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97587 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.826907314, 2024-09-15T00:12:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033973642999999998, 2024-09-15T00:12:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97587 2024-09-15T00:12:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853479+0000", 2024-09-15T00:12:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:12:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853481+0000", 2024-09-15T00:12:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.136e-06 2024-09-15T00:12:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853487+0000", 2024-09-15T00:12:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2439999999999998e-06 2024-09-15T00:12:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853492+0000", 2024-09-15T00:12:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:12:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887284+0000", 2024-09-15T00:12:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033791827000000003 2024-09-15T00:12:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887361+0000", 2024-09-15T00:12:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6983000000000005e-05 2024-09-15T00:12:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887452+0000", 2024-09-15T00:12:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1236999999999994e-05 2024-09-15T00:12:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97588 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.826887157000002, 2024-09-15T00:12:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034134617999999999, 2024-09-15T00:12:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97588 2024-09-15T00:12:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853499+0000", 2024-09-15T00:12:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:12:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853501+0000", 2024-09-15T00:12:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.26e-06 2024-09-15T00:12:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853507+0000", 2024-09-15T00:12:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2899999999999999e-06 2024-09-15T00:12:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853512+0000", 2024-09-15T00:12:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8330000000000001e-06 2024-09-15T00:12:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887495+0000", 2024-09-15T00:12:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033982529999999997 2024-09-15T00:12:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887548+0000", 2024-09-15T00:12:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3445000000000001e-05 2024-09-15T00:12:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887633+0000", 2024-09-15T00:12:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4890999999999997e-05 2024-09-15T00:12:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126866 3.6 3.973c9fa6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.723565212, 2024-09-15T00:12:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153301000000003, 2024-09-15T00:12:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126866 2024-09-15T00:12:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956821+0000", 2024-09-15T00:12:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.91e-07 2024-09-15T00:12:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956822+0000", 2024-09-15T00:12:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.705e-06 2024-09-15T00:12:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956828+0000", 2024-09-15T00:12:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6760000000000004e-06 2024-09-15T00:12:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956833+0000", 2024-09-15T00:12:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5379999999999996e-06 2024-09-15T00:12:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989970+0000", 2024-09-15T00:12:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033137560000000003 2024-09-15T00:12:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989974+0000", 2024-09-15T00:12:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-15T00:12:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126951 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.723439529, 2024-09-15T00:12:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033152585999999998, 2024-09-15T00:12:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126951 2024-09-15T00:12:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-07 2024-09-15T00:12:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956953+0000", 2024-09-15T00:12:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9310000000000002e-06 2024-09-15T00:12:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956957+0000", 2024-09-15T00:12:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5379999999999998e-06 2024-09-15T00:12:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956959+0000", 2024-09-15T00:12:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6599999999999999e-06 2024-09-15T00:12:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990095+0000", 2024-09-15T00:12:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033135855999999998 2024-09-15T00:12:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990099+0000", 2024-09-15T00:12:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4010000000000001e-06 2024-09-15T00:12:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126952 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.72342254, 2024-09-15T00:12:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033156066999999997, 2024-09-15T00:12:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126952 2024-09-15T00:12:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:12:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956965+0000", 2024-09-15T00:12:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3379999999999999e-06 2024-09-15T00:12:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956968+0000", 2024-09-15T00:12:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0419999999999998e-06 2024-09-15T00:12:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956970+0000", 2024-09-15T00:12:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4899999999999999e-06 2024-09-15T00:12:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990113+0000", 2024-09-15T00:12:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033142642 2024-09-15T00:12:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990119+0000", 2024-09-15T00:12:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2979999999999997e-06 2024-09-15T00:12:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126954 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.723377044999999, 2024-09-15T00:12:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033158449, 2024-09-15T00:12:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126954 2024-09-15T00:12:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-15T00:12:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957010+0000", 2024-09-15T00:12:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3659999999999999e-06 2024-09-15T00:12:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957013+0000", 2024-09-15T00:12:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1140000000000002e-06 2024-09-15T00:12:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957016+0000", 2024-09-15T00:12:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1209999999999998e-06 2024-09-15T00:12:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990163+0000", 2024-09-15T00:12:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033146451 2024-09-15T00:12:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990167+0000", 2024-09-15T00:12:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0579999999999999e-06 2024-09-15T00:12:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126955 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.723365831000001, 2024-09-15T00:12:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033160617000000003, 2024-09-15T00:12:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126955 2024-09-15T00:12:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-15T00:12:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957021+0000", 2024-09-15T00:12:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-15T00:12:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957024+0000", 2024-09-15T00:12:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1599999999999998e-06 2024-09-15T00:12:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957027+0000", 2024-09-15T00:12:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000001e-06 2024-09-15T00:12:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990177+0000", 2024-09-15T00:12:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033149741000000003 2024-09-15T00:12:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990180+0000", 2024-09-15T00:12:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4910000000000001e-06 2024-09-15T00:12:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126956 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.72335571, 2024-09-15T00:12:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033163633999999997, 2024-09-15T00:12:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126956 2024-09-15T00:12:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957031+0000", 2024-09-15T00:12:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-06 2024-09-15T00:12:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957034+0000", 2024-09-15T00:12:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9960000000000002e-06 2024-09-15T00:12:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957037+0000", 2024-09-15T00:12:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6419999999999999e-06 2024-09-15T00:12:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990190+0000", 2024-09-15T00:12:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153152999999998 2024-09-15T00:12:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990193+0000", 2024-09-15T00:12:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6069999999999999e-06 2024-09-15T00:12:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126957 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.723346060000001, 2024-09-15T00:12:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033166442999999997, 2024-09-15T00:12:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126957 2024-09-15T00:12:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957040+0000", 2024-09-15T00:12:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:12:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957041+0000", 2024-09-15T00:12:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0440000000000001e-06 2024-09-15T00:12:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957044+0000", 2024-09-15T00:12:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0170000000000001e-06 2024-09-15T00:12:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957048+0000", 2024-09-15T00:12:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0319999999999997e-06 2024-09-15T00:12:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990202+0000", 2024-09-15T00:12:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033154640999999999 2024-09-15T00:12:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990206+0000", 2024-09-15T00:12:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.507e-06 2024-09-15T00:12:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126958 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.723334830000001, 2024-09-15T00:12:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033167613999999998, 2024-09-15T00:12:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126958 2024-09-15T00:12:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:12:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957053+0000", 2024-09-15T00:12:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.756e-06 2024-09-15T00:12:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957059+0000", 2024-09-15T00:12:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2770000000000002e-06 2024-09-15T00:12:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957063+0000", 2024-09-15T00:12:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2459999999999997e-06 2024-09-15T00:12:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990215+0000", 2024-09-15T00:12:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033151509000000003 2024-09-15T00:12:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990218+0000", 2024-09-15T00:12:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5099999999999999e-06 2024-09-15T00:12:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126959 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.723315826, 2024-09-15T00:12:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033161673000000003, 2024-09-15T00:12:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126959 2024-09-15T00:12:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:12:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957072+0000", 2024-09-15T00:12:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2239999999999998e-06 2024-09-15T00:12:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957079+0000", 2024-09-15T00:12:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.19e-06 2024-09-15T00:12:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957083+0000", 2024-09-15T00:12:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:12:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990228+0000", 2024-09-15T00:12:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033144818 2024-09-15T00:12:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990231+0000", 2024-09-15T00:12:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:12:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126960 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.723297019, 2024-09-15T00:12:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033155166, 2024-09-15T00:12:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126960 2024-09-15T00:12:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:12:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957091+0000", 2024-09-15T00:12:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:12:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957097+0000", 2024-09-15T00:12:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6050000000000003e-06 2024-09-15T00:12:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957102+0000", 2024-09-15T00:12:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5539999999999999e-06 2024-09-15T00:12:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990240+0000", 2024-09-15T00:12:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033138289000000001 2024-09-15T00:12:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990244+0000", 2024-09-15T00:12:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4740000000000001e-06 2024-09-15T00:12:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:35.775 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:35.776 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:35.778 INFO:teuthology.orchestra.run.smithi171.stdout:4742 write 0xc04bffb thru 0xc04d643 (0x1649 bytes) 2024-09-15T00:12:35.778 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:35.778 INFO:teuthology.orchestra.run.smithi171.stdout:4746 read 0x9f3a9d2 thru 0x9f3f782 (0x4db1 bytes) 2024-09-15T00:12:35.778 INFO:teuthology.orchestra.run.smithi171.stdout:4747 writesame 0x30780fd thru 0x30869ee data_size 0x9(0xe8f2 bytes) 2024-09-15T00:12:35.778 INFO:teuthology.orchestra.run.smithi171.stdout:4748 write 0x14b8b76 thru 0x14c507f (0xc50a bytes) 2024-09-15T00:12:35.778 INFO:teuthology.orchestra.run.smithi171.stdout:4749 punch from 0x57436ee to 0x574f1bf, (0xbad1 bytes) 2024-09-15T00:12:35.778 INFO:teuthology.orchestra.run.smithi171.stdout:4751 punch from 0x6d696df to 0x6d6a314, (0xc35 bytes) 2024-09-15T00:12:35.778 INFO:teuthology.orchestra.run.smithi171.stdout:4752 write 0x675f613 thru 0x676906d (0x9a5b bytes) 2024-09-15T00:12:35.778 INFO:teuthology.orchestra.run.smithi171.stdout:4753 read 0x675b65b thru 0x6764476 (0x8e1c bytes) 2024-09-15T00:12:35.778 INFO:teuthology.orchestra.run.smithi171.stdout:4754 write 0x42941f0 thru 0x42a2d1a (0xeb2b bytes) 2024-09-15T00:12:35.778 INFO:teuthology.orchestra.run.smithi171.stdout:4755 write 0xaf55281 thru 0xaf61679 (0xc3f9 bytes) 2024-09-15T00:12:35.779 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:35.774+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:35.785 INFO:teuthology.orchestra.run.smithi171.stdout:4756 read 0x16de6ec thru 0x16eacf8 (0xc60d bytes) 2024-09-15T00:12:35.785 INFO:teuthology.orchestra.run.smithi171.stdout:4757 punch from 0xa0c1f7d to 0xa0cbe62, (0x9ee5 bytes) 2024-09-15T00:12:35.785 INFO:teuthology.orchestra.run.smithi171.stdout:4758 clone 39 order 19 su 131072 sc 4 2024-09-15T00:12:35.877 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:35.876+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:35.888 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:35.888 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:35.896 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:35.978 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:35.976+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:36.000 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:36.000 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:36.000 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:36.000 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:36.000 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:36.009 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:36.079 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:36.078+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:36.114 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:36.114 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:36.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:36.114 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:36.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.114 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:12:36.114 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:36.114 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.868625723000001, 2024-09-15T00:12:36.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:12:36.115 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.115 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.115 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:36.115 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.115 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.115 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.115 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:12:36.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.115 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:36.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:36.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:12:36.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:12:36.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:12:36.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:12:36.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:12:36.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:12:36.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:12:36.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:12:36.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:12:36.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:12:36.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:12:36.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:12:36.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:36.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:12:36.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:12:36.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:36.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:12:36.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:12:36.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:36.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:12:36.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:12:36.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:36.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:12:36.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:12:36.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:12:36.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:12:36.120 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.120 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.120 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.120 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58878 3.8 3:12456482:::rbd_data.1139766160da.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:36.120 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:36.120 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.822320138999999, 2024-09-15T00:12:36.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061956199000000003, 2024-09-15T00:12:36.121 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.121 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.121 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:36.121 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.121 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.121 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.121 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58878 2024-09-15T00:12:36.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.121 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:36.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:36.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289151+0000", 2024-09-15T00:12:36.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:12:36.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289473+0000", 2024-09-15T00:12:36.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032183800000000002 2024-09-15T00:12:36.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289487+0000", 2024-09-15T00:12:36.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-05 2024-09-15T00:12:36.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289493+0000", 2024-09-15T00:12:36.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4489999999999996e-06 2024-09-15T00:12:36.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289542+0000", 2024-09-15T00:12:36.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8844999999999997e-05 2024-09-15T00:12:36.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289642+0000", 2024-09-15T00:12:36.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9607999999999999e-05 2024-09-15T00:12:36.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:36.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.293908+0000", 2024-09-15T00:12:36.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659350000000002 2024-09-15T00:12:36.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:36.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351022+0000", 2024-09-15T00:12:36.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057114535000000001 2024-09-15T00:12:36.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:36.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351033+0000", 2024-09-15T00:12:36.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0263e-05 2024-09-15T00:12:36.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:36.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351070+0000", 2024-09-15T00:12:36.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7666000000000002e-05 2024-09-15T00:12:36.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351106+0000", 2024-09-15T00:12:36.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999997e-05 2024-09-15T00:12:36.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.127 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.127 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58879 3.7 3:e169894d:::rbd_data.1139766160da.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:36.127 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:36.127 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.819968651000003, 2024-09-15T00:12:36.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043716294000000003, 2024-09-15T00:12:36.128 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.128 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.128 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:36.128 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.128 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.128 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.128 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58879 2024-09-15T00:12:36.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.128 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:36.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:36.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291502+0000", 2024-09-15T00:12:36.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-15T00:12:36.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291664+0000", 2024-09-15T00:12:36.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016216599999999999 2024-09-15T00:12:36.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291674+0000", 2024-09-15T00:12:36.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0276e-05 2024-09-15T00:12:36.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291680+0000", 2024-09-15T00:12:36.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2580000000000003e-06 2024-09-15T00:12:36.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291732+0000", 2024-09-15T00:12:36.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2114999999999998e-05 2024-09-15T00:12:36.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291866+0000", 2024-09-15T00:12:36.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013416900000000001 2024-09-15T00:12:36.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.296426+0000", 2024-09-15T00:12:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045595779999999999 2024-09-15T00:12:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335075+0000", 2024-09-15T00:12:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038649347000000001 2024-09-15T00:12:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335097+0000", 2024-09-15T00:12:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2565000000000002e-05 2024-09-15T00:12:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:36.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335161+0000", 2024-09-15T00:12:36.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3928000000000002e-05 2024-09-15T00:12:36.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335217+0000", 2024-09-15T00:12:36.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6042000000000003e-05 2024-09-15T00:12:36.133 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.133 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.133 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.134 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58880 3.8 3:13d91cfd:::rbd_data.1139766160da.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:36.134 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:36.134 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.816576544, 2024-09-15T00:12:36.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056331463999999998, 2024-09-15T00:12:36.134 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.134 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.134 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:36.134 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.134 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.134 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.134 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58880 2024-09-15T00:12:36.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.134 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:36.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:36.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294896+0000", 2024-09-15T00:12:36.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.356e-06 2024-09-15T00:12:36.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295288+0000", 2024-09-15T00:12:36.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039223200000000001 2024-09-15T00:12:36.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295304+0000", 2024-09-15T00:12:36.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5937e-05 2024-09-15T00:12:36.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295312+0000", 2024-09-15T00:12:36.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1489999999999994e-06 2024-09-15T00:12:36.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333162+0000", 2024-09-15T00:12:36.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037849840000000003 2024-09-15T00:12:36.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333269+0000", 2024-09-15T00:12:36.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010749800000000001 2024-09-15T00:12:36.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:36.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.336198+0000", 2024-09-15T00:12:36.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029284739999999999 2024-09-15T00:12:36.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:36.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351137+0000", 2024-09-15T00:12:36.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014939103 2024-09-15T00:12:36.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:36.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351150+0000", 2024-09-15T00:12:36.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3542e-05 2024-09-15T00:12:36.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:36.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351191+0000", 2024-09-15T00:12:36.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0791000000000003e-05 2024-09-15T00:12:36.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351225+0000", 2024-09-15T00:12:36.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3541999999999998e-05 2024-09-15T00:12:36.139 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.139 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.140 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.140 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58878 3.8 e56/23 ondisk, result = 0)", 2024-09-15T00:12:36.140 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:36.140 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.810046761000002, 2024-09-15T00:12:36.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049689096000000002, 2024-09-15T00:12:36.140 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.140 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.140 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:36.140 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:36.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:36.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301425+0000", 2024-09-15T00:12:36.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.782e-06 2024-09-15T00:12:36.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301433+0000", 2024-09-15T00:12:36.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689999999999994e-06 2024-09-15T00:12:36.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301449+0000", 2024-09-15T00:12:36.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5767e-05 2024-09-15T00:12:36.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301458+0000", 2024-09-15T00:12:36.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:12:36.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351009+0000", 2024-09-15T00:12:36.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049551523 2024-09-15T00:12:36.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351017+0000", 2024-09-15T00:12:36.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8359999999999992e-06 2024-09-15T00:12:36.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351112+0000", 2024-09-15T00:12:36.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5173999999999994e-05 2024-09-15T00:12:36.143 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.144 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.144 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127004 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 6881280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:36.144 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:36.144 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.143141297, 2024-09-15T00:12:36.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735720000000003, 2024-09-15T00:12:36.144 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.144 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.144 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:36.144 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.144 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.145 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127004 2024-09-15T00:12:36.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.145 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:36.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:36.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:36.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:12:36.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968331+0000", 2024-09-15T00:12:36.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.849e-06 2024-09-15T00:12:36.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968334+0000", 2024-09-15T00:12:36.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7689999999999998e-06 2024-09-15T00:12:36.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968338+0000", 2024-09-15T00:12:36.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2490000000000001e-06 2024-09-15T00:12:36.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008987+0000", 2024-09-15T00:12:36.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040649117999999998 2024-09-15T00:12:36.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009027+0000", 2024-09-15T00:12:36.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0357e-05 2024-09-15T00:12:36.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009064+0000", 2024-09-15T00:12:36.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7036000000000003e-05 2024-09-15T00:12:36.148 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.148 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.149 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127005 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7004160~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:36.149 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:36.149 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.143128745, 2024-09-15T00:12:36.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040824358999999998, 2024-09-15T00:12:36.149 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.149 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.149 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:36.149 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.149 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.149 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.150 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127005 2024-09-15T00:12:36.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.150 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:36.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:36.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:36.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-15T00:12:36.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968343+0000", 2024-09-15T00:12:36.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3430000000000001e-06 2024-09-15T00:12:36.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968349+0000", 2024-09-15T00:12:36.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3080000000000004e-06 2024-09-15T00:12:36.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968355+0000", 2024-09-15T00:12:36.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9499999999999998e-06 2024-09-15T00:12:36.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009091+0000", 2024-09-15T00:12:36.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735645000000001 2024-09-15T00:12:36.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009130+0000", 2024-09-15T00:12:36.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8951e-05 2024-09-15T00:12:36.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009165+0000", 2024-09-15T00:12:36.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5926000000000001e-05 2024-09-15T00:12:36.153 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.153 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.153 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.154 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127006 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7127040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:36.154 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:36.154 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.143111686999999, 2024-09-15T00:12:36.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040912783000000001, 2024-09-15T00:12:36.154 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.154 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.154 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:36.154 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.154 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.154 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.154 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127006 2024-09-15T00:12:36.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.154 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:36.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:36.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:36.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:36.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968359+0000", 2024-09-15T00:12:36.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0419999999999999e-06 2024-09-15T00:12:36.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968363+0000", 2024-09-15T00:12:36.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5999999999999998e-06 2024-09-15T00:12:36.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968366+0000", 2024-09-15T00:12:36.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5179999999999999e-06 2024-09-15T00:12:36.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009196+0000", 2024-09-15T00:12:36.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040830313 2024-09-15T00:12:36.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009234+0000", 2024-09-15T00:12:36.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8562000000000001e-05 2024-09-15T00:12:36.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009271+0000", 2024-09-15T00:12:36.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6538000000000002e-05 2024-09-15T00:12:36.158 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.158 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.158 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.158 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127007 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7249920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:36.158 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:12:36.158 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.143101487999999, 2024-09-15T00:12:36.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041019218000000003, 2024-09-15T00:12:36.158 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.159 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.159 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:36.159 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.159 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.159 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.159 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127007 2024-09-15T00:12:36.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.159 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:12:36.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:12:36.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968369+0000", 2024-09-15T00:12:36.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-15T00:12:36.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968370+0000", 2024-09-15T00:12:36.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.012e-06 2024-09-15T00:12:36.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968374+0000", 2024-09-15T00:12:36.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3839999999999999e-06 2024-09-15T00:12:36.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968376+0000", 2024-09-15T00:12:36.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5339999999999998e-06 2024-09-15T00:12:36.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009292+0000", 2024-09-15T00:12:36.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040915092 2024-09-15T00:12:36.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009333+0000", 2024-09-15T00:12:36.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1105e-05 2024-09-15T00:12:36.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009388+0000", 2024-09-15T00:12:36.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4868999999999999e-05 2024-09-15T00:12:36.162 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.163 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.163 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127008 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7364608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:36.163 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:12:36.163 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.143090366999999, 2024-09-15T00:12:36.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041115290999999998, 2024-09-15T00:12:36.163 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.163 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.163 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:36.163 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.163 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.163 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.164 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127008 2024-09-15T00:12:36.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.164 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:12:36.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:12:36.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968380+0000", 2024-09-15T00:12:36.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3999999999999998e-07 2024-09-15T00:12:36.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968381+0000", 2024-09-15T00:12:36.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8700000000000004e-07 2024-09-15T00:12:36.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968385+0000", 2024-09-15T00:12:36.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9260000000000002e-06 2024-09-15T00:12:36.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968389+0000", 2024-09-15T00:12:36.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.685e-06 2024-09-15T00:12:36.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009419+0000", 2024-09-15T00:12:36.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041029852999999998 2024-09-15T00:12:36.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009458+0000", 2024-09-15T00:12:36.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8538000000000003e-05 2024-09-15T00:12:36.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009495+0000", 2024-09-15T00:12:36.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7061999999999999e-05 2024-09-15T00:12:36.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.167 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.167 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127009 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7487488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:36.167 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:36.167 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.143056250000001, 2024-09-15T00:12:36.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041233063, 2024-09-15T00:12:36.167 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.168 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.168 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:36.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.168 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127009 2024-09-15T00:12:36.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.168 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:36.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7199999999999999e-07 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968417+0000", 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3810000000000002e-06 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968423+0000", 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8820000000000003e-06 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968433+0000", 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0005e-05 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009527+0000", 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041093750999999998 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009601+0000", 2024-09-15T00:12:36.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3979000000000004e-05 2024-09-15T00:12:36.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009647+0000", 2024-09-15T00:12:36.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5493e-05 2024-09-15T00:12:36.171 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.171 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.171 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.171 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127010 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7610368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:36.171 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:36.171 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.143027246000001, 2024-09-15T00:12:36.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041325924999999999, 2024-09-15T00:12:36.172 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.172 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.172 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:36.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.172 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127010 2024-09-15T00:12:36.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.172 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:36.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:36.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:36.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:12:36.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968445+0000", 2024-09-15T00:12:36.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:12:36.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968452+0000", 2024-09-15T00:12:36.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-15T00:12:36.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968456+0000", 2024-09-15T00:12:36.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5210000000000004e-06 2024-09-15T00:12:36.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009664+0000", 2024-09-15T00:12:36.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041207404000000003 2024-09-15T00:12:36.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009723+0000", 2024-09-15T00:12:36.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9036000000000002e-05 2024-09-15T00:12:36.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009768+0000", 2024-09-15T00:12:36.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5578000000000003e-05 2024-09-15T00:12:36.176 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.176 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.176 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.176 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127011 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7733248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:36.176 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:36.176 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.143009011, 2024-09-15T00:12:36.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041482441000000002, 2024-09-15T00:12:36.176 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.176 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.176 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:36.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.177 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127011 2024-09-15T00:12:36.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.177 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:36.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:36.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:36.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-15T00:12:36.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968466+0000", 2024-09-15T00:12:36.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:12:36.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968470+0000", 2024-09-15T00:12:36.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6469999999999997e-06 2024-09-15T00:12:36.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968473+0000", 2024-09-15T00:12:36.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.977e-06 2024-09-15T00:12:36.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009800+0000", 2024-09-15T00:12:36.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041326918999999997 2024-09-15T00:12:36.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009869+0000", 2024-09-15T00:12:36.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8935000000000003e-05 2024-09-15T00:12:36.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009943+0000", 2024-09-15T00:12:36.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4188000000000002e-05 2024-09-15T00:12:36.180 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.180 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.180 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.180 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127012 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7856128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:36.181 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:12:36.181 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.142993509, 2024-09-15T00:12:36.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041640756000000001, 2024-09-15T00:12:36.181 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.181 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.181 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:36.181 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.181 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.181 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.181 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127012 2024-09-15T00:12:36.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.181 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:12:36.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:12:36.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968477+0000", 2024-09-15T00:12:36.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:36.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968482+0000", 2024-09-15T00:12:36.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6180000000000003e-06 2024-09-15T00:12:36.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968486+0000", 2024-09-15T00:12:36.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.501e-06 2024-09-15T00:12:36.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968489+0000", 2024-09-15T00:12:36.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1530000000000001e-06 2024-09-15T00:12:36.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009983+0000", 2024-09-15T00:12:36.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041493980999999999 2024-09-15T00:12:36.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010051+0000", 2024-09-15T00:12:36.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8039000000000004e-05 2024-09-15T00:12:36.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010117+0000", 2024-09-15T00:12:36.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6254000000000006e-05 2024-09-15T00:12:36.184 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.185 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.185 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.185 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127013 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7979008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:36.185 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:12:36.185 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.142977689, 2024-09-15T00:12:36.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041763863999999998, 2024-09-15T00:12:36.185 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.185 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.185 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:36.185 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.185 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.186 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.186 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127013 2024-09-15T00:12:36.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.186 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:12:36.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:12:36.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969767+0000", 2024-09-15T00:12:36.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001275006 2024-09-15T00:12:36.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969772+0000", 2024-09-15T00:12:36.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4769999999999997e-06 2024-09-15T00:12:36.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969781+0000", 2024-09-15T00:12:36.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1959999999999997e-06 2024-09-15T00:12:36.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969788+0000", 2024-09-15T00:12:36.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6359999999999994e-06 2024-09-15T00:12:36.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010162+0000", 2024-09-15T00:12:36.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040373291999999998 2024-09-15T00:12:36.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010212+0000", 2024-09-15T00:12:36.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0543e-05 2024-09-15T00:12:36.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010256+0000", 2024-09-15T00:12:36.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3714000000000002e-05 2024-09-15T00:12:36.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.189 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.189 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127015 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8224768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:36.189 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:12:36.189 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.141638753000001, 2024-09-15T00:12:36.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040710186000000002, 2024-09-15T00:12:36.190 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.190 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.190 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:36.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.190 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127015 2024-09-15T00:12:36.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.190 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:12:36.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:12:36.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969832+0000", 2024-09-15T00:12:36.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:12:36.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969834+0000", 2024-09-15T00:12:36.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2919999999999998e-06 2024-09-15T00:12:36.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969843+0000", 2024-09-15T00:12:36.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0480000000000004e-06 2024-09-15T00:12:36.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969848+0000", 2024-09-15T00:12:36.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-15T00:12:36.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010457+0000", 2024-09-15T00:12:36.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040609054999999998 2024-09-15T00:12:36.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010501+0000", 2024-09-15T00:12:36.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3396000000000003e-05 2024-09-15T00:12:36.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010541+0000", 2024-09-15T00:12:36.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0599000000000002e-05 2024-09-15T00:12:36.194 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.194 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.194 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.194 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127016 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:36.194 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:36.194 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.141588127, 2024-09-15T00:12:36.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040766216000000001, 2024-09-15T00:12:36.194 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.195 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.195 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:36.195 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.195 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.195 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.195 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127016 2024-09-15T00:12:36.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.195 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:36.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:36.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:36.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:12:36.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969886+0000", 2024-09-15T00:12:36.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8449999999999996e-06 2024-09-15T00:12:36.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969897+0000", 2024-09-15T00:12:36.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-15T00:12:36.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969904+0000", 2024-09-15T00:12:36.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1169999999999998e-06 2024-09-15T00:12:36.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010561+0000", 2024-09-15T00:12:36.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040657470000000001 2024-09-15T00:12:36.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010607+0000", 2024-09-15T00:12:36.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6108e-05 2024-09-15T00:12:36.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010648+0000", 2024-09-15T00:12:36.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0479000000000003e-05 2024-09-15T00:12:36.199 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.199 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.199 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.199 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127694 3.9 3:9390b205:::rbd_data.11396481e1ec.00000000000000df:head [set-alloc-hint object_size 1048576 write_size 1048576,write 819200~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T00:12:36.199 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:12:36.199 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.88686901, 2024-09-15T00:12:36.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90376419399999997, 2024-09-15T00:12:36.199 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.199 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.199 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:36.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.200 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.200 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.200 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127694 2024-09-15T00:12:36.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.200 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:12:36.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:12:36.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224602+0000", 2024-09-15T00:12:36.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5899999999999995e-07 2024-09-15T00:12:36.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224610+0000", 2024-09-15T00:12:36.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9130000000000003e-06 2024-09-15T00:12:36.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224620+0000", 2024-09-15T00:12:36.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0426e-05 2024-09-15T00:12:36.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224627+0000", 2024-09-15T00:12:36.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4390000000000003e-06 2024-09-15T00:12:36.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224675+0000", 2024-09-15T00:12:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7636000000000003e-05 2024-09-15T00:12:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:12:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224681+0000", 2024-09-15T00:12:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1460000000000001e-06 2024-09-15T00:12:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126388+0000", 2024-09-15T00:12:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90170724999999996 2024-09-15T00:12:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126526+0000", 2024-09-15T00:12:36.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013730599999999999 2024-09-15T00:12:36.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:36.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126684+0000", 2024-09-15T00:12:36.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000157935 2024-09-15T00:12:36.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:36.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.127932+0000", 2024-09-15T00:12:36.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012479240000000001 2024-09-15T00:12:36.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:36.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128269+0000", 2024-09-15T00:12:36.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033761099999999998 2024-09-15T00:12:36.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:36.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128324+0000", 2024-09-15T00:12:36.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5244999999999997e-05 2024-09-15T00:12:36.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128365+0000", 2024-09-15T00:12:36.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0503999999999997e-05 2024-09-15T00:12:36.205 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.206 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.206 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.206 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:134707 3.15 3:af1fbdb5:::rbd_data.113933a4f79a.000000000000001c:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T00:12:36.206 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:12:36.206 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.4189018960000004, 2024-09-15T00:12:36.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057328626000000001, 2024-09-15T00:12:36.206 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.206 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.206 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:36.206 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:36.206 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:36.206 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.207 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 134707 2024-09-15T00:12:36.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.207 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:12:36.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:12:36.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692569+0000", 2024-09-15T00:12:36.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.339e-06 2024-09-15T00:12:36.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692574+0000", 2024-09-15T00:12:36.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-15T00:12:36.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692584+0000", 2024-09-15T00:12:36.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0363000000000001e-05 2024-09-15T00:12:36.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692590+0000", 2024-09-15T00:12:36.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.818e-06 2024-09-15T00:12:36.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692633+0000", 2024-09-15T00:12:36.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.265e-05 2024-09-15T00:12:36.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692718+0000", 2024-09-15T00:12:36.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5279999999999997e-05 2024-09-15T00:12:36.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:36.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.695940+0000", 2024-09-15T00:12:36.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003222052 2024-09-15T00:12:36.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:36.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749714+0000", 2024-09-15T00:12:36.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.053774126999999998 2024-09-15T00:12:36.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:36.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749788+0000", 2024-09-15T00:12:36.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3693000000000001e-05 2024-09-15T00:12:36.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:36.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749847+0000", 2024-09-15T00:12:36.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175999999999997e-05 2024-09-15T00:12:36.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749897+0000", 2024-09-15T00:12:36.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9311999999999998e-05 2024-09-15T00:12:36.211 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.211 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.212 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.212 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:134707 3.15 e86/55 ondisk, result = 0)", 2024-09-15T00:12:36.212 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:12:36.212 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.4103405469999997, 2024-09-15T00:12:36.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048596066, 2024-09-15T00:12:36.212 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:36.212 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:36.212 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:36.212 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:36.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:36.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:12:36.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:36.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:12:36.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:36.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:36.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701131+0000", 2024-09-15T00:12:36.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3549999999999999e-06 2024-09-15T00:12:36.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:36.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701139+0000", 2024-09-15T00:12:36.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1650000000000006e-06 2024-09-15T00:12:36.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:36.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701156+0000", 2024-09-15T00:12:36.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6730000000000001e-05 2024-09-15T00:12:36.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701167+0000", 2024-09-15T00:12:36.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1504e-05 2024-09-15T00:12:36.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:36.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749629+0000", 2024-09-15T00:12:36.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048461952000000003 2024-09-15T00:12:36.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:36.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749640+0000", 2024-09-15T00:12:36.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0874e-05 2024-09-15T00:12:36.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:36.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:36.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:36.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749725+0000", 2024-09-15T00:12:36.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5486000000000001e-05 2024-09-15T00:12:36.216 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.216 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.216 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.216 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:36.216 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:36.216 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:36.217 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:36.218 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is unset 2024-09-15T00:12:36.219 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:36.215+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:36.242 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone38 from 0xc04d644 (overlap 0xb86e7) to 0xa5fcf 2024-09-15T00:12:36.306 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-15T00:12:36.318 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:36.317+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:36.329 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:36.329 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:36.329 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:36.329 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:36.338 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:36.422 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:36.421+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:36.482 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:36.483 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:36.483 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:36.483 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:36.483 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:36.491 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:36.532 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:36.530+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:36.569 INFO:teuthology.orchestra.run.smithi171.stdout:4759 write 0x184c24f thru 0x184c983 (0x735 bytes) 2024-09-15T00:12:36.582 INFO:teuthology.orchestra.run.smithi171.stdout:4760 trunc from 0xc04d644 to 0x798f71a 2024-09-15T00:12:36.599 INFO:teuthology.orchestra.run.smithi171.stdout:4761 read 0x77ee62b thru 0x77fbd0d (0xd6e3 bytes) 2024-09-15T00:12:36.599 INFO:teuthology.orchestra.run.smithi171.stdout:4762 punch from 0x36949e1 to 0x36a3326, (0xe945 bytes) 2024-09-15T00:12:36.614 INFO:teuthology.orchestra.run.smithi171.stdout:4763 writesame 0xe5b79c2 thru 0xe5c158d data_size 0x3b(0x9bcc bytes) 2024-09-15T00:12:36.614 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:36.614 INFO:teuthology.orchestra.run.smithi171.stdout:4766 write 0x5ea336b thru 0x5eab720 (0x83b6 bytes) 2024-09-15T00:12:36.619 INFO:teuthology.orchestra.run.smithi171.stdout:4767 punch from 0xb357bee to 0xb362565, (0xa977 bytes) 2024-09-15T00:12:36.619 INFO:teuthology.orchestra.run.smithi171.stdout:4768 read 0x1602a25 thru 0x160a3d1 (0x79ad bytes) 2024-09-15T00:12:36.620 INFO:teuthology.orchestra.run.smithi171.stdout:4773 punch from 0x7ea5c8b to 0x7ea7dbd, (0x2132 bytes) 2024-09-15T00:12:36.620 INFO:teuthology.orchestra.run.smithi171.stdout:4774 write 0x1a5e050 thru 0x1a6ad79 (0xcd2a bytes) 2024-09-15T00:12:36.621 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:36.619+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:36.625 INFO:teuthology.orchestra.run.smithi171.stdout:4775 read 0xbb0813 thru 0xbb2097 (0x1885 bytes) 2024-09-15T00:12:36.633 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 0] out_osds: [2, 7] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-09-15T00:12:36.634 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:12:36.634 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [1, 3, 4, 5, 6, 0] 2024-09-15T00:12:36.634 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-15T00:12:36.640 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 71.065383464000007, 2024-09-15T00:12:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:12:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:12:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:12:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:12:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:12:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:12:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:12:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:12:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:12:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:12:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:12:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:12:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:12:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:12:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:12:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:12:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:12:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:12:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:12:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:12:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:12:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:12:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:12:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:12:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 71.065348771999993, 2024-09-15T00:12:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:12:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:12:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:12:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:12:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:12:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:12:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:12:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:12:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:12:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:12:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:12:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:12:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:12:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:12:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:12:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:12:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:12:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:12:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:12:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:12:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:12:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:12:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:12:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48937 3.1a 3:5e7e71d0:::rbd_data.11395b9829f3.0000000000000000:e [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:12:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.025255817000001, 2024-09-15T00:12:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072322336000000001, 2024-09-15T00:12:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48937 2024-09-15T00:12:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:12:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:12:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612043+0000", 2024-09-15T00:12:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:12:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612045+0000", 2024-09-15T00:12:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2749999999999998e-06 2024-09-15T00:12:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612053+0000", 2024-09-15T00:12:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0439999999999996e-06 2024-09-15T00:12:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612062+0000", 2024-09-15T00:12:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6829999999999997e-06 2024-09-15T00:12:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684078+0000", 2024-09-15T00:12:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072016017000000002 2024-09-15T00:12:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684274+0000", 2024-09-15T00:12:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019636600000000001 2024-09-15T00:12:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684364+0000", 2024-09-15T00:12:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0420000000000005e-05 2024-09-15T00:12:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.024675084000002, 2024-09-15T00:12:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:12:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:12:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:12:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:12:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:12:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:12:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:12:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:12:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:12:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:12:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:12:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:12:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:12:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:12:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:12:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.024658553000002, 2024-09-15T00:12:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:12:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:12:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:12:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:12:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:12:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:12:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:12:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:12:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:12:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:12:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:12:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:12:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:12:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:12:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:12:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:12:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.024641774000003, 2024-09-15T00:12:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:12:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:12:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:12:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:12:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:12:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:12:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:12:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:12:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:12:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.024624043999999, 2024-09-15T00:12:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:12:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:12:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:12:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:12:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:12:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:12:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:12:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:12:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:12:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:12:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:12:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:12:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:12:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:12:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:12:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.02460688, 2024-09-15T00:12:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:12:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:12:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:12:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:12:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:12:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:12:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:12:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:12:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:12:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:12:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:12:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:12:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:12:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:12:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:12:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.024585203999997, 2024-09-15T00:12:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:12:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:12:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:12:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:12:36.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:12:36.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:12:36.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:12:36.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:12:36.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:12:36.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.684 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.684 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:12:36.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:12:36.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.684 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.684 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:12:36.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:12:36.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.684 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.684 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:12:36.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:12:36.685 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.685 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.685 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.685 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.685 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:36.685 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.024539079999997, 2024-09-15T00:12:36.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:12:36.685 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.685 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.685 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.685 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.685 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.686 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.686 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:12:36.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.686 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:36.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:36.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:36.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:12:36.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:12:36.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:12:36.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.688 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:12:36.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:12:36.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.688 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.688 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:12:36.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:12:36.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.688 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.688 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:12:36.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:12:36.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:12:36.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:12:36.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.689 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:12:36.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:12:36.689 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.689 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.689 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.689 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.689 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.690 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:36.690 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.024509322999997, 2024-09-15T00:12:36.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:12:36.690 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.690 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.690 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.690 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.690 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.690 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.691 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:12:36.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.691 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.691 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:36.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.691 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:36.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.692 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:12:36.692 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:12:36.692 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.692 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:12:36.692 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:12:36.692 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.692 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:12:36.693 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:12:36.693 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.693 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.693 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.693 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:12:36.693 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:12:36.693 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.693 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.693 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.693 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:12:36.693 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:12:36.693 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.693 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.694 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:12:36.694 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:12:36.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.694 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:12:36.694 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:12:36.694 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.694 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.694 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.694 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.695 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:36.695 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.024483003999997, 2024-09-15T00:12:36.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:12:36.695 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.695 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.695 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.695 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.695 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.695 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.695 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:12:36.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.695 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:36.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.696 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:36.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.696 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:36.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:12:36.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.697 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.697 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:12:36.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:12:36.697 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.697 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.697 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:12:36.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:12:36.697 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.697 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.697 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:12:36.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:12:36.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:12:36.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:12:36.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:12:36.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:12:36.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:12:36.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:12:36.699 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.699 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.699 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.699 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.699 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:36.699 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.024447142, 2024-09-15T00:12:36.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:12:36.699 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.699 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.700 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.700 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.700 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.700 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.700 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:12:36.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.700 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:36.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:36.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:36.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:12:36.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:12:36.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:12:36.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.702 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:12:36.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:12:36.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.702 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:12:36.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:12:36.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.702 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:12:36.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:12:36.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:12:36.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:12:36.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:12:36.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:12:36.703 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.703 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.704 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.704 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.704 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:36.704 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.024425424, 2024-09-15T00:12:36.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:12:36.704 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.704 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.704 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.704 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.704 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.704 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.705 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:12:36.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.705 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:36.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:36.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:12:36.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:12:36.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:12:36.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:12:36.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:12:36.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:12:36.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:12:36.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:12:36.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:12:36.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:12:36.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:12:36.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:12:36.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:12:36.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:12:36.708 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.708 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.708 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.709 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.709 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:36.709 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.024228956999998, 2024-09-15T00:12:36.709 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:12:36.709 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.709 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.709 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.709 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.709 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.709 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.709 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:12:36.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.709 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:36.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:36.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:36.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:12:36.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:12:36.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:12:36.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:12:36.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:12:36.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:12:36.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:12:36.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:12:36.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:12:36.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:12:36.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:12:36.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.713 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.713 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.713 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:12:36.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:12:36.713 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.713 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.713 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.713 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.713 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.713 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:36.713 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.024203118000003, 2024-09-15T00:12:36.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:12:36.713 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.714 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.714 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.714 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.714 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.714 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.714 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:12:36.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.714 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.714 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.714 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:36.714 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:36.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:36.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:12:36.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:12:36.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:12:36.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:12:36.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:12:36.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:12:36.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:12:36.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:12:36.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:12:36.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:12:36.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:12:36.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:12:36.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:12:36.717 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.717 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.718 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.718 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.718 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.718 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:36.718 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.024141471999997, 2024-09-15T00:12:36.718 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:12:36.718 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.718 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.718 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.718 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.718 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.718 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.718 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:12:36.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.719 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:36.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:36.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:12:36.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:12:36.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.720 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:12:36.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:12:36.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.720 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:12:36.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:12:36.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:12:36.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:12:36.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:12:36.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:12:36.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:12:36.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:12:36.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.722 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.722 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:12:36.722 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:12:36.722 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.722 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.722 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.722 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.722 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:36.722 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.024102143999997, 2024-09-15T00:12:36.722 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:12:36.723 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.723 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.723 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.723 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.723 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.723 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.723 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:12:36.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.723 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.723 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:36.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.724 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:36.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.724 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:12:36.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:12:36.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.724 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:12:36.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:12:36.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.725 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.725 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:12:36.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:12:36.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.725 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.725 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:12:36.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:12:36.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.725 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:12:36.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:12:36.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:12:36.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:12:36.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:12:36.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:12:36.726 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.727 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.727 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.727 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.727 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:36.727 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.022376743999999, 2024-09-15T00:12:36.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:12:36.727 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.727 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.727 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.727 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.727 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.728 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.728 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:12:36.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.728 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:36.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:36.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:12:36.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:12:36.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:12:36.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:12:36.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:12:36.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:12:36.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:12:36.730 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:12:36.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:12:36.730 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:12:36.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:12:36.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:12:36.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.731 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.731 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:12:36.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:12:36.731 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.731 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.731 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.732 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:36.732 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:36.732 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.022354583000002, 2024-09-15T00:12:36.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:12:36.732 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:36.732 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:36.732 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:36.732 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:36.732 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:36.732 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:36.732 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:12:36.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.733 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:36.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:36.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:36.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:36.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:36.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:36.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:36.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:36.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:12:36.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:36.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:12:36.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:12:36.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:36.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:12:36.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:12:36.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:36.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:12:36.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:12:36.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.735 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:36.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:12:36.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:12:36.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.735 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:36.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:12:36.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:12:36.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:36.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:36.735 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:36.736 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:12:36.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:12:36.736 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.736 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.736 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.736 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:36.736 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:36.736 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:36.738 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:36.739 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:36.736+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:36.739 INFO:teuthology.orchestra.run.smithi171.stdout:4776 trunc from 0xe5c158e to 0x13585f1 2024-09-15T00:12:36.740 INFO:teuthology.orchestra.run.smithi171.stdout:4777 write 0xbf39677 thru 0xbf3b358 (0x1ce2 bytes) 2024-09-15T00:12:36.740 INFO:teuthology.orchestra.run.smithi171.stdout:4778 write 0xeafcd3b thru 0xeb04f0e (0x81d4 bytes) 2024-09-15T00:12:36.740 INFO:teuthology.orchestra.run.smithi171.stdout:4779 punch from 0xb8d55fc to 0xb8e031b, (0xad1f bytes) 2024-09-15T00:12:36.740 INFO:teuthology.orchestra.run.smithi171.stdout:4780 trunc from 0xeb04f0f to 0x590e382 2024-09-15T00:12:36.740 INFO:teuthology.orchestra.run.smithi171.stdout:4784 read 0x19e0de2 thru 0x19e252b (0x174a bytes) 2024-09-15T00:12:36.740 INFO:teuthology.orchestra.run.smithi171.stdout:4785 trunc from 0x590e382 to 0xa9f80b2 2024-09-15T00:12:36.740 INFO:teuthology.orchestra.run.smithi171.stdout:4786 read 0x90e581d thru 0x90f00d9 (0xa8bd bytes) 2024-09-15T00:12:36.740 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:36.740 INFO:teuthology.orchestra.run.smithi171.stdout:4789 write 0xa51ad6f thru 0xa5203c8 (0x565a bytes) 2024-09-15T00:12:36.740 INFO:teuthology.orchestra.run.smithi171.stdout:4790 write 0x97e6a7 thru 0x98d46a (0xedc4 bytes) 2024-09-15T00:12:36.740 INFO:teuthology.orchestra.run.smithi171.stdout:4791 write 0x20ef1d7 thru 0x20f2e54 (0x3c7e bytes) 2024-09-15T00:12:36.741 INFO:teuthology.orchestra.run.smithi171.stdout:4792 read 0x2c60424 thru 0x2c6fcea (0xf8c7 bytes) 2024-09-15T00:12:36.741 INFO:teuthology.orchestra.run.smithi171.stdout:4793 read 0x6816448 thru 0x681ca0a (0x65c3 bytes) 2024-09-15T00:12:36.741 INFO:teuthology.orchestra.run.smithi171.stdout:4794 trunc from 0xa9f80b2 to 0xdd1d690 2024-09-15T00:12:36.745 INFO:teuthology.orchestra.run.smithi171.stdout:4795 read 0x679e3a2 thru 0x67a2e40 (0x4a9f bytes) 2024-09-15T00:12:36.751 INFO:teuthology.orchestra.run.smithi171.stdout:4796 trunc from 0xdd1d690 to 0x8caf1dc 2024-09-15T00:12:36.762 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:36.762 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:36.763 INFO:teuthology.orchestra.run.smithi171.stdout:4800 read 0x7695a54 thru 0x76a312d (0xd6da bytes) 2024-09-15T00:12:36.763 INFO:teuthology.orchestra.run.smithi171.stdout:4801 read 0x45bf752 thru 0x45c2e37 (0x36e6 bytes) 2024-09-15T00:12:36.763 INFO:teuthology.orchestra.run.smithi171.stdout:4802 read 0x2c739d3 thru 0x2c7a14c (0x677a bytes) 2024-09-15T00:12:36.769 INFO:teuthology.orchestra.run.smithi171.stdout:4803 trunc from 0x8caf1dc to 0x30f228f 2024-09-15T00:12:36.776 INFO:teuthology.orchestra.run.smithi171.stdout:4804 writesame 0x2e5c5d5 thru 0x2e66ea0 data_size 0x4e(0xa8cc bytes) 2024-09-15T00:12:36.783 INFO:teuthology.orchestra.run.smithi171.stdout:4806 write 0x1b57994 thru 0x1b5bb54 (0x41c1 bytes) 2024-09-15T00:12:36.802 INFO:teuthology.orchestra.run.smithi171.stdout:4807 trunc from 0x30f228f to 0xe9ec61d 2024-09-15T00:12:36.807 INFO:teuthology.orchestra.run.smithi171.stdout:4808 read 0x86e364d thru 0x86eaa68 (0x741c bytes) 2024-09-15T00:12:36.807 INFO:teuthology.orchestra.run.smithi171.stdout:4809 read 0x6aaae2f thru 0x6aaec9a (0x3e6c bytes) 2024-09-15T00:12:36.816 INFO:teuthology.orchestra.run.smithi171.stdout:4810 trunc from 0xe9ec61d to 0x6c74aa7 2024-09-15T00:12:36.822 INFO:teuthology.orchestra.run.smithi171.stdout:4811 write 0x39f47d thru 0x3a46ba (0x523e bytes) 2024-09-15T00:12:36.829 INFO:teuthology.orchestra.run.smithi171.stdout:4812 punch from 0x26e3f8f to 0x26e7769, (0x37da bytes) 2024-09-15T00:12:36.838 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:36.836+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:36.841 INFO:teuthology.orchestra.run.smithi171.stdout:4814 writesame 0xb765ea2 thru 0xb76652f data_size 0x1(0x68e bytes) 2024-09-15T00:12:36.841 INFO:teuthology.orchestra.run.smithi171.stdout:4817 write 0x258c682 thru 0x2599539 (0xceb8 bytes) 2024-09-15T00:12:36.848 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:36.848 INFO:teuthology.orchestra.run.smithi171.stdout:4819 read 0x95dcd58 thru 0x95e0eed (0x4196 bytes) 2024-09-15T00:12:36.848 INFO:teuthology.orchestra.run.smithi171.stdout:4820 punch from 0xb756982 to 0xb75e9aa, (0x8028 bytes) 2024-09-15T00:12:36.861 INFO:teuthology.orchestra.run.smithi171.stdout:4821 trunc from 0xb766530 to 0x1c5f56d 2024-09-15T00:12:36.874 INFO:teuthology.orchestra.run.smithi171.stdout:4822 punch from 0x8ba1e to 0x9199f, (0x5f81 bytes) 2024-09-15T00:12:36.886 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:36.886 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:36.886 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:36.886 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:36.889 INFO:teuthology.orchestra.run.smithi171.stdout:4823 write 0x8a03a0b thru 0x8a0b1f7 (0x77ed bytes) 2024-09-15T00:12:36.892 INFO:teuthology.orchestra.run.smithi171.stdout:4824 writesame 0x13086b1 thru 0x1311808 data_size 0x4(0x9158 bytes) 2024-09-15T00:12:36.896 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:36.917 INFO:teuthology.orchestra.run.smithi171.stdout:4825 writesame 0xa1b507f thru 0xa1c3f95 data_size 0x1(0xef17 bytes) 2024-09-15T00:12:36.917 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:36.918 INFO:teuthology.orchestra.run.smithi171.stdout:4827 trunc from 0xa1c3f96 to 0xaedd7ad 2024-09-15T00:12:36.923 INFO:teuthology.orchestra.run.smithi171.stdout:4828 read 0xae5683b thru 0xae5ce4c (0x6612 bytes) 2024-09-15T00:12:36.923 INFO:teuthology.orchestra.run.smithi171.stdout:4829 read 0x4b1d649 thru 0x4b26f2f (0x98e7 bytes) 2024-09-15T00:12:36.923 INFO:teuthology.orchestra.run.smithi171.stdout:4830 punch from 0x1157ecf to 0x115f81e, (0x794f bytes) 2024-09-15T00:12:36.934 INFO:teuthology.orchestra.run.smithi171.stdout:4831 writesame 0xed5e48a thru 0xed68714 data_size 0x1(0xa28b bytes) 2024-09-15T00:12:36.934 INFO:teuthology.orchestra.run.smithi171.stdout:4832 read 0xbac01fa thru 0xbacdc44 (0xda4b bytes) 2024-09-15T00:12:36.934 INFO:teuthology.orchestra.run.smithi171.stdout:4833 write 0x95ab621 thru 0x95b9b22 (0xe502 bytes) 2024-09-15T00:12:36.939 INFO:teuthology.orchestra.run.smithi171.stdout:4834 write 0x9522fa0 thru 0x95271cb (0x422c bytes) 2024-09-15T00:12:36.942 INFO:teuthology.orchestra.run.smithi171.stdout:4835 trunc from 0xed68715 to 0xe7015b7 2024-09-15T00:12:36.948 INFO:teuthology.orchestra.run.smithi171.stdout:4836 punch from 0xb9aac60 to 0xb9b2989, (0x7d29 bytes) 2024-09-15T00:12:36.955 INFO:teuthology.orchestra.run.smithi171.stdout:4837 trunc from 0xe7015b7 to 0x97a37c0 2024-09-15T00:12:36.961 INFO:teuthology.orchestra.run.smithi171.stdout:4838 writesame 0xa18fc0 thru 0xa1c7a1 data_size 0x17(0x37e2 bytes) 2024-09-15T00:12:36.964 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:36.964 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:36.973 INFO:teuthology.orchestra.run.smithi171.stdout:4841 trunc from 0x97a37c0 to 0x2ee6467 2024-09-15T00:12:36.979 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:36.978+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:36.985 INFO:teuthology.orchestra.run.smithi171.stdout:4842 punch from 0xf55e0f to 0xf5bf9e, (0x618f bytes) 2024-09-15T00:12:36.985 INFO:teuthology.orchestra.run.smithi171.stdout:4845 read 0x43c116 thru 0x4440aa (0x7f95 bytes) 2024-09-15T00:12:36.985 INFO:teuthology.orchestra.run.smithi171.stdout:4847 write 0x1f38cb2 thru 0x1f3a58d (0x18dc bytes) 2024-09-15T00:12:36.988 INFO:teuthology.orchestra.run.smithi171.stdout:4848 clone 40 order 22 su 16384 sc 14 2024-09-15T00:12:36.996 INFO:tasks.ceph.mon.a.smithi171.stderr:2024-09-15T00:12:36.994+0000 7f0425d35640 -1 mon.a@0(leader).osd e90 definitely_dead 0 2024-09-15T00:12:37.039 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:37.040 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:37.040 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:37.040 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:37.040 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:37.040 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:37.038+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:37.048 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:37.054 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:12:37.141 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:37.139+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:37.174 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:37.174 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:37.192 INFO:teuthology.orchestra.run.smithi171.stderr:marked out osd.1. 2024-09-15T00:12:37.192 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:37.192 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:37.193 INFO:teuthology.orchestra.run.smithi171.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T00:12:37.195 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:37.195 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:37.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:37.195 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:37.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.195 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:47986 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.195 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:37.195 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.720885512999999, 2024-09-15T00:12:37.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045287635, 2024-09-15T00:12:37.195 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.196 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.196 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:37.196 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.196 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.196 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.196 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 47986 2024-09-15T00:12:37.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.196 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:37.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:37.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:37.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8400000000000004e-07 2024-09-15T00:12:37.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474928+0000", 2024-09-15T00:12:37.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033009430000000002 2024-09-15T00:12:37.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474943+0000", 2024-09-15T00:12:37.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5172e-05 2024-09-15T00:12:37.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474951+0000", 2024-09-15T00:12:37.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1779999999999997e-06 2024-09-15T00:12:37.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475058+0000", 2024-09-15T00:12:37.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000107595 2024-09-15T00:12:37.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475200+0000", 2024-09-15T00:12:37.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014180400000000001 2024-09-15T00:12:37.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:37.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.478046+0000", 2024-09-15T00:12:37.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028464150000000001 2024-09-15T00:12:37.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:37.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516790+0000", 2024-09-15T00:12:37.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038743278999999999 2024-09-15T00:12:37.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:37.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516832+0000", 2024-09-15T00:12:37.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1974000000000002e-05 2024-09-15T00:12:37.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:37.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516872+0000", 2024-09-15T00:12:37.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9928999999999997e-05 2024-09-15T00:12:37.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516914+0000", 2024-09-15T00:12:37.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2762000000000002e-05 2024-09-15T00:12:37.201 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.201 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.201 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.202 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48957 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 4456448~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.202 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:12:37.202 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.583447001000003, 2024-09-15T00:12:37.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044512426000000001, 2024-09-15T00:12:37.202 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.202 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.202 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.202 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.202 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.202 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.202 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48957 2024-09-15T00:12:37.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.203 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:12:37.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:12:37.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609066+0000", 2024-09-15T00:12:37.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:12:37.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609068+0000", 2024-09-15T00:12:37.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9700000000000002e-06 2024-09-15T00:12:37.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609073+0000", 2024-09-15T00:12:37.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7939999999999999e-06 2024-09-15T00:12:37.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609078+0000", 2024-09-15T00:12:37.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3209999999999998e-06 2024-09-15T00:12:37.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653082+0000", 2024-09-15T00:12:37.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044003852000000003 2024-09-15T00:12:37.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653136+0000", 2024-09-15T00:12:37.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4024000000000003e-05 2024-09-15T00:12:37.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653578+0000", 2024-09-15T00:12:37.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044211599999999999 2024-09-15T00:12:37.206 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.206 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.206 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48958 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.583429361999997, 2024-09-15T00:12:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044724939999999998, 2024-09-15T00:12:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48958 2024-09-15T00:12:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:37.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:37.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:37.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:12:37.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609085+0000", 2024-09-15T00:12:37.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6610000000000001e-06 2024-09-15T00:12:37.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609090+0000", 2024-09-15T00:12:37.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2719999999999997e-06 2024-09-15T00:12:37.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609094+0000", 2024-09-15T00:12:37.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.907e-06 2024-09-15T00:12:37.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653618+0000", 2024-09-15T00:12:37.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044523759000000003 2024-09-15T00:12:37.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653703+0000", 2024-09-15T00:12:37.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5550000000000006e-05 2024-09-15T00:12:37.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653808+0000", 2024-09-15T00:12:37.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010445700000000001 2024-09-15T00:12:37.212 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.212 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.212 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.212 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48959 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.212 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:37.213 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.583413423000003, 2024-09-15T00:12:37.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044884741999999998, 2024-09-15T00:12:37.213 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.213 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.213 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.213 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.213 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.213 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.213 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48959 2024-09-15T00:12:37.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.213 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2899999999999999e-07 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609101+0000", 2024-09-15T00:12:37.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.826e-06 2024-09-15T00:12:37.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609108+0000", 2024-09-15T00:12:37.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.165e-06 2024-09-15T00:12:37.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609113+0000", 2024-09-15T00:12:37.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7849999999999999e-06 2024-09-15T00:12:37.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653842+0000", 2024-09-15T00:12:37.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044728954000000001 2024-09-15T00:12:37.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653896+0000", 2024-09-15T00:12:37.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4593999999999999e-05 2024-09-15T00:12:37.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653984+0000", 2024-09-15T00:12:37.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7089000000000006e-05 2024-09-15T00:12:37.216 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.216 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.217 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.217 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48960 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 9961472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.217 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:37.217 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.583392904, 2024-09-15T00:12:37.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045084194000000001, 2024-09-15T00:12:37.217 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.217 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.217 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.217 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.217 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.217 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.217 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48960 2024-09-15T00:12:37.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609120+0000", 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7099999999999997e-07 2024-09-15T00:12:37.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609122+0000", 2024-09-15T00:12:37.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:12:37.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609129+0000", 2024-09-15T00:12:37.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4159999999999998e-06 2024-09-15T00:12:37.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609134+0000", 2024-09-15T00:12:37.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7210000000000002e-06 2024-09-15T00:12:37.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654023+0000", 2024-09-15T00:12:37.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044888979000000002 2024-09-15T00:12:37.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654104+0000", 2024-09-15T00:12:37.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1108999999999999e-05 2024-09-15T00:12:37.221 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654204+0000", 2024-09-15T00:12:37.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9630000000000007e-05 2024-09-15T00:12:37.222 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.222 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.222 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.223 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48961 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.223 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:37.223 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.583372083, 2024-09-15T00:12:37.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045197821999999999, 2024-09-15T00:12:37.223 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.223 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.224 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.224 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.224 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.224 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.224 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48961 2024-09-15T00:12:37.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.224 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.225 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.225 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:37.225 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.225 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:37.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609141+0000", 2024-09-15T00:12:37.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:12:37.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609143+0000", 2024-09-15T00:12:37.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-15T00:12:37.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.228 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609150+0000", 2024-09-15T00:12:37.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2420000000000003e-06 2024-09-15T00:12:37.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609155+0000", 2024-09-15T00:12:37.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.028e-06 2024-09-15T00:12:37.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654232+0000", 2024-09-15T00:12:37.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045076702000000003 2024-09-15T00:12:37.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654281+0000", 2024-09-15T00:12:37.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9798e-05 2024-09-15T00:12:37.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654338+0000", 2024-09-15T00:12:37.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6498e-05 2024-09-15T00:12:37.231 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.232 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.232 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.232 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48962 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 13631488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.232 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:37.233 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.583351033, 2024-09-15T00:12:37.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045312458, 2024-09-15T00:12:37.233 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.233 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.233 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.234 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.234 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.234 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.234 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48962 2024-09-15T00:12:37.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.234 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:37.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:37.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609162+0000", 2024-09-15T00:12:37.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:37.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609164+0000", 2024-09-15T00:12:37.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0099999999999998e-06 2024-09-15T00:12:37.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.238 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609171+0000", 2024-09-15T00:12:37.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7530000000000001e-06 2024-09-15T00:12:37.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.238 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609178+0000", 2024-09-15T00:12:37.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1709999999999999e-06 2024-09-15T00:12:37.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654365+0000", 2024-09-15T00:12:37.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045187867999999999 2024-09-15T00:12:37.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654420+0000", 2024-09-15T00:12:37.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4376000000000003e-05 2024-09-15T00:12:37.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.241 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654474+0000", 2024-09-15T00:12:37.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3879999999999999e-05 2024-09-15T00:12:37.241 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.241 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.241 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.242 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48971 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.242 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:37.242 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.583320039, 2024-09-15T00:12:37.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045896151000000003, 2024-09-15T00:12:37.242 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.243 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.243 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.243 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.243 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.243 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.243 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48971 2024-09-15T00:12:37.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.244 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:37.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:37.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609193+0000", 2024-09-15T00:12:37.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7599999999999997e-07 2024-09-15T00:12:37.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609195+0000", 2024-09-15T00:12:37.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:12:37.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609210+0000", 2024-09-15T00:12:37.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4956999999999999e-05 2024-09-15T00:12:37.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609505+0000", 2024-09-15T00:12:37.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000295078 2024-09-15T00:12:37.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654498+0000", 2024-09-15T00:12:37.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044992277999999997 2024-09-15T00:12:37.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654541+0000", 2024-09-15T00:12:37.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3525000000000001e-05 2024-09-15T00:12:37.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655088+0000", 2024-09-15T00:12:37.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054726799999999995 2024-09-15T00:12:37.247 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.247 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.247 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.248 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48972 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 2097152~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.248 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:37.248 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.582993465000001, 2024-09-15T00:12:37.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046150885000000003, 2024-09-15T00:12:37.248 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.248 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.248 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.248 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.248 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.248 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.249 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48972 2024-09-15T00:12:37.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.249 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:37.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:37.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:37.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6400000000000002e-07 2024-09-15T00:12:37.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609523+0000", 2024-09-15T00:12:37.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5300000000000001e-06 2024-09-15T00:12:37.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609533+0000", 2024-09-15T00:12:37.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0023999999999999e-05 2024-09-15T00:12:37.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609539+0000", 2024-09-15T00:12:37.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3119999999999999e-06 2024-09-15T00:12:37.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655115+0000", 2024-09-15T00:12:37.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045576094999999997 2024-09-15T00:12:37.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655197+0000", 2024-09-15T00:12:37.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1129000000000002e-05 2024-09-15T00:12:37.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655670+0000", 2024-09-15T00:12:37.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00047313100000000001 2024-09-15T00:12:37.252 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.252 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.252 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.252 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.252 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:37.253 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.582965203000001, 2024-09-15T00:12:37.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:12:37.253 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.253 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.253 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.253 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.253 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.253 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.253 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:12:37.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.253 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:37.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:37.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:37.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:12:37.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:12:37.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:12:37.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:12:37.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:12:37.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:12:37.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:12:37.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:12:37.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:12:37.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:12:37.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:12:37.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:12:37.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:12:37.257 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.257 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.258 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.258 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.258 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:37.258 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.582943974000003, 2024-09-15T00:12:37.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:12:37.258 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.258 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.258 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.258 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.258 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.258 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.258 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:12:37.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.259 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:37.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:37.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:12:37.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:12:37.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:12:37.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:12:37.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:12:37.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:12:37.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:12:37.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:12:37.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:12:37.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:12:37.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:12:37.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:12:37.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:12:37.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:12:37.262 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.262 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.262 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.262 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.262 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:37.262 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.582918433000003, 2024-09-15T00:12:37.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:12:37.263 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.263 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.263 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.263 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.263 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.263 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.263 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:12:37.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.263 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:37.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:37.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:37.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:12:37.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:12:37.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:12:37.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:12:37.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:12:37.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:12:37.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:12:37.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:12:37.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:12:37.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:12:37.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:12:37.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:12:37.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:12:37.267 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.267 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.267 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.267 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.267 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:37.267 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.582895958000002, 2024-09-15T00:12:37.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:12:37.268 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.268 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.268 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.268 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.268 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.268 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.268 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:12:37.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.268 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:37.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:37.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:12:37.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:12:37.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:12:37.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:12:37.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:12:37.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:12:37.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:12:37.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:12:37.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:12:37.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:12:37.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.272 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:12:37.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:12:37.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.272 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.272 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:12:37.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:12:37.272 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.272 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.272 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.272 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.273 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:37.273 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.582875448000003, 2024-09-15T00:12:37.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:12:37.273 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.273 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.273 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.273 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.273 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.273 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.273 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:12:37.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.273 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:37.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:37.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:37.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:12:37.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:12:37.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:12:37.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:12:37.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:12:37.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:12:37.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:12:37.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.276 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:12:37.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:12:37.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:12:37.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:12:37.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:12:37.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:12:37.278 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.278 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.278 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.279 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.279 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:37.279 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.582855133000002, 2024-09-15T00:12:37.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:12:37.279 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.279 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.279 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.279 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.279 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.280 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.280 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:12:37.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.280 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:37.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:37.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:12:37.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:12:37.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:12:37.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:12:37.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:12:37.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:12:37.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:12:37.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:12:37.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:12:37.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:12:37.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:12:37.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:12:37.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:12:37.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:12:37.283 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.284 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.284 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.284 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.284 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:37.284 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.582830356999999, 2024-09-15T00:12:37.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:12:37.284 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.284 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.284 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.284 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.284 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.285 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.285 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:12:37.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.285 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:37.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:37.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:37.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:12:37.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:12:37.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:12:37.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:12:37.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:12:37.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:12:37.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:12:37.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:12:37.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:12:37.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:12:37.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:12:37.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:12:37.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:12:37.289 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.289 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.289 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.290 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:37.290 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:37.290 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.403488969999998, 2024-09-15T00:12:37.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:12:37.290 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.290 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.291 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:37.291 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.291 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.291 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.291 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:12:37.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.292 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:37.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:37.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:12:37.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:12:37.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:12:37.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:12:37.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:12:37.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:12:37.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:12:37.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:12:37.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:12:37.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:12:37.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:12:37.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:12:37.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:37.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:12:37.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:12:37.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:37.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:12:37.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:12:37.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:37.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:12:37.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:12:37.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:37.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:12:37.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:12:37.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:12:37.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:12:37.302 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.302 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.302 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.303 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:12:37.303 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:37.303 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.386535518000002, 2024-09-15T00:12:37.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:12:37.304 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.304 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.304 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.304 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:37.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:37.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:12:37.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:12:37.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:12:37.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:12:37.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:12:37.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:12:37.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:12:37.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:12:37.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:12:37.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:12:37.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:12:37.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:12:37.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:12:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:12:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:12:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 39.092104611000003, 2024-09-15T00:12:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:12:37.310 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:12:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:12:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:12:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:12:37.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:12:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:12:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:12:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:12:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:12:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:12:37.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:12:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:12:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:12:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:12:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:12:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:12:37.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:12:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:12:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:12:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:12:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:12:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:12:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:12:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:12:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 39.079648796999997, 2024-09-15T00:12:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:12:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:37.316 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:12:37.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:12:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:12:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:12:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:12:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:12:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:12:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:12:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:12:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:12:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:12:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:12:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:12:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:12:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:37.320 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:37.320 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:37.322 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:37.323 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:37.323 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:37.320+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:37.323 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:37.323 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:37.324 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:37.324 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:37.324 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:37.325 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:37.325 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:37.325 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:37.326 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:12:37.326 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:37.326 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:12:37.423 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:37.421+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:37.467 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:37.467 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:37.476 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:37.523 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:37.522+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:37.579 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:37.579 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:37.589 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:37.632 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:37.631+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:37.692 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:37.692 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:37.700 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:37.741 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:37.740+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:37.826 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:37.824+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:37.845 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:37.845 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:37.853 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:37.927 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:37.925+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:37.996 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:37.996 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:38.005 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:38.049 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:38.047+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:38.108 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:38.108 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:38.116 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:38.158 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:38.156+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:38.219 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:12:38.229 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:38.228+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:38.238 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:38.238 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:38.248 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:38.330 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:38.329+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:38.399 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:38.399 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:38.408 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:38.451 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:38.450+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:38.512 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:38.513 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:38.521 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:38.532 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:38.531+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:38.633 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:38.632+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:38.666 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:38.666 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:38.674 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:38.734 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:38.732+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:38.777 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:38.777 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:38.786 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:38.871 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:38.869+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:38.930 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:38.930 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:38.935 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:38.934+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:38.938 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:39.036 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:39.035+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:39.083 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:39.083 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:39.092 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:39.138 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:39.136+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:39.196 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is unset 2024-09-15T00:12:39.202 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:39.202 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:39.219 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:39.271 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:39.270+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:39.283 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone39 from 0x2ee6467 (overlap 0x13585f1) to 0x1ef3cb 2024-09-15T00:12:39.327 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:12:39.340 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:39.338+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:39.348 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:39.348 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:39.359 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:39.395 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-15T00:12:39.399 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:39.399 INFO:teuthology.orchestra.run.smithi171.stdout:4850 punch from 0x2e6f37f to 0x2e7727f, (0x7f00 bytes) 2024-09-15T00:12:39.403 INFO:teuthology.orchestra.run.smithi171.stdout:4851 read 0x3da1db thru 0x3df6d5 (0x54fb bytes) 2024-09-15T00:12:39.418 INFO:teuthology.orchestra.run.smithi171.stdout:4853 writesame 0xa5113f9 thru 0xa519557 data_size 0x1(0x815f bytes) 2024-09-15T00:12:39.418 INFO:teuthology.orchestra.run.smithi171.stdout:4854 writesame 0x3652e79 thru 0x365f8d1 data_size 0x1f(0xca59 bytes) 2024-09-15T00:12:39.440 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:39.439+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:39.451 INFO:teuthology.orchestra.run.smithi171.stdout:4855 writesame 0x3679ab1 thru 0x367b569 data_size 0x1(0x1ab9 bytes) 2024-09-15T00:12:39.474 INFO:teuthology.orchestra.run.smithi171.stdout:4856 punch from 0x2de40d8 to 0x2defc44, (0xbb6c bytes) 2024-09-15T00:12:39.474 INFO:teuthology.orchestra.run.smithi171.stdout:4858 write 0x983ced6 thru 0x9849858 (0xc983 bytes) 2024-09-15T00:12:39.480 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:39.480 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:39.487 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:39.488 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:39.488 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:39.493 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:39.493 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:39.498 INFO:teuthology.orchestra.run.smithi171.stdout:4863 write 0xdaaf317 thru 0xdab36e7 (0x43d1 bytes) 2024-09-15T00:12:39.505 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:39.505 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:39.505 INFO:teuthology.orchestra.run.smithi171.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T00:12:39.506 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:39.506 INFO:teuthology.orchestra.run.smithi171.stdout:4864 write 0x77a2d12 thru 0x77adf0f (0xb1fe bytes) 2024-09-15T00:12:39.511 INFO:teuthology.orchestra.run.smithi171.stdout:4865 read 0x18cec93 thru 0x18de80b (0xfb79 bytes) 2024-09-15T00:12:39.512 INFO:teuthology.orchestra.run.smithi171.stdout:4866 write 0xfae6e0 thru 0xfb1bc3 (0x34e4 bytes) 2024-09-15T00:12:39.524 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:39.525 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:39.535 INFO:teuthology.orchestra.run.smithi171.stdout:4868 punch from 0x8df1793 to 0x8df1ccc, (0x539 bytes) 2024-09-15T00:12:39.535 INFO:teuthology.orchestra.run.smithi171.stdout:4869 write 0x1a22ad4 thru 0x1a30f55 (0xe482 bytes) 2024-09-15T00:12:39.541 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:39.539+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:39.546 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:39.546 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:39.555 INFO:teuthology.orchestra.run.smithi171.stdout:4870 clone 41 order 24 su 4194304 sc 7 2024-09-15T00:12:39.563 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:39.564 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:39.582 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:39.582 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:39.600 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:39.600 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:39.604 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:12:39.604 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:12:39.625 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:39.626 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:39.635 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:39.679 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:39.678+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:39.743 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:39.742+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:39.755 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:39.755 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:39.755 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:39.755 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:39.755 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.756 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:39.756 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:39.756 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.180809991000004, 2024-09-15T00:12:39.756 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:12:39.756 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.756 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.756 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:39.756 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.756 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.756 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.756 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:12:39.756 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.756 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.757 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.757 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.757 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:39.757 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.757 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.757 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.757 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.757 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:39.757 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.757 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.757 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.757 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.757 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:12:39.757 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:12:39.758 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.758 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.758 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.758 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:12:39.758 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:12:39.758 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.758 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.758 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.758 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:12:39.758 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:12:39.758 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.758 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.758 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.759 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:12:39.759 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:12:39.759 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.759 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.759 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.759 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:12:39.759 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:12:39.759 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.759 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.759 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.759 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:12:39.759 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:12:39.760 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.760 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.760 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:39.760 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:12:39.760 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:12:39.760 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.760 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.760 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:39.760 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:12:39.760 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:12:39.760 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.760 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.760 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:39.760 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:12:39.761 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:12:39.761 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.761 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.761 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:39.761 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:12:39.761 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:12:39.761 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.761 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.761 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.761 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:12:39.761 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:12:39.761 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.761 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.762 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.762 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.762 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.762 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:39.762 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:39.762 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.180775299000004, 2024-09-15T00:12:39.762 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:12:39.762 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.762 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.762 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:39.762 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.762 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.762 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.762 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:12:39.763 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.763 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.763 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.763 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.763 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:39.763 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.763 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.763 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.763 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.763 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:39.763 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.763 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.763 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.764 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.764 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:39.764 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:12:39.764 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.764 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.764 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.764 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:12:39.764 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:12:39.764 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.764 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.764 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.764 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:12:39.764 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:12:39.765 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.765 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.765 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.765 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:12:39.765 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:12:39.765 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.765 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.765 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.765 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:12:39.765 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:12:39.765 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.765 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.765 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.765 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:12:39.766 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:12:39.766 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.766 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.766 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:39.766 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:12:39.766 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:12:39.766 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.766 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.766 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:39.766 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:12:39.766 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:12:39.766 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.766 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.766 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:39.767 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:12:39.767 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:12:39.767 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.767 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.767 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:39.767 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:12:39.767 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:12:39.767 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.767 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.767 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.767 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:12:39.767 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:12:39.767 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.767 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.768 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.768 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.768 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.768 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48937 3.1a 3:5e7e71d0:::rbd_data.11395b9829f3.0000000000000000:e [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.768 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:12:39.768 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.140682343999998, 2024-09-15T00:12:39.768 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072322336000000001, 2024-09-15T00:12:39.768 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.768 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.768 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.768 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.768 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.768 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.769 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48937 2024-09-15T00:12:39.769 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.769 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.769 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.769 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.769 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:12:39.769 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.769 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.769 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.769 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.769 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612042+0000", 2024-09-15T00:12:39.769 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.769 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.770 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.770 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.770 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612043+0000", 2024-09-15T00:12:39.770 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:12:39.770 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.770 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.770 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.770 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612045+0000", 2024-09-15T00:12:39.770 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2749999999999998e-06 2024-09-15T00:12:39.770 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.770 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.770 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.770 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612053+0000", 2024-09-15T00:12:39.770 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0439999999999996e-06 2024-09-15T00:12:39.771 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.771 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.771 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.771 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612062+0000", 2024-09-15T00:12:39.771 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6829999999999997e-06 2024-09-15T00:12:39.771 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.771 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.771 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.771 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684078+0000", 2024-09-15T00:12:39.771 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072016017000000002 2024-09-15T00:12:39.771 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.771 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.771 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.772 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684274+0000", 2024-09-15T00:12:39.772 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019636600000000001 2024-09-15T00:12:39.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.772 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.772 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.772 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684364+0000", 2024-09-15T00:12:39.772 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0420000000000005e-05 2024-09-15T00:12:39.772 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.772 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.772 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.772 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.772 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.772 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:39.773 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.140101610999999, 2024-09-15T00:12:39.773 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:12:39.773 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.773 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.773 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.773 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.773 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.773 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.773 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:12:39.773 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.773 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.773 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.773 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:39.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.774 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:39.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.774 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:39.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:12:39.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.775 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.775 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:12:39.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:12:39.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.775 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.775 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.775 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:12:39.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:12:39.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.775 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.775 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.775 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:12:39.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:12:39.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:12:39.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:12:39.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:12:39.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:12:39.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:12:39.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:12:39.777 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.777 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.777 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.777 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.777 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:39.777 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.140085079999999, 2024-09-15T00:12:39.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:12:39.777 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.777 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.777 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.777 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.777 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.778 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.778 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:12:39.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.778 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:39.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.779 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.779 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:39.779 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.779 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.779 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.779 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.779 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:12:39.779 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:12:39.779 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.779 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.779 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.779 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:12:39.779 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:12:39.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.780 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.780 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:12:39.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:12:39.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.780 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.780 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:12:39.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:12:39.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.780 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:12:39.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:12:39.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:12:39.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:12:39.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:12:39.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:12:39.781 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.781 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.781 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.782 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.782 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:39.782 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.140068300999999, 2024-09-15T00:12:39.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:12:39.782 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.782 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.782 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.782 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.782 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.782 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.782 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:12:39.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.783 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:39.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:39.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:39.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:12:39.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:12:39.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:12:39.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:12:39.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:12:39.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:12:39.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:12:39.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:12:39.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:12:39.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:12:39.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:12:39.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.786 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.786 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:12:39.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:12:39.786 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.786 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.786 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.786 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.786 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:39.786 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.140050571000003, 2024-09-15T00:12:39.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:12:39.786 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.787 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.787 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.787 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.787 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.787 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.787 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:12:39.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.787 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:39.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:39.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:39.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:12:39.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:12:39.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:12:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:12:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:12:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:12:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:12:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:12:39.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:12:39.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:12:39.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:12:39.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:12:39.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:12:39.790 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.790 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.790 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.791 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.791 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:39.791 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.140033406999997, 2024-09-15T00:12:39.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:12:39.791 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.791 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.791 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.791 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.791 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.791 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.791 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:12:39.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.792 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:39.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:39.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:39.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:12:39.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:12:39.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:12:39.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:12:39.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:12:39.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:12:39.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:12:39.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:12:39.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:12:39.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:12:39.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:12:39.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:12:39.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:12:39.795 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.795 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.795 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.796 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.796 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:39.796 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.140011731000001, 2024-09-15T00:12:39.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:12:39.796 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.796 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.796 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.796 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.796 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.796 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.796 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:12:39.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.796 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:39.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:39.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:39.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:12:39.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.798 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:12:39.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:12:39.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.798 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:12:39.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:12:39.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:12:39.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:12:39.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:12:39.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:12:39.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:12:39.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:12:39.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:12:39.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:12:39.800 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.800 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.800 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.800 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.800 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:39.801 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.139965607000001, 2024-09-15T00:12:39.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:12:39.801 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.801 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.801 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.801 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.801 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.801 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.801 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:12:39.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.801 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:39.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:39.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:39.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:12:39.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:12:39.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:12:39.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:12:39.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:12:39.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:12:39.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:12:39.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:12:39.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:12:39.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:12:39.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:12:39.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:12:39.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:12:39.805 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.805 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.805 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.805 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.805 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:39.805 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.139935850000001, 2024-09-15T00:12:39.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:12:39.805 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.805 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.806 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.806 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.806 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.806 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.806 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:12:39.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.806 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:39.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:39.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:12:39.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:12:39.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:12:39.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:12:39.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:12:39.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:12:39.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:12:39.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:12:39.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:12:39.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:12:39.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:12:39.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:12:39.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:12:39.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:12:39.809 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.809 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.809 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.810 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.810 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:39.810 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.139909531000001, 2024-09-15T00:12:39.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:12:39.810 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.810 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.810 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.810 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.810 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.810 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.810 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:12:39.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.811 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:39.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:39.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:39.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:12:39.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:12:39.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:12:39.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:12:39.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:12:39.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:12:39.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:12:39.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:12:39.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:12:39.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:12:39.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:12:39.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:12:39.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:12:39.814 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.814 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.814 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.814 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.814 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:39.814 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.139873669000004, 2024-09-15T00:12:39.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:12:39.815 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.815 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.815 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.815 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.815 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.815 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.815 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:12:39.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.815 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:39.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:39.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:39.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:12:39.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:12:39.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:12:39.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:12:39.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:12:39.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:12:39.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:12:39.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:12:39.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:12:39.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:12:39.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:12:39.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:12:39.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:12:39.819 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.819 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.819 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.819 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.819 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:39.819 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.139851950999997, 2024-09-15T00:12:39.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:12:39.819 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.819 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.819 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.819 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.819 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.820 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.820 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:12:39.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.820 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:39.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:39.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:12:39.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:12:39.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:12:39.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:12:39.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:12:39.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:12:39.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:12:39.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:12:39.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:12:39.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:12:39.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:12:39.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:12:39.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:12:39.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:12:39.823 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.823 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.823 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.823 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.824 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:39.824 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.139655484000002, 2024-09-15T00:12:39.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:12:39.824 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.824 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.824 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.824 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.824 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.824 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.824 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:12:39.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.825 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:39.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:39.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:39.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:12:39.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:12:39.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:12:39.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:12:39.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:12:39.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:12:39.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:12:39.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:12:39.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:12:39.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:12:39.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:12:39.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:12:39.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:12:39.828 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.828 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.828 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.828 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.828 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:39.828 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.139629644999999, 2024-09-15T00:12:39.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:12:39.829 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.829 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.829 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.829 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.829 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.829 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.829 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:12:39.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.829 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:39.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:39.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:39.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:12:39.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:12:39.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:12:39.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:12:39.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:12:39.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:12:39.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:12:39.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:12:39.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:12:39.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:12:39.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:12:39.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:12:39.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:12:39.832 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.833 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.833 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.833 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.833 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:39.833 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.139567999, 2024-09-15T00:12:39.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:12:39.833 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.833 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.833 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.833 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.833 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.833 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.834 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:12:39.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.834 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:39.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:39.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:12:39.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:12:39.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:12:39.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:12:39.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:12:39.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:12:39.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:12:39.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:12:39.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:12:39.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:12:39.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:12:39.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:12:39.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:12:39.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:12:39.837 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.837 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.837 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.837 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.837 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:39.837 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.139528671000001, 2024-09-15T00:12:39.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:12:39.838 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.838 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.838 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.838 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:12:39.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.838 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:39.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:39.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:12:39.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:12:39.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:12:39.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:12:39.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:12:39.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:12:39.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:12:39.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:12:39.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:12:39.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:12:39.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:12:39.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:12:39.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:12:39.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:12:39.841 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.841 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.842 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.842 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.842 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:39.842 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.137803271000003, 2024-09-15T00:12:39.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:12:39.842 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.842 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.842 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.842 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.842 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.842 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.842 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:12:39.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.843 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:39.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:39.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:12:39.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:12:39.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:12:39.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:12:39.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:12:39.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:12:39.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:12:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:12:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:12:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:12:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:12:39.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:12:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:12:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:12:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.137781109999999, 2024-09-15T00:12:39.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:12:39.847 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:39.847 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:39.847 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:39.847 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:39.847 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:39.847 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:39.847 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:12:39.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.847 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:39.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:39.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:39.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:39.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:39.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:39.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:39.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:39.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:12:39.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:39.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:12:39.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:12:39.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:39.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:12:39.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:12:39.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:39.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:12:39.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:12:39.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:39.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:12:39.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:12:39.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:39.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:12:39.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:12:39.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:39.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:39.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:39.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:12:39.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:12:39.850 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.850 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.851 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.851 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:39.851 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:39.851 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:39.852 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:39.853 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:39.850+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:39.952 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:39.951+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:40.015 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:40.016 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:40.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.016 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "MOSDPGPushReply(3.16 93/92 [PushReplyOp(3:6cafa6e5:::rbd_data.1139578eccad.0000000000000004:2a)])", 2024-09-15T00:12:40.016 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:40.013947+0000", 2024-09-15T00:12:40.016 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0841000000000003e-05, 2024-09-15T00:12:40.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1228000000000003e-05, 2024-09-15T00:12:40.016 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.016 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.016 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-15T00:12:40.016 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.013947+0000", 2024-09-15T00:12:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.013947+0000", 2024-09-15T00:12:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.013948+0000", 2024-09-15T00:12:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:12:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.013953+0000", 2024-09-15T00:12:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:12:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.013966+0000", 2024-09-15T00:12:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2621e-05 2024-09-15T00:12:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.013974+0000", 2024-09-15T00:12:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1040000000000007e-06 2024-09-15T00:12:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: ], 2024-09-15T00:12:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 1 2024-09-15T00:12:40.019 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:40.025 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:40.069 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:40.067+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:40.139 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:40.139 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:40.139 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:40.139 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:40.139 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:40.149 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:40.154 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:40.152+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:40.255 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:40.253+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:40.303 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:40.304 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:40.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:40.304 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:40.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.304 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:47986 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.304 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:40.304 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.829419076999997, 2024-09-15T00:12:40.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045287635, 2024-09-15T00:12:40.304 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.304 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.305 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:40.305 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.305 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.305 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.305 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 47986 2024-09-15T00:12:40.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.305 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:40.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:40.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:40.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8400000000000004e-07 2024-09-15T00:12:40.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474928+0000", 2024-09-15T00:12:40.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033009430000000002 2024-09-15T00:12:40.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474943+0000", 2024-09-15T00:12:40.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5172e-05 2024-09-15T00:12:40.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474951+0000", 2024-09-15T00:12:40.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1779999999999997e-06 2024-09-15T00:12:40.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475058+0000", 2024-09-15T00:12:40.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000107595 2024-09-15T00:12:40.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475200+0000", 2024-09-15T00:12:40.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014180400000000001 2024-09-15T00:12:40.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:40.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.478046+0000", 2024-09-15T00:12:40.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028464150000000001 2024-09-15T00:12:40.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:40.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516790+0000", 2024-09-15T00:12:40.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038743278999999999 2024-09-15T00:12:40.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:40.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516832+0000", 2024-09-15T00:12:40.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1974000000000002e-05 2024-09-15T00:12:40.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:40.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516872+0000", 2024-09-15T00:12:40.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9928999999999997e-05 2024-09-15T00:12:40.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516914+0000", 2024-09-15T00:12:40.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2762000000000002e-05 2024-09-15T00:12:40.310 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.310 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.310 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.310 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48957 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 4456448~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.310 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:12:40.310 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.691980565000001, 2024-09-15T00:12:40.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044512426000000001, 2024-09-15T00:12:40.311 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.311 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.311 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.311 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48957 2024-09-15T00:12:40.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.311 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:12:40.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609065+0000", 2024-09-15T00:12:40.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609066+0000", 2024-09-15T00:12:40.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:12:40.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609068+0000", 2024-09-15T00:12:40.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9700000000000002e-06 2024-09-15T00:12:40.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609073+0000", 2024-09-15T00:12:40.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7939999999999999e-06 2024-09-15T00:12:40.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609078+0000", 2024-09-15T00:12:40.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3209999999999998e-06 2024-09-15T00:12:40.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653082+0000", 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044003852000000003 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653136+0000", 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4024000000000003e-05 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653578+0000", 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044211599999999999 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.314 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.315 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48958 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.315 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:40.315 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.691962926000002, 2024-09-15T00:12:40.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044724939999999998, 2024-09-15T00:12:40.315 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.315 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.315 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.315 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.315 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.315 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.315 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48958 2024-09-15T00:12:40.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.316 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:40.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:40.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609083+0000", 2024-09-15T00:12:40.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:12:40.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609085+0000", 2024-09-15T00:12:40.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6610000000000001e-06 2024-09-15T00:12:40.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609090+0000", 2024-09-15T00:12:40.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2719999999999997e-06 2024-09-15T00:12:40.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609094+0000", 2024-09-15T00:12:40.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.907e-06 2024-09-15T00:12:40.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653618+0000", 2024-09-15T00:12:40.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044523759000000003 2024-09-15T00:12:40.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653703+0000", 2024-09-15T00:12:40.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5550000000000006e-05 2024-09-15T00:12:40.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653808+0000", 2024-09-15T00:12:40.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010445700000000001 2024-09-15T00:12:40.319 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.319 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.319 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.319 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48959 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.319 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:40.319 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.691946987000001, 2024-09-15T00:12:40.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044884741999999998, 2024-09-15T00:12:40.319 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.320 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.320 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.320 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.320 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.320 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.320 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48959 2024-09-15T00:12:40.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.320 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:40.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:40.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:40.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2899999999999999e-07 2024-09-15T00:12:40.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609101+0000", 2024-09-15T00:12:40.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.826e-06 2024-09-15T00:12:40.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609108+0000", 2024-09-15T00:12:40.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.165e-06 2024-09-15T00:12:40.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609113+0000", 2024-09-15T00:12:40.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7849999999999999e-06 2024-09-15T00:12:40.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653842+0000", 2024-09-15T00:12:40.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044728954000000001 2024-09-15T00:12:40.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653896+0000", 2024-09-15T00:12:40.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4593999999999999e-05 2024-09-15T00:12:40.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653984+0000", 2024-09-15T00:12:40.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7089000000000006e-05 2024-09-15T00:12:40.323 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.323 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.323 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.323 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48960 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 9961472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.324 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:40.324 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.691926467999998, 2024-09-15T00:12:40.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045084194000000001, 2024-09-15T00:12:40.324 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.324 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.324 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.324 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.324 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.324 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.324 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48960 2024-09-15T00:12:40.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.324 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:40.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:40.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609120+0000", 2024-09-15T00:12:40.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7099999999999997e-07 2024-09-15T00:12:40.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609122+0000", 2024-09-15T00:12:40.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:12:40.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609129+0000", 2024-09-15T00:12:40.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4159999999999998e-06 2024-09-15T00:12:40.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609134+0000", 2024-09-15T00:12:40.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7210000000000002e-06 2024-09-15T00:12:40.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654023+0000", 2024-09-15T00:12:40.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044888979000000002 2024-09-15T00:12:40.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654104+0000", 2024-09-15T00:12:40.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1108999999999999e-05 2024-09-15T00:12:40.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654204+0000", 2024-09-15T00:12:40.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9630000000000007e-05 2024-09-15T00:12:40.331 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.331 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.331 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.332 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48961 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.332 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:40.332 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.691905646999999, 2024-09-15T00:12:40.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045197821999999999, 2024-09-15T00:12:40.332 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.332 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.332 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.332 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.332 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.332 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.332 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48961 2024-09-15T00:12:40.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.332 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:40.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:40.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609141+0000", 2024-09-15T00:12:40.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:12:40.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609143+0000", 2024-09-15T00:12:40.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-15T00:12:40.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609150+0000", 2024-09-15T00:12:40.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2420000000000003e-06 2024-09-15T00:12:40.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609155+0000", 2024-09-15T00:12:40.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.028e-06 2024-09-15T00:12:40.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654232+0000", 2024-09-15T00:12:40.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045076702000000003 2024-09-15T00:12:40.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654281+0000", 2024-09-15T00:12:40.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9798e-05 2024-09-15T00:12:40.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654338+0000", 2024-09-15T00:12:40.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6498e-05 2024-09-15T00:12:40.336 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.336 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.336 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.336 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48962 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 13631488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.336 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:40.336 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.691884596999998, 2024-09-15T00:12:40.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045312458, 2024-09-15T00:12:40.336 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.336 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.336 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.336 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.336 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.337 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.337 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48962 2024-09-15T00:12:40.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.337 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:40.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:40.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609162+0000", 2024-09-15T00:12:40.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:40.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609164+0000", 2024-09-15T00:12:40.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0099999999999998e-06 2024-09-15T00:12:40.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609171+0000", 2024-09-15T00:12:40.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7530000000000001e-06 2024-09-15T00:12:40.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609178+0000", 2024-09-15T00:12:40.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1709999999999999e-06 2024-09-15T00:12:40.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654365+0000", 2024-09-15T00:12:40.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045187867999999999 2024-09-15T00:12:40.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654420+0000", 2024-09-15T00:12:40.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4376000000000003e-05 2024-09-15T00:12:40.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654474+0000", 2024-09-15T00:12:40.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3879999999999999e-05 2024-09-15T00:12:40.340 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.340 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.340 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.340 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48971 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.340 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:40.340 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.691853602999998, 2024-09-15T00:12:40.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045896151000000003, 2024-09-15T00:12:40.340 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.341 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.341 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.341 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.341 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.341 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.341 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48971 2024-09-15T00:12:40.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.341 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:40.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:40.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609193+0000", 2024-09-15T00:12:40.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7599999999999997e-07 2024-09-15T00:12:40.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609195+0000", 2024-09-15T00:12:40.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:12:40.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609210+0000", 2024-09-15T00:12:40.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4956999999999999e-05 2024-09-15T00:12:40.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609505+0000", 2024-09-15T00:12:40.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000295078 2024-09-15T00:12:40.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654498+0000", 2024-09-15T00:12:40.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044992277999999997 2024-09-15T00:12:40.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654541+0000", 2024-09-15T00:12:40.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3525000000000001e-05 2024-09-15T00:12:40.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655088+0000", 2024-09-15T00:12:40.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054726799999999995 2024-09-15T00:12:40.344 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.344 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.344 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.345 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48972 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 2097152~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.345 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:40.345 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.691527029, 2024-09-15T00:12:40.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046150885000000003, 2024-09-15T00:12:40.345 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.345 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.345 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.345 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.345 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.345 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.345 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48972 2024-09-15T00:12:40.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.345 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:40.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:40.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:40.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6400000000000002e-07 2024-09-15T00:12:40.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609523+0000", 2024-09-15T00:12:40.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5300000000000001e-06 2024-09-15T00:12:40.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609533+0000", 2024-09-15T00:12:40.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0023999999999999e-05 2024-09-15T00:12:40.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609539+0000", 2024-09-15T00:12:40.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3119999999999999e-06 2024-09-15T00:12:40.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655115+0000", 2024-09-15T00:12:40.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045576094999999997 2024-09-15T00:12:40.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655197+0000", 2024-09-15T00:12:40.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1129000000000002e-05 2024-09-15T00:12:40.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655670+0000", 2024-09-15T00:12:40.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00047313100000000001 2024-09-15T00:12:40.349 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.349 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.349 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.349 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.349 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:40.349 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.691498766999999, 2024-09-15T00:12:40.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:12:40.349 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.349 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.349 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.349 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.350 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.350 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.350 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:12:40.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.350 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:40.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:40.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:40.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:12:40.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:12:40.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:12:40.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:12:40.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:12:40.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:12:40.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:12:40.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:12:40.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:12:40.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:12:40.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:12:40.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:12:40.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:12:40.353 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.353 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.353 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.353 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.353 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:40.353 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.691477538000001, 2024-09-15T00:12:40.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:12:40.354 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.354 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.354 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.354 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.354 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.354 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.354 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:12:40.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.354 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:40.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:40.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:12:40.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:12:40.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:12:40.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:12:40.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:12:40.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:12:40.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:12:40.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:12:40.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:12:40.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:12:40.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:12:40.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:12:40.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:12:40.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:12:40.357 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.357 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.357 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.358 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.358 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:40.358 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.691451997000001, 2024-09-15T00:12:40.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:12:40.358 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.358 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.358 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.358 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.358 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.358 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.358 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:12:40.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.359 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:40.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:40.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:40.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:12:40.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:12:40.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:12:40.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:12:40.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:12:40.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:12:40.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:12:40.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:12:40.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:12:40.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:12:40.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:12:40.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:12:40.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:12:40.362 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.362 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.362 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.362 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.362 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:40.362 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.691429522, 2024-09-15T00:12:40.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:12:40.362 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.362 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.362 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.363 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.363 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.363 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.363 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:12:40.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.363 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:40.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:40.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:12:40.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:12:40.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:12:40.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:12:40.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:12:40.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:12:40.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:12:40.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:12:40.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:12:40.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:12:40.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:12:40.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:12:40.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:12:40.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:12:40.366 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.366 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.366 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.367 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.367 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:40.367 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.691409012000001, 2024-09-15T00:12:40.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:12:40.367 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.367 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.367 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.367 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.367 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.367 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.367 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:12:40.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.368 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:40.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:40.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:40.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:12:40.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:12:40.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:12:40.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:12:40.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:12:40.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:12:40.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:12:40.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:12:40.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:12:40.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:12:40.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:12:40.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:12:40.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:12:40.372 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.372 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.372 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.372 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.372 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:40.372 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.691388697000001, 2024-09-15T00:12:40.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:12:40.372 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.372 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.372 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.373 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.373 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.373 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.373 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:12:40.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.373 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:40.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:40.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:12:40.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:12:40.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:12:40.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:12:40.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:12:40.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:12:40.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:12:40.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:12:40.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:12:40.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:12:40.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:12:40.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:12:40.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:12:40.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:12:40.376 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.376 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.376 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.377 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.377 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:40.377 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.691363920999997, 2024-09-15T00:12:40.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:12:40.377 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.377 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.377 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.377 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.377 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.377 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.377 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:12:40.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.378 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:40.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:40.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:40.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:12:40.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:12:40.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:12:40.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:12:40.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:12:40.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:12:40.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:12:40.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:12:40.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:12:40.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:12:40.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:12:40.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:12:40.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:12:40.381 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.381 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.381 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.381 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.382 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:40.382 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.512022534000003, 2024-09-15T00:12:40.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:12:40.382 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.382 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.382 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:40.382 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.382 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.382 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.382 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:12:40.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.382 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:40.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:40.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:12:40.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:12:40.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:12:40.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:12:40.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:12:40.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:12:40.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:12:40.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:12:40.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:12:40.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:12:40.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:12:40.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:12:40.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:40.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:12:40.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:12:40.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:40.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:12:40.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:12:40.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:40.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:12:40.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:12:40.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:40.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:12:40.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:12:40.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:12:40.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:12:40.387 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.387 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.387 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.388 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:12:40.388 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:40.388 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.495069082000001, 2024-09-15T00:12:40.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:12:40.388 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.388 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.388 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.388 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:40.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:40.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:12:40.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:12:40.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:12:40.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:12:40.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:12:40.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:12:40.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:12:40.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:12:40.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:12:40.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:12:40.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:12:40.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:12:40.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:12:40.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:12:40.391 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.391 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.392 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.392 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:12:40.392 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:40.392 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.200638175000002, 2024-09-15T00:12:40.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:12:40.392 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.392 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.392 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:40.392 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.392 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.392 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.392 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:12:40.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.393 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:40.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:40.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:12:40.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:12:40.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:12:40.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:12:40.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:12:40.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:12:40.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:12:40.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:12:40.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:12:40.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:12:40.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:12:40.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:12:40.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:40.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:12:40.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:12:40.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:40.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:12:40.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:12:40.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:40.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:12:40.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:12:40.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:40.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:12:40.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:12:40.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:12:40.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:12:40.397 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.397 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.397 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.398 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:12:40.398 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:40.398 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.188182361000003, 2024-09-15T00:12:40.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:12:40.398 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.398 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.398 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.398 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:40.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:40.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:12:40.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:12:40.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:12:40.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:12:40.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:12:40.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:12:40.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:12:40.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:12:40.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:12:40.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:12:40.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:12:40.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:12:40.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:12:40.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:12:40.401 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.401 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.401 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.402 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.402 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.402 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:40.403 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:40.404 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:40.401+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:40.504 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:40.502+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:40.547 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:40.547 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:40.547 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:40.547 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:40.557 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:40.605 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:40.603+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:40.660 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:40.660 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:40.669 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:40.706 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:40.704+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:40.772 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:19308 3.10 3:0ee1524d:::rbd_data.11398e1a091a.0000000000000017:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-15T00:12:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.785122115, 2024-09-15T00:12:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02547147, 2024-09-15T00:12:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 19308 2024-09-15T00:12:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985182+0000", 2024-09-15T00:12:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6500000000000001e-06 2024-09-15T00:12:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985189+0000", 2024-09-15T00:12:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9949999999999999e-06 2024-09-15T00:12:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985205+0000", 2024-09-15T00:12:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6055000000000001e-05 2024-09-15T00:12:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985212+0000", 2024-09-15T00:12:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:12:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985261+0000", 2024-09-15T00:12:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8380000000000001e-05 2024-09-15T00:12:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985311+0000", 2024-09-15T00:12:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0763999999999997e-05 2024-09-15T00:12:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.986489+0000", 2024-09-15T00:12:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0011773420000000001 2024-09-15T00:12:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010514+0000", 2024-09-15T00:12:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024025392999999999 2024-09-15T00:12:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010535+0000", 2024-09-15T00:12:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0652000000000001e-05 2024-09-15T00:12:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010597+0000", 2024-09-15T00:12:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2411999999999998e-05 2024-09-15T00:12:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010652+0000", 2024-09-15T00:12:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5074999999999997e-05 2024-09-15T00:12:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.102945147, 2024-09-15T00:12:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:12:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.780 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:40.780 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.780 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.780 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.780 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:12:40.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.780 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.780 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.780 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:40.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:40.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:12:40.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:12:40.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:12:40.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:12:40.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.782 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.782 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:12:40.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:12:40.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.782 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.782 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:12:40.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:12:40.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.782 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.782 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:12:40.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:12:40.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:12:40.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:12:40.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:40.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:12:40.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:12:40.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:40.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:12:40.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:12:40.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:40.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:12:40.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:12:40.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:40.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:12:40.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:12:40.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:12:40.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:12:40.785 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.785 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.785 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.785 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65520 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:40.785 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:40.785 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.887890091999999, 2024-09-15T00:12:40.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024513282000000001, 2024-09-15T00:12:40.786 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.786 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.786 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:40.786 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.786 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.786 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.786 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65520 2024-09-15T00:12:40.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.786 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.786 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.786 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:40.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:40.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882413+0000", 2024-09-15T00:12:40.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.39e-07 2024-09-15T00:12:40.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882415+0000", 2024-09-15T00:12:40.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8020000000000001e-06 2024-09-15T00:12:40.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882421+0000", 2024-09-15T00:12:40.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7119999999999997e-06 2024-09-15T00:12:40.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882425+0000", 2024-09-15T00:12:40.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2660000000000003e-06 2024-09-15T00:12:40.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906919+0000", 2024-09-15T00:12:40.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024494438 2024-09-15T00:12:40.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.906926+0000", 2024-09-15T00:12:40.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.725e-06 2024-09-15T00:12:40.789 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.789 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.789 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.790 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:65525 3.a 3.48b1ed2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:40.790 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:40.790 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.887775224000002, 2024-09-15T00:12:40.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024517753, 2024-09-15T00:12:40.790 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.790 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.790 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:12:40.790 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.790 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.790 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.790 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 65525 2024-09-15T00:12:40.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.791 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.791 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.791 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:40.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.791 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.791 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:40.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882528+0000", 2024-09-15T00:12:40.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:12:40.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882530+0000", 2024-09-15T00:12:40.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.804e-06 2024-09-15T00:12:40.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882535+0000", 2024-09-15T00:12:40.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5500000000000002e-06 2024-09-15T00:12:40.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.882540+0000", 2024-09-15T00:12:40.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6909999999999996e-06 2024-09-15T00:12:40.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907039+0000", 2024-09-15T00:12:40.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024498809 2024-09-15T00:12:40.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.907045+0000", 2024-09-15T00:12:40.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5459999999999997e-06 2024-09-15T00:12:40.794 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.794 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.794 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.794 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66541 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1449984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:40.794 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:12:40.794 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.856167634999998, 2024-09-15T00:12:40.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024574262999999999, 2024-09-15T00:12:40.794 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.794 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.794 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.795 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.795 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.795 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.795 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66541 2024-09-15T00:12:40.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.795 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:12:40.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914135+0000", 2024-09-15T00:12:40.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914136+0000", 2024-09-15T00:12:40.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3599999999999999e-07 2024-09-15T00:12:40.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914137+0000", 2024-09-15T00:12:40.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9309999999999998e-06 2024-09-15T00:12:40.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914143+0000", 2024-09-15T00:12:40.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0120000000000002e-06 2024-09-15T00:12:40.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914148+0000", 2024-09-15T00:12:40.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.707e-06 2024-09-15T00:12:40.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938536+0000", 2024-09-15T00:12:40.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024387671999999999 2024-09-15T00:12:40.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.798 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938618+0000", 2024-09-15T00:12:40.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1811999999999995e-05 2024-09-15T00:12:40.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938709+0000", 2024-09-15T00:12:40.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1792999999999993e-05 2024-09-15T00:12:40.799 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.799 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.799 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.799 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66542 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1560576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:40.799 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:40.799 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.856148494000003, 2024-09-15T00:12:40.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024740410000000001, 2024-09-15T00:12:40.799 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.800 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.800 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.800 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.800 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.800 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.800 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66542 2024-09-15T00:12:40.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.800 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:40.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:40.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914155+0000", 2024-09-15T00:12:40.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:12:40.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914157+0000", 2024-09-15T00:12:40.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:12:40.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914162+0000", 2024-09-15T00:12:40.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:40.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914167+0000", 2024-09-15T00:12:40.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3789999999999999e-06 2024-09-15T00:12:40.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938730+0000", 2024-09-15T00:12:40.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024562694 2024-09-15T00:12:40.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938803+0000", 2024-09-15T00:12:40.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3325999999999996e-05 2024-09-15T00:12:40.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938895+0000", 2024-09-15T00:12:40.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1884000000000003e-05 2024-09-15T00:12:40.804 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.804 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.804 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.804 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66543 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1572864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:40.804 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:40.804 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.856118234999997, 2024-09-15T00:12:40.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024893976000000002, 2024-09-15T00:12:40.804 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.804 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.804 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.805 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.805 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.805 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.805 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66543 2024-09-15T00:12:40.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.805 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:40.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:40.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:40.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:40.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914188+0000", 2024-09-15T00:12:40.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:12:40.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914193+0000", 2024-09-15T00:12:40.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4129999999999997e-06 2024-09-15T00:12:40.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914198+0000", 2024-09-15T00:12:40.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9309999999999999e-06 2024-09-15T00:12:40.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938914+0000", 2024-09-15T00:12:40.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024716380999999999 2024-09-15T00:12:40.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938993+0000", 2024-09-15T00:12:40.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8763000000000005e-05 2024-09-15T00:12:40.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939079+0000", 2024-09-15T00:12:40.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5372999999999997e-05 2024-09-15T00:12:40.808 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.808 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.808 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.809 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66544 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1683456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:40.809 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:40.809 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.856095832000001, 2024-09-15T00:12:40.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025051225, 2024-09-15T00:12:40.809 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.809 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.809 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.809 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.809 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.809 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.809 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66544 2024-09-15T00:12:40.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.810 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:40.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:40.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:40.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:12:40.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914210+0000", 2024-09-15T00:12:40.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2450000000000001e-06 2024-09-15T00:12:40.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914215+0000", 2024-09-15T00:12:40.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7869999999999998e-06 2024-09-15T00:12:40.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914220+0000", 2024-09-15T00:12:40.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7899999999999999e-06 2024-09-15T00:12:40.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939105+0000", 2024-09-15T00:12:40.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024884381000000001 2024-09-15T00:12:40.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939177+0000", 2024-09-15T00:12:40.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2138999999999995e-05 2024-09-15T00:12:40.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939258+0000", 2024-09-15T00:12:40.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1462e-05 2024-09-15T00:12:40.813 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.813 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.813 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.813 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66545 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1695744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:40.813 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:40.813 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.856073557999999, 2024-09-15T00:12:40.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025209788, 2024-09-15T00:12:40.813 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.814 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.814 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.814 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.814 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.814 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.814 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66545 2024-09-15T00:12:40.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.814 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:40.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:40.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914230+0000", 2024-09-15T00:12:40.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1099999999999996e-07 2024-09-15T00:12:40.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914232+0000", 2024-09-15T00:12:40.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3080000000000002e-06 2024-09-15T00:12:40.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914238+0000", 2024-09-15T00:12:40.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.02e-06 2024-09-15T00:12:40.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914243+0000", 2024-09-15T00:12:40.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1200000000000001e-06 2024-09-15T00:12:40.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939285+0000", 2024-09-15T00:12:40.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025042212000000001 2024-09-15T00:12:40.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939361+0000", 2024-09-15T00:12:40.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5558999999999999e-05 2024-09-15T00:12:40.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939439+0000", 2024-09-15T00:12:40.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8058000000000004e-05 2024-09-15T00:12:40.817 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.817 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.817 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.818 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66546 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:40.818 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:40.818 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.856053582999998, 2024-09-15T00:12:40.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025375960999999999, 2024-09-15T00:12:40.818 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.818 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.818 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.818 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.818 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.818 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.818 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66546 2024-09-15T00:12:40.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.819 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:40.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:40.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914250+0000", 2024-09-15T00:12:40.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:12:40.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914252+0000", 2024-09-15T00:12:40.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.052e-06 2024-09-15T00:12:40.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914258+0000", 2024-09-15T00:12:40.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9039999999999997e-06 2024-09-15T00:12:40.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914262+0000", 2024-09-15T00:12:40.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4719999999999997e-06 2024-09-15T00:12:40.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939466+0000", 2024-09-15T00:12:40.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025203550000000002 2024-09-15T00:12:40.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939537+0000", 2024-09-15T00:12:40.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1780999999999996e-05 2024-09-15T00:12:40.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939625+0000", 2024-09-15T00:12:40.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7828e-05 2024-09-15T00:12:40.822 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.822 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.822 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.822 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66547 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1818624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:40.822 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:40.822 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.856033910000001, 2024-09-15T00:12:40.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025552638999999999, 2024-09-15T00:12:40.823 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.823 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.823 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.823 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.823 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.823 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.823 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66547 2024-09-15T00:12:40.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.823 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:40.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:40.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:40.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:40.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914271+0000", 2024-09-15T00:12:40.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8920000000000001e-06 2024-09-15T00:12:40.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914277+0000", 2024-09-15T00:12:40.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:40.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914282+0000", 2024-09-15T00:12:40.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7190000000000001e-06 2024-09-15T00:12:40.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939654+0000", 2024-09-15T00:12:40.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025372372000000001 2024-09-15T00:12:40.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939734+0000", 2024-09-15T00:12:40.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9946999999999999e-05 2024-09-15T00:12:40.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939822+0000", 2024-09-15T00:12:40.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7680000000000001e-05 2024-09-15T00:12:40.826 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.827 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.827 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.827 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66548 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:40.827 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:40.827 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.856003522999998, 2024-09-15T00:12:40.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025712727000000001, 2024-09-15T00:12:40.827 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.827 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.827 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.827 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.827 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.827 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.828 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66548 2024-09-15T00:12:40.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.828 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:40.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:40.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914300+0000", 2024-09-15T00:12:40.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3599999999999999e-07 2024-09-15T00:12:40.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914302+0000", 2024-09-15T00:12:40.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3369999999999998e-06 2024-09-15T00:12:40.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914313+0000", 2024-09-15T00:12:40.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1399999999999999e-05 2024-09-15T00:12:40.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914319+0000", 2024-09-15T00:12:40.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9100000000000002e-06 2024-09-15T00:12:40.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939850+0000", 2024-09-15T00:12:40.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025531087000000001 2024-09-15T00:12:40.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939932+0000", 2024-09-15T00:12:40.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1534999999999998e-05 2024-09-15T00:12:40.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940012+0000", 2024-09-15T00:12:40.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0022000000000001e-05 2024-09-15T00:12:40.832 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.832 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.832 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.833 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66549 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1941504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:40.833 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:40.833 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.855938432000002, 2024-09-15T00:12:40.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025813836999999999, 2024-09-15T00:12:40.833 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.833 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.833 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.833 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.833 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.833 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.833 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66549 2024-09-15T00:12:40.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.834 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:40.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:40.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914365+0000", 2024-09-15T00:12:40.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2799999999999995e-07 2024-09-15T00:12:40.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914368+0000", 2024-09-15T00:12:40.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.92e-06 2024-09-15T00:12:40.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914376+0000", 2024-09-15T00:12:40.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8110000000000003e-06 2024-09-15T00:12:40.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914381+0000", 2024-09-15T00:12:40.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5119999999999999e-06 2024-09-15T00:12:40.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940040+0000", 2024-09-15T00:12:40.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025658271999999999 2024-09-15T00:12:40.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940103+0000", 2024-09-15T00:12:40.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2891000000000005e-05 2024-09-15T00:12:40.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940178+0000", 2024-09-15T00:12:40.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5703000000000003e-05 2024-09-15T00:12:40.837 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.837 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.837 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.837 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66550 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:40.837 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:40.838 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.855794852000002, 2024-09-15T00:12:40.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025820701000000001, 2024-09-15T00:12:40.838 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.838 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.838 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.838 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66550 2024-09-15T00:12:40.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.838 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:40.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:40.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914509+0000", 2024-09-15T00:12:40.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1999999999999999e-07 2024-09-15T00:12:40.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914513+0000", 2024-09-15T00:12:40.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1640000000000003e-06 2024-09-15T00:12:40.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914522+0000", 2024-09-15T00:12:40.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7639999999999994e-06 2024-09-15T00:12:40.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914527+0000", 2024-09-15T00:12:40.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9170000000000003e-06 2024-09-15T00:12:40.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940199+0000", 2024-09-15T00:12:40.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025672037000000002 2024-09-15T00:12:40.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940269+0000", 2024-09-15T00:12:40.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9208999999999994e-05 2024-09-15T00:12:40.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940329+0000", 2024-09-15T00:12:40.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.999e-05 2024-09-15T00:12:40.842 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.842 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.842 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.842 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66551 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:40.842 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:40.842 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.855738105, 2024-09-15T00:12:40.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025917345000000001, 2024-09-15T00:12:40.843 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.843 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.843 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.843 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.843 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.843 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.843 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66551 2024-09-15T00:12:40.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.843 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:40.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:40.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:40.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1699999999999997e-07 2024-09-15T00:12:40.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914569+0000", 2024-09-15T00:12:40.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.225e-06 2024-09-15T00:12:40.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914582+0000", 2024-09-15T00:12:40.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3648999999999999e-05 2024-09-15T00:12:40.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914588+0000", 2024-09-15T00:12:40.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8449999999999999e-06 2024-09-15T00:12:40.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940355+0000", 2024-09-15T00:12:40.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025767110999999999 2024-09-15T00:12:40.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940423+0000", 2024-09-15T00:12:40.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7574e-05 2024-09-15T00:12:40.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940482+0000", 2024-09-15T00:12:40.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9224000000000001e-05 2024-09-15T00:12:40.847 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.847 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.847 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.847 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:88237 3.a 3:52924f23:::rbd_data.11396cb1f0e4.0000000000000001:16 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-15T00:12:40.847 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:40.847 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.492573141999998, 2024-09-15T00:12:40.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024612426999999999, 2024-09-15T00:12:40.847 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.847 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.847 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.848 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88237 2024-09-15T00:12:40.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.848 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:40.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:40.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277730+0000", 2024-09-15T00:12:40.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7899999999999999e-07 2024-09-15T00:12:40.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277732+0000", 2024-09-15T00:12:40.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1679999999999998e-06 2024-09-15T00:12:40.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277738+0000", 2024-09-15T00:12:40.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.207e-06 2024-09-15T00:12:40.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277743+0000", 2024-09-15T00:12:40.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.639e-06 2024-09-15T00:12:40.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.301683+0000", 2024-09-15T00:12:40.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.023939750999999999 2024-09-15T00:12:40.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.301765+0000", 2024-09-15T00:12:40.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2566999999999997e-05 2024-09-15T00:12:40.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302342+0000", 2024-09-15T00:12:40.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00057671600000000001 2024-09-15T00:12:40.852 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.852 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.852 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.852 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:88238 3.a 3:52924f23:::rbd_data.11396cb1f0e4.0000000000000001:16 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-15T00:12:40.852 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:40.853 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.492553807999997, 2024-09-15T00:12:40.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024851132000000001, 2024-09-15T00:12:40.853 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.853 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.853 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:40.853 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.853 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.853 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.853 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88238 2024-09-15T00:12:40.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.853 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:40.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:40.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277750+0000", 2024-09-15T00:12:40.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3300000000000002e-07 2024-09-15T00:12:40.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277751+0000", 2024-09-15T00:12:40.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:12:40.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277757+0000", 2024-09-15T00:12:40.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.767e-06 2024-09-15T00:12:40.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277769+0000", 2024-09-15T00:12:40.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1874e-05 2024-09-15T00:12:40.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302374+0000", 2024-09-15T00:12:40.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024604669999999999 2024-09-15T00:12:40.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302460+0000", 2024-09-15T00:12:40.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6489999999999994e-05 2024-09-15T00:12:40.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302600+0000", 2024-09-15T00:12:40.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013985700000000001 2024-09-15T00:12:40.857 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.857 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.857 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.858 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125618 3.10 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:40.858 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:12:40.858 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.931806065, 2024-09-15T00:12:40.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030860942999999998, 2024-09-15T00:12:40.858 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.858 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.858 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:40.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.859 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125618 2024-09-15T00:12:40.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.859 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:12:40.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:12:40.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838499+0000", 2024-09-15T00:12:40.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6189999999999998e-06 2024-09-15T00:12:40.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839599+0000", 2024-09-15T00:12:40.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001099764 2024-09-15T00:12:40.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839620+0000", 2024-09-15T00:12:40.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0764e-05 2024-09-15T00:12:40.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839631+0000", 2024-09-15T00:12:40.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1335000000000001e-05 2024-09-15T00:12:40.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839725+0000", 2024-09-15T00:12:40.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4217999999999998e-05 2024-09-15T00:12:40.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839910+0000", 2024-09-15T00:12:40.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000184298 2024-09-15T00:12:40.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:40.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843171+0000", 2024-09-15T00:12:40.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032613540000000002 2024-09-15T00:12:40.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:40.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869187+0000", 2024-09-15T00:12:40.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026015467 2024-09-15T00:12:40.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:40.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869240+0000", 2024-09-15T00:12:40.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3733000000000002e-05 2024-09-15T00:12:40.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:40.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869309+0000", 2024-09-15T00:12:40.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8381999999999997e-05 2024-09-15T00:12:40.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869358+0000", 2024-09-15T00:12:40.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-15T00:12:40.864 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.864 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.864 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.865 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125619 3.a 3:507b29bf:::rbd_data.11396481e1ec.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:40.865 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:12:40.865 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.930643935999999, 2024-09-15T00:12:40.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045768870000000003, 2024-09-15T00:12:40.865 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.865 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.865 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:40.865 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.865 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.865 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.865 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125619 2024-09-15T00:12:40.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.866 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:12:40.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:12:40.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839660+0000", 2024-09-15T00:12:40.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.22e-06 2024-09-15T00:12:40.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840511+0000", 2024-09-15T00:12:40.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000851284 2024-09-15T00:12:40.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840522+0000", 2024-09-15T00:12:40.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0569e-05 2024-09-15T00:12:40.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840527+0000", 2024-09-15T00:12:40.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.412e-06 2024-09-15T00:12:40.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840573+0000", 2024-09-15T00:12:40.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5247000000000002e-05 2024-09-15T00:12:40.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840682+0000", 2024-09-15T00:12:40.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109368 2024-09-15T00:12:40.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:40.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843976+0000", 2024-09-15T00:12:40.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294445 2024-09-15T00:12:40.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:40.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870833+0000", 2024-09-15T00:12:40.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026856944000000001 2024-09-15T00:12:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885344+0000", 2024-09-15T00:12:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014510744000000001 2024-09-15T00:12:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885382+0000", 2024-09-15T00:12:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7821000000000003e-05 2024-09-15T00:12:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885428+0000", 2024-09-15T00:12:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5816000000000003e-05 2024-09-15T00:12:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125621 3.a 3:53594db9:::rbd_data.11396481e1ec.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:12:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.917120557000001, 2024-09-15T00:12:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032481797, 2024-09-15T00:12:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:40.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:40.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:40.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:40.872 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125621 2024-09-15T00:12:40.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.872 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:40.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:40.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:12:40.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:40.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:12:40.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:40.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:40.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853185+0000", 2024-09-15T00:12:40.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.43e-06 2024-09-15T00:12:40.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:40.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854125+0000", 2024-09-15T00:12:40.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000940539 2024-09-15T00:12:40.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:40.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854141+0000", 2024-09-15T00:12:40.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5568000000000001e-05 2024-09-15T00:12:40.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:40.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854148+0000", 2024-09-15T00:12:40.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6529999999999994e-06 2024-09-15T00:12:40.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:40.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870872+0000", 2024-09-15T00:12:40.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016723135 2024-09-15T00:12:40.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:40.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870975+0000", 2024-09-15T00:12:40.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010314099999999999 2024-09-15T00:12:40.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:40.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.873082+0000", 2024-09-15T00:12:40.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021069859999999999 2024-09-15T00:12:40.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:40.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885439+0000", 2024-09-15T00:12:40.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012357343999999999 2024-09-15T00:12:40.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:40.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885569+0000", 2024-09-15T00:12:40.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012995199999999999 2024-09-15T00:12:40.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:40.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885624+0000", 2024-09-15T00:12:40.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5281999999999997e-05 2024-09-15T00:12:40.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:40.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:40.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:40.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885664+0000", 2024-09-15T00:12:40.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9767000000000001e-05 2024-09-15T00:12:40.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.877 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:40.877 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:40.877 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:40.879 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:40.880 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:40.877+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:40.979 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:40.978+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:40.982 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:40.982 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:40.982 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:40.982 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:40.991 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:41.080 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:41.078+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:41.134 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:41.134 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:41.143 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:41.188 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:41.186+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:41.231 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:12:41.247 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:41.247 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:41.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:41.248 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:41.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.248 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:41.248 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:41.248 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 75.166787936999995, 2024-09-15T00:12:41.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:12:41.248 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.248 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.248 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:41.248 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.248 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.248 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.249 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:12:41.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.249 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:41.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:41.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:12:41.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:12:41.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:12:41.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:12:41.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:12:41.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:12:41.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:12:41.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:12:41.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:12:41.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:12:41.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:12:41.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:12:41.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:41.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:12:41.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:12:41.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:41.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:12:41.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:12:41.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:41.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:12:41.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:12:41.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:41.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:12:41.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:12:41.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:12:41.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:12:41.254 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.254 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.254 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.254 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:12:41.254 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:41.254 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 75.154047395000006, 2024-09-15T00:12:41.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:12:41.254 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.254 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.254 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:41.254 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:41.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:41.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:12:41.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:12:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:12:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:12:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:12:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:12:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:12:41.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:12:41.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:12:41.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:12:41.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:12:41.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:12:41.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:12:41.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:12:41.259 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.259 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.259 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.259 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:41.259 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:41.259 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 75.021691636, 2024-09-15T00:12:41.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:12:41.259 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.259 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.259 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:41.259 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.260 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.260 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.260 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:12:41.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.260 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:41.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:41.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:12:41.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:12:41.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:12:41.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:12:41.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:12:41.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:12:41.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:12:41.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:12:41.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:12:41.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:12:41.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:12:41.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:12:41.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:41.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:12:41.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:12:41.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:41.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:12:41.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:12:41.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:41.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:12:41.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:12:41.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:41.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:12:41.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:12:41.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:12:41.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:12:41.265 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.265 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.265 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.266 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:12:41.266 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:41.266 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 75.008229990999993, 2024-09-15T00:12:41.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:12:41.266 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.266 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.266 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:41.266 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:41.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:41.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:12:41.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:12:41.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:12:41.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:12:41.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:12:41.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:12:41.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:12:41.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:12:41.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:12:41.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:12:41.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:12:41.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:12:41.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:12:41.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:12:41.269 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.269 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.269 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.823228508, 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:41.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:41.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:12:41.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:12:41.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.272 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.272 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:12:41.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:12:41.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.272 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.272 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:12:41.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:12:41.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.272 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.272 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:12:41.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:12:41.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:12:41.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:12:41.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:12:41.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:12:41.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:41.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:12:41.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:12:41.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:41.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:12:41.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:12:41.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:41.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:12:41.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:12:41.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:41.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:12:41.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:12:41.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:12:41.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:12:41.275 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.275 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.275 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.276 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:12:41.276 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:41.276 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.810352241000004, 2024-09-15T00:12:41.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:12:41.276 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.276 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.276 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:41.276 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.276 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:41.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:41.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:12:41.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:12:41.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:12:41.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:12:41.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:12:41.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:12:41.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:12:41.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:12:41.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:12:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:12:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:12:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:12:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:12:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:12:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:12:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 60.351837097000001, 2024-09-15T00:12:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:12:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:12:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:12:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:12:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:12:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:12:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:12:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:12:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:12:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:12:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:12:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:12:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:12:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:12:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:12:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:12:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:12:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:12:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:12:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:12:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:12:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:12:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:12:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.634326537, 2024-09-15T00:12:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:12:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:12:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:12:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:12:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:12:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:12:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:12:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:12:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:12:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:12:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:12:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:12:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:12:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:12:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.634307927999998, 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:12:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:12:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:12:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:12:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:12:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:12:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:12:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:12:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:12:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:12:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:12:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:12:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.634282603999999, 2024-09-15T00:12:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:12:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:12:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:12:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:12:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:12:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:12:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:12:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:12:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:12:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:12:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:12:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:12:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:12:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:12:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:12:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.634264279999996, 2024-09-15T00:12:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:12:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:12:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:12:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:12:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:12:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:12:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:12:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:12:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:12:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:12:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:12:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:12:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:12:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:12:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:12:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.634081707, 2024-09-15T00:12:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:12:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:12:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:12:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:12:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:12:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:12:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:12:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:12:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:12:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:12:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:12:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:12:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:12:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:12:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.634052865999998, 2024-09-15T00:12:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:12:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:12:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:12:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:12:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:12:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:12:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:12:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:12:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:12:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:12:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:12:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:12:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:12:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:12:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.634030654999997, 2024-09-15T00:12:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:12:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:12:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:12:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:12:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:12:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:12:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:12:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:12:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:12:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:12:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:12:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:12:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:12:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:12:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.634009495999997, 2024-09-15T00:12:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:12:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:12:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:12:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:12:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:12:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:12:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:12:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:12:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:12:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:12:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:12:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:12:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:12:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:12:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.633979547999999, 2024-09-15T00:12:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:12:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:12:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:12:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:12:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:12:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:12:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:12:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:12:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:12:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:12:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:12:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.633957584000001, 2024-09-15T00:12:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:12:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:12:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:12:41.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:12:41.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:12:41.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:12:41.328 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.328 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.328 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.328 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:41.328 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:41.328 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.633940156999998, 2024-09-15T00:12:41.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:12:41.329 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.329 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.329 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:41.329 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.329 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.329 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.329 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:12:41.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.329 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:41.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:41.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:12:41.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:12:41.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:12:41.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:12:41.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:12:41.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:41.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:12:41.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:12:41.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:12:41.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:12:41.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:12:41.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:12:41.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:12:41.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:12:41.332 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.332 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.332 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.333 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:41.333 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:41.333 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.633918358000003, 2024-09-15T00:12:41.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:12:41.333 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.333 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.333 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:41.333 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.333 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.333 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.333 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:12:41.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:12:41.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:12:41.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:12:41.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:12:41.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:12:41.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:12:41.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:12:41.336 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.336 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.337 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.337 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:41.337 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:41.337 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.584689998999998, 2024-09-15T00:12:41.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:12:41.337 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:41.337 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:41.337 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:41.337 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:41.337 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:41.337 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:41.337 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:12:41.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:12:41.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:41.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:12:41.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:12:41.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:41.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:12:41.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:12:41.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:41.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:12:41.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:12:41.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:41.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:12:41.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:12:41.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:41.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:12:41.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:12:41.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:41.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:12:41.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:12:41.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:41.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:41.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:41.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:12:41.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:12:41.342 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.342 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.342 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.342 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:41.342 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:41.342 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:41.344 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:41.342+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:41.344 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:41.445 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:41.443+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:41.449 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:41.450 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:41.450 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:41.450 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:41.459 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:41.545 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:41.544+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:41.562 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:41.563 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:41.563 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:41.563 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:41.563 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:41.571 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:41.605 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:12:41.658 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:41.657+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:41.679 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:41.679 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:41.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:41.679 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:41.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.679 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-15T00:12:41.679 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:41.680 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.151656544000005, 2024-09-15T00:12:41.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077021529999999998, 2024-09-15T00:12:41.680 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.680 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.680 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:41.680 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:41.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:41.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524698+0000", 2024-09-15T00:12:41.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6300000000000001e-07 2024-09-15T00:12:41.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524778+0000", 2024-09-15T00:12:41.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0598000000000003e-05 2024-09-15T00:12:41.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524785+0000", 2024-09-15T00:12:41.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6359999999999997e-06 2024-09-15T00:12:41.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524788+0000", 2024-09-15T00:12:41.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2540000000000001e-06 2024-09-15T00:12:41.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524817+0000", 2024-09-15T00:12:41.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8784000000000001e-05 2024-09-15T00:12:41.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524828+0000", 2024-09-15T00:12:41.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-15T00:12:41.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:41.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532360+0000", 2024-09-15T00:12:41.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007531757 2024-09-15T00:12:41.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532400+0000", 2024-09-15T00:12:41.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9835000000000001e-05 2024-09-15T00:12:41.685 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.685 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.685 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.685 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=16'106)", 2024-09-15T00:12:41.685 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:41.685 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.150714390999994, 2024-09-15T00:12:41.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067923050000000002, 2024-09-15T00:12:41.685 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.686 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.686 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:41.686 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:41.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:41.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525640+0000", 2024-09-15T00:12:41.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-15T00:12:41.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525644+0000", 2024-09-15T00:12:41.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.709e-06 2024-09-15T00:12:41.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525651+0000", 2024-09-15T00:12:41.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7970000000000003e-06 2024-09-15T00:12:41.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525654+0000", 2024-09-15T00:12:41.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.433e-06 2024-09-15T00:12:41.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525685+0000", 2024-09-15T00:12:41.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0970000000000003e-05 2024-09-15T00:12:41.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.525697+0000", 2024-09-15T00:12:41.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1759e-05 2024-09-15T00:12:41.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:41.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532401+0000", 2024-09-15T00:12:41.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006704425 2024-09-15T00:12:41.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532432+0000", 2024-09-15T00:12:41.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0709e-05 2024-09-15T00:12:41.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.690 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.690 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [2097152~420557,2629632~4096,2650112~4096,2670592~4096,2691072~4096,2711552~4096,2732032~4096,2756608~12288,2777088~12288,2797568~12288,2818048~12288,2838528~12288,2859008~12288,2879488~529715], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3409203, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:41.691 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:41.691 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.162767660999997, 2024-09-15T00:12:41.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0071155660000000003, 2024-09-15T00:12:41.691 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.691 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.691 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:41.691 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:41.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:41.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513587+0000", 2024-09-15T00:12:41.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4899999999999995e-07 2024-09-15T00:12:41.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514349+0000", 2024-09-15T00:12:41.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00076184799999999997 2024-09-15T00:12:41.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514368+0000", 2024-09-15T00:12:41.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9507000000000001e-05 2024-09-15T00:12:41.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514391+0000", 2024-09-15T00:12:41.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2497000000000002e-05 2024-09-15T00:12:41.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514491+0000", 2024-09-15T00:12:41.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9838000000000002e-05 2024-09-15T00:12:41.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514500+0000", 2024-09-15T00:12:41.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4860000000000003e-06 2024-09-15T00:12:41.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.520702+0000", 2024-09-15T00:12:41.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062017410000000002 2024-09-15T00:12:41.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.696 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.696 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.696 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [3409203~654029,4194304~266240,4476928~4096,4497408~4096,4517888~4096,4538368~4096,4558848~4096,4579328~4096,4587520~103731], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4691251, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3409203, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:41.696 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:41.696 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.151909584000002, 2024-09-15T00:12:41.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079084649999999999, 2024-09-15T00:12:41.696 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.696 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.696 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:41.697 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:41.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:41.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524446+0000", 2024-09-15T00:12:41.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.243e-06 2024-09-15T00:12:41.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525115+0000", 2024-09-15T00:12:41.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00066896999999999996 2024-09-15T00:12:41.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525135+0000", 2024-09-15T00:12:41.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0137999999999999e-05 2024-09-15T00:12:41.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525148+0000", 2024-09-15T00:12:41.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3591000000000001e-05 2024-09-15T00:12:41.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525242+0000", 2024-09-15T00:12:41.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3603999999999994e-05 2024-09-15T00:12:41.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525258+0000", 2024-09-15T00:12:41.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6150999999999998e-05 2024-09-15T00:12:41.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.532353+0000", 2024-09-15T00:12:41.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0070947680000000004 2024-09-15T00:12:41.700 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.700 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.700 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.701 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [4691251~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5739827, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4691251, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:41.701 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:41.701 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.138226023000001, 2024-09-15T00:12:41.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0085554080000000005, 2024-09-15T00:12:41.701 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.701 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.701 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:41.701 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:41.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:41.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538129+0000", 2024-09-15T00:12:41.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:12:41.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539029+0000", 2024-09-15T00:12:41.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090036599999999999 2024-09-15T00:12:41.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539047+0000", 2024-09-15T00:12:41.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7776999999999999e-05 2024-09-15T00:12:41.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539058+0000", 2024-09-15T00:12:41.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1070000000000001e-05 2024-09-15T00:12:41.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539128+0000", 2024-09-15T00:12:41.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0531999999999995e-05 2024-09-15T00:12:41.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539138+0000", 2024-09-15T00:12:41.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3910000000000004e-06 2024-09-15T00:12:41.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.546683+0000", 2024-09-15T00:12:41.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075455519999999996 2024-09-15T00:12:41.707 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.707 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.708 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.708 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:6618a9b3:::rbd_data.113947ee3a89.0000000000000009:15, version: 59'176, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:6618a9b3:::rbd_data.113947ee3a89.0000000000000009:15@59'176, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:41.708 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:41.708 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.068908227000001, 2024-09-15T00:12:41.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073968319999999999, 2024-09-15T00:12:41.708 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.708 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.708 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:41.708 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:41.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:41.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607447+0000", 2024-09-15T00:12:41.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2100000000000001e-06 2024-09-15T00:12:41.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607778+0000", 2024-09-15T00:12:41.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033101199999999999 2024-09-15T00:12:41.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607800+0000", 2024-09-15T00:12:41.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2323e-05 2024-09-15T00:12:41.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607815+0000", 2024-09-15T00:12:41.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5073e-05 2024-09-15T00:12:41.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607926+0000", 2024-09-15T00:12:41.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110538 2024-09-15T00:12:41.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607942+0000", 2024-09-15T00:12:41.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5662999999999999e-05 2024-09-15T00:12:41.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.614843+0000", 2024-09-15T00:12:41.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0069010130000000001 2024-09-15T00:12:41.711 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.711 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.712 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.712 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:62885eaf:::rbd_data.1139e6d65aff.0000000000000005:head, version: 62'226, data_included: [0~262144,393216~131072,655360~655360], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:62885eaf:::rbd_data.1139e6d65aff.0000000000000005:head@62'226, size: 1572864, copy_subset: [(0, 1572864)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1310720, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:41.712 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.639003+0000", 2024-09-15T00:12:41.712 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.037350664999998, 2024-09-15T00:12:41.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0070761069999999999, 2024-09-15T00:12:41.712 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.712 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.712 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:41.712 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639003+0000", 2024-09-15T00:12:41.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639003+0000", 2024-09-15T00:12:41.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639004+0000", 2024-09-15T00:12:41.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:12:41.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639567+0000", 2024-09-15T00:12:41.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00056333800000000003 2024-09-15T00:12:41.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639583+0000", 2024-09-15T00:12:41.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5906999999999999e-05 2024-09-15T00:12:41.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639594+0000", 2024-09-15T00:12:41.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1093e-05 2024-09-15T00:12:41.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639667+0000", 2024-09-15T00:12:41.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3089000000000004e-05 2024-09-15T00:12:41.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.639680+0000", 2024-09-15T00:12:41.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2689e-05 2024-09-15T00:12:41.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.646079+0000", 2024-09-15T00:12:41.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063994990000000003 2024-09-15T00:12:41.715 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.715 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.715 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.716 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:125615 3.f e77/68 3:f382b49c:::rbd_data.11396481e1ec.0000000000000004:head v 77'241, mlcod=74'239)", 2024-09-15T00:12:41.716 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.839171+0000", 2024-09-15T00:12:41.716 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.83718339, 2024-09-15T00:12:41.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0072384550000000004, 2024-09-15T00:12:41.716 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.716 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.716 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:41.716 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.839171+0000", 2024-09-15T00:12:41.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.839171+0000", 2024-09-15T00:12:41.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.839172+0000", 2024-09-15T00:12:41.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3769999999999999e-06 2024-09-15T00:12:41.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840892+0000", 2024-09-15T00:12:41.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001720271 2024-09-15T00:12:41.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840905+0000", 2024-09-15T00:12:41.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2461e-05 2024-09-15T00:12:41.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840913+0000", 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1990000000000006e-06 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840962+0000", 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8760000000000001e-05 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840979+0000", 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7280999999999999e-05 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:41.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.846347+0000", 2024-09-15T00:12:41.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005367925 2024-09-15T00:12:41.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.846409+0000", 2024-09-15T00:12:41.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2181000000000007e-05 2024-09-15T00:12:41.720 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.720 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.720 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.720 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133834 3.1b e85/69 3:da9a78a3:::rbd_data.11395dfcf2b8.0000000000000000:head v 85'356, mlcod=85'354)", 2024-09-15T00:12:41.720 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:41.720 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 11.113017657, 2024-09-15T00:12:41.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026079054000000001, 2024-09-15T00:12:41.721 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.721 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.721 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:41.721 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:41.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:41.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563338+0000", 2024-09-15T00:12:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.418e-06 2024-09-15T00:12:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569385+0000", 2024-09-15T00:12:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006047335 2024-09-15T00:12:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569418+0000", 2024-09-15T00:12:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2957999999999999e-05 2024-09-15T00:12:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569432+0000", 2024-09-15T00:12:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3718e-05 2024-09-15T00:12:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569519+0000", 2024-09-15T00:12:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7022999999999997e-05 2024-09-15T00:12:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569540+0000", 2024-09-15T00:12:41.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1426000000000001e-05 2024-09-15T00:12:41.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:41.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589362+0000", 2024-09-15T00:12:41.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019821398 2024-09-15T00:12:41.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589415+0000", 2024-09-15T00:12:41.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3777999999999999e-05 2024-09-15T00:12:41.724 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.724 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.724 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.725 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133836 3.1 e85/67 3:849f9a98:::rbd_data.11395dfcf2b8.0000000000000002:head v 85'304, mlcod=84'301)", 2024-09-15T00:12:41.725 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:41.725 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 11.105322747000001, 2024-09-15T00:12:41.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023231600000000002, 2024-09-15T00:12:41.725 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.725 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.725 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:41.725 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:41.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:41.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571032+0000", 2024-09-15T00:12:41.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-15T00:12:41.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576635+0000", 2024-09-15T00:12:41.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056028700000000002 2024-09-15T00:12:41.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576650+0000", 2024-09-15T00:12:41.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.505e-05 2024-09-15T00:12:41.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576658+0000", 2024-09-15T00:12:41.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6879999999999999e-06 2024-09-15T00:12:41.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576717+0000", 2024-09-15T00:12:41.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175000000000002e-05 2024-09-15T00:12:41.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576742+0000", 2024-09-15T00:12:41.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4734000000000001e-05 2024-09-15T00:12:41.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:41.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594234+0000", 2024-09-15T00:12:41.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017492527000000001 2024-09-15T00:12:41.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594263+0000", 2024-09-15T00:12:41.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8629e-05 2024-09-15T00:12:41.729 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.729 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.729 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.729 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:144980 3.f e93/68 3:f76a968b:::rbd_data.1139646a7d4e.0000000000000009:head v 93'285, mlcod=93'283)", 2024-09-15T00:12:41.729 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:39.520500+0000", 2024-09-15T00:12:41.729 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.155853472, 2024-09-15T00:12:41.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012994957, 2024-09-15T00:12:41.729 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.729 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.729 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:41.729 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.520500+0000", 2024-09-15T00:12:41.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.520500+0000", 2024-09-15T00:12:41.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.520501+0000", 2024-09-15T00:12:41.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2999999999999999e-07 2024-09-15T00:12:41.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.521926+0000", 2024-09-15T00:12:41.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001424224 2024-09-15T00:12:41.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.521939+0000", 2024-09-15T00:12:41.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3049e-05 2024-09-15T00:12:41.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.521946+0000", 2024-09-15T00:12:41.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0770000000000002e-06 2024-09-15T00:12:41.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.521997+0000", 2024-09-15T00:12:41.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1035999999999997e-05 2024-09-15T00:12:41.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.522014+0000", 2024-09-15T00:12:41.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6956000000000002e-05 2024-09-15T00:12:41.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:41.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.533455+0000", 2024-09-15T00:12:41.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011441504 2024-09-15T00:12:41.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.533495+0000", 2024-09-15T00:12:41.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0181e-05 2024-09-15T00:12:41.733 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.733 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.733 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.733 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:144981 3.18 e93/69 3:1880ca23:::rbd_data.1139646a7d4e.000000000000000a:head v 93'485, mlcod=93'483)", 2024-09-15T00:12:41.733 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:39.521311+0000", 2024-09-15T00:12:41.733 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.1550429420000001, 2024-09-15T00:12:41.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012659617, 2024-09-15T00:12:41.733 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.734 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.734 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:41.734 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.521311+0000", 2024-09-15T00:12:41.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.521311+0000", 2024-09-15T00:12:41.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.521312+0000", 2024-09-15T00:12:41.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1699999999999997e-07 2024-09-15T00:12:41.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.522402+0000", 2024-09-15T00:12:41.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001089998 2024-09-15T00:12:41.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.522411+0000", 2024-09-15T00:12:41.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8589999999999994e-06 2024-09-15T00:12:41.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.522416+0000", 2024-09-15T00:12:41.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6359999999999999e-06 2024-09-15T00:12:41.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.522460+0000", 2024-09-15T00:12:41.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4066999999999997e-05 2024-09-15T00:12:41.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.522480+0000", 2024-09-15T00:12:41.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.012e-05 2024-09-15T00:12:41.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:41.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.533933+0000", 2024-09-15T00:12:41.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011452301 2024-09-15T00:12:41.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.533971+0000", 2024-09-15T00:12:41.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7919000000000001e-05 2024-09-15T00:12:41.737 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.737 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.737 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.737 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 93/92 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:41.738 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:39.531770+0000", 2024-09-15T00:12:41.738 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.144584332, 2024-09-15T00:12:41.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007635551, 2024-09-15T00:12:41.738 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.738 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.738 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:41.738 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.531770+0000", 2024-09-15T00:12:41.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.531770+0000", 2024-09-15T00:12:41.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.531770+0000", 2024-09-15T00:12:41.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9500000000000003e-07 2024-09-15T00:12:41.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.532741+0000", 2024-09-15T00:12:41.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097131099999999998 2024-09-15T00:12:41.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.532767+0000", 2024-09-15T00:12:41.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5335999999999999e-05 2024-09-15T00:12:41.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.532789+0000", 2024-09-15T00:12:41.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1848000000000002e-05 2024-09-15T00:12:41.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.532869+0000", 2024-09-15T00:12:41.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0857000000000002e-05 2024-09-15T00:12:41.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.532883+0000", 2024-09-15T00:12:41.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3456e-05 2024-09-15T00:12:41.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.539405+0000", 2024-09-15T00:12:41.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0065222479999999996 2024-09-15T00:12:41.741 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.741 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.741 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.741 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:145028 3.4 e93/92 3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head v 93'455, mlcod=29'3)", 2024-09-15T00:12:41.741 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:39.543212+0000", 2024-09-15T00:12:41.742 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.133142324, 2024-09-15T00:12:41.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011183056, 2024-09-15T00:12:41.742 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.742 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.742 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:41.742 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.543212+0000", 2024-09-15T00:12:41.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.543212+0000", 2024-09-15T00:12:41.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.543212+0000", 2024-09-15T00:12:41.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0200000000000001e-07 2024-09-15T00:12:41.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.544503+0000", 2024-09-15T00:12:41.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001290334 2024-09-15T00:12:41.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.544511+0000", 2024-09-15T00:12:41.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.354e-06 2024-09-15T00:12:41.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.544516+0000", 2024-09-15T00:12:41.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7140000000000001e-06 2024-09-15T00:12:41.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.545605+0000", 2024-09-15T00:12:41.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010889809999999999 2024-09-15T00:12:41.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.545627+0000", 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2345e-05 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.554362+0000", 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0087352139999999998 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.554395+0000", 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2412000000000001e-05 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.746 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1a 93/92 [PushOp(3:5e7e71d0:::rbd_data.11395b9829f3.0000000000000000:e, version: 47'181, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:5e7e71d0:::rbd_data.11395b9829f3.0000000000000000:e@47'181, size: 15335424, copy_subset: [(0, 15335424)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:41.746 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:39.760937+0000", 2024-09-15T00:12:41.746 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.9154166849999998, 2024-09-15T00:12:41.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007391491, 2024-09-15T00:12:41.746 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.746 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.746 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:41.746 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.760937+0000", 2024-09-15T00:12:41.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.760937+0000", 2024-09-15T00:12:41.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.760938+0000", 2024-09-15T00:12:41.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:12:41.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.762175+0000", 2024-09-15T00:12:41.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012376539999999999 2024-09-15T00:12:41.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.762190+0000", 2024-09-15T00:12:41.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4786e-05 2024-09-15T00:12:41.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.762200+0000", 2024-09-15T00:12:41.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7929999999999992e-06 2024-09-15T00:12:41.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.762261+0000", 2024-09-15T00:12:41.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1024999999999999e-05 2024-09-15T00:12:41.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.762269+0000", 2024-09-15T00:12:41.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6839999999999996e-06 2024-09-15T00:12:41.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.768329+0000", 2024-09-15T00:12:41.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006060139 2024-09-15T00:12:41.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.749 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.750 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1a 93/92 [PushOp(3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e, version: 47'183, data_included: [4550067~3380,4554752~4096,4575232~4096,4587520~1037004], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e@47'183, size: 15258735, copy_subset: [(0, 15258735)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5624524, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4550067, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:41.750 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:39.854503+0000", 2024-09-15T00:12:41.750 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.8218507989999999, 2024-09-15T00:12:41.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0068086960000000004, 2024-09-15T00:12:41.750 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.750 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.750 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:41.750 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.854503+0000", 2024-09-15T00:12:41.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.854503+0000", 2024-09-15T00:12:41.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.854504+0000", 2024-09-15T00:12:41.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.06e-07 2024-09-15T00:12:41.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.855445+0000", 2024-09-15T00:12:41.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00094099099999999996 2024-09-15T00:12:41.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.855470+0000", 2024-09-15T00:12:41.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5086e-05 2024-09-15T00:12:41.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.855481+0000", 2024-09-15T00:12:41.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1642e-05 2024-09-15T00:12:41.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.855546+0000", 2024-09-15T00:12:41.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4639000000000002e-05 2024-09-15T00:12:41.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.855555+0000", 2024-09-15T00:12:41.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8510000000000005e-06 2024-09-15T00:12:41.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.861312+0000", 2024-09-15T00:12:41.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057568810000000002 2024-09-15T00:12:41.753 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.753 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.753 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.753 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1a 93/92 [PushOp(3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e, version: 47'185, data_included: [2097152~4096,2117632~4096,2138112~4096,2158592~4096,2179072~4096,2199552~4096,2220032~4096,2228224~262144,2576713~35894,2621440~721866], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e@47'185, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3343306, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:41.754 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:39.889465+0000", 2024-09-15T00:12:41.754 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.7868892830000001, 2024-09-15T00:12:41.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073346749999999997, 2024-09-15T00:12:41.754 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.754 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.754 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:41.754 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.889465+0000", 2024-09-15T00:12:41.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.889465+0000", 2024-09-15T00:12:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.889466+0000", 2024-09-15T00:12:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9000000000000005e-07 2024-09-15T00:12:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.890475+0000", 2024-09-15T00:12:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010089700000000001 2024-09-15T00:12:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.890501+0000", 2024-09-15T00:12:41.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6520999999999999e-05 2024-09-15T00:12:41.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.890525+0000", 2024-09-15T00:12:41.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4113000000000001e-05 2024-09-15T00:12:41.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.890633+0000", 2024-09-15T00:12:41.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107735 2024-09-15T00:12:41.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.890650+0000", 2024-09-15T00:12:41.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7048e-05 2024-09-15T00:12:41.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.896799+0000", 2024-09-15T00:12:41.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061492980000000001 2024-09-15T00:12:41.757 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.757 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.757 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.757 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.e 94/92 [PushOp(3:73ec49f1:::rbd_data.113947ee3a89.0000000000000003:head, version: 58'1013, data_included: [0~983040], data_size: 983040, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:73ec49f1:::rbd_data.113947ee3a89.0000000000000003:head@58'1013, size: 983040, copy_subset: [(0, 983040)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 983040, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:41.757 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:40.194203+0000", 2024-09-15T00:12:41.757 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.482151309, 2024-09-15T00:12:41.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0087426819999999999, 2024-09-15T00:12:41.758 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.758 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.758 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:41.758 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.194203+0000", 2024-09-15T00:12:41.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.194203+0000", 2024-09-15T00:12:41.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.194203+0000", 2024-09-15T00:12:41.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:12:41.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.195084+0000", 2024-09-15T00:12:41.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088122500000000002 2024-09-15T00:12:41.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.195098+0000", 2024-09-15T00:12:41.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3385e-05 2024-09-15T00:12:41.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.195124+0000", 2024-09-15T00:12:41.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6451000000000001e-05 2024-09-15T00:12:41.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.195197+0000", 2024-09-15T00:12:41.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2418999999999999e-05 2024-09-15T00:12:41.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.195209+0000", 2024-09-15T00:12:41.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2926e-05 2024-09-15T00:12:41.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.202945+0000", 2024-09-15T00:12:41.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077358690000000003 2024-09-15T00:12:41.761 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.761 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.761 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.764 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.e 94/92 [PushOp(3:7588bc7b:::rbd_data.1139e6d65aff.0000000000000001:head, version: 62'1027, data_included: [0~524288,655360~524288], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:7588bc7b:::rbd_data.1139e6d65aff.0000000000000001:head@62'1027, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:41.764 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:40.234057+0000", 2024-09-15T00:12:41.764 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4422967199999999, 2024-09-15T00:12:41.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010390427000000001, 2024-09-15T00:12:41.764 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.764 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.764 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:41.764 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.234057+0000", 2024-09-15T00:12:41.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.234057+0000", 2024-09-15T00:12:41.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.234067+0000", 2024-09-15T00:12:41.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8500000000000006e-06 2024-09-15T00:12:41.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.234910+0000", 2024-09-15T00:12:41.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00084323500000000001 2024-09-15T00:12:41.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.234927+0000", 2024-09-15T00:12:41.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6662000000000001e-05 2024-09-15T00:12:41.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.234937+0000", 2024-09-15T00:12:41.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0125999999999999e-05 2024-09-15T00:12:41.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.235019+0000", 2024-09-15T00:12:41.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1954999999999997e-05 2024-09-15T00:12:41.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.235032+0000", 2024-09-15T00:12:41.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3388e-05 2024-09-15T00:12:41.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.244448+0000", 2024-09-15T00:12:41.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094152109999999997 2024-09-15T00:12:41.769 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.769 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.769 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.769 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.e 94/92 [PushOp(3:743f56f7:::rbd_data.11395dfcf2b8.0000000000000001:head, version: 85'1145, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:743f56f7:::rbd_data.11395dfcf2b8.0000000000000001:head@85'1145, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:41.769 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:40.301476+0000", 2024-09-15T00:12:41.769 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.3748777409999999, 2024-09-15T00:12:41.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0072086509999999999, 2024-09-15T00:12:41.769 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:41.769 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:41.769 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:41.769 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:41.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:41.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.301476+0000", 2024-09-15T00:12:41.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:41.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.301476+0000", 2024-09-15T00:12:41.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:41.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:41.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.301477+0000", 2024-09-15T00:12:41.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7899999999999998e-07 2024-09-15T00:12:41.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:41.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.302440+0000", 2024-09-15T00:12:41.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000963221 2024-09-15T00:12:41.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:41.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.302454+0000", 2024-09-15T00:12:41.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.413e-05 2024-09-15T00:12:41.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:41.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.302463+0000", 2024-09-15T00:12:41.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1009999999999998e-06 2024-09-15T00:12:41.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:41.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.302538+0000", 2024-09-15T00:12:41.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4289000000000007e-05 2024-09-15T00:12:41.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:41.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.302544+0000", 2024-09-15T00:12:41.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.793e-06 2024-09-15T00:12:41.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:41.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:41.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:41.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.308685+0000", 2024-09-15T00:12:41.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061405380000000001 2024-09-15T00:12:41.772 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.773 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.773 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.773 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:41.773 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:41.773 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:41.774 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:41.775 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:41.775 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:41.775 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:41.776 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:41.776 INFO:teuthology.orchestra.run.smithi171.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T00:12:41.776 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:41.776 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:41.777 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:41.772+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:41.786 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:41.786 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:41.803 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:41.803 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:41.820 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:41.820 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:41.836 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:41.837 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:41.840 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:12:41.840 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:12:41.874 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:41.873+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:41.891 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:41.891 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:41.901 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:41.975 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:41.974+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:42.004 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:42.004 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:42.013 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:42.017 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is set 2024-09-15T00:12:42.076 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:42.075+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:42.097 INFO:teuthology.orchestra.run.smithi171.stdout:flattening image image_client.0-clone40 2024-09-15T00:12:42.139 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:42.139 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:42.150 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:42.177 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:42.176+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:42.259 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:42.260 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:42.269 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:42.306 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-15T00:12:42.306 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:42.305+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:42.313 INFO:teuthology.orchestra.run.smithi171.stdout:4872 read 0x447eb7 thru 0x450da1 (0x8eeb bytes) 2024-09-15T00:12:42.315 INFO:teuthology.orchestra.run.smithi171.stdout:4874 write 0x1862200 thru 0x1863aeb (0x18ec bytes) 2024-09-15T00:12:42.323 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 0] out_osds: [2, 7, 1] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-09-15T00:12:42.323 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:12:42.323 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-15T00:12:42.323 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.000000 2024-09-15T00:12:42.324 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0 2024-09-15T00:12:42.332 INFO:teuthology.orchestra.run.smithi171.stdout:4875 writesame 0x6f7a985 thru 0x6f84c22 data_size 0x17(0xa29e bytes) 2024-09-15T00:12:42.335 INFO:teuthology.orchestra.run.smithi171.stdout:4881 write 0x947507a thru 0x947528c (0x213 bytes) 2024-09-15T00:12:42.356 INFO:teuthology.orchestra.run.smithi171.stdout:4882 trunc from 0xdab36e8 to 0x184b4b3 2024-09-15T00:12:42.379 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:42.377+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:42.395 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:42.395 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:42.405 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:42.440 INFO:teuthology.orchestra.run.smithi171.stdout:4883 punch from 0xfd4908 to 0xfd8d07, (0x43ff bytes) 2024-09-15T00:12:42.440 INFO:teuthology.orchestra.run.smithi171.stdout:4884 read 0xce6c24 thru 0xcf0247 (0x9624 bytes) 2024-09-15T00:12:42.440 INFO:teuthology.orchestra.run.smithi171.stdout:4885 punch from 0x183341a to 0x1838137, (0x4d1d bytes) 2024-09-15T00:12:42.440 INFO:teuthology.orchestra.run.smithi171.stdout:4886 read 0x10be7c4 thru 0x10c7b5a (0x9397 bytes) 2024-09-15T00:12:42.442 INFO:teuthology.orchestra.run.smithi171.stdout:4887 read 0x17ce1fd thru 0x17d096b (0x276f bytes) 2024-09-15T00:12:42.451 INFO:teuthology.orchestra.run.smithi171.stdout:4888 write 0x4546ac5 thru 0x4549eb9 (0x33f5 bytes) 2024-09-15T00:12:42.471 INFO:teuthology.orchestra.run.smithi171.stdout:4889 write 0xeb32f78 thru 0xeb3f453 (0xc4dc bytes) 2024-09-15T00:12:42.480 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:42.479+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:42.489 INFO:teuthology.orchestra.run.smithi171.stdout:4890 trunc from 0xeb3f454 to 0x26313ca 2024-09-15T00:12:42.519 INFO:teuthology.orchestra.run.smithi171.stdout:4891 writesame 0xb6a293c thru 0xb6a8b7d data_size 0x1(0x6242 bytes) 2024-09-15T00:12:42.527 INFO:teuthology.orchestra.run.smithi171.stdout:4892 write 0xdceff7b thru 0xdcf74f3 (0x7579 bytes) 2024-09-15T00:12:42.530 INFO:teuthology.orchestra.run.smithi171.stdout:4893 read 0x6c30ee3 thru 0x6c379b4 (0x6ad2 bytes) 2024-09-15T00:12:42.531 INFO:teuthology.orchestra.run.smithi171.stdout:4894 write 0xab25c25 thru 0xab3441e (0xe7fa bytes) 2024-09-15T00:12:42.537 INFO:teuthology.orchestra.run.smithi171.stdout:4898 read 0x829f58d thru 0x82a8240 (0x8cb4 bytes) 2024-09-15T00:12:42.537 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:42.537 INFO:teuthology.orchestra.run.smithi171.stdout:4902 read 0x7e12276 thru 0x7e214bd (0xf248 bytes) 2024-09-15T00:12:42.537 INFO:teuthology.orchestra.run.smithi171.stdout:4903 write 0x1172961 thru 0x1181c8c (0xf32c bytes) 2024-09-15T00:12:42.541 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:42.541 INFO:teuthology.orchestra.run.smithi171.stdout:4905 writesame 0x7f42146 thru 0x7f4426a data_size 0x1(0x2125 bytes) 2024-09-15T00:12:42.548 INFO:teuthology.orchestra.run.smithi171.stdout:4906 write 0x3de25cd thru 0x3df1717 (0xf14b bytes) 2024-09-15T00:12:42.553 INFO:teuthology.orchestra.run.smithi171.stdout:4908 writesame 0x1bf2780 thru 0x1bfdd9b data_size 0x15(0xb61c bytes) 2024-09-15T00:12:42.559 INFO:teuthology.orchestra.run.smithi171.stdout:4909 punch from 0xbf78e8b to 0xbf7af70, (0x20e5 bytes) 2024-09-15T00:12:42.561 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:42.561 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:42.565 INFO:teuthology.orchestra.run.smithi171.stdout:4911 writesame 0xe99300f thru 0xe9971fb data_size 0x1(0x41ed bytes) 2024-09-15T00:12:42.573 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:42.573 INFO:teuthology.orchestra.run.smithi171.stdout:4912 trunc from 0xe9971fc to 0x8d70c5b 2024-09-15T00:12:42.581 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:42.582 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:42.582 INFO:teuthology.orchestra.run.smithi171.stdout:4917 writesame 0x4de5616 thru 0x4de71d9 data_size 0x1(0x1bc4 bytes) 2024-09-15T00:12:42.582 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:42.580+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:42.586 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:42.586 INFO:teuthology.orchestra.run.smithi171.stdout:4919 punch from 0x6ea4660 to 0x6eb2f09, (0xe8a9 bytes) 2024-09-15T00:12:42.597 INFO:teuthology.orchestra.run.smithi171.stdout:4920 writesame 0xde6de1c thru 0xde7d4be data_size 0x1(0xf6a3 bytes) 2024-09-15T00:12:42.599 INFO:teuthology.orchestra.run.smithi171.stdout:4922 read 0x8c410 thru 0x9a617 (0xe208 bytes) 2024-09-15T00:12:42.600 INFO:teuthology.orchestra.run.smithi171.stdout:4923 punch from 0x957f0eb to 0x9581ee6, (0x2dfb bytes) 2024-09-15T00:12:42.600 INFO:teuthology.orchestra.run.smithi171.stdout:4924 write 0x87be5a3 thru 0x87ca982 (0xc3e0 bytes) 2024-09-15T00:12:42.605 INFO:teuthology.orchestra.run.smithi171.stdout:4925 trunc from 0xde7d4bf to 0xdd1dc1f 2024-09-15T00:12:42.612 INFO:teuthology.orchestra.run.smithi171.stdout:4927 punch from 0x48adb90 to 0x48bcf36, (0xf3a6 bytes) 2024-09-15T00:12:42.614 INFO:teuthology.orchestra.run.smithi171.stdout:4928 trunc from 0xdd1dc1f to 0xc1a4497 2024-09-15T00:12:42.625 INFO:teuthology.orchestra.run.smithi171.stdout:4932 write 0x31ee5f4 thru 0x31fa71d (0xc12a bytes) 2024-09-15T00:12:42.629 INFO:teuthology.orchestra.run.smithi171.stdout:4933 writesame 0x2c8be4a thru 0x2c9a792 data_size 0x11(0xe949 bytes) 2024-09-15T00:12:42.635 INFO:teuthology.orchestra.run.smithi171.stdout:4934 read 0x569371e thru 0x56a35b9 (0xfe9c bytes) 2024-09-15T00:12:42.636 INFO:teuthology.orchestra.run.smithi171.stdout:4935 read 0xaf40d56 thru 0xaf41c2f (0xeda bytes) 2024-09-15T00:12:42.642 INFO:teuthology.orchestra.run.smithi171.stdout:4936 trunc from 0xc1a4497 to 0x6ea5013 2024-09-15T00:12:42.657 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:42.658 INFO:teuthology.orchestra.run.smithi171.stdout:4940 write 0x6e4d991 thru 0x6e543bf (0x6a2f bytes) 2024-09-15T00:12:42.659 INFO:teuthology.orchestra.run.smithi171.stdout:4941 write 0x513b286 thru 0x513d409 (0x2184 bytes) 2024-09-15T00:12:42.671 INFO:teuthology.orchestra.run.smithi171.stdout:4942 write 0x95672e3 thru 0x9567667 (0x385 bytes) 2024-09-15T00:12:42.675 INFO:teuthology.orchestra.run.smithi171.stdout:4943 read 0x4da5bad thru 0x4da7fde (0x2432 bytes) 2024-09-15T00:12:42.676 INFO:teuthology.orchestra.run.smithi171.stdout:4945 read 0x9008e37 thru 0x900e8e7 (0x5ab1 bytes) 2024-09-15T00:12:42.676 INFO:teuthology.orchestra.run.smithi171.stdout:4947 read 0x8461594 thru 0x84620f4 (0xb61 bytes) 2024-09-15T00:12:42.676 INFO:teuthology.orchestra.run.smithi171.stdout:4948 read 0x28f0c4b thru 0x28f0f1c (0x2d2 bytes) 2024-09-15T00:12:42.676 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:42.682 INFO:teuthology.orchestra.run.smithi171.stdout:4950 writesame 0xa15fd9c thru 0xa16a667 data_size 0x27(0xa8cc bytes) 2024-09-15T00:12:42.682 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:42.680+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:42.689 INFO:teuthology.orchestra.run.smithi171.stdout:4952 write 0xb6265a2 thru 0xb631b12 (0xb571 bytes) 2024-09-15T00:12:42.693 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:42.694 INFO:teuthology.orchestra.run.smithi171.stdout:4953 writesame 0x838cecb thru 0x83981e6 data_size 0x6(0xb31c bytes) 2024-09-15T00:12:42.715 INFO:teuthology.orchestra.run.smithi171.stdout:4954 write 0xb871268 thru 0xb878655 (0x73ee bytes) 2024-09-15T00:12:42.726 INFO:teuthology.orchestra.run.smithi171.stdout:4955 trunc from 0xb878656 to 0x489020b 2024-09-15T00:12:42.750 INFO:teuthology.orchestra.run.smithi171.stdout:4956 write 0x8696996 thru 0x86a3780 (0xcdeb bytes) 2024-09-15T00:12:42.754 INFO:teuthology.orchestra.run.smithi171.stdout:4959 punch from 0x4d298cf to 0x4d38b67, (0xf298 bytes) 2024-09-15T00:12:42.754 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:42.766 INFO:teuthology.orchestra.run.smithi171.stdout:4964 write 0xe512a3f thru 0xe51bafc (0x90be bytes) 2024-09-15T00:12:42.770 INFO:teuthology.orchestra.run.smithi171.stdout:4965 read 0xbc05d29 thru 0xbc0680e (0xae6 bytes) 2024-09-15T00:12:42.770 INFO:teuthology.orchestra.run.smithi171.stdout:4966 read 0xab74881 thru 0xab7b504 (0x6c84 bytes) 2024-09-15T00:12:42.771 INFO:teuthology.orchestra.run.smithi171.stdout:4967 write 0x8e8aafc thru 0x8e8e1c6 (0x36cb bytes) 2024-09-15T00:12:42.775 INFO:teuthology.orchestra.run.smithi171.stdout:4968 writesame 0xd8c830b thru 0xd8cab71 data_size 0x1(0x2867 bytes) 2024-09-15T00:12:42.780 INFO:teuthology.orchestra.run.smithi171.stdout:4970 writesame 0x5b80689 thru 0x5b86a56 data_size 0x23(0x63ce bytes) 2024-09-15T00:12:42.784 INFO:teuthology.orchestra.run.smithi171.stdout:4971 write 0x9b68a8d thru 0x9b750d8 (0xc64c bytes) 2024-09-15T00:12:42.785 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:42.783+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:42.788 INFO:teuthology.orchestra.run.smithi171.stdout:4972 punch from 0xc6b543f to 0xc6b8d89, (0x394a bytes) 2024-09-15T00:12:42.788 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:42.788 INFO:teuthology.orchestra.run.smithi171.stdout:4975 read 0x734a3ae thru 0x73565cd (0xc220 bytes) 2024-09-15T00:12:42.789 INFO:teuthology.orchestra.run.smithi171.stdout:4976 write 0x7a1131e thru 0x7a1cb40 (0xb823 bytes) 2024-09-15T00:12:42.793 INFO:teuthology.orchestra.run.smithi171.stdout:4978 read 0x72db61e thru 0x72e3fdc (0x89bf bytes) 2024-09-15T00:12:42.794 INFO:teuthology.orchestra.run.smithi171.stdout:4979 punch from 0xab0bdc6 to 0xab18a8a, (0xccc4 bytes) 2024-09-15T00:12:42.794 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:42.794 INFO:teuthology.orchestra.run.smithi171.stdout:4982 read 0x99c4328 thru 0x99c9a5d (0x5736 bytes) 2024-09-15T00:12:42.794 INFO:teuthology.orchestra.run.smithi171.stdout:4983 writesame 0x777b019 thru 0x777d2b0 data_size 0x48(0x2298 bytes) 2024-09-15T00:12:42.797 INFO:teuthology.orchestra.run.smithi171.stdout:4984 writesame 0x10e4d69 thru 0x10f4d64 data_size 0xac(0xfffc bytes) 2024-09-15T00:12:42.813 INFO:teuthology.orchestra.run.smithi171.stdout:4985 trunc from 0xe51bafd to 0x2ac11e9 2024-09-15T00:12:42.828 INFO:teuthology.orchestra.run.smithi171.stdout:4986 punch from 0x549d88 to 0x551df7, (0x806f bytes) 2024-09-15T00:12:42.828 INFO:teuthology.orchestra.run.smithi171.stdout:4990 read 0xeccd80 thru 0xecf786 (0x2a07 bytes) 2024-09-15T00:12:42.838 INFO:teuthology.orchestra.run.smithi171.stdout:4991 writesame 0x63875a0 thru 0x638a330 data_size 0x1(0x2d91 bytes) 2024-09-15T00:12:42.845 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:42.845 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:42.848 INFO:teuthology.orchestra.run.smithi171.stdout:4993 write 0x8371fbb thru 0x83768b0 (0x48f6 bytes) 2024-09-15T00:12:42.854 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:42.864 INFO:teuthology.orchestra.run.smithi171.stdout:4994 write 0xd45a83d thru 0xd465266 (0xaa2a bytes) 2024-09-15T00:12:42.865 INFO:teuthology.orchestra.run.smithi171.stdout:4995 writesame 0x3271e6b thru 0x3272fc7 data_size 0x7(0x115d bytes) 2024-09-15T00:12:42.867 INFO:teuthology.orchestra.run.smithi171.stdout:4996 write 0x1d9a30a thru 0x1da7423 (0xd11a bytes) 2024-09-15T00:12:42.871 INFO:teuthology.orchestra.run.smithi171.stdout:4997 read 0xb7f5393 thru 0xb7f9292 (0x3f00 bytes) 2024-09-15T00:12:42.871 INFO:teuthology.orchestra.run.smithi171.stdout:4998 read 0x867faa6 thru 0x8688f29 (0x9484 bytes) 2024-09-15T00:12:42.871 INFO:teuthology.orchestra.run.smithi171.stdout:4999 write 0x453e481 thru 0x4544aa8 (0x6628 bytes) 2024-09-15T00:12:42.873 INFO:teuthology.orchestra.run.smithi171.stdout:5000 read 0x1d1d4b8 thru 0x1d2bfae (0xeaf7 bytes) 2024-09-15T00:12:42.874 INFO:teuthology.orchestra.run.smithi171.stdout:5002 read 0xbfa08d1 thru 0xbfae43c (0xdb6c bytes) 2024-09-15T00:12:42.874 INFO:teuthology.orchestra.run.smithi171.stdout:5004 punch from 0x9a3f315 to 0x9a4c025, (0xcd10 bytes) 2024-09-15T00:12:42.874 INFO:teuthology.orchestra.run.smithi171.stdout:5005 write 0xb7d9a8e thru 0xb7e464b (0xabbe bytes) 2024-09-15T00:12:42.879 INFO:teuthology.orchestra.run.smithi171.stdout:5006 read 0x7bd3904 thru 0x7bd49c1 (0x10be bytes) 2024-09-15T00:12:42.892 INFO:teuthology.orchestra.run.smithi171.stdout:5007 trunc from 0xd465267 to 0x237b640 2024-09-15T00:12:42.897 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:42.896+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:42.918 INFO:teuthology.orchestra.run.smithi171.stdout:5008 writesame 0x7a5475a thru 0x7a588fd data_size 0x2(0x41a4 bytes) 2024-09-15T00:12:42.919 INFO:teuthology.orchestra.run.smithi171.stdout:5010 punch from 0x1bee011 to 0x1bf3805, (0x57f4 bytes) 2024-09-15T00:12:42.919 INFO:teuthology.orchestra.run.smithi171.stdout:5011 write 0x748adeb thru 0x748dabc (0x2cd2 bytes) 2024-09-15T00:12:42.929 INFO:teuthology.orchestra.run.smithi171.stdout:5013 trunc from 0x7a588fe to 0x191da4c 2024-09-15T00:12:42.943 INFO:teuthology.orchestra.run.smithi171.stdout:5014 read 0x8ce4fe thru 0x8dadcf (0xc8d2 bytes) 2024-09-15T00:12:42.944 INFO:teuthology.orchestra.run.smithi171.stdout:5015 clone 42 order 24 su 65536 sc 4 2024-09-15T00:12:42.962 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:42.962 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:42.971 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:43.013 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:43.011+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:43.026 INFO:teuthology.orchestra.run.smithi171.stderr:set osd.4 primary-affinity to 0 (802) 2024-09-15T00:12:43.087 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:43.086+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:43.116 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:43.117 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:43.125 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:43.188 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:43.187+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:43.229 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:43.229 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:43.238 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:43.323 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:43.322+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:43.382 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:43.382 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:43.391 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:43.391 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:43.389+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:43.492 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:43.490+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:43.534 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:43.534 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:43.543 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:43.593 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:43.591+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:43.687 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:43.687 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:43.694 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:43.692+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:43.695 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:43.795 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:43.794+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:43.839 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:43.839 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:43.841 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:12:43.848 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:43.896 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:43.895+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:43.966 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:43.966 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:43.982 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:43.983 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:43.983 INFO:teuthology.orchestra.run.smithi171.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T00:12:43.992 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:43.992 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:43.997 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:43.995+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:43.999 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:43.999 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:44.000 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:44.015 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:44.016 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:44.034 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:44.034 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:44.037 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:12:44.054 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:44.055 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:44.076 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:44.076 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:44.079 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:12:44.079 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:12:44.092 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone41 from 0x191da4c (overlap 0x184b4b3) to 0x26dede 2024-09-15T00:12:44.098 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:44.096+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:44.116 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:44.116 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:44.125 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:44.150 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-15T00:12:44.199 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:44.197+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:44.233 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.152598377999993, 2024-09-15T00:12:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:12:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:12:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:12:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:12:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:12:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:12:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:12:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:12:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:12:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:12:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:12:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:12:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:12:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:12:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:12:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:12:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:12:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:12:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:12:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:12:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:12:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:12:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:12:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:12:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:12:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.139857836000004, 2024-09-15T00:12:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:12:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:12:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:12:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:12:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:12:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:12:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:12:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:12:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:12:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:12:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:12:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:12:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:12:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:12:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:12:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.007502076999998, 2024-09-15T00:12:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:12:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:12:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:12:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:12:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:12:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:12:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:12:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:12:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:12:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:12:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:12:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:12:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:12:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:12:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:12:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:12:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:12:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:12:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:12:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:12:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:12:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:12:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:12:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:12:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:12:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 77.994040432000006, 2024-09-15T00:12:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:12:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:12:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:12:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:12:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:12:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:12:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:12:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:12:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:12:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:12:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:12:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:12:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:12:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:12:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:12:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.809038948999998, 2024-09-15T00:12:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:12:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:12:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:12:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:12:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:12:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:12:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:12:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:12:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:12:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:12:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:12:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:12:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:12:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:12:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:12:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:12:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:12:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:12:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:12:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:12:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:12:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:12:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:12:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:12:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:12:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.796162682000002, 2024-09-15T00:12:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:12:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:12:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:12:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:12:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:12:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:12:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:12:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:12:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:12:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:12:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:12:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:12:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:12:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:12:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:12:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:12:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 63.337647537999999, 2024-09-15T00:12:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:12:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:12:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:12:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:12:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:12:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:12:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:12:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:12:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:12:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:12:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:12:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:12:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:12:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:44.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:12:44.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:12:44.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:44.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:12:44.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:12:44.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:44.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:12:44.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:12:44.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:44.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:12:44.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:12:44.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:12:44.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:12:44.270 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.270 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.270 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.271 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:44.271 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:44.271 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.620136977999998, 2024-09-15T00:12:44.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:12:44.271 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.271 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.271 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:44.271 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.271 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.271 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.271 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:12:44.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.271 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.272 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.272 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:44.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.272 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.272 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:44.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.272 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.272 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:12:44.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:12:44.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:12:44.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:12:44.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:12:44.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:12:44.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:12:44.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:12:44.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:12:44.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:12:44.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:12:44.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:12:44.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:12:44.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:12:44.275 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.275 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.275 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.275 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:44.275 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:44.275 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.620118368999997, 2024-09-15T00:12:44.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:12:44.275 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.275 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.275 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:44.275 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.276 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.276 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.276 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:12:44.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.276 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.276 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:44.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.276 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:44.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:44.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:44.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:12:44.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:12:44.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:12:44.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:12:44.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:12:44.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:12:44.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:12:44.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:12:44.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:12:44.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:12:44.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:12:44.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:12:44.279 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.279 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.279 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.280 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:44.280 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:44.280 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.620093044999997, 2024-09-15T00:12:44.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:12:44.280 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.280 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.280 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:44.280 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.280 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.280 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.280 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:12:44.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.281 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:44.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:44.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:44.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:12:44.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:12:44.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:12:44.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:12:44.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:12:44.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:12:44.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:12:44.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:12:44.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:12:44.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:12:44.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:12:44.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:12:44.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:12:44.284 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.284 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.284 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.284 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:44.285 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:44.285 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.620074721000002, 2024-09-15T00:12:44.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:12:44.285 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.285 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.285 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:44.285 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.285 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.285 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.285 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:12:44.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.286 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:44.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:44.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:12:44.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:12:44.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:12:44.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:12:44.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:12:44.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:44.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:12:44.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:12:44.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:12:44.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:12:44.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:12:44.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:12:44.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:12:44.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:12:44.289 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.289 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.289 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.289 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:44.289 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:44.289 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.619892147999998, 2024-09-15T00:12:44.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:12:44.290 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.290 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.290 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:44.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.290 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:12:44.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.290 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.290 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.290 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:44.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:44.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:44.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:12:44.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:12:44.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:12:44.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:12:44.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:12:44.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:12:44.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:44.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:12:44.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:12:44.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:12:44.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:12:44.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:12:44.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:12:44.293 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.293 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.293 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.294 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:44.294 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:44.294 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.619863307000003, 2024-09-15T00:12:44.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:12:44.294 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.294 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.294 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:44.294 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.294 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.294 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.294 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:12:44.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.295 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:44.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:44.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:44.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:12:44.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:12:44.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:12:44.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:12:44.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:12:44.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:12:44.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:12:44.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:12:44.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:12:44.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:12:44.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:12:44.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:12:44.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:12:44.298 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.298 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.298 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.298 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:44.298 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:44.298 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.619841096000002, 2024-09-15T00:12:44.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:12:44.298 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.298 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.299 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:44.299 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.299 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.299 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.299 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:12:44.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.299 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:44.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:44.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:44.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:12:44.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:12:44.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:12:44.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:12:44.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:12:44.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:12:44.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:44.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:12:44.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:12:44.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:12:44.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:12:44.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:12:44.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:12:44.302 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.302 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.302 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.303 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:44.303 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:44.303 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.619819937000003, 2024-09-15T00:12:44.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:12:44.303 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.303 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.303 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:44.303 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.303 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.303 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.303 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:12:44.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.303 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:44.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:44.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:44.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:44.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:12:44.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:12:44.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:12:44.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:12:44.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:12:44.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:12:44.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:12:44.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:12:44.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:12:44.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:12:44.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:12:44.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:12:44.307 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.307 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.307 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.307 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:44.307 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:44.307 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.619789988999997, 2024-09-15T00:12:44.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:12:44.307 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.307 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.308 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:44.308 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.308 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.308 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.308 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:12:44.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.308 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:44.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:44.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:44.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:12:44.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:12:44.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:12:44.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:12:44.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:12:44.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:12:44.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:12:44.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:12:44.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:12:44.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:12:44.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:12:44.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:12:44.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:12:44.311 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.311 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.311 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.312 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:44.312 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:44.312 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.619768024999999, 2024-09-15T00:12:44.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:12:44.312 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.312 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.312 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:44.312 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.312 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.312 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.312 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:12:44.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.312 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:44.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:44.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:44.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:12:44.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:12:44.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:12:44.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:12:44.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:12:44.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:12:44.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:12:44.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:12:44.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:12:44.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:12:44.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:12:44.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:12:44.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:12:44.316 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.316 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.316 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.316 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:44.316 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:44.316 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.619750598000003, 2024-09-15T00:12:44.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:12:44.317 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.317 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.317 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:44.317 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.317 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.317 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.317 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:12:44.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.317 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:44.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:44.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:12:44.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:12:44.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:12:44.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:12:44.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:12:44.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:44.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:12:44.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:12:44.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:12:44.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:12:44.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:12:44.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:12:44.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:12:44.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:12:44.321 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.321 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.321 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.321 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:44.321 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:44.321 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.619728799000001, 2024-09-15T00:12:44.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:12:44.321 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.321 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.321 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:44.321 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.322 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.322 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.322 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:12:44.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.322 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:44.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:44.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:12:44.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:12:44.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:12:44.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:12:44.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:12:44.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:12:44.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:12:44.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:12:44.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:12:44.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:12:44.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:12:44.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:12:44.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:12:44.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:12:44.325 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.325 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.325 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.325 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:44.326 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:44.326 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.570500440000004, 2024-09-15T00:12:44.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:12:44.326 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:44.326 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:44.326 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:44.326 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:44.326 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:44.326 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.326 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:12:44.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.326 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:44.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:44.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:44.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:44.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:44.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:44.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:44.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:12:44.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:12:44.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:44.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:12:44.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:12:44.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:44.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:12:44.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:12:44.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:12:44.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:12:44.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:44.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:12:44.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:12:44.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:44.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:12:44.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:12:44.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:44.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:12:44.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:12:44.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:44.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:12:44.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:12:44.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:44.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:12:44.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:12:44.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:44.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:12:44.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:12:44.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:44.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:44.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:44.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:12:44.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:12:44.331 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.331 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.331 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.331 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:44.331 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:44.332 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:44.333 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:44.334 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:44.331+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:44.434 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:44.432+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:44.437 INFO:teuthology.orchestra.run.smithi171.stdout:5016 write 0xeb16742 thru 0xeb22bd8 (0xc497 bytes) 2024-09-15T00:12:44.441 INFO:teuthology.orchestra.run.smithi171.stdout:5017 read 0x965e649 thru 0x9669f3d (0xb8f5 bytes) 2024-09-15T00:12:44.446 INFO:teuthology.orchestra.run.smithi171.stdout:5018 trunc from 0xeb22bd9 to 0xae27069 2024-09-15T00:12:44.446 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:44.446 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:44.455 INFO:teuthology.orchestra.run.smithi171.stdout:5020 read 0x27a39fa thru 0x27ad49d (0x9aa4 bytes) 2024-09-15T00:12:44.455 INFO:teuthology.orchestra.run.smithi171.stdout:5021 write 0x3358941 thru 0x33683f4 (0xfab4 bytes) 2024-09-15T00:12:44.456 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:44.528 INFO:teuthology.orchestra.run.smithi171.stdout:5022 read 0x542b3df thru 0x5430ef7 (0x5b19 bytes) 2024-09-15T00:12:44.528 INFO:teuthology.orchestra.run.smithi171.stdout:5023 write 0x9c56210 thru 0x9c5da71 (0x7862 bytes) 2024-09-15T00:12:44.535 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:44.533+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:44.542 INFO:teuthology.orchestra.run.smithi171.stdout:5025 write 0xe8b5b47 thru 0xe8b8286 (0x2740 bytes) 2024-09-15T00:12:44.545 INFO:teuthology.orchestra.run.smithi171.stdout:5026 read 0x23aa0e7 thru 0x23ad0ae (0x2fc8 bytes) 2024-09-15T00:12:44.545 INFO:teuthology.orchestra.run.smithi171.stdout:5030 read 0x3cafb99 thru 0x3cb5199 (0x5601 bytes) 2024-09-15T00:12:44.560 INFO:teuthology.orchestra.run.smithi171.stdout:5031 trunc from 0xe8b8287 to 0x1b7e227 2024-09-15T00:12:44.620 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:44.621 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:44.621 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:44.621 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:44.621 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:44.630 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:44.631 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:44.631 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:44.631 INFO:teuthology.orchestra.run.smithi171.stdout:5034 punch from 0x21bb04 to 0x220af0, (0x4fec bytes) 2024-09-15T00:12:44.635 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:44.634+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:44.641 INFO:teuthology.orchestra.run.smithi171.stdout:5035 writesame 0x5bf36b3 thru 0x5c01a1f data_size 0x3(0xe36d bytes) 2024-09-15T00:12:44.643 INFO:teuthology.orchestra.run.smithi171.stdout:5036 trunc from 0x5c01a20 to 0x46deef1 2024-09-15T00:12:44.665 INFO:teuthology.orchestra.run.smithi171.stdout:5038 writesame 0xdf02bc0 thru 0xdf0c39a data_size 0x19(0x97db bytes) 2024-09-15T00:12:44.665 INFO:teuthology.orchestra.run.smithi171.stdout:5039 punch from 0xc6b45a2 to 0xc6b8920, (0x437e bytes) 2024-09-15T00:12:44.665 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:44.665 INFO:teuthology.orchestra.run.smithi171.stdout:5041 write 0x1849f94 thru 0x18550c1 (0xb12e bytes) 2024-09-15T00:12:44.670 INFO:teuthology.orchestra.run.smithi171.stdout:5042 writesame 0xb7b4006 thru 0xb7be6dc data_size 0x17(0xa6d7 bytes) 2024-09-15T00:12:44.676 INFO:teuthology.orchestra.run.smithi171.stdout:5043 write 0x17a6543 thru 0x17b4184 (0xdc42 bytes) 2024-09-15T00:12:44.679 INFO:teuthology.orchestra.run.smithi171.stdout:5044 write 0xcd1c8a8 thru 0xcd1fb58 (0x32b1 bytes) 2024-09-15T00:12:44.683 INFO:teuthology.orchestra.run.smithi171.stdout:5045 read 0x623f361 thru 0x624ed25 (0xf9c5 bytes) 2024-09-15T00:12:44.683 INFO:teuthology.orchestra.run.smithi171.stdout:5046 write 0x1d5728b thru 0x1d628c7 (0xb63d bytes) 2024-09-15T00:12:44.689 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:44.689 INFO:teuthology.orchestra.run.smithi171.stdout:5048 read 0xb3145c thru 0xb3a501 (0x90a6 bytes) 2024-09-15T00:12:44.690 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:44.690 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:44.705 INFO:teuthology.orchestra.run.smithi171.stdout:5051 trunc from 0xdf0c39b to 0x1080d56 2024-09-15T00:12:44.718 INFO:teuthology.orchestra.run.smithi171.stdout:5052 read 0x1011db9 thru 0x1015e31 (0x4079 bytes) 2024-09-15T00:12:44.733 INFO:teuthology.orchestra.run.smithi171.stdout:5053 write 0x7d22175 thru 0x7d2b25f (0x90eb bytes) 2024-09-15T00:12:44.736 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:44.734+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:44.736 INFO:teuthology.orchestra.run.smithi171.stdout:5054 read 0x44906cd thru 0x4490c90 (0x5c4 bytes) 2024-09-15T00:12:44.745 INFO:teuthology.orchestra.run.smithi171.stdout:5055 writesame 0xa84a52c thru 0xa8508a0 data_size 0x3(0x6375 bytes) 2024-09-15T00:12:44.745 INFO:teuthology.orchestra.run.smithi171.stdout:5056 punch from 0x66a148d to 0x66a1acb, (0x63e bytes) 2024-09-15T00:12:44.745 INFO:teuthology.orchestra.run.smithi171.stdout:5057 trunc from 0xa8508a1 to 0xaddaa06 2024-09-15T00:12:44.750 INFO:teuthology.orchestra.run.smithi171.stdout:5058 write 0x9966124 thru 0x996b0d5 (0x4fb2 bytes) 2024-09-15T00:12:44.752 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:44.752 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:44.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:44.753 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:44.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.753 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4215.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-15T00:12:44.753 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:44.753 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 98.225243704999997, 2024-09-15T00:12:44.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077021529999999998, 2024-09-15T00:12:44.753 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.753 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.753 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:44.753 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:44.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524697+0000", 2024-09-15T00:12:44.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524698+0000", 2024-09-15T00:12:44.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6300000000000001e-07 2024-09-15T00:12:44.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524778+0000", 2024-09-15T00:12:44.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0598000000000003e-05 2024-09-15T00:12:44.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524785+0000", 2024-09-15T00:12:44.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6359999999999997e-06 2024-09-15T00:12:44.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524788+0000", 2024-09-15T00:12:44.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2540000000000001e-06 2024-09-15T00:12:44.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524817+0000", 2024-09-15T00:12:44.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8784000000000001e-05 2024-09-15T00:12:44.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.524828+0000", 2024-09-15T00:12:44.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-15T00:12:44.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:44.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532360+0000", 2024-09-15T00:12:44.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007531757 2024-09-15T00:12:44.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:06.532400+0000", 2024-09-15T00:12:44.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9835000000000001e-05 2024-09-15T00:12:44.757 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.757 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.757 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.757 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [2097152~420557,2629632~4096,2650112~4096,2670592~4096,2691072~4096,2711552~4096,2732032~4096,2756608~12288,2777088~12288,2797568~12288,2818048~12288,2838528~12288,2859008~12288,2879488~529715], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3409203, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:44.757 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:44.758 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.236354822000003, 2024-09-15T00:12:44.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0071155660000000003, 2024-09-15T00:12:44.758 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.758 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.758 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:44.758 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:44.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513586+0000", 2024-09-15T00:12:44.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.513587+0000", 2024-09-15T00:12:44.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4899999999999995e-07 2024-09-15T00:12:44.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514349+0000", 2024-09-15T00:12:44.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00076184799999999997 2024-09-15T00:12:44.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514368+0000", 2024-09-15T00:12:44.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9507000000000001e-05 2024-09-15T00:12:44.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514391+0000", 2024-09-15T00:12:44.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2497000000000002e-05 2024-09-15T00:12:44.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514491+0000", 2024-09-15T00:12:44.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9838000000000002e-05 2024-09-15T00:12:44.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.514500+0000", 2024-09-15T00:12:44.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4860000000000003e-06 2024-09-15T00:12:44.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.520702+0000", 2024-09-15T00:12:44.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062017410000000002 2024-09-15T00:12:44.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.762 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.762 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [3409203~654029,4194304~266240,4476928~4096,4497408~4096,4517888~4096,4538368~4096,4558848~4096,4579328~4096,4587520~103731], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4691251, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3409203, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:44.762 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:44.762 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.225496745000001, 2024-09-15T00:12:44.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079084649999999999, 2024-09-15T00:12:44.762 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.762 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.763 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:44.763 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:44.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524444+0000", 2024-09-15T00:12:44.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.524446+0000", 2024-09-15T00:12:44.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.243e-06 2024-09-15T00:12:44.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525115+0000", 2024-09-15T00:12:44.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00066896999999999996 2024-09-15T00:12:44.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525135+0000", 2024-09-15T00:12:44.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0137999999999999e-05 2024-09-15T00:12:44.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525148+0000", 2024-09-15T00:12:44.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3591000000000001e-05 2024-09-15T00:12:44.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525242+0000", 2024-09-15T00:12:44.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3603999999999994e-05 2024-09-15T00:12:44.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.525258+0000", 2024-09-15T00:12:44.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6150999999999998e-05 2024-09-15T00:12:44.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.532353+0000", 2024-09-15T00:12:44.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0070947680000000004 2024-09-15T00:12:44.766 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.766 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.766 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.766 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e, version: 47'82, data_included: [4691251~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:63e862a3:::rbd_data.11395b9829f3.0000000000000004:e@47'82, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5739827, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4691251, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:44.766 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:44.767 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.211813184, 2024-09-15T00:12:44.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0085554080000000005, 2024-09-15T00:12:44.767 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.767 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.767 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:44.767 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:44.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538128+0000", 2024-09-15T00:12:44.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.538129+0000", 2024-09-15T00:12:44.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:12:44.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539029+0000", 2024-09-15T00:12:44.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090036599999999999 2024-09-15T00:12:44.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539047+0000", 2024-09-15T00:12:44.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7776999999999999e-05 2024-09-15T00:12:44.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539058+0000", 2024-09-15T00:12:44.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1070000000000001e-05 2024-09-15T00:12:44.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539128+0000", 2024-09-15T00:12:44.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0531999999999995e-05 2024-09-15T00:12:44.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.539138+0000", 2024-09-15T00:12:44.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3910000000000004e-06 2024-09-15T00:12:44.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.546683+0000", 2024-09-15T00:12:44.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075455519999999996 2024-09-15T00:12:44.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.770 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.771 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.6 66/65 [PushOp(3:6618a9b3:::rbd_data.113947ee3a89.0000000000000009:15, version: 59'176, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:6618a9b3:::rbd_data.113947ee3a89.0000000000000009:15@59'176, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:44.771 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:44.771 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.142495388, 2024-09-15T00:12:44.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073968319999999999, 2024-09-15T00:12:44.771 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.771 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.771 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:44.771 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:44.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607446+0000", 2024-09-15T00:12:44.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607447+0000", 2024-09-15T00:12:44.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2100000000000001e-06 2024-09-15T00:12:44.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607778+0000", 2024-09-15T00:12:44.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033101199999999999 2024-09-15T00:12:44.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607800+0000", 2024-09-15T00:12:44.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2323e-05 2024-09-15T00:12:44.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607815+0000", 2024-09-15T00:12:44.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5073e-05 2024-09-15T00:12:44.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607926+0000", 2024-09-15T00:12:44.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110538 2024-09-15T00:12:44.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.607942+0000", 2024-09-15T00:12:44.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5662999999999999e-05 2024-09-15T00:12:44.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.614843+0000", 2024-09-15T00:12:44.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0069010130000000001 2024-09-15T00:12:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:125615 3.f e77/68 3:f382b49c:::rbd_data.11396481e1ec.0000000000000004:head v 77'241, mlcod=74'239)", 2024-09-15T00:12:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.839171+0000", 2024-09-15T00:12:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.910770550999999, 2024-09-15T00:12:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0072384550000000004, 2024-09-15T00:12:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.839171+0000", 2024-09-15T00:12:44.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.839171+0000", 2024-09-15T00:12:44.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.839172+0000", 2024-09-15T00:12:44.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3769999999999999e-06 2024-09-15T00:12:44.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840892+0000", 2024-09-15T00:12:44.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001720271 2024-09-15T00:12:44.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840905+0000", 2024-09-15T00:12:44.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2461e-05 2024-09-15T00:12:44.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840913+0000", 2024-09-15T00:12:44.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1990000000000006e-06 2024-09-15T00:12:44.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840962+0000", 2024-09-15T00:12:44.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8760000000000001e-05 2024-09-15T00:12:44.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.840979+0000", 2024-09-15T00:12:44.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7280999999999999e-05 2024-09-15T00:12:44.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:44.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.846347+0000", 2024-09-15T00:12:44.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005367925 2024-09-15T00:12:44.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.846409+0000", 2024-09-15T00:12:44.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2181000000000007e-05 2024-09-15T00:12:44.779 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.779 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.779 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.779 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133834 3.1b e85/69 3:da9a78a3:::rbd_data.11395dfcf2b8.0000000000000000:head v 85'356, mlcod=85'354)", 2024-09-15T00:12:44.780 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:44.780 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.186604817999999, 2024-09-15T00:12:44.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026079054000000001, 2024-09-15T00:12:44.780 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.780 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.780 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:44.780 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:44.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:44.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563338+0000", 2024-09-15T00:12:44.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.418e-06 2024-09-15T00:12:44.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569385+0000", 2024-09-15T00:12:44.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006047335 2024-09-15T00:12:44.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569418+0000", 2024-09-15T00:12:44.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2957999999999999e-05 2024-09-15T00:12:44.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569432+0000", 2024-09-15T00:12:44.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3718e-05 2024-09-15T00:12:44.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569519+0000", 2024-09-15T00:12:44.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7022999999999997e-05 2024-09-15T00:12:44.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569540+0000", 2024-09-15T00:12:44.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1426000000000001e-05 2024-09-15T00:12:44.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:44.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589362+0000", 2024-09-15T00:12:44.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019821398 2024-09-15T00:12:44.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589415+0000", 2024-09-15T00:12:44.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3777999999999999e-05 2024-09-15T00:12:44.784 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.784 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.784 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.784 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133836 3.1 e85/67 3:849f9a98:::rbd_data.11395dfcf2b8.0000000000000002:head v 85'304, mlcod=84'301)", 2024-09-15T00:12:44.784 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:44.784 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.178909908, 2024-09-15T00:12:44.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023231600000000002, 2024-09-15T00:12:44.784 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.784 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.785 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:44.785 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:44.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:44.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571032+0000", 2024-09-15T00:12:44.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-15T00:12:44.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576635+0000", 2024-09-15T00:12:44.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056028700000000002 2024-09-15T00:12:44.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576650+0000", 2024-09-15T00:12:44.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.505e-05 2024-09-15T00:12:44.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576658+0000", 2024-09-15T00:12:44.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6879999999999999e-06 2024-09-15T00:12:44.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576717+0000", 2024-09-15T00:12:44.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175000000000002e-05 2024-09-15T00:12:44.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576742+0000", 2024-09-15T00:12:44.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4734000000000001e-05 2024-09-15T00:12:44.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:44.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594234+0000", 2024-09-15T00:12:44.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017492527000000001 2024-09-15T00:12:44.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594263+0000", 2024-09-15T00:12:44.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8629e-05 2024-09-15T00:12:44.788 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.788 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.788 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.789 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:144980 3.f e93/68 3:f76a968b:::rbd_data.1139646a7d4e.0000000000000009:head v 93'285, mlcod=93'283)", 2024-09-15T00:12:44.789 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:39.520500+0000", 2024-09-15T00:12:44.789 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.2294406330000003, 2024-09-15T00:12:44.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012994957, 2024-09-15T00:12:44.789 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.789 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.789 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:44.789 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.520500+0000", 2024-09-15T00:12:44.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.520500+0000", 2024-09-15T00:12:44.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.520501+0000", 2024-09-15T00:12:44.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2999999999999999e-07 2024-09-15T00:12:44.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.521926+0000", 2024-09-15T00:12:44.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001424224 2024-09-15T00:12:44.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.521939+0000", 2024-09-15T00:12:44.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3049e-05 2024-09-15T00:12:44.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.521946+0000", 2024-09-15T00:12:44.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0770000000000002e-06 2024-09-15T00:12:44.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.521997+0000", 2024-09-15T00:12:44.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1035999999999997e-05 2024-09-15T00:12:44.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.522014+0000", 2024-09-15T00:12:44.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6956000000000002e-05 2024-09-15T00:12:44.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:44.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.533455+0000", 2024-09-15T00:12:44.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011441504 2024-09-15T00:12:44.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.533495+0000", 2024-09-15T00:12:44.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0181e-05 2024-09-15T00:12:44.793 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.793 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.793 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.793 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:144981 3.18 e93/69 3:1880ca23:::rbd_data.1139646a7d4e.000000000000000a:head v 93'485, mlcod=93'483)", 2024-09-15T00:12:44.793 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:39.521311+0000", 2024-09-15T00:12:44.793 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.2286301030000004, 2024-09-15T00:12:44.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012659617, 2024-09-15T00:12:44.793 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.793 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.793 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:44.794 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.521311+0000", 2024-09-15T00:12:44.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.521311+0000", 2024-09-15T00:12:44.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.521312+0000", 2024-09-15T00:12:44.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1699999999999997e-07 2024-09-15T00:12:44.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.522402+0000", 2024-09-15T00:12:44.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001089998 2024-09-15T00:12:44.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.522411+0000", 2024-09-15T00:12:44.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8589999999999994e-06 2024-09-15T00:12:44.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.522416+0000", 2024-09-15T00:12:44.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6359999999999999e-06 2024-09-15T00:12:44.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.522460+0000", 2024-09-15T00:12:44.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4066999999999997e-05 2024-09-15T00:12:44.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.522480+0000", 2024-09-15T00:12:44.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.012e-05 2024-09-15T00:12:44.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:44.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.533933+0000", 2024-09-15T00:12:44.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011452301 2024-09-15T00:12:44.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.533971+0000", 2024-09-15T00:12:44.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7919000000000001e-05 2024-09-15T00:12:44.797 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.797 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.798 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.798 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 93/92 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:44.798 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:39.531770+0000", 2024-09-15T00:12:44.798 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.2181714929999998, 2024-09-15T00:12:44.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007635551, 2024-09-15T00:12:44.798 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.798 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.798 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:44.798 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.531770+0000", 2024-09-15T00:12:44.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.531770+0000", 2024-09-15T00:12:44.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.531770+0000", 2024-09-15T00:12:44.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9500000000000003e-07 2024-09-15T00:12:44.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.532741+0000", 2024-09-15T00:12:44.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097131099999999998 2024-09-15T00:12:44.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.532767+0000", 2024-09-15T00:12:44.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5335999999999999e-05 2024-09-15T00:12:44.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.532789+0000", 2024-09-15T00:12:44.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1848000000000002e-05 2024-09-15T00:12:44.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.532869+0000", 2024-09-15T00:12:44.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0857000000000002e-05 2024-09-15T00:12:44.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.532883+0000", 2024-09-15T00:12:44.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3456e-05 2024-09-15T00:12:44.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.539405+0000", 2024-09-15T00:12:44.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0065222479999999996 2024-09-15T00:12:44.802 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.802 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.802 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.802 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:145028 3.4 e93/92 3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head v 93'455, mlcod=29'3)", 2024-09-15T00:12:44.802 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:39.543212+0000", 2024-09-15T00:12:44.802 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.2067294850000003, 2024-09-15T00:12:44.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011183056, 2024-09-15T00:12:44.802 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.802 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.802 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:44.802 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.543212+0000", 2024-09-15T00:12:44.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.543212+0000", 2024-09-15T00:12:44.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.543212+0000", 2024-09-15T00:12:44.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0200000000000001e-07 2024-09-15T00:12:44.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.544503+0000", 2024-09-15T00:12:44.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001290334 2024-09-15T00:12:44.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.544511+0000", 2024-09-15T00:12:44.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.354e-06 2024-09-15T00:12:44.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.544516+0000", 2024-09-15T00:12:44.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7140000000000001e-06 2024-09-15T00:12:44.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.545605+0000", 2024-09-15T00:12:44.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010889809999999999 2024-09-15T00:12:44.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.545627+0000", 2024-09-15T00:12:44.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2345e-05 2024-09-15T00:12:44.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:44.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.554362+0000", 2024-09-15T00:12:44.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0087352139999999998 2024-09-15T00:12:44.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.554395+0000", 2024-09-15T00:12:44.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2412000000000001e-05 2024-09-15T00:12:44.806 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.806 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.806 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.806 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1a 93/92 [PushOp(3:5e7e71d0:::rbd_data.11395b9829f3.0000000000000000:e, version: 47'181, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:5e7e71d0:::rbd_data.11395b9829f3.0000000000000000:e@47'181, size: 15335424, copy_subset: [(0, 15335424)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:44.806 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:39.760937+0000", 2024-09-15T00:12:44.806 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.9890038460000001, 2024-09-15T00:12:44.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007391491, 2024-09-15T00:12:44.806 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.807 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.807 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:44.807 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.760937+0000", 2024-09-15T00:12:44.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.760937+0000", 2024-09-15T00:12:44.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.760938+0000", 2024-09-15T00:12:44.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:12:44.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.762175+0000", 2024-09-15T00:12:44.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012376539999999999 2024-09-15T00:12:44.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.762190+0000", 2024-09-15T00:12:44.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4786e-05 2024-09-15T00:12:44.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.762200+0000", 2024-09-15T00:12:44.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7929999999999992e-06 2024-09-15T00:12:44.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.762261+0000", 2024-09-15T00:12:44.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1024999999999999e-05 2024-09-15T00:12:44.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.762269+0000", 2024-09-15T00:12:44.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6839999999999996e-06 2024-09-15T00:12:44.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.768329+0000", 2024-09-15T00:12:44.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006060139 2024-09-15T00:12:44.810 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.810 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.810 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.810 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1a 93/92 [PushOp(3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e, version: 47'185, data_included: [2097152~4096,2117632~4096,2138112~4096,2158592~4096,2179072~4096,2199552~4096,2220032~4096,2228224~262144,2576713~35894,2621440~721866], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e@47'185, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3343306, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:12:44.810 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:39.889465+0000", 2024-09-15T00:12:44.810 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8604764439999997, 2024-09-15T00:12:44.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073346749999999997, 2024-09-15T00:12:44.810 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.811 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.811 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:44.811 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.889465+0000", 2024-09-15T00:12:44.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.889465+0000", 2024-09-15T00:12:44.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.889466+0000", 2024-09-15T00:12:44.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9000000000000005e-07 2024-09-15T00:12:44.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.890475+0000", 2024-09-15T00:12:44.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010089700000000001 2024-09-15T00:12:44.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.890501+0000", 2024-09-15T00:12:44.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6520999999999999e-05 2024-09-15T00:12:44.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.890525+0000", 2024-09-15T00:12:44.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4113000000000001e-05 2024-09-15T00:12:44.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.890633+0000", 2024-09-15T00:12:44.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107735 2024-09-15T00:12:44.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.890650+0000", 2024-09-15T00:12:44.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7048e-05 2024-09-15T00:12:44.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:39.896799+0000", 2024-09-15T00:12:44.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061492980000000001 2024-09-15T00:12:44.814 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.814 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.814 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.814 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.e 94/92 [PushOp(3:73ec49f1:::rbd_data.113947ee3a89.0000000000000003:head, version: 58'1013, data_included: [0~983040], data_size: 983040, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:73ec49f1:::rbd_data.113947ee3a89.0000000000000003:head@58'1013, size: 983040, copy_subset: [(0, 983040)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 983040, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:44.814 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:40.194203+0000", 2024-09-15T00:12:44.814 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5557384699999997, 2024-09-15T00:12:44.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0087426819999999999, 2024-09-15T00:12:44.815 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.815 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.815 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:44.815 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.194203+0000", 2024-09-15T00:12:44.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.194203+0000", 2024-09-15T00:12:44.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.194203+0000", 2024-09-15T00:12:44.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:12:44.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.195084+0000", 2024-09-15T00:12:44.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088122500000000002 2024-09-15T00:12:44.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.195098+0000", 2024-09-15T00:12:44.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3385e-05 2024-09-15T00:12:44.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.195124+0000", 2024-09-15T00:12:44.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6451000000000001e-05 2024-09-15T00:12:44.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.195197+0000", 2024-09-15T00:12:44.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2418999999999999e-05 2024-09-15T00:12:44.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.195209+0000", 2024-09-15T00:12:44.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2926e-05 2024-09-15T00:12:44.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.202945+0000", 2024-09-15T00:12:44.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077358690000000003 2024-09-15T00:12:44.818 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.818 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.818 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.818 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.e 94/92 [PushOp(3:7588bc7b:::rbd_data.1139e6d65aff.0000000000000001:head, version: 62'1027, data_included: [0~524288,655360~524288], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:7588bc7b:::rbd_data.1139e6d65aff.0000000000000001:head@62'1027, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:44.818 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:40.234057+0000", 2024-09-15T00:12:44.818 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5158838809999997, 2024-09-15T00:12:44.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010390427000000001, 2024-09-15T00:12:44.819 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.819 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.819 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:44.819 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.234057+0000", 2024-09-15T00:12:44.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.234057+0000", 2024-09-15T00:12:44.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.234067+0000", 2024-09-15T00:12:44.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8500000000000006e-06 2024-09-15T00:12:44.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.234910+0000", 2024-09-15T00:12:44.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00084323500000000001 2024-09-15T00:12:44.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.234927+0000", 2024-09-15T00:12:44.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6662000000000001e-05 2024-09-15T00:12:44.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.234937+0000", 2024-09-15T00:12:44.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0125999999999999e-05 2024-09-15T00:12:44.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.235019+0000", 2024-09-15T00:12:44.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1954999999999997e-05 2024-09-15T00:12:44.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.235032+0000", 2024-09-15T00:12:44.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3388e-05 2024-09-15T00:12:44.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.244448+0000", 2024-09-15T00:12:44.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094152109999999997 2024-09-15T00:12:44.822 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.822 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.822 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.822 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.e 94/92 [PushOp(3:743f56f7:::rbd_data.11395dfcf2b8.0000000000000001:head, version: 85'1145, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:743f56f7:::rbd_data.11395dfcf2b8.0000000000000001:head@85'1145, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:12:44.822 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:40.301476+0000", 2024-09-15T00:12:44.823 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.4484649019999996, 2024-09-15T00:12:44.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0072086509999999999, 2024-09-15T00:12:44.823 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.823 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.823 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:44.823 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.301476+0000", 2024-09-15T00:12:44.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.301476+0000", 2024-09-15T00:12:44.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.301477+0000", 2024-09-15T00:12:44.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7899999999999998e-07 2024-09-15T00:12:44.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.302440+0000", 2024-09-15T00:12:44.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000963221 2024-09-15T00:12:44.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.302454+0000", 2024-09-15T00:12:44.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.413e-05 2024-09-15T00:12:44.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.302463+0000", 2024-09-15T00:12:44.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1009999999999998e-06 2024-09-15T00:12:44.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.302538+0000", 2024-09-15T00:12:44.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4289000000000007e-05 2024-09-15T00:12:44.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.302544+0000", 2024-09-15T00:12:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.793e-06 2024-09-15T00:12:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:40.308685+0000", 2024-09-15T00:12:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061405380000000001 2024-09-15T00:12:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146609 3.b e98/69 3:d39b42ba:::rbd_data.11394501364.0000000000000001:head v 98'283, mlcod=93'281)", 2024-09-15T00:12:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.26919179300000001, 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020892945, 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480751+0000", 2024-09-15T00:12:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6509999999999999e-06 2024-09-15T00:12:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482738+0000", 2024-09-15T00:12:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019867410000000002 2024-09-15T00:12:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482754+0000", 2024-09-15T00:12:44.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6171000000000002e-05 2024-09-15T00:12:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482769+0000", 2024-09-15T00:12:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4688000000000001e-05 2024-09-15T00:12:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482835+0000", 2024-09-15T00:12:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6526000000000006e-05 2024-09-15T00:12:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482858+0000", 2024-09-15T00:12:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2959000000000002e-05 2024-09-15T00:12:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501587+0000", 2024-09-15T00:12:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018728471 2024-09-15T00:12:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501642+0000", 2024-09-15T00:12:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5738000000000001e-05 2024-09-15T00:12:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.834 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146835 3.12 e98/68 3:4eee39d6:::rbd_data.11394501364.0000000000000000:head v 98'474, mlcod=93'472)", 2024-09-15T00:12:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:12:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.16056452299999999, 2024-09-15T00:12:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020080434000000001, 2024-09-15T00:12:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:12:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:12:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589378+0000", 2024-09-15T00:12:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7770000000000001e-06 2024-09-15T00:12:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590678+0000", 2024-09-15T00:12:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012991750000000001 2024-09-15T00:12:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590699+0000", 2024-09-15T00:12:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1353e-05 2024-09-15T00:12:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590710+0000", 2024-09-15T00:12:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1245e-05 2024-09-15T00:12:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590800+0000", 2024-09-15T00:12:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9963000000000006e-05 2024-09-15T00:12:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590823+0000", 2024-09-15T00:12:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3289e-05 2024-09-15T00:12:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609368+0000", 2024-09-15T00:12:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018545078 2024-09-15T00:12:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609457+0000", 2024-09-15T00:12:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8554000000000006e-05 2024-09-15T00:12:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146837 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7201319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:12:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:12:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.14978506, 2024-09-15T00:12:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024751038, 2024-09-15T00:12:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:44.839 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146837 2024-09-15T00:12:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:12:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:12:44.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600157+0000", 2024-09-15T00:12:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:12:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601965+0000", 2024-09-15T00:12:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001807613 2024-09-15T00:12:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:44.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601985+0000", 2024-09-15T00:12:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0679e-05 2024-09-15T00:12:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601998+0000", 2024-09-15T00:12:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2955999999999999e-05 2024-09-15T00:12:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602070+0000", 2024-09-15T00:12:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1958000000000005e-05 2024-09-15T00:12:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602237+0000", 2024-09-15T00:12:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016621299999999999 2024-09-15T00:12:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602799+0000", 2024-09-15T00:12:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000562723 2024-09-15T00:12:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.620811+0000", 2024-09-15T00:12:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018011207000000001 2024-09-15T00:12:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624793+0000", 2024-09-15T00:12:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0039828770000000001 2024-09-15T00:12:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624855+0000", 2024-09-15T00:12:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.143e-05 2024-09-15T00:12:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624907+0000", 2024-09-15T00:12:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2262000000000002e-05 2024-09-15T00:12:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:44.845 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:44.845 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:44.846 INFO:teuthology.orchestra.run.smithi171.stdout:5059 read 0x825c789 thru 0x8263bc1 (0x7439 bytes) 2024-09-15T00:12:44.846 INFO:teuthology.orchestra.run.smithi171.stdout:5060 writesame 0x232808c thru 0x23377da data_size 0x1(0xf74f bytes) 2024-09-15T00:12:44.847 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:44.847 INFO:teuthology.orchestra.run.smithi171.stdout:5062 read 0x7a299f4 thru 0x7a2b9d5 (0x1fe2 bytes) 2024-09-15T00:12:44.848 INFO:teuthology.orchestra.run.smithi171.stdout:5063 trunc from 0xaddaa06 to 0x79e00ec 2024-09-15T00:12:44.848 INFO:teuthology.orchestra.run.smithi171.stdout:5065 writesame 0x9c07145 thru 0x9c0ba53 data_size 0x1(0x490f bytes) 2024-09-15T00:12:44.848 INFO:teuthology.orchestra.run.smithi171.stdout:5067 punch from 0x1a97d85 to 0x1aa0431, (0x86ac bytes) 2024-09-15T00:12:44.848 INFO:teuthology.orchestra.run.smithi171.stdout:5069 read 0xe5bbb8 thru 0xe64817 (0x8c60 bytes) 2024-09-15T00:12:44.848 INFO:teuthology.orchestra.run.smithi171.stdout:5070 punch from 0x127f796 to 0x128c46c, (0xccd6 bytes) 2024-09-15T00:12:44.848 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:44.844+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:44.849 INFO:teuthology.orchestra.run.smithi171.stdout:5072 write 0xd8a9ae2 thru 0xd8b1533 (0x7a52 bytes) 2024-09-15T00:12:44.852 INFO:teuthology.orchestra.run.smithi171.stdout:5073 read 0xba68ee8 thru 0xba7193b (0x8a54 bytes) 2024-09-15T00:12:44.853 INFO:teuthology.orchestra.run.smithi171.stdout:5075 write 0x60f75b2 thru 0x6101894 (0xa2e3 bytes) 2024-09-15T00:12:44.858 INFO:teuthology.orchestra.run.smithi171.stdout:5076 trunc from 0xd8b1534 to 0xcde6b58 2024-09-15T00:12:44.866 INFO:teuthology.orchestra.run.smithi171.stdout:5078 writesame 0x5df4b05 thru 0x5df5f8c data_size 0x24(0x1488 bytes) 2024-09-15T00:12:44.867 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:44.867 INFO:teuthology.orchestra.run.smithi171.stdout:5080 punch from 0x98cda06 to 0x98d2b24, (0x511e bytes) 2024-09-15T00:12:44.875 INFO:teuthology.orchestra.run.smithi171.stdout:5081 trunc from 0xcde6b58 to 0x5d547fb 2024-09-15T00:12:44.887 INFO:teuthology.orchestra.run.smithi171.stdout:5082 trunc from 0x5d547fb to 0x663b939 2024-09-15T00:12:44.892 INFO:teuthology.orchestra.run.smithi171.stdout:5085 write 0x3a257d8 thru 0x3a2b2fd (0x5b26 bytes) 2024-09-15T00:12:44.894 INFO:teuthology.orchestra.run.smithi171.stdout:5086 write 0x4f3c44a thru 0x4f3fb9d (0x3754 bytes) 2024-09-15T00:12:44.896 INFO:teuthology.orchestra.run.smithi171.stdout:5087 punch from 0x361c015 to 0x361cb88, (0xb73 bytes) 2024-09-15T00:12:44.905 INFO:teuthology.orchestra.run.smithi171.stdout:5091 writesame 0x892f49f thru 0x892faa0 data_size 0x1(0x602 bytes) 2024-09-15T00:12:44.905 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:44.905 INFO:teuthology.orchestra.run.smithi171.stdout:5093 read 0x42029da thru 0x420724f (0x4876 bytes) 2024-09-15T00:12:44.905 INFO:teuthology.orchestra.run.smithi171.stdout:5094 read 0x7a3b7e0 thru 0x7a44e0e (0x962f bytes) 2024-09-15T00:12:44.906 INFO:teuthology.orchestra.run.smithi171.stdout:5095 read 0x3cc7cda thru 0x3cd3376 (0xb69d bytes) 2024-09-15T00:12:44.906 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:44.906 INFO:teuthology.orchestra.run.smithi171.stdout:5097 trunc from 0x892faa1 to 0x956ae4a 2024-09-15T00:12:44.922 INFO:teuthology.orchestra.run.smithi171.stdout:5098 write 0xc2eb3f2 thru 0xc2f62c8 (0xaed7 bytes) 2024-09-15T00:12:44.929 INFO:teuthology.orchestra.run.smithi171.stdout:5099 trunc from 0xc2f62c9 to 0xebb0984 2024-09-15T00:12:44.935 INFO:teuthology.orchestra.run.smithi171.stdout:5100 write 0x11af3e thru 0x1205ac (0x566f bytes) 2024-09-15T00:12:44.937 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:44.938 INFO:teuthology.orchestra.run.smithi171.stdout:5103 writesame 0x993d5d2 thru 0x994bacb data_size 0x6a(0xe4fa bytes) 2024-09-15T00:12:44.941 INFO:teuthology.orchestra.run.smithi171.stdout:5104 writesame 0x4a67155 thru 0x4a699f9 data_size 0x1(0x28a5 bytes) 2024-09-15T00:12:44.946 INFO:teuthology.orchestra.run.smithi171.stdout:5105 read 0xe89c60b thru 0xe8a7a77 (0xb46d bytes) 2024-09-15T00:12:44.947 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:44.945+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:44.947 INFO:teuthology.orchestra.run.smithi171.stdout:5107 read 0x635dc1f thru 0x6360d5f (0x3141 bytes) 2024-09-15T00:12:44.947 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:44.947 INFO:teuthology.orchestra.run.smithi171.stdout:5109 punch from 0x510309e to 0x5106ce3, (0x3c45 bytes) 2024-09-15T00:12:44.948 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:44.948 INFO:teuthology.orchestra.run.smithi171.stdout:5113 write 0x38a7b1 thru 0x3914c0 (0x6d10 bytes) 2024-09-15T00:12:44.949 INFO:teuthology.orchestra.run.smithi171.stdout:5114 writesame 0x7387fa5 thru 0x738e2bd data_size 0x1(0x6319 bytes) 2024-09-15T00:12:44.966 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:44.967 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:44.967 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:44.967 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:44.976 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:45.014 INFO:teuthology.orchestra.run.smithi171.stdout:5118 trunc from 0xebb0984 to 0x1e4318 2024-09-15T00:12:45.037 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is set 2024-09-15T00:12:45.048 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:45.046+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:45.059 INFO:teuthology.orchestra.run.smithi171.stdout:5119 write 0xcfbfdfe thru 0xcfcddd5 (0xdfd8 bytes) 2024-09-15T00:12:45.063 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:45.064 INFO:teuthology.orchestra.run.smithi171.stdout:5121 write 0x9e46867 thru 0x9e53c3e (0xd3d8 bytes) 2024-09-15T00:12:45.068 INFO:teuthology.orchestra.run.smithi171.stdout:5123 writesame 0x7243cae thru 0x72471cc data_size 0x3(0x351f bytes) 2024-09-15T00:12:45.075 INFO:teuthology.orchestra.run.smithi171.stdout:5124 punch from 0x3d502a7 to 0x3d518b2, (0x160b bytes) 2024-09-15T00:12:45.076 INFO:teuthology.orchestra.run.smithi171.stdout:5125 punch from 0x9998f20 to 0x99a16e6, (0x87c6 bytes) 2024-09-15T00:12:45.076 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:45.076 INFO:teuthology.orchestra.run.smithi171.stdout:5128 writesame 0x7267a68 thru 0x726a6c3 data_size 0x22(0x2c5c bytes) 2024-09-15T00:12:45.079 INFO:teuthology.orchestra.run.smithi171.stdout:5129 write 0xa420c8f thru 0xa421bed (0xf5f bytes) 2024-09-15T00:12:45.082 INFO:teuthology.orchestra.run.smithi171.stdout:5130 write 0x4ca5a15 thru 0x4cabe29 (0x6415 bytes) 2024-09-15T00:12:45.084 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:45.084 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:45.085 INFO:teuthology.orchestra.run.smithi171.stdout:5135 punch from 0x4e6c38f to 0x4e71661, (0x52d2 bytes) 2024-09-15T00:12:45.085 INFO:teuthology.orchestra.run.smithi171.stdout:5136 read 0x7cc8f4d thru 0x7ccdf55 (0x5009 bytes) 2024-09-15T00:12:45.085 INFO:teuthology.orchestra.run.smithi171.stdout:5137 clone 43 order 18 su 131072 sc 9 2024-09-15T00:12:45.127 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:45.127 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:45.136 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:45.177 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:45.175+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:45.249 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:45.247+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:45.281 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:45.281 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:45.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:45.282 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:45.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.282 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:45.282 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:45.282 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.099363233999995, 2024-09-15T00:12:45.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:12:45.282 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.282 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.282 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:45.282 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.282 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.282 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.282 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:12:45.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.283 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:45.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:45.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:12:45.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:12:45.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:12:45.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:12:45.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:12:45.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:12:45.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:12:45.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:12:45.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:12:45.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:12:45.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:12:45.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:12:45.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:45.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:12:45.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:12:45.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:45.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:12:45.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:12:45.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:45.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:12:45.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:12:45.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:45.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:12:45.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:12:45.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:12:45.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:12:45.287 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.287 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.287 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.288 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:45.288 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:45.288 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.369522055000004, 2024-09-15T00:12:45.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:12:45.288 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.288 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.288 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:45.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.288 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:12:45.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.288 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:45.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:45.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:12:45.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:12:45.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:12:45.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:12:45.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:12:45.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:12:45.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:12:45.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:12:45.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:12:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:12:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:12:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:12:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:12:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:12:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:45.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:12:45.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:12:45.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:45.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:12:45.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:12:45.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:45.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:12:45.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:12:45.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:12:45.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:12:45.293 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.293 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.293 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.293 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:12:45.293 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:45.293 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.362563691999995, 2024-09-15T00:12:45.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:12:45.294 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.294 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.294 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.294 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:45.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:45.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:12:45.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:12:45.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:12:45.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:12:45.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:12:45.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:12:45.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:12:45.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:12:45.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:12:45.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:12:45.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:12:45.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:12:45.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:12:45.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:12:45.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.297 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.297 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:45.298 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:45.298 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.615374854999999, 2024-09-15T00:12:45.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:12:45.298 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.298 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.298 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:45.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.298 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:12:45.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.298 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:45.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:45.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:12:45.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:12:45.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:12:45.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:12:45.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:12:45.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:12:45.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:12:45.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:12:45.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:12:45.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:12:45.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:12:45.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:12:45.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:45.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:12:45.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:12:45.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:45.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:12:45.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:12:45.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:45.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:12:45.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:12:45.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:45.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:12:45.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:12:45.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:12:45.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:12:45.303 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.303 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.303 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.304 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97493 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:45.304 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:45.304 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.425792135000002, 2024-09-15T00:12:45.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033259141999999998, 2024-09-15T00:12:45.304 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.304 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.304 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.304 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97493 2024-09-15T00:12:45.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.304 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:45.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:45.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:45.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-15T00:12:45.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853401+0000", 2024-09-15T00:12:45.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:12:45.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853407+0000", 2024-09-15T00:12:45.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-15T00:12:45.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853411+0000", 2024-09-15T00:12:45.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.481e-06 2024-09-15T00:12:45.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886544+0000", 2024-09-15T00:12:45.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033132861999999999 2024-09-15T00:12:45.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886587+0000", 2024-09-15T00:12:45.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2942000000000003e-05 2024-09-15T00:12:45.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886655+0000", 2024-09-15T00:12:45.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7711000000000003e-05 2024-09-15T00:12:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97494 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.425771011999998, 2024-09-15T00:12:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033392390000000001, 2024-09-15T00:12:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.308 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97494 2024-09-15T00:12:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:45.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-15T00:12:45.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853419+0000", 2024-09-15T00:12:45.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.917e-06 2024-09-15T00:12:45.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853425+0000", 2024-09-15T00:12:45.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:12:45.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853433+0000", 2024-09-15T00:12:45.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8120000000000008e-06 2024-09-15T00:12:45.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886680+0000", 2024-09-15T00:12:45.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033247207000000001 2024-09-15T00:12:45.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886726+0000", 2024-09-15T00:12:45.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5902000000000002e-05 2024-09-15T00:12:45.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886809+0000", 2024-09-15T00:12:45.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3475999999999998e-05 2024-09-15T00:12:45.312 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.312 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.312 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.312 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97495 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:45.312 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:45.313 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.425750059000002, 2024-09-15T00:12:45.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033640503000000002, 2024-09-15T00:12:45.313 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.313 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.313 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.313 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.313 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.313 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.313 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97495 2024-09-15T00:12:45.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.313 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:45.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:45.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:45.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:12:45.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853440+0000", 2024-09-15T00:12:45.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.804e-06 2024-09-15T00:12:45.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853446+0000", 2024-09-15T00:12:45.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5559999999999998e-06 2024-09-15T00:12:45.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853450+0000", 2024-09-15T00:12:45.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9519999999999996e-06 2024-09-15T00:12:45.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886904+0000", 2024-09-15T00:12:45.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033454207999999999 2024-09-15T00:12:45.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887007+0000", 2024-09-15T00:12:45.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102904 2024-09-15T00:12:45.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887078+0000", 2024-09-15T00:12:45.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1719999999999998e-05 2024-09-15T00:12:45.316 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.316 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.317 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.317 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97496 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:45.317 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:45.317 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.425732961000001, 2024-09-15T00:12:45.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033802431000000001, 2024-09-15T00:12:45.317 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.317 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.317 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.317 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.317 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.317 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.317 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97496 2024-09-15T00:12:45.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.318 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:45.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:45.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.53e-07 2024-09-15T00:12:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853462+0000", 2024-09-15T00:12:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-15T00:12:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853468+0000", 2024-09-15T00:12:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3609999999999997e-06 2024-09-15T00:12:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853473+0000", 2024-09-15T00:12:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-15T00:12:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887111+0000", 2024-09-15T00:12:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033638376999999997 2024-09-15T00:12:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887178+0000", 2024-09-15T00:12:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6877000000000004e-05 2024-09-15T00:12:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887257+0000", 2024-09-15T00:12:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9616999999999993e-05 2024-09-15T00:12:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97587 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.425709726999997, 2024-09-15T00:12:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033973642999999998, 2024-09-15T00:12:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97587 2024-09-15T00:12:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853479+0000", 2024-09-15T00:12:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:12:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853481+0000", 2024-09-15T00:12:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.136e-06 2024-09-15T00:12:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853487+0000", 2024-09-15T00:12:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2439999999999998e-06 2024-09-15T00:12:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853492+0000", 2024-09-15T00:12:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:12:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887284+0000", 2024-09-15T00:12:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033791827000000003 2024-09-15T00:12:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887361+0000", 2024-09-15T00:12:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6983000000000005e-05 2024-09-15T00:12:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887452+0000", 2024-09-15T00:12:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1236999999999994e-05 2024-09-15T00:12:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97588 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.425689570000003, 2024-09-15T00:12:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034134617999999999, 2024-09-15T00:12:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97588 2024-09-15T00:12:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853499+0000", 2024-09-15T00:12:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:12:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853501+0000", 2024-09-15T00:12:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.26e-06 2024-09-15T00:12:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853507+0000", 2024-09-15T00:12:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2899999999999999e-06 2024-09-15T00:12:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853512+0000", 2024-09-15T00:12:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8330000000000001e-06 2024-09-15T00:12:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887495+0000", 2024-09-15T00:12:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033982529999999997 2024-09-15T00:12:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887548+0000", 2024-09-15T00:12:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3445000000000001e-05 2024-09-15T00:12:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887633+0000", 2024-09-15T00:12:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4890999999999997e-05 2024-09-15T00:12:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126866 3.6 3.973c9fa6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.322367624999998, 2024-09-15T00:12:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153301000000003, 2024-09-15T00:12:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126866 2024-09-15T00:12:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956821+0000", 2024-09-15T00:12:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.91e-07 2024-09-15T00:12:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956822+0000", 2024-09-15T00:12:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.705e-06 2024-09-15T00:12:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956828+0000", 2024-09-15T00:12:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6760000000000004e-06 2024-09-15T00:12:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956833+0000", 2024-09-15T00:12:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5379999999999996e-06 2024-09-15T00:12:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989970+0000", 2024-09-15T00:12:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033137560000000003 2024-09-15T00:12:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989974+0000", 2024-09-15T00:12:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-15T00:12:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126951 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.322241942000002, 2024-09-15T00:12:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033152585999999998, 2024-09-15T00:12:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126951 2024-09-15T00:12:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-07 2024-09-15T00:12:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956953+0000", 2024-09-15T00:12:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9310000000000002e-06 2024-09-15T00:12:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956957+0000", 2024-09-15T00:12:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5379999999999998e-06 2024-09-15T00:12:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956959+0000", 2024-09-15T00:12:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6599999999999999e-06 2024-09-15T00:12:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990095+0000", 2024-09-15T00:12:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033135855999999998 2024-09-15T00:12:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990099+0000", 2024-09-15T00:12:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4010000000000001e-06 2024-09-15T00:12:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126952 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.322224952999999, 2024-09-15T00:12:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033156066999999997, 2024-09-15T00:12:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126952 2024-09-15T00:12:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:12:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956965+0000", 2024-09-15T00:12:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3379999999999999e-06 2024-09-15T00:12:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956968+0000", 2024-09-15T00:12:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0419999999999998e-06 2024-09-15T00:12:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956970+0000", 2024-09-15T00:12:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4899999999999999e-06 2024-09-15T00:12:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990113+0000", 2024-09-15T00:12:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033142642 2024-09-15T00:12:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990119+0000", 2024-09-15T00:12:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2979999999999997e-06 2024-09-15T00:12:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.341 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126954 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.322179458000001, 2024-09-15T00:12:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033158449, 2024-09-15T00:12:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.342 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126954 2024-09-15T00:12:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-15T00:12:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957010+0000", 2024-09-15T00:12:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3659999999999999e-06 2024-09-15T00:12:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957013+0000", 2024-09-15T00:12:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1140000000000002e-06 2024-09-15T00:12:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957016+0000", 2024-09-15T00:12:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1209999999999998e-06 2024-09-15T00:12:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990163+0000", 2024-09-15T00:12:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033146451 2024-09-15T00:12:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990167+0000", 2024-09-15T00:12:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0579999999999999e-06 2024-09-15T00:12:45.345 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126955 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.322168244, 2024-09-15T00:12:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033160617000000003, 2024-09-15T00:12:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.346 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126955 2024-09-15T00:12:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-15T00:12:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957021+0000", 2024-09-15T00:12:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-15T00:12:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957024+0000", 2024-09-15T00:12:45.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1599999999999998e-06 2024-09-15T00:12:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957027+0000", 2024-09-15T00:12:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000001e-06 2024-09-15T00:12:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990177+0000", 2024-09-15T00:12:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033149741000000003 2024-09-15T00:12:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990180+0000", 2024-09-15T00:12:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4910000000000001e-06 2024-09-15T00:12:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126956 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.322158123000001, 2024-09-15T00:12:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033163633999999997, 2024-09-15T00:12:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.350 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126956 2024-09-15T00:12:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:45.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957031+0000", 2024-09-15T00:12:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-06 2024-09-15T00:12:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957034+0000", 2024-09-15T00:12:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9960000000000002e-06 2024-09-15T00:12:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957037+0000", 2024-09-15T00:12:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6419999999999999e-06 2024-09-15T00:12:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990190+0000", 2024-09-15T00:12:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153152999999998 2024-09-15T00:12:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990193+0000", 2024-09-15T00:12:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6069999999999999e-06 2024-09-15T00:12:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126957 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.322148472999999, 2024-09-15T00:12:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033166442999999997, 2024-09-15T00:12:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:45.354 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126957 2024-09-15T00:12:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957040+0000", 2024-09-15T00:12:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:12:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957041+0000", 2024-09-15T00:12:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0440000000000001e-06 2024-09-15T00:12:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957044+0000", 2024-09-15T00:12:45.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0170000000000001e-06 2024-09-15T00:12:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957048+0000", 2024-09-15T00:12:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0319999999999997e-06 2024-09-15T00:12:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990202+0000", 2024-09-15T00:12:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033154640999999999 2024-09-15T00:12:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990206+0000", 2024-09-15T00:12:45.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.507e-06 2024-09-15T00:12:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126958 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.322137243, 2024-09-15T00:12:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033167613999999998, 2024-09-15T00:12:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126958 2024-09-15T00:12:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:45.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:12:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957053+0000", 2024-09-15T00:12:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.756e-06 2024-09-15T00:12:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957059+0000", 2024-09-15T00:12:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2770000000000002e-06 2024-09-15T00:12:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957063+0000", 2024-09-15T00:12:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2459999999999997e-06 2024-09-15T00:12:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990215+0000", 2024-09-15T00:12:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033151509000000003 2024-09-15T00:12:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990218+0000", 2024-09-15T00:12:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5099999999999999e-06 2024-09-15T00:12:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126959 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.322118239000002, 2024-09-15T00:12:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033161673000000003, 2024-09-15T00:12:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:45.362 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126959 2024-09-15T00:12:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:12:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957072+0000", 2024-09-15T00:12:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2239999999999998e-06 2024-09-15T00:12:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957079+0000", 2024-09-15T00:12:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.19e-06 2024-09-15T00:12:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957083+0000", 2024-09-15T00:12:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:12:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990228+0000", 2024-09-15T00:12:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033144818 2024-09-15T00:12:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990231+0000", 2024-09-15T00:12:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:12:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126960 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.366 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.322099432000002, 2024-09-15T00:12:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033155166, 2024-09-15T00:12:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126960 2024-09-15T00:12:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:12:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957091+0000", 2024-09-15T00:12:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:12:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957097+0000", 2024-09-15T00:12:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6050000000000003e-06 2024-09-15T00:12:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957102+0000", 2024-09-15T00:12:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5539999999999999e-06 2024-09-15T00:12:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990240+0000", 2024-09-15T00:12:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033138289000000001 2024-09-15T00:12:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990244+0000", 2024-09-15T00:12:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4740000000000001e-06 2024-09-15T00:12:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.370 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.370 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:45.372 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:45.372 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:45.370+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:45.472 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:45.471+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:45.517 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:45.517 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:45.517 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:45.517 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:45.526 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:45.573 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:45.571+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:45.628 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:45.628 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:45.628 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:45.628 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:45.628 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:45.636 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:45.674 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:45.672+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:45.742 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:45.742 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:45.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:12:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 57.497019801999997, 2024-09-15T00:12:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:12:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:12:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:12:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:12:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:12:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:12:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:12:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:12:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:12:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:12:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:12:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:12:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:12:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:12:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:12:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:12:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:12:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:12:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:12:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:12:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:12:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:12:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:12:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:12:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58878 3.8 3:12456482:::rbd_data.1139766160da.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 52.450714218000002, 2024-09-15T00:12:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061956199000000003, 2024-09-15T00:12:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58878 2024-09-15T00:12:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289151+0000", 2024-09-15T00:12:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:12:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289473+0000", 2024-09-15T00:12:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032183800000000002 2024-09-15T00:12:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289487+0000", 2024-09-15T00:12:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-05 2024-09-15T00:12:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289493+0000", 2024-09-15T00:12:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4489999999999996e-06 2024-09-15T00:12:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289542+0000", 2024-09-15T00:12:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8844999999999997e-05 2024-09-15T00:12:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289642+0000", 2024-09-15T00:12:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9607999999999999e-05 2024-09-15T00:12:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.293908+0000", 2024-09-15T00:12:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659350000000002 2024-09-15T00:12:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351022+0000", 2024-09-15T00:12:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057114535000000001 2024-09-15T00:12:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351033+0000", 2024-09-15T00:12:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0263e-05 2024-09-15T00:12:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351070+0000", 2024-09-15T00:12:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7666000000000002e-05 2024-09-15T00:12:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351106+0000", 2024-09-15T00:12:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999997e-05 2024-09-15T00:12:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58879 3.7 3:e169894d:::rbd_data.1139766160da.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 52.448362729999999, 2024-09-15T00:12:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043716294000000003, 2024-09-15T00:12:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58879 2024-09-15T00:12:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291502+0000", 2024-09-15T00:12:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-15T00:12:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291664+0000", 2024-09-15T00:12:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016216599999999999 2024-09-15T00:12:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291674+0000", 2024-09-15T00:12:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0276e-05 2024-09-15T00:12:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291680+0000", 2024-09-15T00:12:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2580000000000003e-06 2024-09-15T00:12:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291732+0000", 2024-09-15T00:12:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2114999999999998e-05 2024-09-15T00:12:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291866+0000", 2024-09-15T00:12:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013416900000000001 2024-09-15T00:12:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.296426+0000", 2024-09-15T00:12:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045595779999999999 2024-09-15T00:12:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335075+0000", 2024-09-15T00:12:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038649347000000001 2024-09-15T00:12:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335097+0000", 2024-09-15T00:12:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2565000000000002e-05 2024-09-15T00:12:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335161+0000", 2024-09-15T00:12:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3928000000000002e-05 2024-09-15T00:12:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335217+0000", 2024-09-15T00:12:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6042000000000003e-05 2024-09-15T00:12:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58880 3.8 3:13d91cfd:::rbd_data.1139766160da.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 52.444970623000003, 2024-09-15T00:12:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056331463999999998, 2024-09-15T00:12:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58880 2024-09-15T00:12:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294896+0000", 2024-09-15T00:12:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.356e-06 2024-09-15T00:12:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295288+0000", 2024-09-15T00:12:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039223200000000001 2024-09-15T00:12:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295304+0000", 2024-09-15T00:12:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5937e-05 2024-09-15T00:12:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295312+0000", 2024-09-15T00:12:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1489999999999994e-06 2024-09-15T00:12:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333162+0000", 2024-09-15T00:12:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037849840000000003 2024-09-15T00:12:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333269+0000", 2024-09-15T00:12:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010749800000000001 2024-09-15T00:12:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.336198+0000", 2024-09-15T00:12:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029284739999999999 2024-09-15T00:12:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351137+0000", 2024-09-15T00:12:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014939103 2024-09-15T00:12:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351150+0000", 2024-09-15T00:12:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3542e-05 2024-09-15T00:12:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351191+0000", 2024-09-15T00:12:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0791000000000003e-05 2024-09-15T00:12:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351225+0000", 2024-09-15T00:12:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3541999999999998e-05 2024-09-15T00:12:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58878 3.8 e56/23 ondisk, result = 0)", 2024-09-15T00:12:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 52.438440839999998, 2024-09-15T00:12:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049689096000000002, 2024-09-15T00:12:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301425+0000", 2024-09-15T00:12:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.782e-06 2024-09-15T00:12:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301433+0000", 2024-09-15T00:12:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689999999999994e-06 2024-09-15T00:12:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301449+0000", 2024-09-15T00:12:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5767e-05 2024-09-15T00:12:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301458+0000", 2024-09-15T00:12:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:12:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351009+0000", 2024-09-15T00:12:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049551523 2024-09-15T00:12:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351017+0000", 2024-09-15T00:12:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8359999999999992e-06 2024-09-15T00:12:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351112+0000", 2024-09-15T00:12:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5173999999999994e-05 2024-09-15T00:12:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127004 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 6881280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.771535375999999, 2024-09-15T00:12:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735720000000003, 2024-09-15T00:12:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127004 2024-09-15T00:12:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:12:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968331+0000", 2024-09-15T00:12:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.849e-06 2024-09-15T00:12:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968334+0000", 2024-09-15T00:12:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7689999999999998e-06 2024-09-15T00:12:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968338+0000", 2024-09-15T00:12:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2490000000000001e-06 2024-09-15T00:12:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008987+0000", 2024-09-15T00:12:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040649117999999998 2024-09-15T00:12:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009027+0000", 2024-09-15T00:12:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0357e-05 2024-09-15T00:12:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009064+0000", 2024-09-15T00:12:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7036000000000003e-05 2024-09-15T00:12:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127005 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7004160~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.771522824000002, 2024-09-15T00:12:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040824358999999998, 2024-09-15T00:12:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127005 2024-09-15T00:12:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-15T00:12:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968343+0000", 2024-09-15T00:12:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3430000000000001e-06 2024-09-15T00:12:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968349+0000", 2024-09-15T00:12:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3080000000000004e-06 2024-09-15T00:12:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968355+0000", 2024-09-15T00:12:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9499999999999998e-06 2024-09-15T00:12:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009091+0000", 2024-09-15T00:12:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735645000000001 2024-09-15T00:12:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009130+0000", 2024-09-15T00:12:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8951e-05 2024-09-15T00:12:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009165+0000", 2024-09-15T00:12:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5926000000000001e-05 2024-09-15T00:12:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127006 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7127040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.771505766000001, 2024-09-15T00:12:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040912783000000001, 2024-09-15T00:12:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127006 2024-09-15T00:12:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968359+0000", 2024-09-15T00:12:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0419999999999999e-06 2024-09-15T00:12:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968363+0000", 2024-09-15T00:12:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5999999999999998e-06 2024-09-15T00:12:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968366+0000", 2024-09-15T00:12:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5179999999999999e-06 2024-09-15T00:12:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009196+0000", 2024-09-15T00:12:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040830313 2024-09-15T00:12:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009234+0000", 2024-09-15T00:12:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8562000000000001e-05 2024-09-15T00:12:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009271+0000", 2024-09-15T00:12:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6538000000000002e-05 2024-09-15T00:12:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127007 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7249920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:12:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.771495566999999, 2024-09-15T00:12:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041019218000000003, 2024-09-15T00:12:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127007 2024-09-15T00:12:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:12:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:12:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968369+0000", 2024-09-15T00:12:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-15T00:12:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968370+0000", 2024-09-15T00:12:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.012e-06 2024-09-15T00:12:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968374+0000", 2024-09-15T00:12:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3839999999999999e-06 2024-09-15T00:12:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968376+0000", 2024-09-15T00:12:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5339999999999998e-06 2024-09-15T00:12:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009292+0000", 2024-09-15T00:12:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040915092 2024-09-15T00:12:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009333+0000", 2024-09-15T00:12:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1105e-05 2024-09-15T00:12:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009388+0000", 2024-09-15T00:12:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4868999999999999e-05 2024-09-15T00:12:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127008 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7364608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:12:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.771484445999999, 2024-09-15T00:12:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041115290999999998, 2024-09-15T00:12:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127008 2024-09-15T00:12:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:12:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:12:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968380+0000", 2024-09-15T00:12:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3999999999999998e-07 2024-09-15T00:12:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968381+0000", 2024-09-15T00:12:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8700000000000004e-07 2024-09-15T00:12:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968385+0000", 2024-09-15T00:12:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9260000000000002e-06 2024-09-15T00:12:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968389+0000", 2024-09-15T00:12:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.685e-06 2024-09-15T00:12:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009419+0000", 2024-09-15T00:12:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041029852999999998 2024-09-15T00:12:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009458+0000", 2024-09-15T00:12:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8538000000000003e-05 2024-09-15T00:12:45.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009495+0000", 2024-09-15T00:12:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7061999999999999e-05 2024-09-15T00:12:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127009 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7487488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.796 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.771450329, 2024-09-15T00:12:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041233063, 2024-09-15T00:12:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127009 2024-09-15T00:12:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7199999999999999e-07 2024-09-15T00:12:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968417+0000", 2024-09-15T00:12:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3810000000000002e-06 2024-09-15T00:12:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968423+0000", 2024-09-15T00:12:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8820000000000003e-06 2024-09-15T00:12:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968433+0000", 2024-09-15T00:12:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0005e-05 2024-09-15T00:12:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009527+0000", 2024-09-15T00:12:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041093750999999998 2024-09-15T00:12:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009601+0000", 2024-09-15T00:12:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3979000000000004e-05 2024-09-15T00:12:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009647+0000", 2024-09-15T00:12:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5493e-05 2024-09-15T00:12:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.800 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127010 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7610368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.771421324999999, 2024-09-15T00:12:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041325924999999999, 2024-09-15T00:12:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127010 2024-09-15T00:12:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:12:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968445+0000", 2024-09-15T00:12:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:12:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968452+0000", 2024-09-15T00:12:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-15T00:12:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968456+0000", 2024-09-15T00:12:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5210000000000004e-06 2024-09-15T00:12:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009664+0000", 2024-09-15T00:12:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041207404000000003 2024-09-15T00:12:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009723+0000", 2024-09-15T00:12:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9036000000000002e-05 2024-09-15T00:12:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009768+0000", 2024-09-15T00:12:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5578000000000003e-05 2024-09-15T00:12:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127011 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7733248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.77140309, 2024-09-15T00:12:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041482441000000002, 2024-09-15T00:12:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.806 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127011 2024-09-15T00:12:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-15T00:12:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968466+0000", 2024-09-15T00:12:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:12:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968470+0000", 2024-09-15T00:12:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6469999999999997e-06 2024-09-15T00:12:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968473+0000", 2024-09-15T00:12:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.977e-06 2024-09-15T00:12:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009800+0000", 2024-09-15T00:12:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041326918999999997 2024-09-15T00:12:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009869+0000", 2024-09-15T00:12:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8935000000000003e-05 2024-09-15T00:12:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009943+0000", 2024-09-15T00:12:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4188000000000002e-05 2024-09-15T00:12:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.811 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127012 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7856128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:12:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.771387588, 2024-09-15T00:12:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041640756000000001, 2024-09-15T00:12:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.812 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127012 2024-09-15T00:12:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:12:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:12:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968477+0000", 2024-09-15T00:12:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968482+0000", 2024-09-15T00:12:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6180000000000003e-06 2024-09-15T00:12:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968486+0000", 2024-09-15T00:12:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.501e-06 2024-09-15T00:12:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968489+0000", 2024-09-15T00:12:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1530000000000001e-06 2024-09-15T00:12:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009983+0000", 2024-09-15T00:12:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041493980999999999 2024-09-15T00:12:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010051+0000", 2024-09-15T00:12:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8039000000000004e-05 2024-09-15T00:12:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010117+0000", 2024-09-15T00:12:45.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6254000000000006e-05 2024-09-15T00:12:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127013 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7979008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:12:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.771371768000002, 2024-09-15T00:12:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041763863999999998, 2024-09-15T00:12:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.817 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127013 2024-09-15T00:12:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:12:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:12:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969767+0000", 2024-09-15T00:12:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001275006 2024-09-15T00:12:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969772+0000", 2024-09-15T00:12:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4769999999999997e-06 2024-09-15T00:12:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969781+0000", 2024-09-15T00:12:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1959999999999997e-06 2024-09-15T00:12:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969788+0000", 2024-09-15T00:12:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6359999999999994e-06 2024-09-15T00:12:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010162+0000", 2024-09-15T00:12:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040373291999999998 2024-09-15T00:12:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010212+0000", 2024-09-15T00:12:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0543e-05 2024-09-15T00:12:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010256+0000", 2024-09-15T00:12:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3714000000000002e-05 2024-09-15T00:12:45.821 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127015 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8224768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:12:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.770032831999998, 2024-09-15T00:12:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040710186000000002, 2024-09-15T00:12:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127015 2024-09-15T00:12:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:12:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:12:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969832+0000", 2024-09-15T00:12:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:12:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969834+0000", 2024-09-15T00:12:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2919999999999998e-06 2024-09-15T00:12:45.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969843+0000", 2024-09-15T00:12:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0480000000000004e-06 2024-09-15T00:12:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969848+0000", 2024-09-15T00:12:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-15T00:12:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010457+0000", 2024-09-15T00:12:45.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040609054999999998 2024-09-15T00:12:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010501+0000", 2024-09-15T00:12:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3396000000000003e-05 2024-09-15T00:12:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010541+0000", 2024-09-15T00:12:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0599000000000002e-05 2024-09-15T00:12:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.826 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127016 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.769982206000002, 2024-09-15T00:12:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040766216000000001, 2024-09-15T00:12:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127016 2024-09-15T00:12:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:12:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969886+0000", 2024-09-15T00:12:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8449999999999996e-06 2024-09-15T00:12:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969897+0000", 2024-09-15T00:12:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-15T00:12:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969904+0000", 2024-09-15T00:12:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1169999999999998e-06 2024-09-15T00:12:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010561+0000", 2024-09-15T00:12:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040657470000000001 2024-09-15T00:12:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010607+0000", 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6108e-05 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010648+0000", 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0479000000000003e-05 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127694 3.9 3:9390b205:::rbd_data.11396481e1ec.00000000000000df:head [set-alloc-hint object_size 1048576 write_size 1048576,write 819200~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.515263089000001, 2024-09-15T00:12:45.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90376419399999997, 2024-09-15T00:12:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127694 2024-09-15T00:12:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:12:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:12:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224602+0000", 2024-09-15T00:12:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5899999999999995e-07 2024-09-15T00:12:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224610+0000", 2024-09-15T00:12:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9130000000000003e-06 2024-09-15T00:12:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224620+0000", 2024-09-15T00:12:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0426e-05 2024-09-15T00:12:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224627+0000", 2024-09-15T00:12:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4390000000000003e-06 2024-09-15T00:12:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224675+0000", 2024-09-15T00:12:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7636000000000003e-05 2024-09-15T00:12:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:12:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224681+0000", 2024-09-15T00:12:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1460000000000001e-06 2024-09-15T00:12:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126388+0000", 2024-09-15T00:12:45.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90170724999999996 2024-09-15T00:12:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126526+0000", 2024-09-15T00:12:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013730599999999999 2024-09-15T00:12:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126684+0000", 2024-09-15T00:12:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000157935 2024-09-15T00:12:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:45.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.127932+0000", 2024-09-15T00:12:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012479240000000001 2024-09-15T00:12:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128269+0000", 2024-09-15T00:12:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033761099999999998 2024-09-15T00:12:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128324+0000", 2024-09-15T00:12:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5244999999999997e-05 2024-09-15T00:12:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128365+0000", 2024-09-15T00:12:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0503999999999997e-05 2024-09-15T00:12:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:134707 3.15 3:af1fbdb5:::rbd_data.113933a4f79a.000000000000001c:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T00:12:45.838 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:12:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.047295975000001, 2024-09-15T00:12:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057328626000000001, 2024-09-15T00:12:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 134707 2024-09-15T00:12:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:12:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:12:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692569+0000", 2024-09-15T00:12:45.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.339e-06 2024-09-15T00:12:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692574+0000", 2024-09-15T00:12:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-15T00:12:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692584+0000", 2024-09-15T00:12:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0363000000000001e-05 2024-09-15T00:12:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692590+0000", 2024-09-15T00:12:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.818e-06 2024-09-15T00:12:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692633+0000", 2024-09-15T00:12:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.265e-05 2024-09-15T00:12:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692718+0000", 2024-09-15T00:12:45.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5279999999999997e-05 2024-09-15T00:12:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.695940+0000", 2024-09-15T00:12:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003222052 2024-09-15T00:12:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749714+0000", 2024-09-15T00:12:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.053774126999999998 2024-09-15T00:12:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749788+0000", 2024-09-15T00:12:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3693000000000001e-05 2024-09-15T00:12:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749847+0000", 2024-09-15T00:12:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175999999999997e-05 2024-09-15T00:12:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749897+0000", 2024-09-15T00:12:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9311999999999998e-05 2024-09-15T00:12:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:134707 3.15 e86/55 ondisk, result = 0)", 2024-09-15T00:12:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:12:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.038734626, 2024-09-15T00:12:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048596066, 2024-09-15T00:12:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:45.845 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:12:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:12:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701131+0000", 2024-09-15T00:12:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3549999999999999e-06 2024-09-15T00:12:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701139+0000", 2024-09-15T00:12:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1650000000000006e-06 2024-09-15T00:12:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701156+0000", 2024-09-15T00:12:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6730000000000001e-05 2024-09-15T00:12:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701167+0000", 2024-09-15T00:12:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1504e-05 2024-09-15T00:12:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749629+0000", 2024-09-15T00:12:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048461952000000003 2024-09-15T00:12:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749640+0000", 2024-09-15T00:12:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0874e-05 2024-09-15T00:12:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749725+0000", 2024-09-15T00:12:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5486000000000001e-05 2024-09-15T00:12:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:45.849 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:45.849 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:45.851 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:45.851 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:45.849+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:45.951 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:45.949+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:45.968 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:46.053 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:46.051+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:46.080 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:12:46.106 INFO:teuthology.orchestra.run.smithi171.stdout:leaving image image_client.0-clone42 intact 2024-09-15T00:12:46.118 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:46.118 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:46.118 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:46.118 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:46.118 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:46.127 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:46.171 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:46.170+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:46.208 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:46.208 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:46.227 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:46.227 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:46.227 INFO:teuthology.orchestra.run.smithi171.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T00:12:46.235 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:46.235 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:46.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:46.235 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:46.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.235 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:46.235 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:46.235 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 80.660290005999997, 2024-09-15T00:12:46.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:12:46.235 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.235 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.235 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:46.236 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.236 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.236 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.236 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:12:46.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.236 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:46.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:46.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:12:46.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:12:46.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:12:46.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:12:46.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.238 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:12:46.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:12:46.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.238 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:12:46.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:12:46.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:12:46.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:12:46.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:12:46.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:12:46.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:46.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:12:46.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:12:46.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:46.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:12:46.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:12:46.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:46.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:12:46.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:12:46.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:46.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:12:46.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:12:46.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.241 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.241 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:12:46.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:12:46.241 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.241 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.241 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.241 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:46.242 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:46.242 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 80.660255313999997, 2024-09-15T00:12:46.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:12:46.242 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.242 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.242 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:46.242 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.242 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.242 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.242 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:12:46.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.243 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:46.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:46.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:46.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:12:46.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:12:46.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:12:46.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:12:46.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:12:46.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:12:46.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:12:46.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:12:46.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:12:46.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:12:46.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:12:46.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:46.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:12:46.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:12:46.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:46.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:12:46.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:12:46.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:46.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:12:46.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:12:46.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:46.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:12:46.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:12:46.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:12:46.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:12:46.247 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.248 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.248 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.248 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.248 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:46.248 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.619581625999999, 2024-09-15T00:12:46.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:12:46.248 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.248 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.248 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.248 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.248 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.248 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.249 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:12:46.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.249 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:46.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:46.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:46.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:12:46.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:12:46.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:12:46.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:12:46.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:12:46.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:12:46.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:12:46.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:12:46.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:12:46.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:12:46.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:12:46.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:12:46.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:12:46.252 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.252 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.252 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.252 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.252 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:46.253 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.619565094999999, 2024-09-15T00:12:46.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:12:46.253 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.253 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.253 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.253 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.256 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.256 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.256 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:12:46.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.257 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:46.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:46.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:12:46.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:12:46.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:12:46.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:12:46.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:12:46.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:12:46.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:12:46.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:12:46.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:12:46.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:12:46.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:12:46.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:12:46.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:12:46.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:12:46.260 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.260 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.260 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.260 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.260 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:46.260 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.619548315999999, 2024-09-15T00:12:46.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:12:46.261 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.261 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.261 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.261 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.261 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.261 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.261 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:12:46.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.261 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:46.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:46.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:46.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:12:46.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:12:46.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:12:46.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:12:46.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:12:46.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:12:46.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:12:46.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:12:46.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:12:46.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:12:46.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:12:46.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:12:46.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:12:46.265 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.265 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.265 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.265 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.265 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:46.265 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.619530586000003, 2024-09-15T00:12:46.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:12:46.266 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.266 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.266 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.266 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.266 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.266 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.266 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:12:46.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.266 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:46.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:46.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:46.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:12:46.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:12:46.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:12:46.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:12:46.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:12:46.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:12:46.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:12:46.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:12:46.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:12:46.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:12:46.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:12:46.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:12:46.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:12:46.270 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.271 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.271 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.271 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.271 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:46.271 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.619513421999997, 2024-09-15T00:12:46.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:12:46.271 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.271 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.271 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.271 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.271 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.271 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.271 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:12:46.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.272 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.272 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.272 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:46.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.272 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.272 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:46.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.272 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:46.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:12:46.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:12:46.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:12:46.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:12:46.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:12:46.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:12:46.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:12:46.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:12:46.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:12:46.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:12:46.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:12:46.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:12:46.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:12:46.275 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.275 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.275 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.275 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.275 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:46.275 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.619491746000001, 2024-09-15T00:12:46.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:12:46.276 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.276 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.276 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.276 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.276 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.276 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.277 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:12:46.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.277 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:46.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:46.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:46.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:12:46.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:12:46.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:12:46.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:12:46.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:12:46.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:12:46.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:12:46.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:12:46.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:12:46.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:12:46.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:12:46.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:12:46.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:12:46.280 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.280 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.280 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.281 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.281 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:46.281 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.619445622000001, 2024-09-15T00:12:46.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:12:46.281 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.281 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.281 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.281 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.281 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.281 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.281 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:12:46.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.282 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:46.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:46.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:46.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:12:46.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:12:46.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:12:46.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:12:46.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:12:46.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:12:46.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:12:46.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:12:46.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:12:46.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:12:46.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:12:46.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:12:46.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:12:46.285 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.285 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.285 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.285 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.285 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:46.285 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.619415865000001, 2024-09-15T00:12:46.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:12:46.286 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.286 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.286 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.286 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.286 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.286 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.286 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:12:46.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.286 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:46.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:46.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:12:46.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:12:46.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:12:46.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:12:46.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:12:46.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:12:46.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:12:46.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:12:46.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:12:46.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:12:46.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:12:46.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:12:46.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:12:46.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:12:46.289 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.289 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.290 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.290 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.290 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:46.290 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.619389546000001, 2024-09-15T00:12:46.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:12:46.290 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.290 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.290 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.290 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:12:46.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.291 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:46.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:46.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:46.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:12:46.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:12:46.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:12:46.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:12:46.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:12:46.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:12:46.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:12:46.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:12:46.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:12:46.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:12:46.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:12:46.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:12:46.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:12:46.294 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.294 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.294 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.294 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.294 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:46.294 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.619353684000004, 2024-09-15T00:12:46.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:12:46.294 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.295 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.295 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.295 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.295 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.295 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.295 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:12:46.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.295 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:46.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:46.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:46.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:12:46.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:12:46.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:12:46.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:12:46.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:12:46.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:12:46.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:12:46.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:12:46.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:12:46.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:12:46.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:12:46.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:12:46.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:12:46.298 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.298 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.298 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.299 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.299 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:46.299 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.619331965999997, 2024-09-15T00:12:46.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:12:46.299 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.299 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.299 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.299 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.299 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.299 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.299 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:12:46.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.299 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:46.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:46.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:12:46.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:12:46.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:12:46.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:12:46.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:12:46.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:12:46.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:12:46.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:12:46.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:12:46.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:12:46.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:12:46.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:12:46.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:12:46.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:12:46.303 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.303 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.303 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.303 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.303 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:46.303 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.619135499000002, 2024-09-15T00:12:46.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:12:46.303 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.303 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.303 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.304 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.304 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.304 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.304 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:12:46.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.304 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:46.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:46.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:46.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:12:46.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:12:46.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:12:46.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:12:46.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:12:46.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:12:46.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:12:46.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:12:46.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:12:46.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:12:46.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:12:46.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:12:46.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:12:46.307 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.307 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.307 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.307 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.308 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:46.308 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.619109659999999, 2024-09-15T00:12:46.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:12:46.308 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.308 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.308 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.308 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.308 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.308 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.308 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:12:46.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.308 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:46.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:46.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:46.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:12:46.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:12:46.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:12:46.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:12:46.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:12:46.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:12:46.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:12:46.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:12:46.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:12:46.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:12:46.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:12:46.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:12:46.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:12:46.312 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.312 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.312 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.312 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.312 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:46.313 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.619048014000001, 2024-09-15T00:12:46.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:12:46.313 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.313 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.313 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.313 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.313 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.313 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.313 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:12:46.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.313 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:46.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:46.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:12:46.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:12:46.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:12:46.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:12:46.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:12:46.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:12:46.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:12:46.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:12:46.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:12:46.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:12:46.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:12:46.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:12:46.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:12:46.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:12:46.317 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.317 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.317 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.317 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.317 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:46.317 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.619008686000001, 2024-09-15T00:12:46.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:12:46.317 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.317 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.317 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.317 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.318 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.318 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.318 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:12:46.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.318 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:46.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:46.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:12:46.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:12:46.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:12:46.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:12:46.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:12:46.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:12:46.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:12:46.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:12:46.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:12:46.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:12:46.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:12:46.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:12:46.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:12:46.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:12:46.321 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.321 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.321 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.322 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.322 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:46.322 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.617283286000003, 2024-09-15T00:12:46.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:12:46.322 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.322 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.322 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.322 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.322 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.322 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.322 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:12:46.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.322 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:46.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:46.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:12:46.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:12:46.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:12:46.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:12:46.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:12:46.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:12:46.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:12:46.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:12:46.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:12:46.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:12:46.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:12:46.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:12:46.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:12:46.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:12:46.326 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.326 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.326 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.326 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:46.326 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:46.326 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.617261124999999, 2024-09-15T00:12:46.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:12:46.326 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.326 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.326 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:46.326 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.326 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.326 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.327 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:12:46.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.327 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:46.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:46.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:46.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:12:46.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:12:46.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:12:46.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:12:46.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:12:46.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:12:46.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:12:46.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:12:46.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:12:46.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:12:46.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:12:46.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:12:46.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:12:46.330 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.330 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.330 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.330 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:145034 3.6 3:618673df:::rbd_object_map.1139646a7d4e.000000000000002b:head [writefull 0~48] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T00:12:46.330 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:12:46.330 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.2150075210000004, 2024-09-15T00:12:46.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0081079740000001, 2024-09-15T00:12:46.330 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:46.331 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:46.331 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:46.331 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:46.331 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:46.331 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:46.331 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 145034 2024-09-15T00:12:46.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.331 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:46.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:46.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:12:46.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:46.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:12:46.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:46.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:46.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017198+0000", 2024-09-15T00:12:46.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0100000000000004e-07 2024-09-15T00:12:46.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:46.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017202+0000", 2024-09-15T00:12:46.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:12:46.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:46.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017212+0000", 2024-09-15T00:12:46.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7189999999999996e-06 2024-09-15T00:12:46.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:46.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017219+0000", 2024-09-15T00:12:46.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7839999999999998e-06 2024-09-15T00:12:46.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017268+0000", 2024-09-15T00:12:46.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:12:46.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:12:46.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017277+0000", 2024-09-15T00:12:46.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1899999999999995e-06 2024-09-15T00:12:46.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:46.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019139+0000", 2024-09-15T00:12:46.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0018626129999999 2024-09-15T00:12:46.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:46.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019236+0000", 2024-09-15T00:12:46.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6513000000000002e-05 2024-09-15T00:12:46.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:46.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019318+0000", 2024-09-15T00:12:46.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1848000000000006e-05 2024-09-15T00:12:46.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:46.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.023151+0000", 2024-09-15T00:12:46.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038332169999999999 2024-09-15T00:12:46.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:46.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025217+0000", 2024-09-15T00:12:46.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020657610000000002 2024-09-15T00:12:46.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:46.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025267+0000", 2024-09-15T00:12:46.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0198000000000003e-05 2024-09-15T00:12:46.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:46.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:46.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:46.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025305+0000", 2024-09-15T00:12:46.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8065999999999998e-05 2024-09-15T00:12:46.336 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.336 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.336 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.337 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:46.337 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:46.337 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:46.338 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:46.338 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:46.339 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:46.339 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:46.339 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:46.340 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:46.336+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:46.340 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-15T00:12:46.340 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:46.341 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:46.342 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:46.342 INFO:teuthology.orchestra.run.smithi171.stdout:5139 trunc from 0xcfcddd6 to 0xd296cec 2024-09-15T00:12:46.342 INFO:teuthology.orchestra.run.smithi171.stdout:5141 read 0x3024772 thru 0x302ba73 (0x7302 bytes) 2024-09-15T00:12:46.342 INFO:teuthology.orchestra.run.smithi171.stdout:5143 read 0xb450390 thru 0xb453363 (0x2fd4 bytes) 2024-09-15T00:12:46.342 INFO:teuthology.orchestra.run.smithi171.stdout:5145 punch from 0x8cdcd1c to 0x8cdd8f0, (0xbd4 bytes) 2024-09-15T00:12:46.342 INFO:teuthology.orchestra.run.smithi171.stdout:5148 read 0x40ef409 thru 0x40fb9f2 (0xc5ea bytes) 2024-09-15T00:12:46.342 INFO:teuthology.orchestra.run.smithi171.stdout:5150 read 0x388b88a thru 0x3894865 (0x8fdc bytes) 2024-09-15T00:12:46.342 INFO:teuthology.orchestra.run.smithi171.stdout:5151 read 0x5940687 thru 0x5944a44 (0x43be bytes) 2024-09-15T00:12:46.342 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:46.342 INFO:teuthology.orchestra.run.smithi171.stdout:5155 read 0x9cc5e40 thru 0x9cd1f2d (0xc0ee bytes) 2024-09-15T00:12:46.342 INFO:teuthology.orchestra.run.smithi171.stdout:5156 write 0x17b0f4 thru 0x184691 (0x959e bytes) 2024-09-15T00:12:46.343 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:46.343 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:46.343 INFO:teuthology.orchestra.run.smithi171.stdout:5158 writesame 0x4b43389 thru 0x4b44a8f data_size 0xf(0x1707 bytes) 2024-09-15T00:12:46.344 INFO:teuthology.orchestra.run.smithi171.stdout:5159 write 0x623e009 thru 0x624db57 (0xfb4f bytes) 2024-09-15T00:12:46.344 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:46.344 INFO:teuthology.orchestra.run.smithi171.stdout:5161 read 0x785e457 thru 0x7864781 (0x632b bytes) 2024-09-15T00:12:46.344 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:46.344 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:46.345 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:12:46.345 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:46.345 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:12:46.345 INFO:teuthology.orchestra.run.smithi171.stdout:5165 trunc from 0xd296cec to 0x210a008 2024-09-15T00:12:46.360 INFO:teuthology.orchestra.run.smithi171.stdout:5166 writesame 0x6ed778a thru 0x6ee5d1f data_size 0x1(0xe596 bytes) 2024-09-15T00:12:46.372 INFO:teuthology.orchestra.run.smithi171.stdout:5167 writesame 0xc44f64d thru 0xc4579a9 data_size 0x1(0x835d bytes) 2024-09-15T00:12:46.372 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:46.372 INFO:teuthology.orchestra.run.smithi171.stdout:5169 write 0x872216a thru 0x872ed80 (0xcc17 bytes) 2024-09-15T00:12:46.377 INFO:teuthology.orchestra.run.smithi171.stdout:5170 punch from 0x952d568 to 0x952f299, (0x1d31 bytes) 2024-09-15T00:12:46.377 INFO:teuthology.orchestra.run.smithi171.stdout:5171 read 0x3048312 thru 0x304b524 (0x3213 bytes) 2024-09-15T00:12:46.377 INFO:teuthology.orchestra.run.smithi171.stdout:5173 read 0x3962e90 thru 0x3970bc8 (0xdd39 bytes) 2024-09-15T00:12:46.384 INFO:teuthology.orchestra.run.smithi171.stdout:5174 writesame 0xcf15b0b thru 0xcf206b1 data_size 0x1(0xaba7 bytes) 2024-09-15T00:12:46.384 INFO:teuthology.orchestra.run.smithi171.stdout:5176 write 0x8c93b3 thru 0x8d54fa (0xc148 bytes) 2024-09-15T00:12:46.390 INFO:teuthology.orchestra.run.smithi171.stdout:5177 trunc from 0xcf206b2 to 0xe238f06 2024-09-15T00:12:46.396 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:46.396 INFO:teuthology.orchestra.run.smithi171.stdout:5181 write 0xb2667f9 thru 0xb272b73 (0xc37b bytes) 2024-09-15T00:12:46.401 INFO:teuthology.orchestra.run.smithi171.stdout:5184 writesame 0xac91963 thru 0xac931d4 data_size 0x15(0x1872 bytes) 2024-09-15T00:12:46.413 INFO:teuthology.orchestra.run.smithi171.stdout:5185 trunc from 0xe238f06 to 0x736c90b 2024-09-15T00:12:46.430 INFO:teuthology.orchestra.run.smithi171.stdout:5186 trunc from 0x736c90b to 0xacbb9e8 2024-09-15T00:12:46.435 INFO:teuthology.orchestra.run.smithi171.stdout:5188 read 0x7020c91 thru 0x7023a25 (0x2d95 bytes) 2024-09-15T00:12:46.435 INFO:teuthology.orchestra.run.smithi171.stdout:5189 trunc from 0xacbb9e8 to 0xa725fe4 2024-09-15T00:12:46.439 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:46.437+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:46.440 INFO:teuthology.orchestra.run.smithi171.stdout:5192 write 0x87c25e7 thru 0x87d1e72 (0xf88c bytes) 2024-09-15T00:12:46.454 INFO:teuthology.orchestra.run.smithi171.stdout:5193 write 0xc8255b2 thru 0xc82e861 (0x92b0 bytes) 2024-09-15T00:12:46.457 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:46.457 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:46.458 INFO:teuthology.orchestra.run.smithi171.stdout:5198 writesame 0xe214e4 thru 0xe2c5fe data_size 0x33(0xb11b bytes) 2024-09-15T00:12:46.464 INFO:teuthology.orchestra.run.smithi171.stdout:5199 read 0x248f227 thru 0x2499723 (0xa4fd bytes) 2024-09-15T00:12:46.467 INFO:teuthology.orchestra.run.smithi171.stdout:5200 trunc from 0xc82e862 to 0x7637264 2024-09-15T00:12:46.487 INFO:teuthology.orchestra.run.smithi171.stdout:5201 write 0xac224cd thru 0xac2d107 (0xac3b bytes) 2024-09-15T00:12:46.488 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:46.488 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:46.491 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:46.491 INFO:teuthology.orchestra.run.smithi171.stdout:5203 read 0x790027e thru 0x790bee0 (0xbc63 bytes) 2024-09-15T00:12:46.491 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:46.491 INFO:teuthology.orchestra.run.smithi171.stdout:5205 writesame 0x6b5bcc1 thru 0x6b5cde5 data_size 0x21(0x1125 bytes) 2024-09-15T00:12:46.498 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:46.503 INFO:teuthology.orchestra.run.smithi171.stdout:5206 trunc from 0xac2d108 to 0x3f68a0d 2024-09-15T00:12:46.515 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:46.515 INFO:teuthology.orchestra.run.smithi171.stdout:5208 read 0x2831661 thru 0x2831a5b (0x3fb bytes) 2024-09-15T00:12:46.515 INFO:teuthology.orchestra.run.smithi171.stdout:5209 punch from 0x15cc3da to 0x15d770b, (0xb331 bytes) 2024-09-15T00:12:46.528 INFO:teuthology.orchestra.run.smithi171.stdout:5211 write 0xa6c681a thru 0xa6c8a94 (0x227b bytes) 2024-09-15T00:12:46.533 INFO:teuthology.orchestra.run.smithi171.stdout:5212 write 0x722640e thru 0x7235f36 (0xfb29 bytes) 2024-09-15T00:12:46.539 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:46.538+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:46.548 INFO:teuthology.orchestra.run.smithi171.stdout:5213 writesame 0xd677d1b thru 0xd67810b data_size 0x1(0x3f1 bytes) 2024-09-15T00:12:46.548 INFO:teuthology.orchestra.run.smithi171.stdout:5214 read 0xb0098bb thru 0xb017113 (0xd859 bytes) 2024-09-15T00:12:46.548 INFO:teuthology.orchestra.run.smithi171.stdout:5217 read 0x7d2476b thru 0x7d26c3d (0x24d3 bytes) 2024-09-15T00:12:46.548 INFO:teuthology.orchestra.run.smithi171.stdout:5219 read 0x3c08b21 thru 0x3c0b17c (0x265c bytes) 2024-09-15T00:12:46.548 INFO:teuthology.orchestra.run.smithi171.stdout:5221 clone 44 order 22 su 16384 sc 14 2024-09-15T00:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:46.610 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:46.640 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:46.639+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:46.713 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:46.713 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:46.721 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:46.742 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:46.740+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:46.824 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:46.824 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:46.833 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:46.877 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:46.875+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:46.934 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:46.935 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:46.943 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:46.943 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:46.941+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:47.043 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:47.042+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:47.060 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:12:47.093 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:47.093 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:47.102 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:47.123 INFO:teuthology.orchestra.run.smithi171.stdout:flattening image image_client.0-clone43 2024-09-15T00:12:47.144 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:47.143+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:47.217 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:47.217 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:47.226 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:47.245 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:47.244+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:47.326 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-15T00:12:47.332 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:47.332 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:47.341 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:47.384 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:47.383+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:47.444 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:47.444 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:47.447 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:47.445+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:47.453 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:47.548 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:47.546+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:47.556 INFO:teuthology.orchestra.run.smithi171.stdout:5222 writesame 0xa27630d thru 0xa28105b data_size 0x17(0xad4f bytes) 2024-09-15T00:12:47.563 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:47.563 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:47.569 INFO:teuthology.orchestra.run.smithi171.stdout:5224 read 0x51bd208 thru 0x51cbd57 (0xeb50 bytes) 2024-09-15T00:12:47.569 INFO:teuthology.orchestra.run.smithi171.stdout:5225 read 0x44dc80 thru 0x44e0c8 (0x449 bytes) 2024-09-15T00:12:47.569 INFO:teuthology.orchestra.run.smithi171.stdout:5226 writesame 0xc37afb5 thru 0xc3853aa data_size 0x1f(0xa3f6 bytes) 2024-09-15T00:12:47.575 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:47.577 INFO:teuthology.orchestra.run.smithi171.stdout:5227 read 0x5bbe42f thru 0x5bcb756 (0xd328 bytes) 2024-09-15T00:12:47.578 INFO:teuthology.orchestra.run.smithi171.stdout:5228 write 0x5d1a080 thru 0x5d29a9b (0xfa1c bytes) 2024-09-15T00:12:47.586 INFO:teuthology.orchestra.run.smithi171.stdout:5229 read 0xaf2efc1 thru 0xaf3b5df (0xc61f bytes) 2024-09-15T00:12:47.587 INFO:teuthology.orchestra.run.smithi171.stdout:5230 write 0x7e9f2b thru 0x7f5963 (0xba39 bytes) 2024-09-15T00:12:47.598 INFO:teuthology.orchestra.run.smithi171.stdout:5231 read 0x3565f24 thru 0x3575a06 (0xfae3 bytes) 2024-09-15T00:12:47.599 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:47.599 INFO:teuthology.orchestra.run.smithi171.stdout:5235 punch from 0x78aba13 to 0x78ada2d, (0x201a bytes) 2024-09-15T00:12:47.600 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:47.600 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:47.600 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:47.600 INFO:teuthology.orchestra.run.smithi171.stdout:5240 read 0x3a06c78 thru 0x3a09d43 (0x30cc bytes) 2024-09-15T00:12:47.600 INFO:teuthology.orchestra.run.smithi171.stdout:5241 punch from 0xf7ad9f to 0xf81468, (0x66c9 bytes) 2024-09-15T00:12:47.600 INFO:teuthology.orchestra.run.smithi171.stdout:5242 write 0x364f68c thru 0x364f7c9 (0x13e bytes) 2024-09-15T00:12:47.601 INFO:teuthology.orchestra.run.smithi171.stdout:5243 read 0x5c05b06 thru 0x5c10032 (0xa52d bytes) 2024-09-15T00:12:47.607 INFO:teuthology.orchestra.run.smithi171.stdout:5245 writesame 0xd9882d6 thru 0xd98f905 data_size 0x3e(0x7630 bytes) 2024-09-15T00:12:47.612 INFO:teuthology.orchestra.run.smithi171.stdout:5246 punch from 0xa1358b9 to 0xa136673, (0xdba bytes) 2024-09-15T00:12:47.612 INFO:teuthology.orchestra.run.smithi171.stdout:5248 punch from 0x87c9899 to 0x87d0de9, (0x7550 bytes) 2024-09-15T00:12:47.612 INFO:teuthology.orchestra.run.smithi171.stdout:5249 write 0xb714bbc thru 0xb71a82f (0x5c74 bytes) 2024-09-15T00:12:47.617 INFO:teuthology.orchestra.run.smithi171.stdout:5250 write 0xd072acc thru 0xd080059 (0xd58e bytes) 2024-09-15T00:12:47.636 INFO:teuthology.orchestra.run.smithi171.stdout:5251 trunc from 0xd98f906 to 0x2c77d83 2024-09-15T00:12:47.648 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:47.647+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:47.675 INFO:teuthology.orchestra.run.smithi171.stdout:5252 read 0x2409ba9 thru 0x24135f7 (0x9a4f bytes) 2024-09-15T00:12:47.676 INFO:teuthology.orchestra.run.smithi171.stdout:5253 trunc from 0x2c77d83 to 0x363e381 2024-09-15T00:12:47.695 INFO:teuthology.orchestra.run.smithi171.stdout:5255 write 0x9bbdfab thru 0x9bbfe52 (0x1ea8 bytes) 2024-09-15T00:12:47.699 INFO:teuthology.orchestra.run.smithi171.stdout:5256 write 0x22c9af7 thru 0x22d3402 (0x990c bytes) 2024-09-15T00:12:47.703 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:47.703 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:47.707 INFO:teuthology.orchestra.run.smithi171.stdout:5257 trunc from 0x9bbfe53 to 0x4b17a24 2024-09-15T00:12:47.716 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:47.717 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:47.723 INFO:teuthology.orchestra.run.smithi171.stdout:5259 write 0x5ac7e34 thru 0x5ac843c (0x609 bytes) 2024-09-15T00:12:47.727 INFO:teuthology.orchestra.run.smithi171.stdout:5261 write 0xaaf709 thru 0xabebaa (0xf4a2 bytes) 2024-09-15T00:12:47.733 INFO:teuthology.orchestra.run.smithi171.stdout:5262 trunc from 0x5ac843d to 0x8cbe615 2024-09-15T00:12:47.739 INFO:teuthology.orchestra.run.smithi171.stdout:5263 read 0x82b3cbb thru 0x82bb773 (0x7ab9 bytes) 2024-09-15T00:12:47.739 INFO:teuthology.orchestra.run.smithi171.stdout:5265 punch from 0x22481af to 0x224ebaf, (0x6a00 bytes) 2024-09-15T00:12:47.749 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:47.747+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:47.750 INFO:teuthology.orchestra.run.smithi171.stdout:5266 write 0xe1e9731 thru 0xe1eac4b (0x151b bytes) 2024-09-15T00:12:47.753 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:47.753 INFO:teuthology.orchestra.run.smithi171.stdout:5268 write 0x69fce15 thru 0x6a0b73c (0xe928 bytes) 2024-09-15T00:12:47.758 INFO:teuthology.orchestra.run.smithi171.stdout:5269 write 0xd73e75a thru 0xd741af3 (0x339a bytes) 2024-09-15T00:12:47.766 INFO:teuthology.orchestra.run.smithi171.stdout:5270 read 0x914a1eb thru 0x914fe2b (0x5c41 bytes) 2024-09-15T00:12:47.767 INFO:teuthology.orchestra.run.smithi171.stdout:5271 read 0x6f87b06 thru 0x6f8be6b (0x4366 bytes) 2024-09-15T00:12:47.767 INFO:teuthology.orchestra.run.smithi171.stdout:5272 write 0x522d983 thru 0x522ebbe (0x123c bytes) 2024-09-15T00:12:47.768 INFO:teuthology.orchestra.run.smithi171.stdout:5273 read 0x71ad487 thru 0x71ba207 (0xcd81 bytes) 2024-09-15T00:12:47.768 INFO:teuthology.orchestra.run.smithi171.stdout:5274 punch from 0x57c9054 to 0x57d8048, (0xeff4 bytes) 2024-09-15T00:12:47.782 INFO:teuthology.orchestra.run.smithi171.stdout:5275 trunc from 0xe1eac4c to 0x25c1843 2024-09-15T00:12:47.798 INFO:teuthology.orchestra.run.smithi171.stdout:5276 writesame 0xe92bfb thru 0xe96473 data_size 0x3(0x3879 bytes) 2024-09-15T00:12:47.822 INFO:teuthology.orchestra.run.smithi171.stdout:5277 write 0xd1b4479 thru 0xd1c1ee1 (0xda69 bytes) 2024-09-15T00:12:47.833 INFO:teuthology.orchestra.run.smithi171.stdout:5278 read 0x412ad36 thru 0x4137eec (0xd1b7 bytes) 2024-09-15T00:12:47.833 INFO:teuthology.orchestra.run.smithi171.stdout:5279 read 0x52d079d thru 0x52d4e34 (0x4698 bytes) 2024-09-15T00:12:47.833 INFO:teuthology.orchestra.run.smithi171.stdout:5280 read 0x15d3d thru 0x21cd1 (0xbf95 bytes) 2024-09-15T00:12:47.833 INFO:teuthology.orchestra.run.smithi171.stdout:5281 write 0x47852c9 thru 0x478c4d0 (0x7208 bytes) 2024-09-15T00:12:47.840 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:47.840 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:47.842 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:47.842 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:47.847 INFO:teuthology.orchestra.run.smithi171.stdout:5285 write 0xd37245d thru 0xd37472f (0x22d3 bytes) 2024-09-15T00:12:47.850 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:47.848+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:47.852 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:47.852 INFO:teuthology.orchestra.run.smithi171.stdout:5286 read 0x5f3fd7f thru 0x5f44ab9 (0x4d3b bytes) 2024-09-15T00:12:47.852 INFO:teuthology.orchestra.run.smithi171.stdout:5287 read 0x3860017 thru 0x386732a (0x7314 bytes) 2024-09-15T00:12:47.852 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:47.852 INFO:teuthology.orchestra.run.smithi171.stdout:5291 read 0x6719080 thru 0x671a1dc (0x115d bytes) 2024-09-15T00:12:47.853 INFO:teuthology.orchestra.run.smithi171.stdout:5293 clone 45 order 19 su 16384 sc 12 2024-09-15T00:12:47.951 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:47.950+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:47.994 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:47.995 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:48.003 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:48.040 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 0] out_osds: [2, 7, 1] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-09-15T00:12:48.040 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:12:48.040 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-15T00:12:48.041 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T00:12:48.041 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T00:12:48.049 INFO:tasks.ceph.osd.2:Started 2024-09-15T00:12:48.049 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:12:48.052 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:48.050+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:48.060 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is unset 2024-09-15T00:12:48.123 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:48.123 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:48.124 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone44 from 0xd374730 (overlap 0x25c1843) to 0xf1a34e 2024-09-15T00:12:48.134 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:48.152 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:48.151+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:48.165 INFO:teuthology.orchestra.run.smithi171.stderr:no valid command found; 10 closest matches: 2024-09-15T00:12:48.166 INFO:teuthology.orchestra.run.smithi171.stderr:0 2024-09-15T00:12:48.166 INFO:teuthology.orchestra.run.smithi171.stderr:1 2024-09-15T00:12:48.166 INFO:teuthology.orchestra.run.smithi171.stderr:2 2024-09-15T00:12:48.166 INFO:teuthology.orchestra.run.smithi171.stderr:abort 2024-09-15T00:12:48.166 INFO:teuthology.orchestra.run.smithi171.stderr:assert 2024-09-15T00:12:48.166 INFO:teuthology.orchestra.run.smithi171.stderr:config diff 2024-09-15T00:12:48.166 INFO:teuthology.orchestra.run.smithi171.stderr:config diff get 2024-09-15T00:12:48.166 INFO:teuthology.orchestra.run.smithi171.stderr:config get 2024-09-15T00:12:48.166 INFO:teuthology.orchestra.run.smithi171.stderr:config help [] 2024-09-15T00:12:48.166 INFO:teuthology.orchestra.run.smithi171.stderr:config set ... 2024-09-15T00:12:48.166 INFO:teuthology.orchestra.run.smithi171.stderr:admin_socket: invalid command 2024-09-15T00:12:48.168 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T00:12:48.169 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-15T00:12:48.207 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-15T00:12:48.253 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:48.254 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:48.254 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:48.252+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:48.268 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:48.346 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:12:48.354 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:48.352+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:48.420 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:48.420 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:48.430 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:48.454 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:48.453+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:48.475 INFO:teuthology.orchestra.run.smithi171.stdout:5296 trunc from 0xd374730 to 0x4d9e907 2024-09-15T00:12:48.481 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:48.481 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:48.503 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:48.504 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:48.504 INFO:teuthology.orchestra.run.smithi171.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T00:12:48.512 INFO:teuthology.orchestra.run.smithi171.stdout:5297 write 0x5a3e532 thru 0x5a49d40 (0xb80f bytes) 2024-09-15T00:12:48.518 INFO:teuthology.orchestra.run.smithi171.stdout:5299 read 0x2e12dc1 thru 0x2e18227 (0x5467 bytes) 2024-09-15T00:12:48.518 INFO:teuthology.orchestra.run.smithi171.stdout:5302 read 0x2fa9758 thru 0x2fac388 (0x2c31 bytes) 2024-09-15T00:12:48.518 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:48.518 INFO:teuthology.orchestra.run.smithi171.stdout:5304 read 0x4594256 thru 0x459e71f (0xa4ca bytes) 2024-09-15T00:12:48.519 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:48.519 INFO:teuthology.orchestra.run.smithi171.stdout:5308 read 0x554ed5f thru 0x5559b7a (0xae1c bytes) 2024-09-15T00:12:48.519 INFO:teuthology.orchestra.run.smithi171.stdout:5309 punch from 0x1558716 to 0x1566150, (0xda3a bytes) 2024-09-15T00:12:48.519 INFO:teuthology.orchestra.run.smithi171.stdout:5310 read 0x4d2af1a thru 0x4d2b9ba (0xaa1 bytes) 2024-09-15T00:12:48.520 INFO:teuthology.orchestra.run.smithi171.stdout:5311 write 0x58cd178 thru 0x58db7cb (0xe654 bytes) 2024-09-15T00:12:48.523 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:48.523 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:48.527 INFO:teuthology.orchestra.run.smithi171.stdout:5312 write 0x3de3d89 thru 0x3dec979 (0x8bf1 bytes) 2024-09-15T00:12:48.547 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:48.547 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:48.548 INFO:teuthology.orchestra.run.smithi171.stdout:5313 write 0xae977d5 thru 0xae9ee81 (0x76ad bytes) 2024-09-15T00:12:48.552 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:48.553 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:48.553 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:48.553 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:48.553 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:48.555 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:48.553+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:48.556 INFO:teuthology.orchestra.run.smithi171.stdout:5314 read 0x5062a42 thru 0x506b087 (0x8646 bytes) 2024-09-15T00:12:48.556 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:48.556 INFO:teuthology.orchestra.run.smithi171.stdout:5316 read 0xd0583b thru 0xd0f531 (0x9cf7 bytes) 2024-09-15T00:12:48.563 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:48.564 INFO:teuthology.orchestra.run.smithi171.stdout:5317 writesame 0xc62e5fc thru 0xc63bd3f data_size 0x17(0xd744 bytes) 2024-09-15T00:12:48.564 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:48.566 INFO:teuthology.orchestra.run.smithi171.stdout:5319 trunc from 0xc63bd40 to 0xe1bc9b7 2024-09-15T00:12:48.569 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:48.569 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:48.573 INFO:teuthology.orchestra.run.smithi171.stdout:5321 write 0x8728244 thru 0x872907b (0xe38 bytes) 2024-09-15T00:12:48.576 INFO:teuthology.orchestra.run.smithi171.stdout:5322 read 0x3d9bc84 thru 0x3d9ead5 (0x2e52 bytes) 2024-09-15T00:12:48.576 INFO:teuthology.orchestra.run.smithi171.stdout:5323 write 0x1db6bb1 thru 0x1db8e40 (0x2290 bytes) 2024-09-15T00:12:48.584 INFO:teuthology.orchestra.run.smithi171.stdout:5325 read 0xbd7a7ca thru 0xbd7e415 (0x3c4c bytes) 2024-09-15T00:12:48.584 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:48.584 INFO:teuthology.orchestra.run.smithi171.stdout:5327 read 0x6d324e8 thru 0x6d3ff33 (0xda4c bytes) 2024-09-15T00:12:48.584 INFO:teuthology.orchestra.run.smithi171.stdout:5330 write 0x37e744b thru 0x37f170f (0xa2c5 bytes) 2024-09-15T00:12:48.593 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:48.593 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:48.593 INFO:teuthology.orchestra.run.smithi171.stdout:5331 write 0x68686d9 thru 0x687234e (0x9c76 bytes) 2024-09-15T00:12:48.599 INFO:teuthology.orchestra.run.smithi171.stdout:5332 write 0x32ba9e thru 0x32be02 (0x365 bytes) 2024-09-15T00:12:48.610 INFO:teuthology.orchestra.run.smithi171.stdout:5333 read 0x6ccc10d thru 0x6ccc726 (0x61a bytes) 2024-09-15T00:12:48.610 INFO:teuthology.orchestra.run.smithi171.stdout:5334 clone 46 order 22 su 32768 sc 4 2024-09-15T00:12:48.612 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:48.612 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:48.615 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:12:48.615 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:12:48.656 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:48.655+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:48.679 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:48.680 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:48.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:48.680 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:48.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.680 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:12:48.680 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:48.680 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.434344379999999, 2024-09-15T00:12:48.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:12:48.680 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.680 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.680 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:48.680 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.681 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.681 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.681 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:12:48.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.681 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:48.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:48.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:12:48.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:12:48.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:12:48.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:12:48.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:12:48.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:12:48.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:12:48.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:12:48.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:12:48.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:12:48.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:12:48.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:12:48.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:48.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:12:48.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:12:48.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:48.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:12:48.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:12:48.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:48.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:12:48.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:12:48.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:48.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:12:48.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:12:48.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:12:48.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:12:48.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.686 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.686 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58878 3.8 3:12456482:::rbd_data.1139766160da.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:48.686 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:48.686 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.388038796000004, 2024-09-15T00:12:48.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061956199000000003, 2024-09-15T00:12:48.686 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.687 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.687 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:48.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.687 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58878 2024-09-15T00:12:48.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.687 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:48.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:48.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289151+0000", 2024-09-15T00:12:48.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:12:48.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289473+0000", 2024-09-15T00:12:48.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032183800000000002 2024-09-15T00:12:48.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289487+0000", 2024-09-15T00:12:48.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-05 2024-09-15T00:12:48.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289493+0000", 2024-09-15T00:12:48.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4489999999999996e-06 2024-09-15T00:12:48.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289542+0000", 2024-09-15T00:12:48.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8844999999999997e-05 2024-09-15T00:12:48.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289642+0000", 2024-09-15T00:12:48.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9607999999999999e-05 2024-09-15T00:12:48.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:48.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.293908+0000", 2024-09-15T00:12:48.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659350000000002 2024-09-15T00:12:48.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:48.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351022+0000", 2024-09-15T00:12:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057114535000000001 2024-09-15T00:12:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351033+0000", 2024-09-15T00:12:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0263e-05 2024-09-15T00:12:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351070+0000", 2024-09-15T00:12:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7666000000000002e-05 2024-09-15T00:12:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351106+0000", 2024-09-15T00:12:48.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999997e-05 2024-09-15T00:12:48.692 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.692 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.692 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.692 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58879 3.7 3:e169894d:::rbd_data.1139766160da.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:48.692 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:48.692 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.385687308000001, 2024-09-15T00:12:48.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043716294000000003, 2024-09-15T00:12:48.692 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.692 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.693 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:48.693 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.693 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.693 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.693 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58879 2024-09-15T00:12:48.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.693 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:48.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:48.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291502+0000", 2024-09-15T00:12:48.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-15T00:12:48.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291664+0000", 2024-09-15T00:12:48.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016216599999999999 2024-09-15T00:12:48.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291674+0000", 2024-09-15T00:12:48.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0276e-05 2024-09-15T00:12:48.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291680+0000", 2024-09-15T00:12:48.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2580000000000003e-06 2024-09-15T00:12:48.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291732+0000", 2024-09-15T00:12:48.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2114999999999998e-05 2024-09-15T00:12:48.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291866+0000", 2024-09-15T00:12:48.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013416900000000001 2024-09-15T00:12:48.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:48.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.296426+0000", 2024-09-15T00:12:48.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045595779999999999 2024-09-15T00:12:48.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:48.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335075+0000", 2024-09-15T00:12:48.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038649347000000001 2024-09-15T00:12:48.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:48.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335097+0000", 2024-09-15T00:12:48.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2565000000000002e-05 2024-09-15T00:12:48.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:48.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335161+0000", 2024-09-15T00:12:48.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3928000000000002e-05 2024-09-15T00:12:48.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335217+0000", 2024-09-15T00:12:48.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6042000000000003e-05 2024-09-15T00:12:48.698 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.698 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.698 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.699 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58880 3.8 3:13d91cfd:::rbd_data.1139766160da.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:48.699 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:48.699 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.382295200999998, 2024-09-15T00:12:48.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056331463999999998, 2024-09-15T00:12:48.699 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.699 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.699 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:48.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.700 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58880 2024-09-15T00:12:48.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.700 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:48.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:48.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294896+0000", 2024-09-15T00:12:48.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.356e-06 2024-09-15T00:12:48.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295288+0000", 2024-09-15T00:12:48.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039223200000000001 2024-09-15T00:12:48.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295304+0000", 2024-09-15T00:12:48.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5937e-05 2024-09-15T00:12:48.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295312+0000", 2024-09-15T00:12:48.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1489999999999994e-06 2024-09-15T00:12:48.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333162+0000", 2024-09-15T00:12:48.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037849840000000003 2024-09-15T00:12:48.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333269+0000", 2024-09-15T00:12:48.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010749800000000001 2024-09-15T00:12:48.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:48.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.336198+0000", 2024-09-15T00:12:48.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029284739999999999 2024-09-15T00:12:48.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:48.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351137+0000", 2024-09-15T00:12:48.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014939103 2024-09-15T00:12:48.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:48.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351150+0000", 2024-09-15T00:12:48.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3542e-05 2024-09-15T00:12:48.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:48.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351191+0000", 2024-09-15T00:12:48.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0791000000000003e-05 2024-09-15T00:12:48.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351225+0000", 2024-09-15T00:12:48.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3541999999999998e-05 2024-09-15T00:12:48.705 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.705 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.705 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.705 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58878 3.8 e56/23 ondisk, result = 0)", 2024-09-15T00:12:48.705 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:48.705 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.375765418, 2024-09-15T00:12:48.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049689096000000002, 2024-09-15T00:12:48.705 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.705 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.705 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:48.705 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:48.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:48.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301425+0000", 2024-09-15T00:12:48.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.782e-06 2024-09-15T00:12:48.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301433+0000", 2024-09-15T00:12:48.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689999999999994e-06 2024-09-15T00:12:48.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301449+0000", 2024-09-15T00:12:48.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5767e-05 2024-09-15T00:12:48.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301458+0000", 2024-09-15T00:12:48.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:12:48.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351009+0000", 2024-09-15T00:12:48.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049551523 2024-09-15T00:12:48.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351017+0000", 2024-09-15T00:12:48.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8359999999999992e-06 2024-09-15T00:12:48.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351112+0000", 2024-09-15T00:12:48.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5173999999999994e-05 2024-09-15T00:12:48.709 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.709 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.709 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.709 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127004 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 6881280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:48.710 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:48.710 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.708859954000001, 2024-09-15T00:12:48.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735720000000003, 2024-09-15T00:12:48.710 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.710 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.710 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:48.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.710 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127004 2024-09-15T00:12:48.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.710 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:48.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:48.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:48.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:12:48.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968331+0000", 2024-09-15T00:12:48.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.849e-06 2024-09-15T00:12:48.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968334+0000", 2024-09-15T00:12:48.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7689999999999998e-06 2024-09-15T00:12:48.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968338+0000", 2024-09-15T00:12:48.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2490000000000001e-06 2024-09-15T00:12:48.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008987+0000", 2024-09-15T00:12:48.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040649117999999998 2024-09-15T00:12:48.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009027+0000", 2024-09-15T00:12:48.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0357e-05 2024-09-15T00:12:48.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009064+0000", 2024-09-15T00:12:48.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7036000000000003e-05 2024-09-15T00:12:48.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.714 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.714 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127005 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7004160~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:48.714 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:48.714 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.708847402, 2024-09-15T00:12:48.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040824358999999998, 2024-09-15T00:12:48.714 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.714 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.715 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:48.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.715 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127005 2024-09-15T00:12:48.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.715 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:48.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:48.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:48.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-15T00:12:48.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968343+0000", 2024-09-15T00:12:48.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3430000000000001e-06 2024-09-15T00:12:48.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968349+0000", 2024-09-15T00:12:48.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3080000000000004e-06 2024-09-15T00:12:48.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968355+0000", 2024-09-15T00:12:48.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9499999999999998e-06 2024-09-15T00:12:48.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009091+0000", 2024-09-15T00:12:48.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735645000000001 2024-09-15T00:12:48.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009130+0000", 2024-09-15T00:12:48.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8951e-05 2024-09-15T00:12:48.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009165+0000", 2024-09-15T00:12:48.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5926000000000001e-05 2024-09-15T00:12:48.718 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.718 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.719 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.719 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127006 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7127040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:48.719 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:48.719 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.708830343999999, 2024-09-15T00:12:48.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040912783000000001, 2024-09-15T00:12:48.719 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.719 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.719 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:48.719 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.719 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.719 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.719 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127006 2024-09-15T00:12:48.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.720 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:48.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:48.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:48.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:48.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968359+0000", 2024-09-15T00:12:48.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0419999999999999e-06 2024-09-15T00:12:48.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968363+0000", 2024-09-15T00:12:48.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5999999999999998e-06 2024-09-15T00:12:48.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968366+0000", 2024-09-15T00:12:48.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5179999999999999e-06 2024-09-15T00:12:48.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009196+0000", 2024-09-15T00:12:48.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040830313 2024-09-15T00:12:48.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009234+0000", 2024-09-15T00:12:48.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8562000000000001e-05 2024-09-15T00:12:48.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009271+0000", 2024-09-15T00:12:48.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6538000000000002e-05 2024-09-15T00:12:48.726 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.726 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.726 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.726 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127007 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7249920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:48.726 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:12:48.727 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.708820145000001, 2024-09-15T00:12:48.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041019218000000003, 2024-09-15T00:12:48.727 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.727 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.727 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:48.727 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.727 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.727 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.727 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127007 2024-09-15T00:12:48.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.727 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:12:48.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:12:48.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968369+0000", 2024-09-15T00:12:48.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-15T00:12:48.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968370+0000", 2024-09-15T00:12:48.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.012e-06 2024-09-15T00:12:48.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968374+0000", 2024-09-15T00:12:48.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3839999999999999e-06 2024-09-15T00:12:48.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968376+0000", 2024-09-15T00:12:48.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5339999999999998e-06 2024-09-15T00:12:48.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009292+0000", 2024-09-15T00:12:48.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040915092 2024-09-15T00:12:48.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009333+0000", 2024-09-15T00:12:48.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1105e-05 2024-09-15T00:12:48.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009388+0000", 2024-09-15T00:12:48.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4868999999999999e-05 2024-09-15T00:12:48.731 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.731 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.731 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.731 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127008 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7364608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:48.732 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:12:48.732 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.708809024000001, 2024-09-15T00:12:48.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041115290999999998, 2024-09-15T00:12:48.732 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.732 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.732 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:48.732 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.732 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.732 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.732 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127008 2024-09-15T00:12:48.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.732 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:12:48.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:12:48.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968380+0000", 2024-09-15T00:12:48.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3999999999999998e-07 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968381+0000", 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8700000000000004e-07 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968385+0000", 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9260000000000002e-06 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968389+0000", 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.685e-06 2024-09-15T00:12:48.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009419+0000", 2024-09-15T00:12:48.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041029852999999998 2024-09-15T00:12:48.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009458+0000", 2024-09-15T00:12:48.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8538000000000003e-05 2024-09-15T00:12:48.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009495+0000", 2024-09-15T00:12:48.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7061999999999999e-05 2024-09-15T00:12:48.736 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.736 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.736 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.736 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127009 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7487488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:48.736 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:48.736 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.708774906999999, 2024-09-15T00:12:48.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041233063, 2024-09-15T00:12:48.736 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.736 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.736 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:48.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.737 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127009 2024-09-15T00:12:48.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.737 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:48.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:48.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:48.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7199999999999999e-07 2024-09-15T00:12:48.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968417+0000", 2024-09-15T00:12:48.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3810000000000002e-06 2024-09-15T00:12:48.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968423+0000", 2024-09-15T00:12:48.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8820000000000003e-06 2024-09-15T00:12:48.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968433+0000", 2024-09-15T00:12:48.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0005e-05 2024-09-15T00:12:48.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009527+0000", 2024-09-15T00:12:48.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041093750999999998 2024-09-15T00:12:48.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009601+0000", 2024-09-15T00:12:48.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3979000000000004e-05 2024-09-15T00:12:48.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009647+0000", 2024-09-15T00:12:48.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5493e-05 2024-09-15T00:12:48.740 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.740 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.740 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.740 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127010 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7610368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:48.740 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:48.741 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.708745903000001, 2024-09-15T00:12:48.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041325924999999999, 2024-09-15T00:12:48.741 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.741 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.741 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:48.741 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.741 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.741 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.741 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127010 2024-09-15T00:12:48.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.741 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:48.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:48.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:48.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:12:48.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968445+0000", 2024-09-15T00:12:48.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:12:48.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968452+0000", 2024-09-15T00:12:48.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-15T00:12:48.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968456+0000", 2024-09-15T00:12:48.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5210000000000004e-06 2024-09-15T00:12:48.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009664+0000", 2024-09-15T00:12:48.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041207404000000003 2024-09-15T00:12:48.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009723+0000", 2024-09-15T00:12:48.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9036000000000002e-05 2024-09-15T00:12:48.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009768+0000", 2024-09-15T00:12:48.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5578000000000003e-05 2024-09-15T00:12:48.744 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.745 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.745 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127011 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7733248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:48.745 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:48.745 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.708727668000002, 2024-09-15T00:12:48.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041482441000000002, 2024-09-15T00:12:48.745 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.745 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.745 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:48.745 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.745 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.745 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.746 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127011 2024-09-15T00:12:48.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.746 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:48.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:48.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:48.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-15T00:12:48.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968466+0000", 2024-09-15T00:12:48.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:12:48.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968470+0000", 2024-09-15T00:12:48.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6469999999999997e-06 2024-09-15T00:12:48.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968473+0000", 2024-09-15T00:12:48.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.977e-06 2024-09-15T00:12:48.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009800+0000", 2024-09-15T00:12:48.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041326918999999997 2024-09-15T00:12:48.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009869+0000", 2024-09-15T00:12:48.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8935000000000003e-05 2024-09-15T00:12:48.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009943+0000", 2024-09-15T00:12:48.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4188000000000002e-05 2024-09-15T00:12:48.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.749 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.749 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127012 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7856128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:48.749 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:12:48.749 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.708712166000002, 2024-09-15T00:12:48.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041640756000000001, 2024-09-15T00:12:48.749 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.750 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.750 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:48.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.750 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127012 2024-09-15T00:12:48.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.750 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:12:48.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:12:48.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968477+0000", 2024-09-15T00:12:48.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:48.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968482+0000", 2024-09-15T00:12:48.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6180000000000003e-06 2024-09-15T00:12:48.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968486+0000", 2024-09-15T00:12:48.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.501e-06 2024-09-15T00:12:48.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968489+0000", 2024-09-15T00:12:48.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1530000000000001e-06 2024-09-15T00:12:48.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009983+0000", 2024-09-15T00:12:48.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041493980999999999 2024-09-15T00:12:48.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010051+0000", 2024-09-15T00:12:48.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8039000000000004e-05 2024-09-15T00:12:48.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010117+0000", 2024-09-15T00:12:48.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6254000000000006e-05 2024-09-15T00:12:48.753 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.753 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.753 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.754 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127013 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7979008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:48.754 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:12:48.754 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.708696346, 2024-09-15T00:12:48.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041763863999999998, 2024-09-15T00:12:48.754 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.754 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.754 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:48.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.754 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127013 2024-09-15T00:12:48.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.754 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:12:48.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:12:48.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969767+0000", 2024-09-15T00:12:48.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001275006 2024-09-15T00:12:48.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969772+0000", 2024-09-15T00:12:48.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4769999999999997e-06 2024-09-15T00:12:48.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969781+0000", 2024-09-15T00:12:48.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1959999999999997e-06 2024-09-15T00:12:48.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969788+0000", 2024-09-15T00:12:48.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6359999999999994e-06 2024-09-15T00:12:48.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010162+0000", 2024-09-15T00:12:48.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040373291999999998 2024-09-15T00:12:48.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010212+0000", 2024-09-15T00:12:48.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0543e-05 2024-09-15T00:12:48.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010256+0000", 2024-09-15T00:12:48.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3714000000000002e-05 2024-09-15T00:12:48.758 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.758 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.758 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.758 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127015 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8224768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:48.758 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:12:48.758 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.70735741, 2024-09-15T00:12:48.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040710186000000002, 2024-09-15T00:12:48.758 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.759 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.759 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:48.759 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.759 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.759 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.759 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127015 2024-09-15T00:12:48.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.759 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:12:48.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:12:48.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969832+0000", 2024-09-15T00:12:48.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:12:48.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969834+0000", 2024-09-15T00:12:48.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2919999999999998e-06 2024-09-15T00:12:48.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969843+0000", 2024-09-15T00:12:48.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0480000000000004e-06 2024-09-15T00:12:48.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969848+0000", 2024-09-15T00:12:48.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-15T00:12:48.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010457+0000", 2024-09-15T00:12:48.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040609054999999998 2024-09-15T00:12:48.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010501+0000", 2024-09-15T00:12:48.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3396000000000003e-05 2024-09-15T00:12:48.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010541+0000", 2024-09-15T00:12:48.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0599000000000002e-05 2024-09-15T00:12:48.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.762 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.763 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127016 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:48.763 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:48.763 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.707306784, 2024-09-15T00:12:48.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040766216000000001, 2024-09-15T00:12:48.763 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.763 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.763 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:48.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.763 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127016 2024-09-15T00:12:48.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.763 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:48.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:48.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:48.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:12:48.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969886+0000", 2024-09-15T00:12:48.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8449999999999996e-06 2024-09-15T00:12:48.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969897+0000", 2024-09-15T00:12:48.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-15T00:12:48.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969904+0000", 2024-09-15T00:12:48.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1169999999999998e-06 2024-09-15T00:12:48.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010561+0000", 2024-09-15T00:12:48.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040657470000000001 2024-09-15T00:12:48.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010607+0000", 2024-09-15T00:12:48.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6108e-05 2024-09-15T00:12:48.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010648+0000", 2024-09-15T00:12:48.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0479000000000003e-05 2024-09-15T00:12:48.767 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.767 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.767 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.767 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127694 3.9 3:9390b205:::rbd_data.11396481e1ec.00000000000000df:head [set-alloc-hint object_size 1048576 write_size 1048576,write 819200~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T00:12:48.767 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:12:48.767 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.452587667, 2024-09-15T00:12:48.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90376419399999997, 2024-09-15T00:12:48.767 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.767 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.767 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:48.768 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.768 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.768 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.768 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127694 2024-09-15T00:12:48.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.768 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:12:48.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:12:48.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224602+0000", 2024-09-15T00:12:48.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5899999999999995e-07 2024-09-15T00:12:48.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224610+0000", 2024-09-15T00:12:48.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9130000000000003e-06 2024-09-15T00:12:48.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224620+0000", 2024-09-15T00:12:48.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0426e-05 2024-09-15T00:12:48.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224627+0000", 2024-09-15T00:12:48.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4390000000000003e-06 2024-09-15T00:12:48.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224675+0000", 2024-09-15T00:12:48.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7636000000000003e-05 2024-09-15T00:12:48.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:12:48.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224681+0000", 2024-09-15T00:12:48.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1460000000000001e-06 2024-09-15T00:12:48.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126388+0000", 2024-09-15T00:12:48.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90170724999999996 2024-09-15T00:12:48.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126526+0000", 2024-09-15T00:12:48.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013730599999999999 2024-09-15T00:12:48.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:48.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126684+0000", 2024-09-15T00:12:48.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000157935 2024-09-15T00:12:48.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:48.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.127932+0000", 2024-09-15T00:12:48.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012479240000000001 2024-09-15T00:12:48.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:48.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128269+0000", 2024-09-15T00:12:48.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033761099999999998 2024-09-15T00:12:48.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:48.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128324+0000", 2024-09-15T00:12:48.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5244999999999997e-05 2024-09-15T00:12:48.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128365+0000", 2024-09-15T00:12:48.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0503999999999997e-05 2024-09-15T00:12:48.773 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.773 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.773 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.774 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:134707 3.15 3:af1fbdb5:::rbd_data.113933a4f79a.000000000000001c:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T00:12:48.774 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:12:48.774 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.984620552999999, 2024-09-15T00:12:48.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057328626000000001, 2024-09-15T00:12:48.774 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.774 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.774 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:48.774 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:48.774 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:48.774 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:48.774 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 134707 2024-09-15T00:12:48.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.775 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:12:48.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:12:48.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692569+0000", 2024-09-15T00:12:48.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.339e-06 2024-09-15T00:12:48.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692574+0000", 2024-09-15T00:12:48.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-15T00:12:48.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692584+0000", 2024-09-15T00:12:48.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0363000000000001e-05 2024-09-15T00:12:48.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692590+0000", 2024-09-15T00:12:48.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.818e-06 2024-09-15T00:12:48.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692633+0000", 2024-09-15T00:12:48.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.265e-05 2024-09-15T00:12:48.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692718+0000", 2024-09-15T00:12:48.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5279999999999997e-05 2024-09-15T00:12:48.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:48.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.695940+0000", 2024-09-15T00:12:48.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003222052 2024-09-15T00:12:48.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:48.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749714+0000", 2024-09-15T00:12:48.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.053774126999999998 2024-09-15T00:12:48.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:48.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749788+0000", 2024-09-15T00:12:48.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3693000000000001e-05 2024-09-15T00:12:48.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:48.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749847+0000", 2024-09-15T00:12:48.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175999999999997e-05 2024-09-15T00:12:48.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749897+0000", 2024-09-15T00:12:48.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9311999999999998e-05 2024-09-15T00:12:48.779 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.780 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.780 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.780 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:134707 3.15 e86/55 ondisk, result = 0)", 2024-09-15T00:12:48.780 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:12:48.780 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.976059203999998, 2024-09-15T00:12:48.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048596066, 2024-09-15T00:12:48.780 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:48.780 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:48.780 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:48.780 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:48.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:48.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:12:48.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:48.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:12:48.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:48.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:48.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701131+0000", 2024-09-15T00:12:48.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3549999999999999e-06 2024-09-15T00:12:48.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:48.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701139+0000", 2024-09-15T00:12:48.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1650000000000006e-06 2024-09-15T00:12:48.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:48.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701156+0000", 2024-09-15T00:12:48.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6730000000000001e-05 2024-09-15T00:12:48.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:48.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701167+0000", 2024-09-15T00:12:48.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1504e-05 2024-09-15T00:12:48.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:48.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749629+0000", 2024-09-15T00:12:48.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048461952000000003 2024-09-15T00:12:48.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:48.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749640+0000", 2024-09-15T00:12:48.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0874e-05 2024-09-15T00:12:48.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:48.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:48.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:48.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749725+0000", 2024-09-15T00:12:48.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5486000000000001e-05 2024-09-15T00:12:48.784 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.784 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.784 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.784 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:48.784 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:48.784 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:48.785 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:48.786 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:48.784+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:48.886 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:48.885+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:48.891 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:48.891 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:48.891 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:48.891 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:48.900 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:48.987 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:48.986+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:49.003 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:49.003 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:49.003 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:49.003 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:49.003 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:49.012 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:49.088 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:49.086+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:49.123 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:49.123 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:49.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:49.123 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.548749521999994, 2024-09-15T00:12:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:12:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:12:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:49.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:49.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:12:49.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:12:49.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:12:49.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:12:49.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:12:49.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:12:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:12:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:12:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:12:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:12:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:12:49.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:12:49.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:49.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:12:49.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:12:49.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:49.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:12:49.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:12:49.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:49.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:12:49.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:12:49.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:49.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:12:49.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:12:49.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:12:49.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:12:49.132 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.132 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.132 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.132 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:49.132 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:49.132 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.548714829999994, 2024-09-15T00:12:49.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:12:49.133 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.133 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.133 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:49.133 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.133 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.133 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.133 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:12:49.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.133 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:49.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:49.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:49.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:12:49.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:12:49.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:12:49.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:12:49.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:12:49.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.508041141999996, 2024-09-15T00:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.508024610999996, 2024-09-15T00:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.508007832000004, 2024-09-15T00:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.507990101999994, 2024-09-15T00:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.507972937999995, 2024-09-15T00:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.507951262000006, 2024-09-15T00:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.507905137999998, 2024-09-15T00:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.507875381000005, 2024-09-15T00:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.507849062000005, 2024-09-15T00:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.507813200000001, 2024-09-15T00:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.507791482000002, 2024-09-15T00:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.507595015000007, 2024-09-15T00:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.507569176000004, 2024-09-15T00:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.507507529999998, 2024-09-15T00:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.507468201999998, 2024-09-15T00:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.505742802, 2024-09-15T00:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.505720640999996, 2024-09-15T00:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:145034 3.6 3:618673df:::rbd_object_map.1139646a7d4e.000000000000002b:head [writefull 0~48] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T00:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.1034670369999997, 2024-09-15T00:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0081079740000001, 2024-09-15T00:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 145034 2024-09-15T00:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017198+0000", 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0100000000000004e-07 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017202+0000", 2024-09-15T00:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017212+0000", 2024-09-15T00:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7189999999999996e-06 2024-09-15T00:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017219+0000", 2024-09-15T00:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7839999999999998e-06 2024-09-15T00:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017268+0000", 2024-09-15T00:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017277+0000", 2024-09-15T00:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1899999999999995e-06 2024-09-15T00:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019139+0000", 2024-09-15T00:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0018626129999999 2024-09-15T00:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019236+0000", 2024-09-15T00:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6513000000000002e-05 2024-09-15T00:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019318+0000", 2024-09-15T00:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1848000000000006e-05 2024-09-15T00:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.023151+0000", 2024-09-15T00:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038332169999999999 2024-09-15T00:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025217+0000", 2024-09-15T00:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020657610000000002 2024-09-15T00:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025267+0000", 2024-09-15T00:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0198000000000003e-05 2024-09-15T00:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025305+0000", 2024-09-15T00:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8065999999999998e-05 2024-09-15T00:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:49.221 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone45 from 0xe1bc9b7 (overlap 0x4d9e907) to 0x45db81f 2024-09-15T00:12:49.222 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:49.222 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:49.219+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:49.275 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-15T00:12:49.296 INFO:teuthology.orchestra.run.smithi171.stdout:5335 write 0xcd15f25 thru 0xcd2093a (0xaa16 bytes) 2024-09-15T00:12:49.309 INFO:teuthology.orchestra.run.smithi171.stdout:5336 punch from 0xe34b9f to 0xe3d5e9, (0x8a4a bytes) 2024-09-15T00:12:49.309 INFO:teuthology.orchestra.run.smithi171.stdout:5337 writesame 0x33fe8bd thru 0x3400f65 data_size 0x1(0x26a9 bytes) 2024-09-15T00:12:49.321 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:49.320+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:49.363 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:49.363 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:49.363 INFO:teuthology.orchestra.run.smithi171.stdout:5343 write 0xcf442c9 thru 0xcf4af19 (0x6c51 bytes) 2024-09-15T00:12:49.366 INFO:teuthology.orchestra.run.smithi171.stdout:5345 writesame 0xc84b198 thru 0xc84f77c data_size 0x1(0x45e5 bytes) 2024-09-15T00:12:49.375 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:49.375 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:49.375 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:49.375 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:49.384 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:49.392 INFO:teuthology.orchestra.run.smithi171.stdout:5346 punch from 0x499887f to 0x499bd29, (0x34aa bytes) 2024-09-15T00:12:49.401 INFO:teuthology.orchestra.run.smithi171.stdout:5347 trunc from 0xe1bc9b7 to 0x6826631 2024-09-15T00:12:49.422 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:49.421+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:49.432 INFO:teuthology.orchestra.run.smithi171.stdout:5348 read 0x5f785d4 thru 0x5f788ba (0x2e7 bytes) 2024-09-15T00:12:49.442 INFO:teuthology.orchestra.run.smithi171.stdout:5351 write 0xa14e939 thru 0xa15207e (0x3746 bytes) 2024-09-15T00:12:49.445 INFO:teuthology.orchestra.run.smithi171.stdout:5352 writesame 0x5a01649 thru 0x5a0583f data_size 0xd(0x41f7 bytes) 2024-09-15T00:12:49.460 INFO:teuthology.orchestra.run.smithi171.stdout:5353 writesame 0xa68d1ae thru 0xa69a34f data_size 0x1(0xd1a2 bytes) 2024-09-15T00:12:49.462 INFO:teuthology.orchestra.run.smithi171.stdout:5354 write 0x141cff9 thru 0x142957d (0xc585 bytes) 2024-09-15T00:12:49.503 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:49.504 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:49.504 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:49.504 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:49.504 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:49.504 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:49.502+0000 7feb60e0b640 -1 Falling back to public interface 2024-09-15T00:12:49.513 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:49.538 INFO:teuthology.orchestra.run.smithi171.stdout:5356 writesame 0x1409ea5 thru 0x14115bc data_size 0x67(0x7718 bytes) 2024-09-15T00:12:49.557 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:49.555+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:49.597 INFO:teuthology.orchestra.run.smithi171.stdout:5358 writesame 0xea9dfa2 thru 0xeaa69f0 data_size 0x1(0x8a4f bytes) 2024-09-15T00:12:49.597 INFO:teuthology.orchestra.run.smithi171.stdout:5360 write 0x9ec017d thru 0x9ecfea4 (0xfd28 bytes) 2024-09-15T00:12:49.604 INFO:teuthology.orchestra.run.smithi171.stdout:5361 writesame 0x64795e3 thru 0x6487dfc data_size 0x9(0xe81a bytes) 2024-09-15T00:12:49.607 INFO:teuthology.orchestra.run.smithi171.stdout:5362 writesame 0xd4b6dc9 thru 0xd4bd961 data_size 0x7(0x6b99 bytes) 2024-09-15T00:12:49.614 INFO:teuthology.orchestra.run.smithi171.stdout:5363 writesame 0xbf54734 thru 0xbf5bda3 data_size 0xa(0x7670 bytes) 2024-09-15T00:12:49.621 INFO:teuthology.orchestra.run.smithi171.stdout:5364 write 0x98bc028 thru 0x98c6a94 (0xaa6d bytes) 2024-09-15T00:12:49.624 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:49.622+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:49.624 INFO:teuthology.orchestra.run.smithi171.stdout:5366 write 0x37a9f9c thru 0x37aa121 (0x186 bytes) 2024-09-15T00:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:47986 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.154231683999996, 2024-09-15T00:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045287635, 2024-09-15T00:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 47986 2024-09-15T00:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.471627+0000", 2024-09-15T00:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8400000000000004e-07 2024-09-15T00:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474928+0000", 2024-09-15T00:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033009430000000002 2024-09-15T00:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474943+0000", 2024-09-15T00:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5172e-05 2024-09-15T00:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.474951+0000", 2024-09-15T00:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1779999999999997e-06 2024-09-15T00:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475058+0000", 2024-09-15T00:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000107595 2024-09-15T00:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.475200+0000", 2024-09-15T00:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014180400000000001 2024-09-15T00:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.478046+0000", 2024-09-15T00:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028464150000000001 2024-09-15T00:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516790+0000", 2024-09-15T00:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038743278999999999 2024-09-15T00:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516832+0000", 2024-09-15T00:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1974000000000002e-05 2024-09-15T00:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516872+0000", 2024-09-15T00:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9928999999999997e-05 2024-09-15T00:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.516914+0000", 2024-09-15T00:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2762000000000002e-05 2024-09-15T00:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48959 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.016759594000007, 2024-09-15T00:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044884741999999998, 2024-09-15T00:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48959 2024-09-15T00:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609099+0000", 2024-09-15T00:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2899999999999999e-07 2024-09-15T00:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609101+0000", 2024-09-15T00:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.826e-06 2024-09-15T00:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609108+0000", 2024-09-15T00:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.165e-06 2024-09-15T00:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609113+0000", 2024-09-15T00:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7849999999999999e-06 2024-09-15T00:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653842+0000", 2024-09-15T00:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044728954000000001 2024-09-15T00:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653896+0000", 2024-09-15T00:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4593999999999999e-05 2024-09-15T00:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.653984+0000", 2024-09-15T00:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7089000000000006e-05 2024-09-15T00:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48960 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 9961472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.016739075000004, 2024-09-15T00:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045084194000000001, 2024-09-15T00:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48960 2024-09-15T00:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609119+0000", 2024-09-15T00:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609120+0000", 2024-09-15T00:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7099999999999997e-07 2024-09-15T00:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609122+0000", 2024-09-15T00:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609129+0000", 2024-09-15T00:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4159999999999998e-06 2024-09-15T00:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609134+0000", 2024-09-15T00:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7210000000000002e-06 2024-09-15T00:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654023+0000", 2024-09-15T00:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044888979000000002 2024-09-15T00:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654104+0000", 2024-09-15T00:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1108999999999999e-05 2024-09-15T00:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654204+0000", 2024-09-15T00:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9630000000000007e-05 2024-09-15T00:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48961 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.016718253999997, 2024-09-15T00:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045197821999999999, 2024-09-15T00:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48961 2024-09-15T00:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609140+0000", 2024-09-15T00:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609141+0000", 2024-09-15T00:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609143+0000", 2024-09-15T00:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-15T00:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609150+0000", 2024-09-15T00:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2420000000000003e-06 2024-09-15T00:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609155+0000", 2024-09-15T00:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.028e-06 2024-09-15T00:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654232+0000", 2024-09-15T00:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045076702000000003 2024-09-15T00:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654281+0000", 2024-09-15T00:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9798e-05 2024-09-15T00:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654338+0000", 2024-09-15T00:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6498e-05 2024-09-15T00:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48962 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 13631488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.016697203999996, 2024-09-15T00:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045312458, 2024-09-15T00:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48962 2024-09-15T00:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609162+0000", 2024-09-15T00:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609164+0000", 2024-09-15T00:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0099999999999998e-06 2024-09-15T00:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609171+0000", 2024-09-15T00:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7530000000000001e-06 2024-09-15T00:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609178+0000", 2024-09-15T00:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1709999999999999e-06 2024-09-15T00:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654365+0000", 2024-09-15T00:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045187867999999999 2024-09-15T00:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654420+0000", 2024-09-15T00:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4376000000000003e-05 2024-09-15T00:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654474+0000", 2024-09-15T00:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3879999999999999e-05 2024-09-15T00:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48971 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.016666209999997, 2024-09-15T00:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045896151000000003, 2024-09-15T00:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48971 2024-09-15T00:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609193+0000", 2024-09-15T00:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7599999999999997e-07 2024-09-15T00:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609195+0000", 2024-09-15T00:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609210+0000", 2024-09-15T00:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4956999999999999e-05 2024-09-15T00:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609505+0000", 2024-09-15T00:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000295078 2024-09-15T00:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654498+0000", 2024-09-15T00:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044992277999999997 2024-09-15T00:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654541+0000", 2024-09-15T00:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3525000000000001e-05 2024-09-15T00:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655088+0000", 2024-09-15T00:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054726799999999995 2024-09-15T00:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48972 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 2097152~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.016339635999998, 2024-09-15T00:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046150885000000003, 2024-09-15T00:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48972 2024-09-15T00:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6400000000000002e-07 2024-09-15T00:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609523+0000", 2024-09-15T00:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5300000000000001e-06 2024-09-15T00:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609533+0000", 2024-09-15T00:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0023999999999999e-05 2024-09-15T00:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609539+0000", 2024-09-15T00:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3119999999999999e-06 2024-09-15T00:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655115+0000", 2024-09-15T00:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045576094999999997 2024-09-15T00:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655197+0000", 2024-09-15T00:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1129000000000002e-05 2024-09-15T00:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655670+0000", 2024-09-15T00:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00047313100000000001 2024-09-15T00:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.016311373999997, 2024-09-15T00:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.016290144999999, 2024-09-15T00:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.016264604, 2024-09-15T00:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.016242129000005, 2024-09-15T00:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.016221619000007, 2024-09-15T00:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.016201304000006, 2024-09-15T00:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.016176528000003, 2024-09-15T00:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.836835140999995, 2024-09-15T00:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.819881688999999, 2024-09-15T00:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 51.525450782, 2024-09-15T00:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:12:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:12:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:12:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:12:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:12:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:12:49.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:12:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:12:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:12:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:12:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:12:49.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:12:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:12:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:12:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:12:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:12:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:49.738 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:12:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:12:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:12:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:12:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:12:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:12:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.739 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:12:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 51.512994968000001, 2024-09-15T00:12:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:12:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:49.740 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:12:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:12:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:12:49.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:12:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:12:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:12:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:12:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:12:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.742 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:12:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:12:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:12:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:12:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:12:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:12:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.743 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:146935 3.16 3:6c79e092:::rbd_data.11394501364.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:12:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:12:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8722154499999997, 2024-09-15T00:12:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.058987488999999997, 2024-09-15T00:12:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 146935 2024-09-15T00:12:49.744 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:12:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:12:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.745 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753644+0000", 2024-09-15T00:12:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6199999999999997e-07 2024-09-15T00:12:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753648+0000", 2024-09-15T00:12:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-15T00:12:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753659+0000", 2024-09-15T00:12:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0703000000000001e-05 2024-09-15T00:12:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.746 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753664+0000", 2024-09-15T00:12:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.04e-06 2024-09-15T00:12:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753709+0000", 2024-09-15T00:12:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4728000000000003e-05 2024-09-15T00:12:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753756+0000", 2024-09-15T00:12:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7808e-05 2024-09-15T00:12:49.747 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.757051+0000", 2024-09-15T00:12:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294249 2024-09-15T00:12:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812553+0000", 2024-09-15T00:12:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055501807 2024-09-15T00:12:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812564+0000", 2024-09-15T00:12:49.748 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1467999999999999e-05 2024-09-15T00:12:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812598+0000", 2024-09-15T00:12:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3553000000000002e-05 2024-09-15T00:12:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812630+0000", 2024-09-15T00:12:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2895000000000003e-05 2024-09-15T00:12:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.749 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:146935 3.16 e98/95 ondisk, result = 0)", 2024-09-15T00:12:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:12:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8614127539999998, 2024-09-15T00:12:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048111830000000001, 2024-09-15T00:12:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:12:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.750 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:12:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764447+0000", 2024-09-15T00:12:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0500000000000002e-07 2024-09-15T00:12:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764450+0000", 2024-09-15T00:12:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4400000000000001e-06 2024-09-15T00:12:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764462+0000", 2024-09-15T00:12:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1603000000000001e-05 2024-09-15T00:12:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764469+0000", 2024-09-15T00:12:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5089999999999996e-06 2024-09-15T00:12:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:49.752 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812541+0000", 2024-09-15T00:12:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048071585 2024-09-15T00:12:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812548+0000", 2024-09-15T00:12:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9979999999999997e-06 2024-09-15T00:12:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812557+0000", 2024-09-15T00:12:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7899999999999994e-06 2024-09-15T00:12:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.753 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.754 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:49.754 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:49.754 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:49.756 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:49.756 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:49.753+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:49.757 INFO:teuthology.orchestra.run.smithi171.stdout:5368 write 0xe74f3a6 thru 0xe751e55 (0x2ab0 bytes) 2024-09-15T00:12:49.757 INFO:teuthology.orchestra.run.smithi171.stdout:5371 writesame 0x5e9a8fe thru 0x5ea84f2 data_size 0x1(0xdbf5 bytes) 2024-09-15T00:12:49.757 INFO:teuthology.orchestra.run.smithi171.stdout:5373 write 0x1a4f7b1 thru 0x1a5f312 (0xfb62 bytes) 2024-09-15T00:12:49.757 INFO:teuthology.orchestra.run.smithi171.stdout:5374 write 0xb49d075 thru 0xb4aae6f (0xddfb bytes) 2024-09-15T00:12:49.757 INFO:teuthology.orchestra.run.smithi171.stdout:5377 write 0x9516126 thru 0x9524284 (0xe15f bytes) 2024-09-15T00:12:49.757 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:49.757 INFO:teuthology.orchestra.run.smithi171.stdout:5380 write 0xc6ade69 thru 0xc6b09e8 (0x2b80 bytes) 2024-09-15T00:12:49.757 INFO:teuthology.orchestra.run.smithi171.stdout:5381 punch from 0x57306fe to 0x573fb3f, (0xf441 bytes) 2024-09-15T00:12:49.758 INFO:teuthology.orchestra.run.smithi171.stdout:5382 read 0x78064bf thru 0x7808d84 (0x28c6 bytes) 2024-09-15T00:12:49.758 INFO:teuthology.orchestra.run.smithi171.stdout:5383 punch from 0xd3c0d7f to 0xd3c9ce3, (0x8f64 bytes) 2024-09-15T00:12:49.758 INFO:teuthology.orchestra.run.smithi171.stdout:5384 write 0x9201458 thru 0x9209c9f (0x8848 bytes) 2024-09-15T00:12:49.758 INFO:teuthology.orchestra.run.smithi171.stdout:5385 read 0xcf75f58 thru 0xcf84761 (0xe80a bytes) 2024-09-15T00:12:49.758 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:49.758 INFO:teuthology.orchestra.run.smithi171.stdout:5389 writesame 0x3d190f thru 0x3defff data_size 0x19(0xd6f1 bytes) 2024-09-15T00:12:49.812 INFO:teuthology.orchestra.run.smithi171.stdout:5390 read 0x56d0cf1 thru 0x56d81c8 (0x74d8 bytes) 2024-09-15T00:12:49.812 INFO:teuthology.orchestra.run.smithi171.stdout:5391 write 0x9dd8be7 thru 0x9de2657 (0x9a71 bytes) 2024-09-15T00:12:49.815 INFO:teuthology.orchestra.run.smithi171.stdout:5393 punch from 0xd7804e3 to 0xd784b77, (0x4694 bytes) 2024-09-15T00:12:49.815 INFO:teuthology.orchestra.run.smithi171.stdout:5394 punch from 0x89676bf to 0x8970bd5, (0x9516 bytes) 2024-09-15T00:12:49.815 INFO:teuthology.orchestra.run.smithi171.stdout:5395 read 0x4951f3 thru 0x4a10c1 (0xbecf bytes) 2024-09-15T00:12:49.815 INFO:teuthology.orchestra.run.smithi171.stdout:5396 read 0xbffde26 thru 0xc008bfa (0xadd5 bytes) 2024-09-15T00:12:49.816 INFO:teuthology.orchestra.run.smithi171.stdout:5398 writesame 0xcba1553 thru 0xcbab4c3 data_size 0x31(0x9f71 bytes) 2024-09-15T00:12:49.824 INFO:teuthology.orchestra.run.smithi171.stdout:5399 punch from 0xd26c00 to 0xd33aeb, (0xceeb bytes) 2024-09-15T00:12:49.824 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:49.824 INFO:teuthology.orchestra.run.smithi171.stdout:5401 punch from 0x5f335b1 to 0x5f42d26, (0xf775 bytes) 2024-09-15T00:12:49.824 INFO:teuthology.orchestra.run.smithi171.stdout:5402 read 0x874c4d0 thru 0x874ce9e (0x9cf bytes) 2024-09-15T00:12:49.824 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:49.824 INFO:teuthology.orchestra.run.smithi171.stdout:5404 punch from 0x7315e43 to 0x7321f5d, (0xc11a bytes) 2024-09-15T00:12:49.825 INFO:teuthology.orchestra.run.smithi171.stdout:5406 read 0x1a1bf7a thru 0x1a1cb9e (0xc25 bytes) 2024-09-15T00:12:49.825 INFO:teuthology.orchestra.run.smithi171.stdout:5408 write 0xa145633 thru 0xa151b49 (0xc517 bytes) 2024-09-15T00:12:49.829 INFO:teuthology.orchestra.run.smithi171.stdout:5409 write 0x51d6609 thru 0x51d8b1b (0x2513 bytes) 2024-09-15T00:12:49.842 INFO:teuthology.orchestra.run.smithi171.stdout:5410 write 0xac207b thru 0xacdb5a (0xbae0 bytes) 2024-09-15T00:12:49.856 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:49.854+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:49.908 INFO:teuthology.orchestra.run.smithi171.stdout:5412 read 0xd38752b thru 0xd391336 (0x9e0c bytes) 2024-09-15T00:12:49.910 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:49.910 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:49.910 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:49.911 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:49.914 INFO:teuthology.orchestra.run.smithi171.stdout:5413 trunc from 0xeaa69f1 to 0x9e00550 2024-09-15T00:12:49.920 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:49.929 INFO:teuthology.orchestra.run.smithi171.stdout:5414 writesame 0x9988e8a thru 0x998b774 data_size 0x5(0x28eb bytes) 2024-09-15T00:12:49.939 INFO:teuthology.orchestra.run.smithi171.stdout:5415 write 0xc036d6a thru 0xc044034 (0xd2cb bytes) 2024-09-15T00:12:49.949 INFO:teuthology.orchestra.run.smithi171.stdout:5417 read 0x108c4ab thru 0x1094d68 (0x88be bytes) 2024-09-15T00:12:49.950 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:49.962 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:49.960+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:49.964 INFO:teuthology.orchestra.run.smithi171.stdout:5419 trunc from 0xc044035 to 0xc8442 2024-09-15T00:12:49.991 INFO:teuthology.orchestra.run.smithi171.stdout:5420 write 0x540ef2 thru 0x54892e (0x7a3d bytes) 2024-09-15T00:12:49.992 INFO:teuthology.orchestra.run.smithi171.stdout:5421 read 0xfbe4a thru 0xff309 (0x34c0 bytes) 2024-09-15T00:12:49.992 INFO:teuthology.orchestra.run.smithi171.stdout:5422 read 0x408ef9 thru 0x414f4b (0xc053 bytes) 2024-09-15T00:12:50.010 INFO:teuthology.orchestra.run.smithi171.stdout:5423 writesame 0xa9926ab thru 0xa99c716 data_size 0x2(0xa06c bytes) 2024-09-15T00:12:50.010 INFO:teuthology.orchestra.run.smithi171.stdout:5424 punch from 0x5fa0c31 to 0x5fa1036, (0x405 bytes) 2024-09-15T00:12:50.017 INFO:teuthology.orchestra.run.smithi171.stdout:5426 write 0xc5337d6 thru 0xc535c79 (0x24a4 bytes) 2024-09-15T00:12:50.020 INFO:teuthology.orchestra.run.smithi171.stdout:5427 write 0x4de34d4 thru 0x4de43d7 (0xf04 bytes) 2024-09-15T00:12:50.023 INFO:teuthology.orchestra.run.smithi171.stdout:5428 write 0x31e2ea7 thru 0x31e5efe (0x3058 bytes) 2024-09-15T00:12:50.032 INFO:teuthology.orchestra.run.smithi171.stdout:5429 trunc from 0xc535c7a to 0x78487f6 2024-09-15T00:12:50.050 INFO:teuthology.orchestra.run.smithi171.stdout:5432 write 0xb8c7ffc thru 0xb8c9dea (0x1def bytes) 2024-09-15T00:12:50.053 INFO:teuthology.orchestra.run.smithi171.stdout:5433 write 0xb344d13 thru 0xb35444c (0xf73a bytes) 2024-09-15T00:12:50.057 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:50.056+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:50.058 INFO:teuthology.orchestra.run.smithi171.stdout:5435 write 0x8f0ed59 thru 0x8f1216e (0x3416 bytes) 2024-09-15T00:12:50.064 INFO:teuthology.orchestra.run.smithi171.stdout:5437 read 0x29aa0dc thru 0x29b1bbd (0x7ae2 bytes) 2024-09-15T00:12:50.064 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:50.064 INFO:teuthology.orchestra.run.smithi171.stdout:5440 read 0x16dd249 thru 0x16ddc73 (0xa2b bytes) 2024-09-15T00:12:50.064 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:50.064 INFO:teuthology.orchestra.run.smithi171.stdout:5442 write 0xa90357e thru 0xa906490 (0x2f13 bytes) 2024-09-15T00:12:50.069 INFO:teuthology.orchestra.run.smithi171.stdout:5443 trunc from 0xb8c9deb to 0x9e7b7f5 2024-09-15T00:12:50.078 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:12:50.081 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:50.081 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:50.081 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:50.081 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:50.081 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:50.087 INFO:teuthology.orchestra.run.smithi171.stdout:5444 writesame 0xb00706e thru 0xb00758f data_size 0x9(0x522 bytes) 2024-09-15T00:12:50.088 INFO:teuthology.orchestra.run.smithi171.stdout:5445 writesame 0x356d054 thru 0x357896e data_size 0x1(0xb91b bytes) 2024-09-15T00:12:50.091 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:50.159 INFO:teuthology.orchestra.run.smithi171.stdout:5446 write 0x98740d4 thru 0x9877afd (0x3a2a bytes) 2024-09-15T00:12:50.173 INFO:teuthology.orchestra.run.smithi171.stdout:5448 trunc from 0xb007590 to 0x1d30efb 2024-09-15T00:12:50.174 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:50.173+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:50.186 INFO:teuthology.orchestra.run.smithi171.stdout:5450 writesame 0x174947a thru 0x174c7fa data_size 0xf(0x3381 bytes) 2024-09-15T00:12:50.189 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:50.190 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:50.190 INFO:teuthology.orchestra.run.smithi171.stdout:5455 punch from 0xe698b8 to 0xe6cfde, (0x3726 bytes) 2024-09-15T00:12:50.190 INFO:teuthology.orchestra.run.smithi171.stdout:5456 read 0x1a56fba thru 0x1a5bfa7 (0x4fee bytes) 2024-09-15T00:12:50.190 INFO:teuthology.orchestra.run.smithi171.stdout:5457 trunc from 0x1d30efb to 0x17a97d2 2024-09-15T00:12:50.197 INFO:teuthology.orchestra.run.smithi171.stdout:5458 read 0x8a442f thru 0x8a9758 (0x532a bytes) 2024-09-15T00:12:50.197 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:50.197 INFO:teuthology.orchestra.run.smithi171.stdout:5460 read 0x20e3e9 thru 0x211947 (0x355f bytes) 2024-09-15T00:12:50.197 INFO:teuthology.orchestra.run.smithi171.stdout:5463 punch from 0x3e111c to 0x3ecf7c, (0xbe60 bytes) 2024-09-15T00:12:50.206 INFO:teuthology.orchestra.run.smithi171.stdout:5465 write 0x3caadf5 thru 0x3cba5d4 (0xf7e0 bytes) 2024-09-15T00:12:50.220 INFO:teuthology.orchestra.run.smithi171.stdout:5466 writesame 0x4eecfd9 thru 0x4ef6332 data_size 0x3(0x935a bytes) 2024-09-15T00:12:50.220 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:50.220 INFO:teuthology.orchestra.run.smithi171.stdout:5469 read 0x22ae484 thru 0x22bc742 (0xe2bf bytes) 2024-09-15T00:12:50.220 INFO:teuthology.orchestra.run.smithi171.stdout:5471 punch from 0x2aec6bd to 0x2af1a35, (0x5378 bytes) 2024-09-15T00:12:50.220 INFO:teuthology.orchestra.run.smithi171.stdout:5472 punch from 0x334ac6e to 0x334c5f2, (0x1984 bytes) 2024-09-15T00:12:50.228 INFO:teuthology.orchestra.run.smithi171.stdout:5473 trunc from 0x4ef6333 to 0xac2fd7f 2024-09-15T00:12:50.235 INFO:teuthology.orchestra.run.smithi171.stdout:5476 write 0x4ffc670 thru 0x4fff343 (0x2cd4 bytes) 2024-09-15T00:12:50.239 INFO:teuthology.orchestra.run.smithi171.stdout:5477 write 0x49429f1 thru 0x494f7a0 (0xcdb0 bytes) 2024-09-15T00:12:50.244 INFO:teuthology.orchestra.run.smithi171.stdout:5478 writesame 0x9aedfa5 thru 0x9aef3d5 data_size 0x1(0x1431 bytes) 2024-09-15T00:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:19308 3.10 3:0ee1524d:::rbd_data.11398e1a091a.0000000000000017:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-15T00:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 71.257617126, 2024-09-15T00:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02547147, 2024-09-15T00:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 19308 2024-09-15T00:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985182+0000", 2024-09-15T00:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6500000000000001e-06 2024-09-15T00:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985189+0000", 2024-09-15T00:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9949999999999999e-06 2024-09-15T00:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985205+0000", 2024-09-15T00:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6055000000000001e-05 2024-09-15T00:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985212+0000", 2024-09-15T00:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985261+0000", 2024-09-15T00:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8380000000000001e-05 2024-09-15T00:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985311+0000", 2024-09-15T00:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0763999999999997e-05 2024-09-15T00:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.986489+0000", 2024-09-15T00:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0011773420000000001 2024-09-15T00:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010514+0000", 2024-09-15T00:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024025392999999999 2024-09-15T00:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010535+0000", 2024-09-15T00:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0652000000000001e-05 2024-09-15T00:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010597+0000", 2024-09-15T00:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2411999999999998e-05 2024-09-15T00:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010652+0000", 2024-09-15T00:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5074999999999997e-05 2024-09-15T00:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.575440158000006, 2024-09-15T00:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:12:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:12:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:12:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:12:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:12:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:12:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:12:50.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66542 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1560576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.328643505000002, 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024740410000000001, 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.262 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66542 2024-09-15T00:12:50.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.263 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:50.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:50.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914155+0000", 2024-09-15T00:12:50.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:12:50.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914157+0000", 2024-09-15T00:12:50.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:12:50.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914162+0000", 2024-09-15T00:12:50.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:50.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914167+0000", 2024-09-15T00:12:50.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3789999999999999e-06 2024-09-15T00:12:50.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938730+0000", 2024-09-15T00:12:50.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024562694 2024-09-15T00:12:50.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938803+0000", 2024-09-15T00:12:50.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3325999999999996e-05 2024-09-15T00:12:50.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938895+0000", 2024-09-15T00:12:50.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1884000000000003e-05 2024-09-15T00:12:50.266 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.266 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.266 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.267 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66543 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1572864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:50.267 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:50.267 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.328613246000003, 2024-09-15T00:12:50.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024893976000000002, 2024-09-15T00:12:50.267 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.267 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.267 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.267 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.267 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.267 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.268 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66543 2024-09-15T00:12:50.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.268 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:50.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:50.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:50.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:50.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914188+0000", 2024-09-15T00:12:50.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:12:50.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914193+0000", 2024-09-15T00:12:50.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4129999999999997e-06 2024-09-15T00:12:50.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914198+0000", 2024-09-15T00:12:50.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9309999999999999e-06 2024-09-15T00:12:50.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938914+0000", 2024-09-15T00:12:50.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024716380999999999 2024-09-15T00:12:50.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938993+0000", 2024-09-15T00:12:50.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8763000000000005e-05 2024-09-15T00:12:50.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939079+0000", 2024-09-15T00:12:50.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5372999999999997e-05 2024-09-15T00:12:50.271 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.271 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.271 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.272 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66544 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1683456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:50.272 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:50.272 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.328590843000001, 2024-09-15T00:12:50.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025051225, 2024-09-15T00:12:50.272 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.272 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.272 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.272 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.272 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.272 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.272 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66544 2024-09-15T00:12:50.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.273 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:50.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:50.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:50.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:12:50.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914210+0000", 2024-09-15T00:12:50.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2450000000000001e-06 2024-09-15T00:12:50.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914215+0000", 2024-09-15T00:12:50.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7869999999999998e-06 2024-09-15T00:12:50.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.276 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914220+0000", 2024-09-15T00:12:50.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7899999999999999e-06 2024-09-15T00:12:50.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.276 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939105+0000", 2024-09-15T00:12:50.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024884381000000001 2024-09-15T00:12:50.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939177+0000", 2024-09-15T00:12:50.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2138999999999995e-05 2024-09-15T00:12:50.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939258+0000", 2024-09-15T00:12:50.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1462e-05 2024-09-15T00:12:50.277 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.277 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.277 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.278 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66545 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1695744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:50.278 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:50.278 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.328568568999998, 2024-09-15T00:12:50.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025209788, 2024-09-15T00:12:50.278 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.278 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.279 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.279 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.279 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.279 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.279 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66545 2024-09-15T00:12:50.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.279 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:50.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:50.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914230+0000", 2024-09-15T00:12:50.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1099999999999996e-07 2024-09-15T00:12:50.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914232+0000", 2024-09-15T00:12:50.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3080000000000002e-06 2024-09-15T00:12:50.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914238+0000", 2024-09-15T00:12:50.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.02e-06 2024-09-15T00:12:50.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914243+0000", 2024-09-15T00:12:50.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1200000000000001e-06 2024-09-15T00:12:50.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939285+0000", 2024-09-15T00:12:50.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025042212000000001 2024-09-15T00:12:50.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939361+0000", 2024-09-15T00:12:50.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5558999999999999e-05 2024-09-15T00:12:50.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939439+0000", 2024-09-15T00:12:50.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8058000000000004e-05 2024-09-15T00:12:50.283 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.283 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.283 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.283 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66546 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:50.283 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:50.283 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.328548593999997, 2024-09-15T00:12:50.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025375960999999999, 2024-09-15T00:12:50.283 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.284 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.284 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.284 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.284 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.284 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.284 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66546 2024-09-15T00:12:50.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.284 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:50.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:50.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914250+0000", 2024-09-15T00:12:50.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:12:50.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914252+0000", 2024-09-15T00:12:50.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.052e-06 2024-09-15T00:12:50.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914258+0000", 2024-09-15T00:12:50.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9039999999999997e-06 2024-09-15T00:12:50.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914262+0000", 2024-09-15T00:12:50.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4719999999999997e-06 2024-09-15T00:12:50.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939466+0000", 2024-09-15T00:12:50.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025203550000000002 2024-09-15T00:12:50.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939537+0000", 2024-09-15T00:12:50.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1780999999999996e-05 2024-09-15T00:12:50.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939625+0000", 2024-09-15T00:12:50.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7828e-05 2024-09-15T00:12:50.289 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.289 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.289 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.290 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66547 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1818624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:50.290 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:50.290 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.328528921, 2024-09-15T00:12:50.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025552638999999999, 2024-09-15T00:12:50.290 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.290 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.290 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.290 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66547 2024-09-15T00:12:50.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.291 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:50.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:50.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:50.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:50.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914271+0000", 2024-09-15T00:12:50.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8920000000000001e-06 2024-09-15T00:12:50.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914277+0000", 2024-09-15T00:12:50.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:50.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914282+0000", 2024-09-15T00:12:50.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7190000000000001e-06 2024-09-15T00:12:50.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939654+0000", 2024-09-15T00:12:50.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025372372000000001 2024-09-15T00:12:50.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939734+0000", 2024-09-15T00:12:50.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9946999999999999e-05 2024-09-15T00:12:50.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939822+0000", 2024-09-15T00:12:50.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7680000000000001e-05 2024-09-15T00:12:50.294 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.294 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.294 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.295 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66548 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:50.295 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:50.295 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.328498533999998, 2024-09-15T00:12:50.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025712727000000001, 2024-09-15T00:12:50.295 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.295 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.296 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.296 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.296 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.296 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.296 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66548 2024-09-15T00:12:50.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.296 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:50.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:50.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914300+0000", 2024-09-15T00:12:50.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3599999999999999e-07 2024-09-15T00:12:50.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914302+0000", 2024-09-15T00:12:50.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3369999999999998e-06 2024-09-15T00:12:50.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914313+0000", 2024-09-15T00:12:50.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1399999999999999e-05 2024-09-15T00:12:50.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914319+0000", 2024-09-15T00:12:50.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9100000000000002e-06 2024-09-15T00:12:50.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939850+0000", 2024-09-15T00:12:50.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025531087000000001 2024-09-15T00:12:50.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939932+0000", 2024-09-15T00:12:50.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1534999999999998e-05 2024-09-15T00:12:50.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940012+0000", 2024-09-15T00:12:50.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0022000000000001e-05 2024-09-15T00:12:50.300 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.300 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.300 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.300 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66549 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1941504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:50.300 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:50.300 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.328433443000002, 2024-09-15T00:12:50.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025813836999999999, 2024-09-15T00:12:50.300 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.300 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.301 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.301 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.301 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.301 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.301 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66549 2024-09-15T00:12:50.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.301 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:50.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:50.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914365+0000", 2024-09-15T00:12:50.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2799999999999995e-07 2024-09-15T00:12:50.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914368+0000", 2024-09-15T00:12:50.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.92e-06 2024-09-15T00:12:50.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914376+0000", 2024-09-15T00:12:50.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8110000000000003e-06 2024-09-15T00:12:50.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914381+0000", 2024-09-15T00:12:50.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5119999999999999e-06 2024-09-15T00:12:50.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940040+0000", 2024-09-15T00:12:50.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025658271999999999 2024-09-15T00:12:50.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940103+0000", 2024-09-15T00:12:50.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2891000000000005e-05 2024-09-15T00:12:50.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940178+0000", 2024-09-15T00:12:50.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5703000000000003e-05 2024-09-15T00:12:50.304 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.304 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.305 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.305 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66550 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:50.305 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:50.305 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.328289863000002, 2024-09-15T00:12:50.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025820701000000001, 2024-09-15T00:12:50.305 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.305 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.305 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.305 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.305 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.305 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.305 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66550 2024-09-15T00:12:50.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.306 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:50.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:50.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914509+0000", 2024-09-15T00:12:50.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1999999999999999e-07 2024-09-15T00:12:50.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914513+0000", 2024-09-15T00:12:50.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1640000000000003e-06 2024-09-15T00:12:50.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914522+0000", 2024-09-15T00:12:50.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7639999999999994e-06 2024-09-15T00:12:50.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914527+0000", 2024-09-15T00:12:50.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9170000000000003e-06 2024-09-15T00:12:50.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940199+0000", 2024-09-15T00:12:50.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025672037000000002 2024-09-15T00:12:50.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940269+0000", 2024-09-15T00:12:50.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9208999999999994e-05 2024-09-15T00:12:50.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940329+0000", 2024-09-15T00:12:50.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.999e-05 2024-09-15T00:12:50.310 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.310 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.310 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.310 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66551 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:50.310 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:50.310 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.328233116, 2024-09-15T00:12:50.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025917345000000001, 2024-09-15T00:12:50.311 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.311 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.311 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.311 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66551 2024-09-15T00:12:50.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.311 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:50.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:50.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:50.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1699999999999997e-07 2024-09-15T00:12:50.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914569+0000", 2024-09-15T00:12:50.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.225e-06 2024-09-15T00:12:50.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914582+0000", 2024-09-15T00:12:50.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3648999999999999e-05 2024-09-15T00:12:50.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914588+0000", 2024-09-15T00:12:50.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8449999999999999e-06 2024-09-15T00:12:50.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940355+0000", 2024-09-15T00:12:50.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025767110999999999 2024-09-15T00:12:50.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940423+0000", 2024-09-15T00:12:50.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7574e-05 2024-09-15T00:12:50.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940482+0000", 2024-09-15T00:12:50.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9224000000000001e-05 2024-09-15T00:12:50.315 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.315 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.315 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.315 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:88238 3.a 3:52924f23:::rbd_data.11396cb1f0e4.0000000000000001:16 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-15T00:12:50.315 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:50.315 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 47.965048819000003, 2024-09-15T00:12:50.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024851132000000001, 2024-09-15T00:12:50.315 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.315 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.315 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.315 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.315 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.316 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.316 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88238 2024-09-15T00:12:50.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.316 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:50.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:50.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277750+0000", 2024-09-15T00:12:50.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3300000000000002e-07 2024-09-15T00:12:50.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277751+0000", 2024-09-15T00:12:50.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:12:50.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277757+0000", 2024-09-15T00:12:50.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.767e-06 2024-09-15T00:12:50.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277769+0000", 2024-09-15T00:12:50.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1874e-05 2024-09-15T00:12:50.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302374+0000", 2024-09-15T00:12:50.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024604669999999999 2024-09-15T00:12:50.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302460+0000", 2024-09-15T00:12:50.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6489999999999994e-05 2024-09-15T00:12:50.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302600+0000", 2024-09-15T00:12:50.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013985700000000001 2024-09-15T00:12:50.319 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.319 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.319 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.319 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125618 3.10 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:50.319 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:12:50.319 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.404301075999999, 2024-09-15T00:12:50.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030860942999999998, 2024-09-15T00:12:50.320 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.320 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.320 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:50.320 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.320 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.320 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.320 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125618 2024-09-15T00:12:50.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.320 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:12:50.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:12:50.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838499+0000", 2024-09-15T00:12:50.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6189999999999998e-06 2024-09-15T00:12:50.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839599+0000", 2024-09-15T00:12:50.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001099764 2024-09-15T00:12:50.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839620+0000", 2024-09-15T00:12:50.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0764e-05 2024-09-15T00:12:50.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839631+0000", 2024-09-15T00:12:50.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1335000000000001e-05 2024-09-15T00:12:50.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839725+0000", 2024-09-15T00:12:50.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4217999999999998e-05 2024-09-15T00:12:50.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839910+0000", 2024-09-15T00:12:50.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000184298 2024-09-15T00:12:50.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:50.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843171+0000", 2024-09-15T00:12:50.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032613540000000002 2024-09-15T00:12:50.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:50.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869187+0000", 2024-09-15T00:12:50.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026015467 2024-09-15T00:12:50.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:50.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869240+0000", 2024-09-15T00:12:50.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3733000000000002e-05 2024-09-15T00:12:50.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:50.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869309+0000", 2024-09-15T00:12:50.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8381999999999997e-05 2024-09-15T00:12:50.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869358+0000", 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125619 3.a 3:507b29bf:::rbd_data.11396481e1ec.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.403138946999999, 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045768870000000003, 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:50.325 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.326 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.326 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.326 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125619 2024-09-15T00:12:50.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.326 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:12:50.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:12:50.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839660+0000", 2024-09-15T00:12:50.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.22e-06 2024-09-15T00:12:50.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840511+0000", 2024-09-15T00:12:50.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000851284 2024-09-15T00:12:50.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840522+0000", 2024-09-15T00:12:50.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0569e-05 2024-09-15T00:12:50.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840527+0000", 2024-09-15T00:12:50.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.412e-06 2024-09-15T00:12:50.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840573+0000", 2024-09-15T00:12:50.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5247000000000002e-05 2024-09-15T00:12:50.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840682+0000", 2024-09-15T00:12:50.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109368 2024-09-15T00:12:50.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:50.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843976+0000", 2024-09-15T00:12:50.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294445 2024-09-15T00:12:50.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:50.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870833+0000", 2024-09-15T00:12:50.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026856944000000001 2024-09-15T00:12:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885344+0000", 2024-09-15T00:12:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014510744000000001 2024-09-15T00:12:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885382+0000", 2024-09-15T00:12:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7821000000000003e-05 2024-09-15T00:12:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885428+0000", 2024-09-15T00:12:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5816000000000003e-05 2024-09-15T00:12:50.331 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.331 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.331 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.331 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125621 3.a 3:53594db9:::rbd_data.11396481e1ec.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:50.331 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:12:50.331 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.389615568, 2024-09-15T00:12:50.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032481797, 2024-09-15T00:12:50.331 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.331 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.331 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:50.331 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.331 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.332 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.332 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125621 2024-09-15T00:12:50.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.332 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:12:50.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:12:50.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853185+0000", 2024-09-15T00:12:50.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.43e-06 2024-09-15T00:12:50.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854125+0000", 2024-09-15T00:12:50.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000940539 2024-09-15T00:12:50.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854141+0000", 2024-09-15T00:12:50.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5568000000000001e-05 2024-09-15T00:12:50.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854148+0000", 2024-09-15T00:12:50.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6529999999999994e-06 2024-09-15T00:12:50.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870872+0000", 2024-09-15T00:12:50.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016723135 2024-09-15T00:12:50.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870975+0000", 2024-09-15T00:12:50.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010314099999999999 2024-09-15T00:12:50.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:50.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.873082+0000", 2024-09-15T00:12:50.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021069859999999999 2024-09-15T00:12:50.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:50.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885439+0000", 2024-09-15T00:12:50.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012357343999999999 2024-09-15T00:12:50.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:50.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885569+0000", 2024-09-15T00:12:50.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012995199999999999 2024-09-15T00:12:50.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:50.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885624+0000", 2024-09-15T00:12:50.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5281999999999997e-05 2024-09-15T00:12:50.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885664+0000", 2024-09-15T00:12:50.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9767000000000001e-05 2024-09-15T00:12:50.336 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.336 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.336 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.337 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:147075 3.0 3:06ab39f2:::rbd_data.11394501364.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:12:50.337 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:12:50.337 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.2937171870000004, 2024-09-15T00:12:50.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046402906000000001, 2024-09-15T00:12:50.337 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.337 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.337 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:50.337 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.337 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.337 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.337 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 147075 2024-09-15T00:12:50.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.337 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:12:50.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:12:50.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:12:50.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4399999999999999e-07 2024-09-15T00:12:50.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949086+0000", 2024-09-15T00:12:50.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.138e-06 2024-09-15T00:12:50.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949096+0000", 2024-09-15T00:12:50.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0438999999999999e-05 2024-09-15T00:12:50.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949101+0000", 2024-09-15T00:12:50.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8999999999999997e-06 2024-09-15T00:12:50.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949145+0000", 2024-09-15T00:12:50.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4141000000000003e-05 2024-09-15T00:12:50.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949195+0000", 2024-09-15T00:12:50.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9657999999999998e-05 2024-09-15T00:12:50.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:50.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.950923+0000", 2024-09-15T00:12:50.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001728753 2024-09-15T00:12:50.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:50.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995390+0000", 2024-09-15T00:12:50.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044466335000000003 2024-09-15T00:12:50.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:50.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995401+0000", 2024-09-15T00:12:50.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1671000000000001e-05 2024-09-15T00:12:50.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:50.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995432+0000", 2024-09-15T00:12:50.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0704999999999998e-05 2024-09-15T00:12:50.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995484+0000", 2024-09-15T00:12:50.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1422000000000003e-05 2024-09-15T00:12:50.342 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.342 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.342 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.342 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:147075 3.0 e98/94 ondisk, result = 0)", 2024-09-15T00:12:50.343 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:12:50.343 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.2859174659999999, 2024-09-15T00:12:50.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038514459000000001, 2024-09-15T00:12:50.343 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.343 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.343 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.343 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:12:50.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:12:50.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956881+0000", 2024-09-15T00:12:50.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0599999999999999e-07 2024-09-15T00:12:50.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956885+0000", 2024-09-15T00:12:50.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3500000000000001e-06 2024-09-15T00:12:50.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956897+0000", 2024-09-15T00:12:50.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2303999999999999e-05 2024-09-15T00:12:50.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956902+0000", 2024-09-15T00:12:50.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5169999999999999e-06 2024-09-15T00:12:50.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995377+0000", 2024-09-15T00:12:50.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038474544999999999 2024-09-15T00:12:50.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995385+0000", 2024-09-15T00:12:50.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6240000000000002e-06 2024-09-15T00:12:50.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995395+0000", 2024-09-15T00:12:50.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0213e-05 2024-09-15T00:12:50.346 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.346 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.346 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.346 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153172 3.14 3:2a7843e3:::rbd_data.113958cb0e55.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 983040~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:12:50.346 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:12:50.347 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.46502840200000001, 2024-09-15T00:12:50.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.033232668, 2024-09-15T00:12:50.347 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.347 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.347 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:50.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.347 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153172 2024-09-15T00:12:50.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.347 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:12:50.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:12:50.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777770+0000", 2024-09-15T00:12:50.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.017e-06 2024-09-15T00:12:50.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788009+0000", 2024-09-15T00:12:50.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010238146 2024-09-15T00:12:50.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788024+0000", 2024-09-15T00:12:50.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5582000000000001e-05 2024-09-15T00:12:50.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788048+0000", 2024-09-15T00:12:50.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4173e-05 2024-09-15T00:12:50.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788097+0000", 2024-09-15T00:12:50.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8498999999999998e-05 2024-09-15T00:12:50.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788201+0000", 2024-09-15T00:12:50.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010398799999999999 2024-09-15T00:12:50.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:50.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788393+0000", 2024-09-15T00:12:50.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000192337 2024-09-15T00:12:50.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:50.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810637+0000", 2024-09-15T00:12:50.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.022244202000000001 2024-09-15T00:12:50.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:50.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810903+0000", 2024-09-15T00:12:50.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00026574600000000003 2024-09-15T00:12:50.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:50.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810953+0000", 2024-09-15T00:12:50.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9509000000000003e-05 2024-09-15T00:12:50.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.811002+0000", 2024-09-15T00:12:50.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9468999999999997e-05 2024-09-15T00:12:50.352 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.352 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.352 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.352 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153770 3.13 3:cebb4cdf:::rbd_data.113958cb0e55.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 2826363~24453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:12:50.352 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.878288+0000", 2024-09-15T00:12:50.352 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.36451013700000001, 2024-09-15T00:12:50.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029205471, 2024-09-15T00:12:50.353 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.353 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.353 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:50.353 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.353 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.353 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.353 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153770 2024-09-15T00:12:50.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.353 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.878288+0000", 2024-09-15T00:12:50.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.878288+0000", 2024-09-15T00:12:50.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.878295+0000", 2024-09-15T00:12:50.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2409999999999998e-06 2024-09-15T00:12:50.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.885400+0000", 2024-09-15T00:12:50.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0071045780000000003 2024-09-15T00:12:50.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.885421+0000", 2024-09-15T00:12:50.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1664000000000001e-05 2024-09-15T00:12:50.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.885430+0000", 2024-09-15T00:12:50.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5909999999999996e-06 2024-09-15T00:12:50.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.885486+0000", 2024-09-15T00:12:50.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6190000000000002e-05 2024-09-15T00:12:50.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.885595+0000", 2024-09-15T00:12:50.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109478 2024-09-15T00:12:50.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:50.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.885871+0000", 2024-09-15T00:12:50.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00027578800000000001 2024-09-15T00:12:50.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:50.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.906276+0000", 2024-09-15T00:12:50.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02040438 2024-09-15T00:12:50.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:50.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.907422+0000", 2024-09-15T00:12:50.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001146317 2024-09-15T00:12:50.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:50.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.907460+0000", 2024-09-15T00:12:50.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8059999999999998e-05 2024-09-15T00:12:50.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.907493+0000", 2024-09-15T00:12:50.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3183999999999999e-05 2024-09-15T00:12:50.358 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.358 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.358 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.358 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.358 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.358 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:50.360 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:50.360 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:50.357+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:50.362 INFO:teuthology.orchestra.run.smithi171.stdout:5484 trunc from 0xac2fd7f to 0xe636dfd 2024-09-15T00:12:50.362 INFO:teuthology.orchestra.run.smithi171.stdout:5486 punch from 0x8adf67 to 0x8b4367, (0x6400 bytes) 2024-09-15T00:12:50.362 INFO:teuthology.orchestra.run.smithi171.stdout:5487 punch from 0xa29cb7a to 0xa2a42fe, (0x7784 bytes) 2024-09-15T00:12:50.362 INFO:teuthology.orchestra.run.smithi171.stdout:5488 write 0xecbb248 thru 0xecc9cca (0xea83 bytes) 2024-09-15T00:12:50.362 INFO:teuthology.orchestra.run.smithi171.stdout:5489 read 0x677281b thru 0x677ba61 (0x9247 bytes) 2024-09-15T00:12:50.362 INFO:teuthology.orchestra.run.smithi171.stdout:5490 write 0x85bbf38 thru 0x85c5956 (0x9a1f bytes) 2024-09-15T00:12:50.362 INFO:teuthology.orchestra.run.smithi171.stdout:5491 write 0x70df3e thru 0x71b326 (0xd3e9 bytes) 2024-09-15T00:12:50.362 INFO:teuthology.orchestra.run.smithi171.stdout:5492 write 0x8c813c1 thru 0x8c865eb (0x522b bytes) 2024-09-15T00:12:50.362 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:50.362 INFO:teuthology.orchestra.run.smithi171.stdout:5496 write 0x87a085a thru 0x87af969 (0xf110 bytes) 2024-09-15T00:12:50.363 INFO:teuthology.orchestra.run.smithi171.stdout:5497 write 0x88a070c thru 0x88a71b8 (0x6aad bytes) 2024-09-15T00:12:50.363 INFO:teuthology.orchestra.run.smithi171.stdout:5498 write 0xb886b thru 0xbc936 (0x40cc bytes) 2024-09-15T00:12:50.363 INFO:teuthology.orchestra.run.smithi171.stdout:5499 writesame 0xd93e137 thru 0xd94c157 data_size 0x1(0xe021 bytes) 2024-09-15T00:12:50.363 INFO:teuthology.orchestra.run.smithi171.stdout:5500 writesame 0xe8fc8fa thru 0xe90185f data_size 0x1(0x4f66 bytes) 2024-09-15T00:12:50.363 INFO:teuthology.orchestra.run.smithi171.stdout:5502 writesame 0x2e1b680 thru 0x2e1cb9b data_size 0xe(0x151c bytes) 2024-09-15T00:12:50.363 INFO:teuthology.orchestra.run.smithi171.stdout:5506 write 0xbc6913a thru 0xbc6917a (0x41 bytes) 2024-09-15T00:12:50.363 INFO:teuthology.orchestra.run.smithi171.stdout:5507 write 0x6eff6d5 thru 0x6f03b70 (0x449c bytes) 2024-09-15T00:12:50.363 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:50.363 INFO:teuthology.orchestra.run.smithi171.stdout:5509 punch from 0x8073012 to 0x80755cf, (0x25bd bytes) 2024-09-15T00:12:50.363 INFO:teuthology.orchestra.run.smithi171.stdout:5510 writesame 0xcab1569 thru 0xcabb3fb data_size 0x17(0x9e93 bytes) 2024-09-15T00:12:50.364 INFO:teuthology.orchestra.run.smithi171.stdout:5511 writesame 0xa56d0c8 thru 0xa5799f2 data_size 0x7(0xc92b bytes) 2024-09-15T00:12:50.370 INFO:teuthology.orchestra.run.smithi171.stdout:5514 write 0x17d1959 thru 0x17d9838 (0x7ee0 bytes) 2024-09-15T00:12:50.378 INFO:teuthology.orchestra.run.smithi171.stdout:5515 write 0x2d93f63 thru 0x2da2100 (0xe19e bytes) 2024-09-15T00:12:50.394 INFO:teuthology.orchestra.run.smithi171.stdout:5516 trunc from 0xecc9ccb to 0x6daf918 2024-09-15T00:12:50.408 INFO:teuthology.orchestra.run.smithi171.stdout:5517 punch from 0x2123f49 to 0x212894f, (0x4a06 bytes) 2024-09-15T00:12:50.409 INFO:teuthology.orchestra.run.smithi171.stdout:5520 read 0x1e90bc5 thru 0x1e9ddfe (0xd23a bytes) 2024-09-15T00:12:50.409 INFO:teuthology.orchestra.run.smithi171.stdout:5522 read 0x5c822e3 thru 0x5c8d944 (0xb662 bytes) 2024-09-15T00:12:50.409 INFO:teuthology.orchestra.run.smithi171.stdout:5523 read 0x4c05780 thru 0x4c13577 (0xddf8 bytes) 2024-09-15T00:12:50.410 INFO:teuthology.orchestra.run.smithi171.stdout:5525 read 0x1f3e307 thru 0x1f43428 (0x5122 bytes) 2024-09-15T00:12:50.410 INFO:teuthology.orchestra.run.smithi171.stdout:5526 read 0x4116f41 thru 0x4125311 (0xe3d1 bytes) 2024-09-15T00:12:50.410 INFO:teuthology.orchestra.run.smithi171.stdout:5527 write 0x60536f5 thru 0x605390a (0x216 bytes) 2024-09-15T00:12:50.413 INFO:teuthology.orchestra.run.smithi171.stdout:5528 read 0x138c943 thru 0x139a689 (0xdd47 bytes) 2024-09-15T00:12:50.413 INFO:teuthology.orchestra.run.smithi171.stdout:5529 read 0x5cfcc05 thru 0x5d044f8 (0x78f4 bytes) 2024-09-15T00:12:50.414 INFO:teuthology.orchestra.run.smithi171.stdout:5530 read 0x6c22921 thru 0x6c30f07 (0xe5e7 bytes) 2024-09-15T00:12:50.414 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:50.431 INFO:teuthology.orchestra.run.smithi171.stdout:5532 writesame 0xeb26b75 thru 0xeb29183 data_size 0x1(0x260f bytes) 2024-09-15T00:12:50.431 INFO:teuthology.orchestra.run.smithi171.stdout:5533 clone 47 order 18 su 65536 sc 13 2024-09-15T00:12:50.460 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:50.459+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:50.504 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:50.504 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:50.504 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:50.504 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:50.513 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:50.597 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:50.595+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:50.616 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:12:50.656 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:50.656 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:50.656 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:50.656 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:50.656 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:50.662 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:50.661+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:50.664 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:50.737 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:50.737 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:50.753 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:50.754 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:50.754 INFO:teuthology.orchestra.run.smithi171.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T00:12:50.765 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:50.763+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:50.770 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:50.770 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:50.786 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:50.787 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:50.803 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:50.803 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:50.810 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:50.810 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:50.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:50.810 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:50.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.810 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:50.810 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:50.810 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 84.729090397999997, 2024-09-15T00:12:50.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:12:50.810 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.811 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.811 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:50.811 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.811 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.811 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.811 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:12:50.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.811 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:50.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:50.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:12:50.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:12:50.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:12:50.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:12:50.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:12:50.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:12:50.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:12:50.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:12:50.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:12:50.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:12:50.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:12:50.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:12:50.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:50.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:12:50.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:12:50.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:50.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:12:50.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:12:50.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:50.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:12:50.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:12:50.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:50.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:12:50.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:12:50.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:12:50.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:12:50.817 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.817 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.817 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.818 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:12:50.818 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:50.818 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 84.716349855999994, 2024-09-15T00:12:50.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:12:50.818 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.818 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.818 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.818 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:50.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:50.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:12:50.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:12:50.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:12:50.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:12:50.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:12:50.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:12:50.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:12:50.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:12:50.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:12:50.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:12:50.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:12:50.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:12:50.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:12:50.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:12:50.822 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.822 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.822 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.823 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:50.823 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:50.823 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 84.583994097000001, 2024-09-15T00:12:50.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:12:50.823 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.823 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.823 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:50.823 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.823 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.823 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.823 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:12:50.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.824 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:50.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:50.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:12:50.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:12:50.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:12:50.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:12:50.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:12:50.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:12:50.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:12:50.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:12:50.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:12:50.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:12:50.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:12:50.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:12:50.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:50.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:12:50.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:12:50.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:50.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:12:50.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:12:50.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:50.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:12:50.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:12:50.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:50.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:12:50.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:12:50.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:12:50.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:12:50.829 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.830 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.830 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.830 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:12:50.830 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:50.830 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 84.570532451999995, 2024-09-15T00:12:50.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:12:50.830 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.830 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.830 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.831 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:50.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:50.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:12:50.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:12:50.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:12:50.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:12:50.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:12:50.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:12:50.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:12:50.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:12:50.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:12:50.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:12:50.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:12:50.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:12:50.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:12:50.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:12:50.834 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.835 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.835 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.835 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:50.835 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:50.835 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 75.385530969000001, 2024-09-15T00:12:50.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:12:50.835 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.835 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.835 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:50.836 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.836 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.836 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.836 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:12:50.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.836 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:50.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:50.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:12:50.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:12:50.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:12:50.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:12:50.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:12:50.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:12:50.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:12:50.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:12:50.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:12:50.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:12:50.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:12:50.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:12:50.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:50.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:12:50.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:12:50.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:50.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:12:50.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:12:50.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:50.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:12:50.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:12:50.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:50.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:12:50.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:12:50.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:12:50.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:12:50.842 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.842 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.842 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.842 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:12:50.842 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:50.842 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 75.372654702000005, 2024-09-15T00:12:50.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:12:50.842 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.843 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.843 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.843 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:50.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:50.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:12:50.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:12:50.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:12:50.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:12:50.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:12:50.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:12:50.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:12:50.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:12:50.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:12:50.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:12:50.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:12:50.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:12:50.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:12:50.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:12:50.847 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.847 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.847 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.847 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:12:50.847 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:50.847 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.914139558000002, 2024-09-15T00:12:50.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:12:50.847 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.847 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.847 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:50.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.848 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:12:50.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.848 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:50.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:50.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:12:50.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:50.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:12:50.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:12:50.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:12:50.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:12:50.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:12:50.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:12:50.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:12:50.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:12:50.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:12:50.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:12:50.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:50.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:12:50.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:12:50.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:50.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:12:50.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:12:50.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:50.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:12:50.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:12:50.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:50.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:12:50.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:12:50.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:12:50.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:12:50.854 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.854 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.854 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.854 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:50.854 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:50.855 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 66.196628997999994, 2024-09-15T00:12:50.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:12:50.855 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.855 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.855 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.855 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.855 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.855 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.855 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:12:50.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.855 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:50.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:50.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:12:50.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:12:50.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:12:50.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:12:50.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:12:50.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:12:50.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:12:50.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:12:50.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:12:50.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:12:50.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:12:50.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:12:50.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:12:50.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:12:50.859 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.859 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.859 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.860 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:50.860 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:50.860 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 66.196610389, 2024-09-15T00:12:50.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:12:50.860 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.860 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.860 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.860 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.860 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.860 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.860 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:12:50.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.861 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:50.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:50.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:50.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:50.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:12:50.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:12:50.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:12:50.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:12:50.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:12:50.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:12:50.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:12:50.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:12:50.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:12:50.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:12:50.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:12:50.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:12:50.865 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.865 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.865 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.865 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:50.865 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:50.865 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 66.196585064999994, 2024-09-15T00:12:50.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:12:50.865 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.865 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.865 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.866 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:12:50.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.866 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:50.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:50.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:50.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:12:50.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:12:50.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:12:50.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:12:50.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:12:50.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:12:50.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:12:50.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:12:50.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:12:50.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:12:50.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:12:50.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:12:50.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:12:50.870 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.870 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.870 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.870 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:50.870 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:50.870 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 66.196566740999998, 2024-09-15T00:12:50.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:12:50.870 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.871 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.871 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.871 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:12:50.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.871 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:50.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:50.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:12:50.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:12:50.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:12:50.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:12:50.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:12:50.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:50.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:12:50.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:12:50.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:12:50.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:12:50.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:12:50.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:12:50.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:12:50.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:12:50.875 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.875 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.875 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.876 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:50.876 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:50.876 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 66.196384167999994, 2024-09-15T00:12:50.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:12:50.876 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.876 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.876 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.876 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.876 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.876 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.876 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:12:50.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.877 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:50.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:50.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:50.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:12:50.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:12:50.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:12:50.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:12:50.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:12:50.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:12:50.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:50.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:12:50.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:12:50.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:12:50.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:12:50.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:12:50.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:12:50.881 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.881 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.881 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.881 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:50.881 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:50.881 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 66.196355327000006, 2024-09-15T00:12:50.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:12:50.881 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.881 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.882 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.882 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:12:50.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.882 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:50.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:50.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:50.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:12:50.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:12:50.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:12:50.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:12:50.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:12:50.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:12:50.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:12:50.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:12:50.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:12:50.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:12:50.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:12:50.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:12:50.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:12:50.886 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.886 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.887 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.887 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:50.887 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:50.887 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 66.196333116000005, 2024-09-15T00:12:50.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:12:50.887 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.887 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.887 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.887 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.888 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.888 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.888 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:12:50.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.888 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:50.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:12:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:12:50.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:12:50.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:12:50.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:12:50.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:12:50.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:50.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:12:50.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:12:50.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:12:50.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:12:50.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:12:50.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:12:50.892 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.892 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.892 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.892 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:50.892 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:50.892 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 66.196311957000006, 2024-09-15T00:12:50.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:12:50.893 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.893 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.893 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.893 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.893 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.893 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.893 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:12:50.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.893 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:50.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:50.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:50.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:50.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:12:50.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:12:50.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:12:50.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:12:50.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:12:50.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:12:50.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:12:50.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:12:50.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:12:50.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:12:50.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:12:50.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:12:50.897 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.897 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.897 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.898 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:50.898 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:50.898 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 66.196282009000001, 2024-09-15T00:12:50.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:12:50.898 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.898 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.898 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.898 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.898 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.898 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.898 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:12:50.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.899 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:50.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:50.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:50.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:12:50.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:12:50.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:12:50.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:12:50.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:12:50.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:12:50.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:12:50.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:12:50.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:12:50.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:12:50.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:12:50.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:12:50.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:12:50.903 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.903 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.903 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.903 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:50.903 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:50.904 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 66.196260045000002, 2024-09-15T00:12:50.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:12:50.904 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.904 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.904 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.904 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:12:50.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.904 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:50.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:50.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:50.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:12:50.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:12:50.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:12:50.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:12:50.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:12:50.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:12:50.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:12:50.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:12:50.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:12:50.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:12:50.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:12:50.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:12:50.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:12:50.908 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.908 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.908 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.909 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:50.909 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:50.909 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 66.196242617999999, 2024-09-15T00:12:50.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:12:50.909 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.909 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.909 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.909 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.909 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.909 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.909 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:12:50.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.910 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:50.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:50.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:12:50.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:12:50.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:12:50.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:12:50.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:12:50.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:50.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:12:50.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:12:50.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:12:50.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:12:50.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:12:50.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:12:50.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:12:50.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:12:50.915 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.915 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.915 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.915 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:50.915 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:50.916 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 66.196220819000004, 2024-09-15T00:12:50.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:12:50.916 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.916 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.916 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:50.916 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.916 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.916 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.916 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:12:50.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.916 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:50.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:50.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:12:50.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:12:50.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:12:50.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:12:50.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:12:50.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:12:50.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:12:50.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:12:50.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:12:50.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:12:50.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:12:50.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:12:50.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:12:50.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:12:50.920 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.920 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.921 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.921 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:50.921 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:50.921 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 66.146992460000007, 2024-09-15T00:12:50.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:12:50.921 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:50.921 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:50.921 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:50.921 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:50.921 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:50.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:50.922 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:12:50.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.922 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:50.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:50.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:50.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:50.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:50.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:50.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:50.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:12:50.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:12:50.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:50.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:12:50.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:12:50.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:50.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:12:50.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:12:50.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:50.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:12:50.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:12:50.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:50.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:12:50.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:12:50.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:50.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:12:50.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:12:50.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:50.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:12:50.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:12:50.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:50.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:12:50.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:12:50.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:50.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:12:50.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:12:50.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:50.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:12:50.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:12:50.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:50.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:50.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:50.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:12:50.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:12:50.927 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.928 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.928 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.928 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:50.928 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:50.928 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:50.929 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:50.929 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:50.930 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:50.931 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:50.927+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:50.931 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:50.932 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:12:50.932 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:50.933 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:12:51.030 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:51.028+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:51.073 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is unset 2024-09-15T00:12:51.080 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:51.080 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:51.093 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:51.130 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:51.129+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:51.145 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone46 from 0xeb29184 (overlap 0xc8442) to 0x0 2024-09-15T00:12:51.230 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-15T00:12:51.231 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:51.230+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:51.243 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:51.243 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:51.258 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:51.332 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:51.330+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:51.432 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:51.431+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:51.463 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:51.463 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:51.474 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:51.533 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:51.532+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:51.598 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:51.599 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:51.608 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:51.622 INFO:teuthology.orchestra.run.smithi171.stdout:5534 read 0x23794dc thru 0x2384e04 (0xb929 bytes) 2024-09-15T00:12:51.624 INFO:teuthology.orchestra.run.smithi171.stdout:5535 punch from 0xb45e6fa to 0xb45ee06, (0x70c bytes) 2024-09-15T00:12:51.629 INFO:teuthology.orchestra.run.smithi171.stdout:5538 writesame 0x4a6a1aa thru 0x4a71912 data_size 0xb(0x7769 bytes) 2024-09-15T00:12:51.635 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:51.633+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:51.635 INFO:teuthology.orchestra.run.smithi171.stdout:5539 writesame 0x912c35f thru 0x91328b9 data_size 0x1f(0x655b bytes) 2024-09-15T00:12:51.639 INFO:teuthology.orchestra.run.smithi171.stdout:5540 write 0x6f0238a thru 0x6f07ead (0x5b24 bytes) 2024-09-15T00:12:51.643 INFO:teuthology.orchestra.run.smithi171.stdout:5542 punch from 0xd668b5e to 0xd66eb95, (0x6037 bytes) 2024-09-15T00:12:51.644 INFO:teuthology.orchestra.run.smithi171.stdout:5543 read 0x9bb6066 thru 0x9bb8637 (0x25d2 bytes) 2024-09-15T00:12:51.644 INFO:teuthology.orchestra.run.smithi171.stdout:5544 writesame 0x6eee3a4 thru 0x6ef6306 data_size 0x1(0x7f63 bytes) 2024-09-15T00:12:51.697 INFO:teuthology.orchestra.run.smithi171.stdout:5545 writesame 0x5d8b4a6 thru 0x5d90d58 data_size 0x1d(0x58b3 bytes) 2024-09-15T00:12:51.715 INFO:teuthology.orchestra.run.smithi171.stdout:5546 trunc from 0xeb29184 to 0x3ce84e2 2024-09-15T00:12:51.735 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:51.734+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:51.736 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:51.736 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:51.747 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:51.755 INFO:teuthology.orchestra.run.smithi171.stdout:5547 trunc from 0x3ce84e2 to 0x4a11b0d 2024-09-15T00:12:51.761 INFO:teuthology.orchestra.run.smithi171.stdout:5548 punch from 0x2b7fdf9 to 0x2b8a96b, (0xab72 bytes) 2024-09-15T00:12:51.778 INFO:teuthology.orchestra.run.smithi171.stdout:5551 write 0xd686ee6 thru 0xd68d37e (0x6499 bytes) 2024-09-15T00:12:51.784 INFO:teuthology.orchestra.run.smithi171.stdout:5552 write 0x91e2a3f thru 0x91ec6a8 (0x9c6a bytes) 2024-09-15T00:12:51.786 INFO:teuthology.orchestra.run.smithi171.stdout:5553 punch from 0xc8ed2c5 to 0xc8ed97f, (0x6ba bytes) 2024-09-15T00:12:51.793 INFO:teuthology.orchestra.run.smithi171.stdout:5555 write 0xe85970a thru 0xe860ff2 (0x78e9 bytes) 2024-09-15T00:12:51.803 INFO:teuthology.orchestra.run.smithi171.stdout:5556 punch from 0xe6f094f to 0xe6fa02f, (0x96e0 bytes) 2024-09-15T00:12:51.803 INFO:teuthology.orchestra.run.smithi171.stdout:5558 write 0x50e71a5 thru 0x50f44a2 (0xd2fe bytes) 2024-09-15T00:12:51.811 INFO:teuthology.orchestra.run.smithi171.stdout:5559 writesame 0x4a95f05 thru 0x4aa5cef data_size 0x1(0xfdeb bytes) 2024-09-15T00:12:51.836 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:51.834+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:51.865 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:51.865 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:51.874 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:51.893 INFO:teuthology.orchestra.run.smithi171.stdout:5560 punch from 0x21a3f6a to 0x21b02e6, (0xc37c bytes) 2024-09-15T00:12:51.893 INFO:teuthology.orchestra.run.smithi171.stdout:5561 read 0x4dc8943 thru 0x4dd2f52 (0xa610 bytes) 2024-09-15T00:12:51.893 INFO:teuthology.orchestra.run.smithi171.stdout:5562 write 0x48e826c thru 0x48f4536 (0xc2cb bytes) 2024-09-15T00:12:51.898 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:51.898 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:51.898 INFO:teuthology.orchestra.run.smithi171.stdout:5566 writesame 0x674174d thru 0x674fa3c data_size 0x8(0xe2f0 bytes) 2024-09-15T00:12:51.916 INFO:teuthology.orchestra.run.smithi171.stdout:5568 trunc from 0xe860ff3 to 0x5d1f9af 2024-09-15T00:12:51.936 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:51.935+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:51.943 INFO:teuthology.orchestra.run.smithi171.stdout:5569 write 0x8d7e402 thru 0x8d8d811 (0xf410 bytes) 2024-09-15T00:12:51.956 INFO:teuthology.orchestra.run.smithi171.stdout:5570 trunc from 0x8d8d812 to 0x603f1b7 2024-09-15T00:12:51.967 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:51.967 INFO:teuthology.orchestra.run.smithi171.stdout:5572 read 0x1491df5 thru 0x14a096c (0xeb78 bytes) 2024-09-15T00:12:51.967 INFO:teuthology.orchestra.run.smithi171.stdout:5573 clone 48 order 25 su 131072 sc 13 2024-09-15T00:12:52.003 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:52.003 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:52.011 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:52.038 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:52.036+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:52.139 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:52.137+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:52.149 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone47 from 0x603f1b7 (overlap 0x3ce84e2) to 0x0 2024-09-15T00:12:52.164 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:52.164 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:52.177 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:52.240 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:52.238+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:52.328 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:52.328 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:52.340 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:52.341 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:52.340+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:52.442 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:52.440+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:52.465 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:52.465 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:52.475 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:52.478 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-09-15T00:12:52.497 INFO:teuthology.orchestra.run.smithi171.stdout:5575 writesame 0xc7c7f4d thru 0xc7cf67d data_size 0x7(0x7731 bytes) 2024-09-15T00:12:52.531 INFO:teuthology.orchestra.run.smithi171.stdout:5576 writesame 0x867d4e8 thru 0x8687f00 data_size 0x5(0xaa19 bytes) 2024-09-15T00:12:52.542 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:52.541+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:52.573 INFO:teuthology.orchestra.run.smithi171.stdout:5577 punch from 0xb1ef34d to 0xb1fc377, (0xd02a bytes) 2024-09-15T00:12:52.573 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:52.573 INFO:teuthology.orchestra.run.smithi171.stdout:5579 punch from 0xa0a293b to 0xa0b19a0, (0xf065 bytes) 2024-09-15T00:12:52.573 INFO:teuthology.orchestra.run.smithi171.stdout:5581 punch from 0x11919cc to 0x119c9f7, (0xb02b bytes) 2024-09-15T00:12:52.574 INFO:teuthology.orchestra.run.smithi171.stdout:5582 read 0x1be6a31 thru 0x1beac99 (0x4269 bytes) 2024-09-15T00:12:52.574 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:52.582 INFO:teuthology.orchestra.run.smithi171.stdout:5585 trunc from 0xc7cf67e to 0x5748cfd 2024-09-15T00:12:52.643 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:52.643 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:52.644 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:52.642+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:52.654 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:52.722 INFO:teuthology.orchestra.run.smithi171.stdout:5586 trunc from 0x5748cfd to 0x72dafff 2024-09-15T00:12:52.743 INFO:teuthology.orchestra.run.smithi171.stdout:5587 write 0xe8a4fd1 thru 0xe8b153e (0xc56e bytes) 2024-09-15T00:12:52.744 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:52.742+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:52.745 INFO:teuthology.orchestra.run.smithi171.stdout:5588 punch from 0x1049b5 to 0x111db0, (0xd3fb bytes) 2024-09-15T00:12:52.745 INFO:teuthology.orchestra.run.smithi171.stdout:5590 read 0x2691e2 thru 0x26fcb2 (0x6ad1 bytes) 2024-09-15T00:12:52.757 INFO:teuthology.orchestra.run.smithi171.stdout:5592 trunc from 0xe8b153f to 0x47d2662 2024-09-15T00:12:52.765 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:52.780 INFO:teuthology.orchestra.run.smithi171.stdout:5594 write 0xbbfb361 thru 0xbc01fdd (0x6c7d bytes) 2024-09-15T00:12:52.785 INFO:teuthology.orchestra.run.smithi171.stdout:5595 trunc from 0xbc01fde to 0xe4f67f0 2024-09-15T00:12:52.791 INFO:teuthology.orchestra.run.smithi171.stdout:5597 read 0x7d1ab03 thru 0x7d216a2 (0x6ba0 bytes) 2024-09-15T00:12:52.791 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:52.795 INFO:teuthology.orchestra.run.smithi171.stdout:5599 trunc from 0xe4f67f0 to 0xb9da023 2024-09-15T00:12:52.802 INFO:teuthology.orchestra.run.smithi171.stdout:5600 read 0x180f48b thru 0x181566d (0x61e3 bytes) 2024-09-15T00:12:52.802 INFO:teuthology.orchestra.run.smithi171.stdout:5602 read 0x8dd26a2 thru 0x8dd3581 (0xee0 bytes) 2024-09-15T00:12:52.803 INFO:teuthology.orchestra.run.smithi171.stdout:5603 writesame 0xaf9a88f thru 0xafa7673 data_size 0x1(0xcde5 bytes) 2024-09-15T00:12:52.804 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:52.805 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:52.813 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:52.855 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:52.853+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:52.874 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:52.874 INFO:teuthology.orchestra.run.smithi171.stdout:5605 read 0x6a4fd9c thru 0x6a5fa4a (0xfcaf bytes) 2024-09-15T00:12:52.874 INFO:teuthology.orchestra.run.smithi171.stdout:5606 read 0x3d0fa7e thru 0x3d1943f (0x99c2 bytes) 2024-09-15T00:12:52.875 INFO:teuthology.orchestra.run.smithi171.stdout:5607 punch from 0xb95fe16 to 0xb96473f, (0x4929 bytes) 2024-09-15T00:12:52.875 INFO:teuthology.orchestra.run.smithi171.stdout:5608 writesame 0x21eac9b thru 0x21f1ad5 data_size 0x1(0x6e3b bytes) 2024-09-15T00:12:52.924 INFO:teuthology.orchestra.run.smithi171.stdout:5609 write 0x4ca24e6 thru 0x4ca8f07 (0x6a22 bytes) 2024-09-15T00:12:52.929 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:52.929 INFO:teuthology.orchestra.run.smithi171.stdout:5614 read 0x166b718 thru 0x1678d82 (0xd66b bytes) 2024-09-15T00:12:52.929 INFO:teuthology.orchestra.run.smithi171.stdout:5615 write 0xf357bb thru 0xf43836 (0xe07c bytes) 2024-09-15T00:12:52.931 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:52.931 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:52.931 INFO:teuthology.orchestra.run.smithi171.stdout:5619 read 0x12ab94b thru 0x12ad1d5 (0x188b bytes) 2024-09-15T00:12:52.931 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:52.932 INFO:teuthology.orchestra.run.smithi171.stdout:5622 write 0x3dfbc6b thru 0x3e0bc66 (0xfffc bytes) 2024-09-15T00:12:52.933 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:12:52.936 INFO:teuthology.orchestra.run.smithi171.stdout:5624 writesame 0x86bb9fe thru 0x86c0a70 data_size 0x5(0x5073 bytes) 2024-09-15T00:12:52.938 INFO:teuthology.orchestra.run.smithi171.stdout:5626 punch from 0x10c42a to 0x11a452, (0xe028 bytes) 2024-09-15T00:12:52.938 INFO:teuthology.orchestra.run.smithi171.stdout:5627 read 0x74812d4 thru 0x748e23f (0xcf6c bytes) 2024-09-15T00:12:52.944 INFO:teuthology.orchestra.run.smithi171.stdout:5629 write 0xc25f8f0 thru 0xc268d1f (0x9430 bytes) 2024-09-15T00:12:52.945 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:52.944+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:52.946 INFO:teuthology.orchestra.run.smithi171.stdout:5630 write 0x5907e7b thru 0x590c602 (0x4788 bytes) 2024-09-15T00:12:52.960 INFO:teuthology.orchestra.run.smithi171.stdout:5632 trunc from 0xc268d20 to 0x2928ab3 2024-09-15T00:12:52.966 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:52.966 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:52.976 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:52.982 INFO:teuthology.orchestra.run.smithi171.stdout:5634 write 0x5913a93 thru 0x591dd42 (0xa2b0 bytes) 2024-09-15T00:12:52.984 INFO:teuthology.orchestra.run.smithi171.stdout:5635 read 0x30afaa4 thru 0x30b1300 (0x185d bytes) 2024-09-15T00:12:52.987 INFO:teuthology.orchestra.run.smithi171.stdout:5636 trunc from 0x591dd43 to 0x7b034d6 2024-09-15T00:12:52.992 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:52.995 INFO:teuthology.orchestra.run.smithi171.stdout:5640 trunc from 0x7b034d6 to 0x9b69deb 2024-09-15T00:12:53.000 INFO:teuthology.orchestra.run.smithi171.stdout:5641 read 0x516d1c1 thru 0x516f827 (0x2667 bytes) 2024-09-15T00:12:53.000 INFO:teuthology.orchestra.run.smithi171.stdout:5642 read 0x1e17532 thru 0x1e22de0 (0xb8af bytes) 2024-09-15T00:12:53.001 INFO:teuthology.orchestra.run.smithi171.stdout:5644 clone 49 order 21 su 131072 sc 12 2024-09-15T00:12:53.046 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:53.045+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:53.071 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:53.071 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:53.090 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:53.091 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:53.091 INFO:teuthology.orchestra.run.smithi171.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T00:12:53.091 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:12:53.109 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:53.109 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:53.126 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:53.126 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:53.127 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:53.127 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:53.136 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:53.144 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:53.145 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:53.147 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:53.145+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:53.163 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:53.163 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:53.170 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:12:53.179 INFO:teuthology.orchestra.run.smithi171.stdout:flattening image image_client.0-clone48 2024-09-15T00:12:53.182 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:53.182 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:53.185 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:12:53.186 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:12:53.244 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:53.244 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:53.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:53.244 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:53.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.244 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48962 3.17 3:ea893c28:::rbd_data.11395b9829f3.0000000000000003:e [sparse-read 13631488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:53.244 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:53.245 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.632416268, 2024-09-15T00:12:53.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045312458, 2024-09-15T00:12:53.245 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.245 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.245 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.245 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.245 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.245 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.245 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48962 2024-09-15T00:12:53.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.245 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:53.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609161+0000", 2024-09-15T00:12:53.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609162+0000", 2024-09-15T00:12:53.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:53.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609164+0000", 2024-09-15T00:12:53.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0099999999999998e-06 2024-09-15T00:12:53.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609171+0000", 2024-09-15T00:12:53.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7530000000000001e-06 2024-09-15T00:12:53.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609178+0000", 2024-09-15T00:12:53.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1709999999999999e-06 2024-09-15T00:12:53.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654365+0000", 2024-09-15T00:12:53.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045187867999999999 2024-09-15T00:12:53.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654420+0000", 2024-09-15T00:12:53.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4376000000000003e-05 2024-09-15T00:12:53.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654474+0000", 2024-09-15T00:12:53.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3879999999999999e-05 2024-09-15T00:12:53.249 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.249 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.249 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.249 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48971 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:53.249 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:53.250 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.632385274000001, 2024-09-15T00:12:53.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045896151000000003, 2024-09-15T00:12:53.250 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.250 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.250 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.250 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.250 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.250 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.250 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48971 2024-09-15T00:12:53.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.250 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:53.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609192+0000", 2024-09-15T00:12:53.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609193+0000", 2024-09-15T00:12:53.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7599999999999997e-07 2024-09-15T00:12:53.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609195+0000", 2024-09-15T00:12:53.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:12:53.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609210+0000", 2024-09-15T00:12:53.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4956999999999999e-05 2024-09-15T00:12:53.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609505+0000", 2024-09-15T00:12:53.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000295078 2024-09-15T00:12:53.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654498+0000", 2024-09-15T00:12:53.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044992277999999997 2024-09-15T00:12:53.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.654541+0000", 2024-09-15T00:12:53.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3525000000000001e-05 2024-09-15T00:12:53.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655088+0000", 2024-09-15T00:12:53.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054726799999999995 2024-09-15T00:12:53.254 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.254 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.254 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.254 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48972 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 2097152~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:53.254 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:53.254 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.632058700000002, 2024-09-15T00:12:53.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046150885000000003, 2024-09-15T00:12:53.254 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.254 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.254 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.254 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.255 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.255 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.255 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48972 2024-09-15T00:12:53.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.255 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:53.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:53.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609519+0000", 2024-09-15T00:12:53.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6400000000000002e-07 2024-09-15T00:12:53.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609523+0000", 2024-09-15T00:12:53.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5300000000000001e-06 2024-09-15T00:12:53.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609533+0000", 2024-09-15T00:12:53.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0023999999999999e-05 2024-09-15T00:12:53.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609539+0000", 2024-09-15T00:12:53.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3119999999999999e-06 2024-09-15T00:12:53.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655115+0000", 2024-09-15T00:12:53.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045576094999999997 2024-09-15T00:12:53.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655197+0000", 2024-09-15T00:12:53.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1129000000000002e-05 2024-09-15T00:12:53.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655670+0000", 2024-09-15T00:12:53.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00047313100000000001 2024-09-15T00:12:53.258 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.258 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.258 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.259 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:53.259 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:53.259 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.632030438000001, 2024-09-15T00:12:53.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:12:53.259 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.259 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.259 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.259 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.259 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.259 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.259 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:12:53.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.260 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:53.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:53.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:53.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:12:53.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:12:53.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:12:53.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:12:53.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:12:53.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:12:53.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:12:53.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:12:53.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:12:53.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:12:53.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:12:53.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:12:53.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:12:53.263 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.263 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.263 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.263 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:53.263 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:53.263 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.632009209000003, 2024-09-15T00:12:53.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:12:53.263 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.264 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.264 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.264 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.264 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.264 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.264 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:12:53.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.264 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:53.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:53.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:12:53.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:12:53.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:12:53.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:12:53.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:12:53.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:12:53.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:12:53.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:12:53.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:12:53.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:12:53.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:12:53.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:12:53.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:12:53.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:12:53.267 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.267 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.267 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.268 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:53.268 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:53.268 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.631983668000004, 2024-09-15T00:12:53.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:12:53.268 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.268 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.268 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.268 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.268 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.268 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.268 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:12:53.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.268 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:53.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:53.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:53.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:12:53.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:12:53.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:12:53.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:12:53.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:12:53.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:12:53.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:12:53.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:12:53.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:12:53.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:12:53.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:12:53.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:12:53.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:12:53.272 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.272 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.272 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.272 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:53.272 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:53.272 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.631961192999995, 2024-09-15T00:12:53.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:12:53.272 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.272 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.272 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.272 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.273 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.273 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.273 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:12:53.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.273 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:53.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:53.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:12:53.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:12:53.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:12:53.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:12:53.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:12:53.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:12:53.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:12:53.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:12:53.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:12:53.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:12:53.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.276 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:12:53.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:12:53.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.276 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:12:53.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:12:53.276 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.276 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.276 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.276 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:53.277 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:53.277 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.631940682999996, 2024-09-15T00:12:53.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:12:53.277 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.277 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.277 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.277 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.277 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.277 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.277 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:12:53.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.277 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:53.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:53.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:53.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:12:53.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:12:53.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:12:53.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:12:53.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:12:53.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:12:53.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:12:53.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:12:53.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:12:53.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:12:53.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:12:53.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:12:53.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:12:53.281 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.281 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.281 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.281 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:53.281 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:53.281 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.631920367999996, 2024-09-15T00:12:53.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:12:53.281 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.281 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.281 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.281 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.281 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.282 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.282 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:12:53.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.282 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:53.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:53.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:12:53.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:12:53.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:12:53.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:12:53.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:12:53.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:12:53.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:12:53.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:12:53.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:12:53.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:12:53.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:12:53.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:12:53.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:12:53.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:12:53.285 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.285 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.285 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.285 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:53.285 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:53.285 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.631895592000006, 2024-09-15T00:12:53.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:12:53.286 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.286 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.286 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.286 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.286 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.286 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.286 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:12:53.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.286 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:53.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:53.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:53.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:12:53.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:12:53.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:12:53.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:12:53.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:12:53.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:12:53.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:12:53.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:12:53.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:12:53.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:12:53.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:12:53.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:12:53.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:12:53.290 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.290 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.290 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.290 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:53.290 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:53.290 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.452554204999998, 2024-09-15T00:12:53.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:12:53.290 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.290 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.290 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:53.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.291 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:12:53.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.291 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:53.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:53.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:12:53.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:12:53.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:12:53.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:12:53.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:12:53.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:12:53.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:12:53.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:12:53.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:12:53.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:12:53.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:12:53.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:12:53.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:53.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:12:53.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:12:53.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:53.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:12:53.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:12:53.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:53.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:12:53.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:12:53.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:53.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:12:53.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:12:53.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:12:53.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:12:53.296 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.296 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.296 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.296 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:12:53.296 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:53.296 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.435600753000003, 2024-09-15T00:12:53.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:12:53.297 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.297 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.297 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.297 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:53.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:12:53.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:12:53.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:12:53.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:12:53.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:12:53.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:12:53.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:12:53.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:12:53.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:12:53.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:12:53.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:12:53.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:12:53.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:12:53.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:12:53.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:12:53.300 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.300 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.300 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.300 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:12:53.300 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:53.301 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.141169845999997, 2024-09-15T00:12:53.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:12:53.301 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.301 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.301 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:53.301 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.301 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.301 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.301 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:12:53.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.301 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:53.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:12:53.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:12:53.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:12:53.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:12:53.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:12:53.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:12:53.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:12:53.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:12:53.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:12:53.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:12:53.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:12:53.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:12:53.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:12:53.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:53.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:12:53.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:12:53.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:53.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:12:53.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:12:53.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:53.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:12:53.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:12:53.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:53.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:12:53.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:12:53.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:12:53.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:12:53.306 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.306 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.306 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.307 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:12:53.307 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:53.307 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.128714031999998, 2024-09-15T00:12:53.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:12:53.307 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.307 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.307 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.307 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:53.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:12:53.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:12:53.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:12:53.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:12:53.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:12:53.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:12:53.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:12:53.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:12:53.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:12:53.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.311 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:146935 3.16 3:6c79e092:::rbd_data.11394501364.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:12:53.311 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:12:53.311 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4879345139999991, 2024-09-15T00:12:53.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.058987488999999997, 2024-09-15T00:12:53.311 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.311 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.311 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:53.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.311 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 146935 2024-09-15T00:12:53.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.311 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:12:53.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:12:53.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753644+0000", 2024-09-15T00:12:53.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6199999999999997e-07 2024-09-15T00:12:53.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753648+0000", 2024-09-15T00:12:53.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-15T00:12:53.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753659+0000", 2024-09-15T00:12:53.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0703000000000001e-05 2024-09-15T00:12:53.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753664+0000", 2024-09-15T00:12:53.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.04e-06 2024-09-15T00:12:53.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753709+0000", 2024-09-15T00:12:53.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4728000000000003e-05 2024-09-15T00:12:53.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753756+0000", 2024-09-15T00:12:53.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7808e-05 2024-09-15T00:12:53.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:53.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.757051+0000", 2024-09-15T00:12:53.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294249 2024-09-15T00:12:53.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:53.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812553+0000", 2024-09-15T00:12:53.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055501807 2024-09-15T00:12:53.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:53.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812564+0000", 2024-09-15T00:12:53.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1467999999999999e-05 2024-09-15T00:12:53.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:53.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812598+0000", 2024-09-15T00:12:53.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3553000000000002e-05 2024-09-15T00:12:53.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812630+0000", 2024-09-15T00:12:53.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2895000000000003e-05 2024-09-15T00:12:53.316 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.316 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.316 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:146935 3.16 e98/95 ondisk, result = 0)", 2024-09-15T00:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4771318180000002, 2024-09-15T00:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048111830000000001, 2024-09-15T00:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:12:53.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764447+0000", 2024-09-15T00:12:53.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0500000000000002e-07 2024-09-15T00:12:53.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764450+0000", 2024-09-15T00:12:53.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4400000000000001e-06 2024-09-15T00:12:53.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764462+0000", 2024-09-15T00:12:53.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1603000000000001e-05 2024-09-15T00:12:53.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764469+0000", 2024-09-15T00:12:53.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5089999999999996e-06 2024-09-15T00:12:53.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812541+0000", 2024-09-15T00:12:53.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048071585 2024-09-15T00:12:53.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812548+0000", 2024-09-15T00:12:53.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9979999999999997e-06 2024-09-15T00:12:53.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812557+0000", 2024-09-15T00:12:53.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7899999999999994e-06 2024-09-15T00:12:53.320 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.320 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.320 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.321 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:152505 3.17 3:ea3f268c:::rbd_data.113958cb0e55.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:12:53.321 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:12:53.321 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.5750258559999999, 2024-09-15T00:12:53.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.060817959999999997, 2024-09-15T00:12:53.321 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.321 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.321 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:53.321 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.321 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.321 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.321 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 152505 2024-09-15T00:12:53.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.321 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:12:53.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:12:53.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666553+0000", 2024-09-15T00:12:53.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-15T00:12:53.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666562+0000", 2024-09-15T00:12:53.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0909999999999999e-06 2024-09-15T00:12:53.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666572+0000", 2024-09-15T00:12:53.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0214000000000001e-05 2024-09-15T00:12:53.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666577+0000", 2024-09-15T00:12:53.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3800000000000002e-06 2024-09-15T00:12:53.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666617+0000", 2024-09-15T00:12:53.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9304000000000002e-05 2024-09-15T00:12:53.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666672+0000", 2024-09-15T00:12:53.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5206e-05 2024-09-15T00:12:53.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:53.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.670821+0000", 2024-09-15T00:12:53.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041493169999999996 2024-09-15T00:12:53.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:53.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727290+0000", 2024-09-15T00:12:53.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.056468580999999997 2024-09-15T00:12:53.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:53.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727302+0000", 2024-09-15T00:12:53.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2605e-05 2024-09-15T00:12:53.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727337+0000", 2024-09-15T00:12:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4783999999999998e-05 2024-09-15T00:12:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727370+0000", 2024-09-15T00:12:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2366000000000001e-05 2024-09-15T00:12:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:152505 3.17 e103/68 ondisk, result = 0)", 2024-09-15T00:12:53.326 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:12:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.56294742, 2024-09-15T00:12:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048665938999999998, 2024-09-15T00:12:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:12:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:12:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678631+0000", 2024-09-15T00:12:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5099999999999998e-07 2024-09-15T00:12:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678634+0000", 2024-09-15T00:12:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.478e-06 2024-09-15T00:12:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678647+0000", 2024-09-15T00:12:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2085e-05 2024-09-15T00:12:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678653+0000", 2024-09-15T00:12:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2099999999999998e-06 2024-09-15T00:12:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727277+0000", 2024-09-15T00:12:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048624405000000002 2024-09-15T00:12:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727285+0000", 2024-09-15T00:12:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2060000000000007e-06 2024-09-15T00:12:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727296+0000", 2024-09-15T00:12:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0604e-05 2024-09-15T00:12:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153959 3.c 3:326b8454:::rbd_data.113958cb0e55.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:12:53.330 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:12:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.1517194740000001, 2024-09-15T00:12:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.068102041000000002, 2024-09-15T00:12:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153959 2024-09-15T00:12:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:12:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:12:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089859+0000", 2024-09-15T00:12:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1800000000000004e-07 2024-09-15T00:12:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089864+0000", 2024-09-15T00:12:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2079999999999999e-06 2024-09-15T00:12:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089880+0000", 2024-09-15T00:12:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5695000000000001e-05 2024-09-15T00:12:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089889+0000", 2024-09-15T00:12:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.935e-06 2024-09-15T00:12:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089942+0000", 2024-09-15T00:12:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3219e-05 2024-09-15T00:12:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.090066+0000", 2024-09-15T00:12:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012385099999999999 2024-09-15T00:12:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.095262+0000", 2024-09-15T00:12:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051963950000000004 2024-09-15T00:12:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157865+0000", 2024-09-15T00:12:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.062602865999999993 2024-09-15T00:12:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157891+0000", 2024-09-15T00:12:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5778e-05 2024-09-15T00:12:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157924+0000", 2024-09-15T00:12:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3133000000000003e-05 2024-09-15T00:12:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157960+0000", 2024-09-15T00:12:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6043e-05 2024-09-15T00:12:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:153959 3.c e103/23 ondisk, result = 0)", 2024-09-15T00:12:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:12:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.1390638160000002, 2024-09-15T00:12:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055370211000000003, 2024-09-15T00:12:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.336 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:12:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:12:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102515+0000", 2024-09-15T00:12:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.206e-06 2024-09-15T00:12:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102520+0000", 2024-09-15T00:12:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1209999999999998e-06 2024-09-15T00:12:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102537+0000", 2024-09-15T00:12:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:12:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102546+0000", 2024-09-15T00:12:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2329999999999995e-06 2024-09-15T00:12:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157847+0000", 2024-09-15T00:12:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055300798999999998 2024-09-15T00:12:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157857+0000", 2024-09-15T00:12:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.046e-05 2024-09-15T00:12:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157884+0000", 2024-09-15T00:12:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.671e-05 2024-09-15T00:12:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.340 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.340 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:53.342 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T00:12:53.342 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:53.343 INFO:teuthology.orchestra.run.smithi171.stderr:no valid command found; 10 closest matches: 2024-09-15T00:12:53.343 INFO:teuthology.orchestra.run.smithi171.stderr:0 2024-09-15T00:12:53.343 INFO:teuthology.orchestra.run.smithi171.stderr:1 2024-09-15T00:12:53.343 INFO:teuthology.orchestra.run.smithi171.stderr:2 2024-09-15T00:12:53.343 INFO:teuthology.orchestra.run.smithi171.stderr:abort 2024-09-15T00:12:53.343 INFO:teuthology.orchestra.run.smithi171.stderr:assert 2024-09-15T00:12:53.343 INFO:teuthology.orchestra.run.smithi171.stderr:config diff 2024-09-15T00:12:53.344 INFO:teuthology.orchestra.run.smithi171.stderr:config diff get 2024-09-15T00:12:53.344 INFO:teuthology.orchestra.run.smithi171.stderr:config get 2024-09-15T00:12:53.344 INFO:teuthology.orchestra.run.smithi171.stderr:config help [] 2024-09-15T00:12:53.344 INFO:teuthology.orchestra.run.smithi171.stderr:config set ... 2024-09-15T00:12:53.344 INFO:teuthology.orchestra.run.smithi171.stderr:admin_socket: invalid command 2024-09-15T00:12:53.344 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-15T00:12:53.344 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #47, image image_client.0-clone47 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent48 2024-09-15T00:12:53.345 INFO:teuthology.orchestra.run.smithi171.stdout:5645 write 0x518cfa2 thru 0x51922dc (0x533b bytes) 2024-09-15T00:12:53.345 INFO:teuthology.orchestra.run.smithi171.stdout:5646 trunc from 0x9b69deb to 0x225a941 2024-09-15T00:12:53.345 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:53.340+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:53.379 INFO:teuthology.orchestra.run.smithi171.stdout:5647 trunc from 0x225a941 to 0x490100a 2024-09-15T00:12:53.393 INFO:teuthology.orchestra.run.smithi171.stdout:5648 write 0x725a071 thru 0x7262355 (0x82e5 bytes) 2024-09-15T00:12:53.414 INFO:teuthology.orchestra.run.smithi171.stdout:5649 writesame 0xc8b153b thru 0xc8b3342 data_size 0x8(0x1e08 bytes) 2024-09-15T00:12:53.415 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:53.415 INFO:teuthology.orchestra.run.smithi171.stdout:5652 punch from 0xb894a76 to 0xb89eead, (0xa437 bytes) 2024-09-15T00:12:53.415 INFO:teuthology.orchestra.run.smithi171.stdout:5654 writesame 0x5f98146 thru 0x5fa0950 data_size 0x39(0x880b bytes) 2024-09-15T00:12:53.424 INFO:teuthology.orchestra.run.smithi171.stdout:5655 trunc from 0xc8b3343 to 0xb152e8c 2024-09-15T00:12:53.428 INFO:teuthology.orchestra.run.smithi171.stdout:5656 punch from 0x68db6ac to 0x68e3861, (0x81b5 bytes) 2024-09-15T00:12:53.436 INFO:teuthology.orchestra.run.smithi171.stdout:5657 trunc from 0xb152e8c to 0x4d16813 2024-09-15T00:12:53.442 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:53.440+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:53.453 INFO:teuthology.orchestra.run.smithi171.stdout:5658 flatten 2024-09-15T00:12:53.457 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:53.457 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:53.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.457 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:162975 3.0 3:03827783:::rbd_data.1139e085a23a.0000000000000008:33 [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:53.457 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.453614+0000", 2024-09-15T00:12:53.457 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.001075412, 2024-09-15T00:12:53.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0011102849999999999, 2024-09-15T00:12:53.458 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.458 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.458 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.458 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.458 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.458 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.458 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 162975 2024-09-15T00:12:53.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.459 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453614+0000", 2024-09-15T00:12:53.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453614+0000", 2024-09-15T00:12:53.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453614+0000", 2024-09-15T00:12:53.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7000000000000005e-07 2024-09-15T00:12:53.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453617+0000", 2024-09-15T00:12:53.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7149999999999998e-06 2024-09-15T00:12:53.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453624+0000", 2024-09-15T00:12:53.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0559999999999999e-06 2024-09-15T00:12:53.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453629+0000", 2024-09-15T00:12:53.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4700000000000001e-06 2024-09-15T00:12:53.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454584+0000", 2024-09-15T00:12:53.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00095442200000000002 2024-09-15T00:12:53.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454655+0000", 2024-09-15T00:12:53.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1219999999999999e-05 2024-09-15T00:12:53.463 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.463 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.463 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.463 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:162992 3.0 3.c1ee41c0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:53.463 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.453793+0000", 2024-09-15T00:12:53.463 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00089617899999999999, 2024-09-15T00:12:53.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001018857, 2024-09-15T00:12:53.463 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.463 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.463 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-15T00:12:53.464 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.464 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.464 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.464 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 162992 2024-09-15T00:12:53.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.464 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453793+0000", 2024-09-15T00:12:53.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453793+0000", 2024-09-15T00:12:53.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453794+0000", 2024-09-15T00:12:53.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4600000000000004e-07 2024-09-15T00:12:53.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453797+0000", 2024-09-15T00:12:53.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8099999999999999e-06 2024-09-15T00:12:53.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453806+0000", 2024-09-15T00:12:53.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.183e-06 2024-09-15T00:12:53.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453812+0000", 2024-09-15T00:12:53.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6380000000000001e-06 2024-09-15T00:12:53.467 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.467 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.467 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.467 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163000 3.10 3.abab8030 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:53.467 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.453882+0000", 2024-09-15T00:12:53.467 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00080688199999999998, 2024-09-15T00:12:53.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0010033889999999999, 2024-09-15T00:12:53.467 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.467 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.467 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-15T00:12:53.467 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.468 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.468 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.468 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163000 2024-09-15T00:12:53.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.468 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453882+0000", 2024-09-15T00:12:53.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453882+0000", 2024-09-15T00:12:53.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453883+0000", 2024-09-15T00:12:53.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0270000000000001e-06 2024-09-15T00:12:53.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453888+0000", 2024-09-15T00:12:53.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3580000000000002e-06 2024-09-15T00:12:53.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453901+0000", 2024-09-15T00:12:53.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2833000000000001e-05 2024-09-15T00:12:53.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453909+0000", 2024-09-15T00:12:53.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0409999999999998e-06 2024-09-15T00:12:53.470 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.470 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.471 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.471 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163001 3.10 3.abab8030 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:53.471 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.453923+0000", 2024-09-15T00:12:53.471 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00076626299999999999, 2024-09-15T00:12:53.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0010222390000000001, 2024-09-15T00:12:53.471 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.471 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.472 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-15T00:12:53.472 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.472 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.472 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.472 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163001 2024-09-15T00:12:53.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.472 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453923+0000", 2024-09-15T00:12:53.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453923+0000", 2024-09-15T00:12:53.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453923+0000", 2024-09-15T00:12:53.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9200000000000001e-07 2024-09-15T00:12:53.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453932+0000", 2024-09-15T00:12:53.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8049999999999996e-06 2024-09-15T00:12:53.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453940+0000", 2024-09-15T00:12:53.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9880000000000004e-06 2024-09-15T00:12:53.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453946+0000", 2024-09-15T00:12:53.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2960000000000004e-06 2024-09-15T00:12:53.475 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.475 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.475 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.475 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163008 3.0 3.c1ee41c0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:53.475 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.453960+0000", 2024-09-15T00:12:53.476 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00072930900000000003, 2024-09-15T00:12:53.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0010394549999999999, 2024-09-15T00:12:53.476 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.476 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.476 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-15T00:12:53.476 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.476 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.568 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.568 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163008 2024-09-15T00:12:53.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.568 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453960+0000", 2024-09-15T00:12:53.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453960+0000", 2024-09-15T00:12:53.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453960+0000", 2024-09-15T00:12:53.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6199999999999998e-07 2024-09-15T00:12:53.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453963+0000", 2024-09-15T00:12:53.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.689e-06 2024-09-15T00:12:53.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453970+0000", 2024-09-15T00:12:53.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8349999999999998e-06 2024-09-15T00:12:53.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.453975+0000", 2024-09-15T00:12:53.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5369999999999997e-06 2024-09-15T00:12:53.572 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.572 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.572 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.572 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163015 3.10 3.abab8030 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:53.573 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.454030+0000", 2024-09-15T00:12:53.573 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00065897700000000004, 2024-09-15T00:12:53.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0010246980000000001, 2024-09-15T00:12:53.573 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.573 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.573 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-15T00:12:53.573 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.574 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.574 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.574 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163015 2024-09-15T00:12:53.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.574 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454030+0000", 2024-09-15T00:12:53.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454030+0000", 2024-09-15T00:12:53.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454031+0000", 2024-09-15T00:12:53.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9999999999999997e-07 2024-09-15T00:12:53.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454034+0000", 2024-09-15T00:12:53.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7890000000000001e-06 2024-09-15T00:12:53.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454044+0000", 2024-09-15T00:12:53.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3030000000000005e-06 2024-09-15T00:12:53.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454050+0000", 2024-09-15T00:12:53.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0560000000000001e-06 2024-09-15T00:12:53.577 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.578 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.578 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.578 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163016 3.10 3.abab8030 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:53.578 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.454060+0000", 2024-09-15T00:12:53.578 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00062922900000000003, 2024-09-15T00:12:53.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001045453, 2024-09-15T00:12:53.578 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.579 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.579 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-15T00:12:53.579 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.579 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.579 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.579 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163016 2024-09-15T00:12:53.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.579 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454060+0000", 2024-09-15T00:12:53.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454060+0000", 2024-09-15T00:12:53.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454060+0000", 2024-09-15T00:12:53.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5900000000000002e-07 2024-09-15T00:12:53.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454062+0000", 2024-09-15T00:12:53.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:12:53.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454069+0000", 2024-09-15T00:12:53.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9650000000000002e-06 2024-09-15T00:12:53.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454075+0000", 2024-09-15T00:12:53.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.541e-06 2024-09-15T00:12:53.582 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.582 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.582 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.583 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163024 3.0 3.c1ee41c0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:53.583 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.454114+0000", 2024-09-15T00:12:53.583 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00057498, 2024-09-15T00:12:53.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001047252, 2024-09-15T00:12:53.583 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.583 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.583 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-15T00:12:53.583 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.583 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.583 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.583 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163024 2024-09-15T00:12:53.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.584 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454114+0000", 2024-09-15T00:12:53.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454114+0000", 2024-09-15T00:12:53.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454115+0000", 2024-09-15T00:12:53.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2900000000000003e-07 2024-09-15T00:12:53.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454117+0000", 2024-09-15T00:12:53.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7810000000000001e-06 2024-09-15T00:12:53.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454128+0000", 2024-09-15T00:12:53.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0400999999999999e-05 2024-09-15T00:12:53.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454134+0000", 2024-09-15T00:12:53.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1720000000000004e-06 2024-09-15T00:12:53.586 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.586 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.586 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.587 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163031 3.10 3.abab8030 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:53.587 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.454629+0000", 2024-09-15T00:12:53.587 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.0241000000000001e-05, 2024-09-15T00:12:53.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00058974199999999998, 2024-09-15T00:12:53.587 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.588 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.588 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-15T00:12:53.588 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.588 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.588 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.588 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163031 2024-09-15T00:12:53.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.588 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454629+0000", 2024-09-15T00:12:53.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454629+0000", 2024-09-15T00:12:53.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454630+0000", 2024-09-15T00:12:53.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3500000000000005e-07 2024-09-15T00:12:53.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454634+0000", 2024-09-15T00:12:53.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3900000000000003e-06 2024-09-15T00:12:53.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454645+0000", 2024-09-15T00:12:53.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.114e-05 2024-09-15T00:12:53.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454653+0000", 2024-09-15T00:12:53.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.61e-06 2024-09-15T00:12:53.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.591 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.592 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.592 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163040 3.0 3.c1ee41c0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:53.592 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.454665+0000", 2024-09-15T00:12:53.592 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.3890999999999999e-05, 2024-09-15T00:12:53.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00063747299999999995, 2024-09-15T00:12:53.592 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.592 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.592 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-15T00:12:53.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.593 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.593 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163040 2024-09-15T00:12:53.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.593 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454665+0000", 2024-09-15T00:12:53.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454665+0000", 2024-09-15T00:12:53.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454666+0000", 2024-09-15T00:12:53.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1200000000000003e-07 2024-09-15T00:12:53.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454668+0000", 2024-09-15T00:12:53.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6230000000000001e-06 2024-09-15T00:12:53.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454675+0000", 2024-09-15T00:12:53.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7970000000000003e-06 2024-09-15T00:12:53.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454681+0000", 2024-09-15T00:12:53.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5520000000000004e-06 2024-09-15T00:12:53.595 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.595 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.595 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.596 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163047 3.10 3.abab8030 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:53.596 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.454688+0000", 2024-09-15T00:12:53.596 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 7.8999999999999995e-07, 2024-09-15T00:12:53.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00068124500000000003, 2024-09-15T00:12:53.596 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.596 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.596 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-15T00:12:53.596 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.597 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.597 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.597 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163047 2024-09-15T00:12:53.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.597 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454688+0000", 2024-09-15T00:12:53.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454688+0000", 2024-09-15T00:12:53.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454689+0000", 2024-09-15T00:12:53.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:12:53.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454691+0000", 2024-09-15T00:12:53.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3029999999999998e-06 2024-09-15T00:12:53.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454696+0000", 2024-09-15T00:12:53.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.626e-06 2024-09-15T00:12:53.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454701+0000", 2024-09-15T00:12:53.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8690000000000003e-06 2024-09-15T00:12:53.600 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.600 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.600 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.600 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163056 3.0 3.c1ee41c0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:53.600 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.454709+0000", 2024-09-15T00:12:53.600 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4294967295.9999795, 2024-09-15T00:12:53.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00072111199999999999, 2024-09-15T00:12:53.600 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.601 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.601 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-15T00:12:53.601 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.601 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.601 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.601 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163056 2024-09-15T00:12:53.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.601 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454709+0000", 2024-09-15T00:12:53.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454709+0000", 2024-09-15T00:12:53.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454710+0000", 2024-09-15T00:12:53.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8799999999999998e-07 2024-09-15T00:12:53.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454712+0000", 2024-09-15T00:12:53.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3240000000000001e-06 2024-09-15T00:12:53.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454730+0000", 2024-09-15T00:12:53.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8145e-05 2024-09-15T00:12:53.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454736+0000", 2024-09-15T00:12:53.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8930000000000002e-06 2024-09-15T00:12:53.603 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.603 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.603 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.603 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163063 3.10 3.abab8030 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:53.603 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.454744+0000", 2024-09-15T00:12:53.604 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4294967295.9999447, 2024-09-15T00:12:53.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00077468899999999995, 2024-09-15T00:12:53.604 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.604 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.604 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-15T00:12:53.604 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.604 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.604 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.604 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163063 2024-09-15T00:12:53.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.604 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454744+0000", 2024-09-15T00:12:53.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454744+0000", 2024-09-15T00:12:53.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454745+0000", 2024-09-15T00:12:53.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:12:53.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454747+0000", 2024-09-15T00:12:53.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.678e-06 2024-09-15T00:12:53.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454754+0000", 2024-09-15T00:12:53.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4119999999999998e-06 2024-09-15T00:12:53.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454758+0000", 2024-09-15T00:12:53.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7489999999999998e-06 2024-09-15T00:12:53.606 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.606 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.607 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.607 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163072 3.0 3.c1ee41c0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:53.607 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.454765+0000", 2024-09-15T00:12:53.607 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4294967295.9999237, 2024-09-15T00:12:53.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00088502199999999996, 2024-09-15T00:12:53.607 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.607 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.607 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-15T00:12:53.607 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.607 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.607 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.608 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163072 2024-09-15T00:12:53.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.608 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454765+0000", 2024-09-15T00:12:53.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454765+0000", 2024-09-15T00:12:53.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454765+0000", 2024-09-15T00:12:53.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:12:53.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454768+0000", 2024-09-15T00:12:53.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2809999999999998e-06 2024-09-15T00:12:53.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454774+0000", 2024-09-15T00:12:53.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9479999999999996e-06 2024-09-15T00:12:53.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.454778+0000", 2024-09-15T00:12:53.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6959999999999996e-06 2024-09-15T00:12:53.610 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.610 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.610 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.610 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.610 INFO:teuthology.orchestra.run.smithi184.stdout: ], 2024-09-15T00:12:53.610 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 14 2024-09-15T00:12:53.610 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:53.612 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:53.613 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:53.610+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:53.614 INFO:teuthology.orchestra.run.smithi171.stdout:5660 read 0x43e840f thru 0x43eba43 (0x3635 bytes) 2024-09-15T00:12:53.614 INFO:teuthology.orchestra.run.smithi171.stdout:5661 writesame 0x2157203 thru 0x215dd48 data_size 0x3(0x6b46 bytes) 2024-09-15T00:12:53.614 INFO:teuthology.orchestra.run.smithi171.stdout:5662 write 0x7066d70 thru 0x707175e (0xa9ef bytes) 2024-09-15T00:12:53.614 INFO:teuthology.orchestra.run.smithi171.stdout:5663 write 0xdb2298c thru 0xdb27593 (0x4c08 bytes) 2024-09-15T00:12:53.614 INFO:teuthology.orchestra.run.smithi171.stdout:5664 read 0xc22d374 thru 0xc2332b1 (0x5f3e bytes) 2024-09-15T00:12:53.614 INFO:teuthology.orchestra.run.smithi171.stdout:5665 write 0xc44d24a thru 0xc45ab75 (0xd92c bytes) 2024-09-15T00:12:53.614 INFO:teuthology.orchestra.run.smithi171.stdout:5667 write 0x8859820 thru 0x885e520 (0x4d01 bytes) 2024-09-15T00:12:53.614 INFO:teuthology.orchestra.run.smithi171.stdout:5669 punch from 0x7b4c08a to 0x7b5becc, (0xfe42 bytes) 2024-09-15T00:12:53.614 INFO:teuthology.orchestra.run.smithi171.stdout:5671 read 0xae0c2d2 thru 0xae17eea (0xbc19 bytes) 2024-09-15T00:12:53.615 INFO:teuthology.orchestra.run.smithi171.stdout:5672 read 0x92f2f53 thru 0x92f38b8 (0x966 bytes) 2024-09-15T00:12:53.615 INFO:teuthology.orchestra.run.smithi171.stdout:5673 write 0x78ecb72 thru 0x78fc442 (0xf8d1 bytes) 2024-09-15T00:12:53.615 INFO:teuthology.orchestra.run.smithi171.stdout:5674 write 0x49b4afc thru 0x49bfa01 (0xaf06 bytes) 2024-09-15T00:12:53.615 INFO:teuthology.orchestra.run.smithi171.stdout:5675 punch from 0xa53b5e0 to 0xa5499ca, (0xe3ea bytes) 2024-09-15T00:12:53.615 INFO:teuthology.orchestra.run.smithi171.stdout:5677 punch from 0x1dbd730 to 0x1dc95ae, (0xbe7e bytes) 2024-09-15T00:12:53.615 INFO:teuthology.orchestra.run.smithi171.stdout:5678 trunc from 0xdb27594 to 0x4edfa83 2024-09-15T00:12:53.615 INFO:teuthology.orchestra.run.smithi171.stdout:5680 writesame 0x17c0274 thru 0x17c8ceb data_size 0xe(0x8a78 bytes) 2024-09-15T00:12:53.615 INFO:teuthology.orchestra.run.smithi171.stdout:5681 trunc from 0x4edfa83 to 0x57e9a4 2024-09-15T00:12:53.633 INFO:teuthology.orchestra.run.smithi171.stdout:5682 writesame 0x1b9d39 thru 0x1be2bc data_size 0x6(0x4584 bytes) 2024-09-15T00:12:53.638 INFO:teuthology.orchestra.run.smithi171.stdout:5683 read 0x19fe2c thru 0x1a2eab (0x3080 bytes) 2024-09-15T00:12:53.650 INFO:teuthology.orchestra.run.smithi171.stdout:5684 write 0x4394e9e thru 0x439fed5 (0xb038 bytes) 2024-09-15T00:12:53.653 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:53.670 INFO:teuthology.orchestra.run.smithi171.stdout:5686 write 0xbdd92e7 thru 0xbde6c83 (0xd99d bytes) 2024-09-15T00:12:53.675 INFO:teuthology.orchestra.run.smithi171.stdout:5688 writesame 0xb6c8e15 thru 0xb6d057c data_size 0x4(0x7768 bytes) 2024-09-15T00:12:53.685 INFO:teuthology.orchestra.run.smithi171.stdout:5689 read 0xb2db8b8 thru 0xb2e1380 (0x5ac9 bytes) 2024-09-15T00:12:53.685 INFO:teuthology.orchestra.run.smithi171.stdout:5691 writesame 0x25b36e9 thru 0x25b74e0 data_size 0xc(0x3df8 bytes) 2024-09-15T00:12:53.689 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:53.689 INFO:teuthology.orchestra.run.smithi171.stdout:5694 read 0x9d7da7f thru 0x9d8bb90 (0xe112 bytes) 2024-09-15T00:12:53.689 INFO:teuthology.orchestra.run.smithi171.stdout:5695 read 0x14ba01a thru 0x14c8ace (0xeab5 bytes) 2024-09-15T00:12:53.689 INFO:teuthology.orchestra.run.smithi171.stdout:5696 read 0x77affd thru 0x787d37 (0xcd3b bytes) 2024-09-15T00:12:53.696 INFO:teuthology.orchestra.run.smithi171.stdout:5697 writesame 0xc82a2dc thru 0xc8316e3 data_size 0x4(0x7408 bytes) 2024-09-15T00:12:53.696 INFO:teuthology.orchestra.run.smithi171.stdout:5698 write 0x21a2b06 thru 0x21ac57d (0x9a78 bytes) 2024-09-15T00:12:53.701 INFO:teuthology.orchestra.run.smithi171.stdout:5699 write 0x44a799b thru 0x44b57df (0xde45 bytes) 2024-09-15T00:12:53.707 INFO:teuthology.orchestra.run.smithi171.stdout:5700 read 0x85557d9 thru 0x8558765 (0x2f8d bytes) 2024-09-15T00:12:53.712 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:53.710+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:53.713 INFO:teuthology.orchestra.run.smithi171.stdout:5702 write 0xcd875dd thru 0xcd9383f (0xc263 bytes) 2024-09-15T00:12:53.726 INFO:teuthology.orchestra.run.smithi171.stdout:5704 trunc from 0xcd93840 to 0x4e502ee 2024-09-15T00:12:53.736 INFO:teuthology.orchestra.run.smithi171.stdout:5705 punch from 0x4ba8aed to 0x4baa27a, (0x178d bytes) 2024-09-15T00:12:53.753 INFO:teuthology.orchestra.run.smithi171.stdout:5707 write 0xe3f150e thru 0xe3fc91e (0xb411 bytes) 2024-09-15T00:12:53.756 INFO:teuthology.orchestra.run.smithi171.stdout:5708 read 0xb5925ca thru 0xb59e926 (0xc35d bytes) 2024-09-15T00:12:53.757 INFO:teuthology.orchestra.run.smithi171.stdout:5709 write 0xe3b847f thru 0xe3c2609 (0xa18b bytes) 2024-09-15T00:12:53.762 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:53.762 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:53.762 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:53.762 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:53.762 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:53.763 INFO:teuthology.orchestra.run.smithi171.stdout:5712 writesame 0x4d749a1 thru 0x4d74afa data_size 0x1(0x15a bytes) 2024-09-15T00:12:53.766 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:53.767 INFO:teuthology.orchestra.run.smithi171.stdout:5714 punch from 0xbccc59a to 0xbcd660b, (0xa071 bytes) 2024-09-15T00:12:53.767 INFO:teuthology.orchestra.run.smithi171.stdout:5715 writesame 0xd11c09a thru 0xd126c6e data_size 0x1(0xabd5 bytes) 2024-09-15T00:12:53.771 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:53.785 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:53.783+0000 7feb60e0b640 -1 osd.2 79 log_to_monitors true 2024-09-15T00:12:53.815 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:53.814+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:53.819 INFO:teuthology.orchestra.run.smithi171.stdout:5716 read 0xcd5dc91 thru 0xcd65c0a (0x7f7a bytes) 2024-09-15T00:12:53.820 INFO:teuthology.orchestra.run.smithi171.stdout:5717 trunc from 0xe3fc91f to 0xd786a5c 2024-09-15T00:12:53.833 INFO:teuthology.orchestra.run.smithi171.stdout:5718 punch from 0x58a573a to 0x58b0419, (0xacdf bytes) 2024-09-15T00:12:53.833 INFO:teuthology.orchestra.run.smithi171.stdout:5719 write 0xbb4e913 thru 0xbb5a839 (0xbf27 bytes) 2024-09-15T00:12:53.849 INFO:teuthology.orchestra.run.smithi171.stdout:5720 trunc from 0xd786a5c to 0x3ed4675 2024-09-15T00:12:53.879 INFO:teuthology.orchestra.run.smithi171.stdout:5721 write 0xdc5f5d9 thru 0xdc650fc (0x5b24 bytes) 2024-09-15T00:12:53.883 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:53.883 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:53.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:53.884 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:53.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.884 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:19308 3.10 3:0ee1524d:::rbd_data.11398e1a091a.0000000000000017:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-15T00:12:53.884 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:53.884 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.895403088999998, 2024-09-15T00:12:53.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02547147, 2024-09-15T00:12:53.884 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.885 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.885 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:53.885 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.885 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.885 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.885 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 19308 2024-09-15T00:12:53.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.885 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:53.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985181+0000", 2024-09-15T00:12:53.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985182+0000", 2024-09-15T00:12:53.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6500000000000001e-06 2024-09-15T00:12:53.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985189+0000", 2024-09-15T00:12:53.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9949999999999999e-06 2024-09-15T00:12:53.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985205+0000", 2024-09-15T00:12:53.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6055000000000001e-05 2024-09-15T00:12:53.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985212+0000", 2024-09-15T00:12:53.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:12:53.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985261+0000", 2024-09-15T00:12:53.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8380000000000001e-05 2024-09-15T00:12:53.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.985311+0000", 2024-09-15T00:12:53.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0763999999999997e-05 2024-09-15T00:12:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:38.986489+0000", 2024-09-15T00:12:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0011773420000000001 2024-09-15T00:12:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:53.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010514+0000", 2024-09-15T00:12:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024025392999999999 2024-09-15T00:12:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010535+0000", 2024-09-15T00:12:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0652000000000001e-05 2024-09-15T00:12:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010597+0000", 2024-09-15T00:12:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2411999999999998e-05 2024-09-15T00:12:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:39.010652+0000", 2024-09-15T00:12:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5074999999999997e-05 2024-09-15T00:12:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.892 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.213226121000005, 2024-09-15T00:12:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:12:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.893 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:12:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:12:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:12:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:12:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:12:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:12:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:12:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:12:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:12:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:12:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:12:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:12:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:12:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:12:53.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:12:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:12:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:12:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:12:53.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:12:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:12:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:53.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:12:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:12:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:12:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:12:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.901 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66542 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1560576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 58.966429468000001, 2024-09-15T00:12:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024740410000000001, 2024-09-15T00:12:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.902 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66542 2024-09-15T00:12:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914154+0000", 2024-09-15T00:12:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914155+0000", 2024-09-15T00:12:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:12:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914157+0000", 2024-09-15T00:12:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:12:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914162+0000", 2024-09-15T00:12:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914167+0000", 2024-09-15T00:12:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3789999999999999e-06 2024-09-15T00:12:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938730+0000", 2024-09-15T00:12:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024562694 2024-09-15T00:12:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938803+0000", 2024-09-15T00:12:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3325999999999996e-05 2024-09-15T00:12:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938895+0000", 2024-09-15T00:12:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1884000000000003e-05 2024-09-15T00:12:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66543 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1572864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 58.966399209000002, 2024-09-15T00:12:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024893976000000002, 2024-09-15T00:12:53.908 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66543 2024-09-15T00:12:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914185+0000", 2024-09-15T00:12:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914188+0000", 2024-09-15T00:12:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:12:53.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914193+0000", 2024-09-15T00:12:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4129999999999997e-06 2024-09-15T00:12:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914198+0000", 2024-09-15T00:12:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9309999999999999e-06 2024-09-15T00:12:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938914+0000", 2024-09-15T00:12:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024716380999999999 2024-09-15T00:12:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.938993+0000", 2024-09-15T00:12:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8763000000000005e-05 2024-09-15T00:12:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939079+0000", 2024-09-15T00:12:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5372999999999997e-05 2024-09-15T00:12:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.912 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66544 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1683456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 58.966376806, 2024-09-15T00:12:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025051225, 2024-09-15T00:12:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.913 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66544 2024-09-15T00:12:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914207+0000", 2024-09-15T00:12:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:12:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914210+0000", 2024-09-15T00:12:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2450000000000001e-06 2024-09-15T00:12:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914215+0000", 2024-09-15T00:12:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7869999999999998e-06 2024-09-15T00:12:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914220+0000", 2024-09-15T00:12:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7899999999999999e-06 2024-09-15T00:12:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939105+0000", 2024-09-15T00:12:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024884381000000001 2024-09-15T00:12:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939177+0000", 2024-09-15T00:12:53.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2138999999999995e-05 2024-09-15T00:12:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939258+0000", 2024-09-15T00:12:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1462e-05 2024-09-15T00:12:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66545 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1695744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 58.966354531999997, 2024-09-15T00:12:53.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025209788, 2024-09-15T00:12:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66545 2024-09-15T00:12:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:53.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914229+0000", 2024-09-15T00:12:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914230+0000", 2024-09-15T00:12:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1099999999999996e-07 2024-09-15T00:12:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914232+0000", 2024-09-15T00:12:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3080000000000002e-06 2024-09-15T00:12:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914238+0000", 2024-09-15T00:12:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.02e-06 2024-09-15T00:12:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914243+0000", 2024-09-15T00:12:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1200000000000001e-06 2024-09-15T00:12:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939285+0000", 2024-09-15T00:12:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025042212000000001 2024-09-15T00:12:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939361+0000", 2024-09-15T00:12:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5558999999999999e-05 2024-09-15T00:12:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939439+0000", 2024-09-15T00:12:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8058000000000004e-05 2024-09-15T00:12:53.921 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66546 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 58.966334557000003, 2024-09-15T00:12:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025375960999999999, 2024-09-15T00:12:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66546 2024-09-15T00:12:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914249+0000", 2024-09-15T00:12:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914250+0000", 2024-09-15T00:12:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:12:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914252+0000", 2024-09-15T00:12:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.052e-06 2024-09-15T00:12:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914258+0000", 2024-09-15T00:12:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9039999999999997e-06 2024-09-15T00:12:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914262+0000", 2024-09-15T00:12:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4719999999999997e-06 2024-09-15T00:12:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939466+0000", 2024-09-15T00:12:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025203550000000002 2024-09-15T00:12:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939537+0000", 2024-09-15T00:12:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1780999999999996e-05 2024-09-15T00:12:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939625+0000", 2024-09-15T00:12:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7828e-05 2024-09-15T00:12:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.926 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66547 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1818624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 58.966314883999999, 2024-09-15T00:12:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025552638999999999, 2024-09-15T00:12:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66547 2024-09-15T00:12:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914269+0000", 2024-09-15T00:12:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914271+0000", 2024-09-15T00:12:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8920000000000001e-06 2024-09-15T00:12:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914277+0000", 2024-09-15T00:12:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914282+0000", 2024-09-15T00:12:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7190000000000001e-06 2024-09-15T00:12:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939654+0000", 2024-09-15T00:12:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025372372000000001 2024-09-15T00:12:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939734+0000", 2024-09-15T00:12:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9946999999999999e-05 2024-09-15T00:12:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939822+0000", 2024-09-15T00:12:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7680000000000001e-05 2024-09-15T00:12:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66548 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 58.966284496999997, 2024-09-15T00:12:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025712727000000001, 2024-09-15T00:12:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.931 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66548 2024-09-15T00:12:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914299+0000", 2024-09-15T00:12:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914300+0000", 2024-09-15T00:12:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3599999999999999e-07 2024-09-15T00:12:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914302+0000", 2024-09-15T00:12:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3369999999999998e-06 2024-09-15T00:12:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914313+0000", 2024-09-15T00:12:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1399999999999999e-05 2024-09-15T00:12:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914319+0000", 2024-09-15T00:12:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9100000000000002e-06 2024-09-15T00:12:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939850+0000", 2024-09-15T00:12:53.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025531087000000001 2024-09-15T00:12:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.939932+0000", 2024-09-15T00:12:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1534999999999998e-05 2024-09-15T00:12:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940012+0000", 2024-09-15T00:12:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0022000000000001e-05 2024-09-15T00:12:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.935 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66549 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 1941504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 58.966219406, 2024-09-15T00:12:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025813836999999999, 2024-09-15T00:12:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66549 2024-09-15T00:12:53.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914364+0000", 2024-09-15T00:12:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914365+0000", 2024-09-15T00:12:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2799999999999995e-07 2024-09-15T00:12:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914368+0000", 2024-09-15T00:12:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.92e-06 2024-09-15T00:12:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914376+0000", 2024-09-15T00:12:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8110000000000003e-06 2024-09-15T00:12:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914381+0000", 2024-09-15T00:12:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5119999999999999e-06 2024-09-15T00:12:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940040+0000", 2024-09-15T00:12:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025658271999999999 2024-09-15T00:12:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940103+0000", 2024-09-15T00:12:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2891000000000005e-05 2024-09-15T00:12:53.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940178+0000", 2024-09-15T00:12:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5703000000000003e-05 2024-09-15T00:12:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66550 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 58.966075826000001, 2024-09-15T00:12:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025820701000000001, 2024-09-15T00:12:53.940 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66550 2024-09-15T00:12:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914508+0000", 2024-09-15T00:12:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914509+0000", 2024-09-15T00:12:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1999999999999999e-07 2024-09-15T00:12:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914513+0000", 2024-09-15T00:12:53.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1640000000000003e-06 2024-09-15T00:12:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914522+0000", 2024-09-15T00:12:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7639999999999994e-06 2024-09-15T00:12:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914527+0000", 2024-09-15T00:12:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9170000000000003e-06 2024-09-15T00:12:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940199+0000", 2024-09-15T00:12:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025672037000000002 2024-09-15T00:12:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940269+0000", 2024-09-15T00:12:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9208999999999994e-05 2024-09-15T00:12:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940329+0000", 2024-09-15T00:12:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.999e-05 2024-09-15T00:12:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.944 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:66551 3.a 3:54b78d12:::rbd_data.1139766160da.0000000000000009:13 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-15T00:12:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 58.966019078999999, 2024-09-15T00:12:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025917345000000001, 2024-09-15T00:12:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.945 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 66551 2024-09-15T00:12:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914565+0000", 2024-09-15T00:12:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1699999999999997e-07 2024-09-15T00:12:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914569+0000", 2024-09-15T00:12:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.225e-06 2024-09-15T00:12:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914582+0000", 2024-09-15T00:12:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3648999999999999e-05 2024-09-15T00:12:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.914588+0000", 2024-09-15T00:12:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8449999999999999e-06 2024-09-15T00:12:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940355+0000", 2024-09-15T00:12:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025767110999999999 2024-09-15T00:12:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940423+0000", 2024-09-15T00:12:53.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7574e-05 2024-09-15T00:12:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:54.940482+0000", 2024-09-15T00:12:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9224000000000001e-05 2024-09-15T00:12:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:88238 3.a 3:52924f23:::rbd_data.11396cb1f0e4.0000000000000001:16 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-15T00:12:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 51.602834782000002, 2024-09-15T00:12:53.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024851132000000001, 2024-09-15T00:12:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88238 2024-09-15T00:12:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277749+0000", 2024-09-15T00:12:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277750+0000", 2024-09-15T00:12:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3300000000000002e-07 2024-09-15T00:12:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277751+0000", 2024-09-15T00:12:53.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:12:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277757+0000", 2024-09-15T00:12:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.767e-06 2024-09-15T00:12:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.277769+0000", 2024-09-15T00:12:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1874e-05 2024-09-15T00:12:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302374+0000", 2024-09-15T00:12:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024604669999999999 2024-09-15T00:12:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302460+0000", 2024-09-15T00:12:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6489999999999994e-05 2024-09-15T00:12:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:02.302600+0000", 2024-09-15T00:12:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013985700000000001 2024-09-15T00:12:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.953 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125618 3.10 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:12:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.042087039000002, 2024-09-15T00:12:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030860942999999998, 2024-09-15T00:12:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.954 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125618 2024-09-15T00:12:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:12:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:12:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838499+0000", 2024-09-15T00:12:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6189999999999998e-06 2024-09-15T00:12:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839599+0000", 2024-09-15T00:12:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001099764 2024-09-15T00:12:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839620+0000", 2024-09-15T00:12:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0764e-05 2024-09-15T00:12:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839631+0000", 2024-09-15T00:12:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1335000000000001e-05 2024-09-15T00:12:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839725+0000", 2024-09-15T00:12:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4217999999999998e-05 2024-09-15T00:12:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839910+0000", 2024-09-15T00:12:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000184298 2024-09-15T00:12:53.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843171+0000", 2024-09-15T00:12:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032613540000000002 2024-09-15T00:12:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869187+0000", 2024-09-15T00:12:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026015467 2024-09-15T00:12:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869240+0000", 2024-09-15T00:12:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3733000000000002e-05 2024-09-15T00:12:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869309+0000", 2024-09-15T00:12:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8381999999999997e-05 2024-09-15T00:12:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869358+0000", 2024-09-15T00:12:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-15T00:12:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125619 3.a 3:507b29bf:::rbd_data.11396481e1ec.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:12:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.040924910000001, 2024-09-15T00:12:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045768870000000003, 2024-09-15T00:12:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125619 2024-09-15T00:12:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:12:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:12:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839660+0000", 2024-09-15T00:12:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.22e-06 2024-09-15T00:12:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840511+0000", 2024-09-15T00:12:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000851284 2024-09-15T00:12:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840522+0000", 2024-09-15T00:12:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0569e-05 2024-09-15T00:12:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840527+0000", 2024-09-15T00:12:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.412e-06 2024-09-15T00:12:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840573+0000", 2024-09-15T00:12:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5247000000000002e-05 2024-09-15T00:12:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840682+0000", 2024-09-15T00:12:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109368 2024-09-15T00:12:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843976+0000", 2024-09-15T00:12:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294445 2024-09-15T00:12:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870833+0000", 2024-09-15T00:12:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026856944000000001 2024-09-15T00:12:53.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885344+0000", 2024-09-15T00:12:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014510744000000001 2024-09-15T00:12:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885382+0000", 2024-09-15T00:12:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7821000000000003e-05 2024-09-15T00:12:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885428+0000", 2024-09-15T00:12:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5816000000000003e-05 2024-09-15T00:12:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125621 3.a 3:53594db9:::rbd_data.11396481e1ec.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:12:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.027401531000002, 2024-09-15T00:12:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032481797, 2024-09-15T00:12:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:53.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125621 2024-09-15T00:12:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:12:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:12:53.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853185+0000", 2024-09-15T00:12:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.43e-06 2024-09-15T00:12:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854125+0000", 2024-09-15T00:12:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000940539 2024-09-15T00:12:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854141+0000", 2024-09-15T00:12:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5568000000000001e-05 2024-09-15T00:12:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854148+0000", 2024-09-15T00:12:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6529999999999994e-06 2024-09-15T00:12:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870872+0000", 2024-09-15T00:12:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016723135 2024-09-15T00:12:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870975+0000", 2024-09-15T00:12:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010314099999999999 2024-09-15T00:12:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.873082+0000", 2024-09-15T00:12:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021069859999999999 2024-09-15T00:12:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885439+0000", 2024-09-15T00:12:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012357343999999999 2024-09-15T00:12:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885569+0000", 2024-09-15T00:12:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012995199999999999 2024-09-15T00:12:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885624+0000", 2024-09-15T00:12:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5281999999999997e-05 2024-09-15T00:12:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885664+0000", 2024-09-15T00:12:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9767000000000001e-05 2024-09-15T00:12:53.971 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:147075 3.0 3:06ab39f2:::rbd_data.11394501364.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:12:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:12:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9315031499999993, 2024-09-15T00:12:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046402906000000001, 2024-09-15T00:12:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.972 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 147075 2024-09-15T00:12:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:12:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:12:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:12:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4399999999999999e-07 2024-09-15T00:12:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949086+0000", 2024-09-15T00:12:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.138e-06 2024-09-15T00:12:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949096+0000", 2024-09-15T00:12:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0438999999999999e-05 2024-09-15T00:12:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949101+0000", 2024-09-15T00:12:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8999999999999997e-06 2024-09-15T00:12:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949145+0000", 2024-09-15T00:12:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4141000000000003e-05 2024-09-15T00:12:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949195+0000", 2024-09-15T00:12:53.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9657999999999998e-05 2024-09-15T00:12:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.950923+0000", 2024-09-15T00:12:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001728753 2024-09-15T00:12:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995390+0000", 2024-09-15T00:12:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044466335000000003 2024-09-15T00:12:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995401+0000", 2024-09-15T00:12:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1671000000000001e-05 2024-09-15T00:12:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995432+0000", 2024-09-15T00:12:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0704999999999998e-05 2024-09-15T00:12:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995484+0000", 2024-09-15T00:12:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1422000000000003e-05 2024-09-15T00:12:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.977 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:147075 3.0 e98/94 ondisk, result = 0)", 2024-09-15T00:12:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:12:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9237034289999997, 2024-09-15T00:12:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038514459000000001, 2024-09-15T00:12:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:12:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:12:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956881+0000", 2024-09-15T00:12:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0599999999999999e-07 2024-09-15T00:12:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956885+0000", 2024-09-15T00:12:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3500000000000001e-06 2024-09-15T00:12:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956897+0000", 2024-09-15T00:12:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2303999999999999e-05 2024-09-15T00:12:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956902+0000", 2024-09-15T00:12:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5169999999999999e-06 2024-09-15T00:12:53.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995377+0000", 2024-09-15T00:12:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038474544999999999 2024-09-15T00:12:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995385+0000", 2024-09-15T00:12:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6240000000000002e-06 2024-09-15T00:12:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995395+0000", 2024-09-15T00:12:53.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0213e-05 2024-09-15T00:12:53.982 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.982 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.982 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.982 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153172 3.14 3:2a7843e3:::rbd_data.113958cb0e55.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 983040~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:12:53.982 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:12:53.982 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1028143650000004, 2024-09-15T00:12:53.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.033232668, 2024-09-15T00:12:53.982 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.982 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.982 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:53.982 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.982 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.983 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.983 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153172 2024-09-15T00:12:53.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.983 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:12:53.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:12:53.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777770+0000", 2024-09-15T00:12:53.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.017e-06 2024-09-15T00:12:53.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788009+0000", 2024-09-15T00:12:53.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010238146 2024-09-15T00:12:53.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788024+0000", 2024-09-15T00:12:53.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5582000000000001e-05 2024-09-15T00:12:53.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788048+0000", 2024-09-15T00:12:53.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4173e-05 2024-09-15T00:12:53.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788097+0000", 2024-09-15T00:12:53.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8498999999999998e-05 2024-09-15T00:12:53.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788201+0000", 2024-09-15T00:12:53.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010398799999999999 2024-09-15T00:12:53.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:53.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788393+0000", 2024-09-15T00:12:53.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000192337 2024-09-15T00:12:53.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:53.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810637+0000", 2024-09-15T00:12:53.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.022244202000000001 2024-09-15T00:12:53.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:53.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810903+0000", 2024-09-15T00:12:53.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00026574600000000003 2024-09-15T00:12:53.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:53.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810953+0000", 2024-09-15T00:12:53.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9509000000000003e-05 2024-09-15T00:12:53.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.811002+0000", 2024-09-15T00:12:53.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9468999999999997e-05 2024-09-15T00:12:53.987 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.988 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.988 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.988 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153770 3.13 3:cebb4cdf:::rbd_data.113958cb0e55.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 2826363~24453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:12:53.988 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.878288+0000", 2024-09-15T00:12:53.988 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.0022960999999997, 2024-09-15T00:12:53.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029205471, 2024-09-15T00:12:53.988 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:53.988 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:53.988 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:53.988 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:53.988 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:53.989 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:53.989 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153770 2024-09-15T00:12:53.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.989 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:53.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:53.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.878288+0000", 2024-09-15T00:12:53.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:53.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.878288+0000", 2024-09-15T00:12:53.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:53.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:53.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.878295+0000", 2024-09-15T00:12:53.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2409999999999998e-06 2024-09-15T00:12:53.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:53.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.885400+0000", 2024-09-15T00:12:53.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0071045780000000003 2024-09-15T00:12:53.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:53.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.885421+0000", 2024-09-15T00:12:53.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1664000000000001e-05 2024-09-15T00:12:53.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:53.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.885430+0000", 2024-09-15T00:12:53.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5909999999999996e-06 2024-09-15T00:12:53.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:53.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.885486+0000", 2024-09-15T00:12:53.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6190000000000002e-05 2024-09-15T00:12:53.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:53.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.885595+0000", 2024-09-15T00:12:53.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109478 2024-09-15T00:12:53.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:53.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.885871+0000", 2024-09-15T00:12:53.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00027578800000000001 2024-09-15T00:12:53.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:53.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.906276+0000", 2024-09-15T00:12:53.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02040438 2024-09-15T00:12:53.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:53.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.907422+0000", 2024-09-15T00:12:53.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001146317 2024-09-15T00:12:53.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:53.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.907460+0000", 2024-09-15T00:12:53.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8059999999999998e-05 2024-09-15T00:12:53.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:53.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:53.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:53.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.907493+0000", 2024-09-15T00:12:53.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3183999999999999e-05 2024-09-15T00:12:53.993 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.994 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.994 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.994 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:53.994 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:53.994 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:53.996 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:53.997 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:53.993+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:53.997 INFO:teuthology.orchestra.run.smithi171.stdout:5722 write 0x79d9804 thru 0x79e7dc2 (0xe5bf bytes) 2024-09-15T00:12:53.997 INFO:teuthology.orchestra.run.smithi171.stdout:5723 punch from 0x1c205e9 to 0x1c24b8e, (0x45a5 bytes) 2024-09-15T00:12:53.997 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:53.998 INFO:teuthology.orchestra.run.smithi171.stdout:5725 write 0xe306caa thru 0xe30b0b1 (0x4408 bytes) 2024-09-15T00:12:53.998 INFO:teuthology.orchestra.run.smithi171.stdout:5726 trunc from 0xe30b0b2 to 0xb38ea5b 2024-09-15T00:12:53.998 INFO:teuthology.orchestra.run.smithi171.stdout:5727 write 0x6fd03ea thru 0x6fd82a7 (0x7ebe bytes) 2024-09-15T00:12:53.998 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:53.998 INFO:teuthology.orchestra.run.smithi171.stdout:5732 trunc from 0xb38ea5b to 0x12c9695 2024-09-15T00:12:53.998 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:53.998 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:53.998 INFO:teuthology.orchestra.run.smithi171.stdout:5736 trunc from 0x12c9695 to 0x4d37e93 2024-09-15T00:12:53.998 INFO:teuthology.orchestra.run.smithi171.stdout:5737 write 0xdca70ed thru 0xdcb136c (0xa280 bytes) 2024-09-15T00:12:53.998 INFO:teuthology.orchestra.run.smithi171.stdout:5738 read 0xa3c0fcc thru 0xa3c1b16 (0xb4b bytes) 2024-09-15T00:12:53.998 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:53.998 INFO:teuthology.orchestra.run.smithi171.stdout:5742 read 0x34a2c00 thru 0x34a57ef (0x2bf0 bytes) 2024-09-15T00:12:53.998 INFO:teuthology.orchestra.run.smithi171.stdout:5744 trunc from 0xdcb136d to 0xcf2c95d 2024-09-15T00:12:54.013 INFO:teuthology.orchestra.run.smithi171.stdout:5745 punch from 0xbeff231 to 0xbf02f7f, (0x3d4e bytes) 2024-09-15T00:12:54.013 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:54.013 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:54.013 INFO:teuthology.orchestra.run.smithi171.stdout:5749 read 0x2d1075d thru 0x2d10b61 (0x405 bytes) 2024-09-15T00:12:54.013 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:54.013 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:54.013 INFO:teuthology.orchestra.run.smithi171.stdout:5752 write 0x75fe06b thru 0x760c8f7 (0xe88d bytes) 2024-09-15T00:12:54.022 INFO:teuthology.orchestra.run.smithi171.stdout:5753 write 0x4df4a73 thru 0x4dfe5cb (0x9b59 bytes) 2024-09-15T00:12:54.028 INFO:teuthology.orchestra.run.smithi171.stdout:5754 write 0x9500cf3 thru 0x950e1c9 (0xd4d7 bytes) 2024-09-15T00:12:54.035 INFO:teuthology.orchestra.run.smithi171.stdout:5755 write 0x18c933e thru 0x18d398b (0xa64e bytes) 2024-09-15T00:12:54.039 INFO:teuthology.orchestra.run.smithi171.stdout:5756 writesame 0xc62be42 thru 0xc63ba0d data_size 0x37(0xfbcc bytes) 2024-09-15T00:12:54.044 INFO:teuthology.orchestra.run.smithi171.stdout:5757 write 0x2f3ab36 thru 0x2f448f9 (0x9dc4 bytes) 2024-09-15T00:12:54.056 INFO:teuthology.orchestra.run.smithi171.stdout:5761 writesame 0xcfad6a1 thru 0xcfb9d16 data_size 0x26(0xc676 bytes) 2024-09-15T00:12:54.056 INFO:teuthology.orchestra.run.smithi171.stdout:5763 punch from 0x4ba1bc8 to 0x4ba89e8, (0x6e20 bytes) 2024-09-15T00:12:54.056 INFO:teuthology.orchestra.run.smithi171.stdout:5764 read 0x5d72a75 thru 0x5d79ee8 (0x7474 bytes) 2024-09-15T00:12:54.056 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:54.056 INFO:teuthology.orchestra.run.smithi171.stdout:5766 write 0xd559c3 thru 0xd55c92 (0x2d0 bytes) 2024-09-15T00:12:54.057 INFO:teuthology.orchestra.run.smithi171.stdout:5770 read 0x6545aa0 thru 0x654e6b9 (0x8c1a bytes) 2024-09-15T00:12:54.065 INFO:teuthology.orchestra.run.smithi171.stdout:5771 write 0xea30ad0 thru 0xea3732f (0x6860 bytes) 2024-09-15T00:12:54.069 INFO:teuthology.orchestra.run.smithi171.stdout:5772 punch from 0x9c37ebb to 0x9c3e3bd, (0x6502 bytes) 2024-09-15T00:12:54.069 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:54.069 INFO:teuthology.orchestra.run.smithi171.stdout:5777 punch from 0x21ae753 to 0x21bd692, (0xef3f bytes) 2024-09-15T00:12:54.069 INFO:teuthology.orchestra.run.smithi171.stdout:5778 writesame 0x8dc1853 thru 0x8dc74d8 data_size 0xd(0x5c86 bytes) 2024-09-15T00:12:54.073 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:54.073 INFO:teuthology.orchestra.run.smithi171.stdout:5780 write 0x5eac299 thru 0x5ebaeca (0xec32 bytes) 2024-09-15T00:12:54.079 INFO:teuthology.orchestra.run.smithi171.stdout:5782 read 0x9d84e1b thru 0x9d91d11 (0xcef7 bytes) 2024-09-15T00:12:54.086 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is set 2024-09-15T00:12:54.093 INFO:teuthology.orchestra.run.smithi171.stdout:5783 trunc from 0xea37330 to 0x3343284 2024-09-15T00:12:54.096 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:54.094+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:54.111 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:54.109+0000 7feb5466b640 -1 osd.2 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:12:54.122 INFO:teuthology.orchestra.run.smithi171.stdout:5784 trunc from 0x3343284 to 0xdf969db 2024-09-15T00:12:54.131 INFO:teuthology.orchestra.run.smithi171.stdout:5785 write 0xd3f4c59 thru 0xd3f90f4 (0x449c bytes) 2024-09-15T00:12:54.137 INFO:teuthology.orchestra.run.smithi171.stdout:5786 write 0xdb4d8a9 thru 0xdb5a8ad (0xd005 bytes) 2024-09-15T00:12:54.142 INFO:teuthology.orchestra.run.smithi171.stdout:5788 read 0x1e7c931 thru 0x1e89639 (0xcd09 bytes) 2024-09-15T00:12:54.143 INFO:teuthology.orchestra.run.smithi171.stdout:5789 read 0xc29815b thru 0xc2a2ffc (0xaea2 bytes) 2024-09-15T00:12:54.143 INFO:teuthology.orchestra.run.smithi171.stdout:5793 write 0x1631e40 thru 0x1639777 (0x7938 bytes) 2024-09-15T00:12:54.146 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:54.146 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:54.146 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:54.146 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:54.147 INFO:teuthology.orchestra.run.smithi171.stdout:5794 write 0x79dbeaa thru 0x79e306b (0x71c2 bytes) 2024-09-15T00:12:54.153 INFO:teuthology.orchestra.run.smithi171.stdout:5795 write 0xb7b534a thru 0xb7c126d (0xbf24 bytes) 2024-09-15T00:12:54.155 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:54.158 INFO:teuthology.orchestra.run.smithi171.stdout:5797 write 0x6396320 thru 0x639e406 (0x80e7 bytes) 2024-09-15T00:12:54.163 INFO:teuthology.orchestra.run.smithi171.stdout:5798 trunc from 0xdf969db to 0xe79c62b 2024-09-15T00:12:54.169 INFO:teuthology.orchestra.run.smithi171.stdout:5799 writesame 0xcf04023 thru 0xcf0f02b data_size 0x1(0xb009 bytes) 2024-09-15T00:12:54.198 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:54.197+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:54.257 INFO:teuthology.orchestra.run.smithi171.stdout:5800 read 0x5e7c197 thru 0x5e82ae5 (0x694f bytes) 2024-09-15T00:12:54.257 INFO:teuthology.orchestra.run.smithi171.stdout:5802 writesame 0x20ed745 thru 0x20fc1b9 data_size 0xab(0xea75 bytes) 2024-09-15T00:12:54.260 INFO:teuthology.orchestra.run.smithi171.stdout:5803 read 0x204f79c thru 0x205daf3 (0xe358 bytes) 2024-09-15T00:12:54.260 INFO:teuthology.orchestra.run.smithi171.stdout:5804 write 0xdbbb9dd thru 0xdbc7a4a (0xc06e bytes) 2024-09-15T00:12:54.264 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:54.264 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:54.264 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:54.264 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:54.264 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:54.265 INFO:teuthology.orchestra.run.smithi171.stdout:5805 read 0x98be8d7 thru 0x98cad9b (0xc4c5 bytes) 2024-09-15T00:12:54.265 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:54.266 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:54.266 INFO:teuthology.orchestra.run.smithi171.stdout:5808 writesame 0x3b4958e thru 0x3b51cc1 data_size 0x22(0x8734 bytes) 2024-09-15T00:12:54.273 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:54.274 INFO:teuthology.orchestra.run.smithi171.stdout:5809 trunc from 0xe79c62b to 0xbc4aa91 2024-09-15T00:12:54.294 INFO:teuthology.orchestra.run.smithi171.stdout:5811 read 0x158c7bf thru 0x158efcb (0x280d bytes) 2024-09-15T00:12:54.295 INFO:teuthology.orchestra.run.smithi171.stdout:5813 write 0xb32b248 thru 0xb332925 (0x76de bytes) 2024-09-15T00:12:54.298 INFO:teuthology.orchestra.run.smithi171.stdout:5814 read 0xb582b04 thru 0xb58dcde (0xb1db bytes) 2024-09-15T00:12:54.298 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:54.298 INFO:teuthology.orchestra.run.smithi171.stdout:5816 write 0x60660 thru 0x6efa5 (0xe946 bytes) 2024-09-15T00:12:54.299 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:54.296+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:54.302 INFO:teuthology.orchestra.run.smithi171.stdout:5817 punch from 0x89665e5 to 0x89758a3, (0xf2be bytes) 2024-09-15T00:12:54.302 INFO:teuthology.orchestra.run.smithi171.stdout:5818 clone 50 order 18 su 4096 sc 10 2024-09-15T00:12:54.398 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:54.397+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:54.419 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 88.337876582999996, 2024-09-15T00:12:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:12:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:12:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:12:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:12:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:12:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:12:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:12:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:12:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:12:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:12:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:12:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:12:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:12:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:12:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:12:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:12:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:12:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:12:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:12:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:12:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:12:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:12:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:12:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:12:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:12:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:12:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 88.325136040999993, 2024-09-15T00:12:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:12:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:12:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:12:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:12:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:12:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:12:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:12:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:12:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:12:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:12:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:12:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:12:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:12:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:12:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:12:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:12:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 88.192780282000001, 2024-09-15T00:12:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:12:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:12:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:12:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:12:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:12:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:12:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:12:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:12:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:12:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:12:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:12:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:12:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:12:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:12:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:12:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:12:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:12:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:12:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:12:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:12:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:12:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:12:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:12:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:12:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:12:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:12:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 88.179318636999994, 2024-09-15T00:12:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:12:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:12:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:12:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:12:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:12:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:12:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:12:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:12:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:12:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:12:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:12:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:12:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:12:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:12:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:12:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:12:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.994317154000001, 2024-09-15T00:12:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:12:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.440 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:12:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:12:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:12:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:12:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:12:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:12:54.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:12:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:12:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:12:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:12:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:12:54.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:12:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:12:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:12:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:12:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:12:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:54.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:12:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:12:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:12:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:12:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:12:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:12:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:12:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:12:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:12:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:54.448 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.981440887000005, 2024-09-15T00:12:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:12:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:12:54.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:12:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:12:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:12:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:12:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:12:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:12:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:12:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:12:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:12:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:12:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:12:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:12:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:12:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:12:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:12:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 73.522925743000002, 2024-09-15T00:12:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:12:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:12:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.453 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:12:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:12:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:12:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:12:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:12:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:12:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:12:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:12:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:12:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:12:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:12:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:12:54.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:12:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:12:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:12:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:12:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:12:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:12:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:54.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:12:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:12:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:12:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:12:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:12:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:12:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.458 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.805415182999994, 2024-09-15T00:12:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:12:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.459 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:12:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:12:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:12:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:12:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:12:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:12:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:12:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:12:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:12:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:12:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:12:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:12:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:12:54.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:12:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:12:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:12:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:54.463 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.805396574, 2024-09-15T00:12:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:12:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:12:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:12:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:12:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:12:54.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:12:54.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:12:54.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:12:54.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:12:54.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:12:54.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:12:54.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:12:54.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:12:54.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:12:54.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:12:54.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:12:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.805371249999993, 2024-09-15T00:12:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:12:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.468 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:12:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:12:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:12:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:12:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:12:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:12:54.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:12:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:12:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:12:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:12:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:12:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:12:54.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:12:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:12:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:12:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:54.472 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.805352925999998, 2024-09-15T00:12:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:12:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:12:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:54.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:12:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:12:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:12:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:12:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:12:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:12:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:12:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:12:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:12:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:12:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:12:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:12:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:12:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:12:54.476 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.805170352999994, 2024-09-15T00:12:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:12:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.477 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:12:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:54.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:12:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:12:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:12:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:12:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:12:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:12:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:12:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:12:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:12:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:12:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:12:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:12:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:12:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:12:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.481 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.805141512000006, 2024-09-15T00:12:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:12:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:12:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:12:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:12:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:12:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:12:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:12:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:12:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:12:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:12:54.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:12:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:12:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:12:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:12:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:12:54.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:12:54.486 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.486 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.486 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.486 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:54.486 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:54.486 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.805119301000005, 2024-09-15T00:12:54.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:12:54.486 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.486 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.486 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:54.486 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.487 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.487 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.487 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:12:54.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.487 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:54.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:54.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:12:54.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:12:54.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:12:54.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:12:54.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:12:54.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:12:54.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:12:54.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:12:54.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:12:54.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:12:54.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:12:54.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:12:54.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:12:54.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:12:54.490 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.490 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.490 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.490 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:54.491 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:54.491 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.805098142000006, 2024-09-15T00:12:54.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:12:54.491 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.491 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.491 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:54.491 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.491 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.491 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.491 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:12:54.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.491 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:54.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:54.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:12:54.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:12:54.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:12:54.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:12:54.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:12:54.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:12:54.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:12:54.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:12:54.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:12:54.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:12:54.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:12:54.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:12:54.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:12:54.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:12:54.495 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.495 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.495 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.495 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:54.495 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:54.495 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.805068194, 2024-09-15T00:12:54.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:12:54.495 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.495 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.495 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:54.495 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.496 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.496 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.496 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:12:54.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.496 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:54.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:54.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:12:54.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:12:54.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:12:54.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:12:54.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:12:54.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:12:54.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:12:54.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:12:54.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:12:54.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:12:54.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:12:54.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:12:54.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:12:54.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:12:54.499 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.499 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.499 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.499 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:54.500 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:54.500 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.805046230000002, 2024-09-15T00:12:54.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:12:54.500 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.500 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.500 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:54.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.500 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:12:54.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.500 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:54.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:54.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:12:54.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:12:54.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:12:54.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:12:54.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:12:54.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:12:54.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:12:54.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:12:54.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:12:54.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:12:54.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:12:54.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:12:54.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:12:54.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:12:54.504 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.504 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.504 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.504 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:54.504 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:54.504 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.805028802999999, 2024-09-15T00:12:54.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:12:54.504 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.505 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.505 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:54.505 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.505 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.505 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.505 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:12:54.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.505 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:54.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:12:54.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:12:54.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:12:54.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:12:54.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:12:54.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:12:54.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:12:54.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:12:54.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:12:54.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:12:54.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:12:54.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:12:54.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:12:54.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:12:54.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:12:54.508 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.508 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.508 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.509 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:54.509 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:54.509 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.805007004000004, 2024-09-15T00:12:54.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:12:54.509 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.509 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.509 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:54.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.509 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:12:54.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.510 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:54.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:12:54.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:12:54.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:12:54.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:12:54.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:12:54.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:12:54.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:12:54.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:12:54.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:12:54.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:12:54.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:12:54.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:12:54.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:12:54.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:12:54.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:12:54.513 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.513 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.513 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.513 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:54.513 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:54.513 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.755778645000007, 2024-09-15T00:12:54.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:12:54.514 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:54.514 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:54.514 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.514 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:54.514 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:54.514 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.514 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:12:54.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.514 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:54.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:54.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:54.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:54.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:12:54.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:54.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:54.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:12:54.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:12:54.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:54.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:12:54.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:12:54.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:54.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:12:54.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:12:54.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:12:54.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:12:54.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:54.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:12:54.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:12:54.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:54.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:12:54.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:12:54.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:54.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:12:54.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:12:54.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:54.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:12:54.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:12:54.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:54.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:12:54.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:12:54.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:54.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:12:54.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:12:54.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:54.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:54.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:54.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:12:54.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:12:54.519 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.519 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.519 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.519 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:54.519 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:54.519 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:54.521 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:54.519+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:54.521 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:54.621 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:54.620+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:54.626 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:54.626 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:54.626 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:54.626 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:54.635 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:54.722 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:54.721+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:54.778 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:54.779 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:54.779 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:54.779 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:54.779 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:54.788 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:54.824 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:54.822+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:54.894 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:54.895 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:54.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:54.895 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:54.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.895 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133834 3.1b e85/69 3:da9a78a3:::rbd_data.11395dfcf2b8.0000000000000000:head v 85'356, mlcod=85'354)", 2024-09-15T00:12:54.895 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:54.895 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 24.328765713999999, 2024-09-15T00:12:54.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026079054000000001, 2024-09-15T00:12:54.895 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.895 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.896 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.896 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:54.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:54.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563338+0000", 2024-09-15T00:12:54.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.418e-06 2024-09-15T00:12:54.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569385+0000", 2024-09-15T00:12:54.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006047335 2024-09-15T00:12:54.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569418+0000", 2024-09-15T00:12:54.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2957999999999999e-05 2024-09-15T00:12:54.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569432+0000", 2024-09-15T00:12:54.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3718e-05 2024-09-15T00:12:54.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569519+0000", 2024-09-15T00:12:54.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7022999999999997e-05 2024-09-15T00:12:54.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569540+0000", 2024-09-15T00:12:54.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1426000000000001e-05 2024-09-15T00:12:54.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589362+0000", 2024-09-15T00:12:54.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019821398 2024-09-15T00:12:54.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589415+0000", 2024-09-15T00:12:54.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3777999999999999e-05 2024-09-15T00:12:54.899 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.899 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.899 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133836 3.1 e85/67 3:849f9a98:::rbd_data.11395dfcf2b8.0000000000000002:head v 85'304, mlcod=84'301)", 2024-09-15T00:12:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 24.321070804000001, 2024-09-15T00:12:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023231600000000002, 2024-09-15T00:12:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:54.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571032+0000", 2024-09-15T00:12:54.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-15T00:12:54.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576635+0000", 2024-09-15T00:12:54.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056028700000000002 2024-09-15T00:12:54.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576650+0000", 2024-09-15T00:12:54.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.505e-05 2024-09-15T00:12:54.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576658+0000", 2024-09-15T00:12:54.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6879999999999999e-06 2024-09-15T00:12:54.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576717+0000", 2024-09-15T00:12:54.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175000000000002e-05 2024-09-15T00:12:54.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576742+0000", 2024-09-15T00:12:54.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4734000000000001e-05 2024-09-15T00:12:54.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594234+0000", 2024-09-15T00:12:54.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017492527000000001 2024-09-15T00:12:54.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594263+0000", 2024-09-15T00:12:54.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8629e-05 2024-09-15T00:12:54.904 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.904 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.904 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.904 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146609 3.b e98/69 3:d39b42ba:::rbd_data.11394501364.0000000000000001:head v 98'283, mlcod=93'281)", 2024-09-15T00:12:54.904 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:12:54.904 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.411352688999999, 2024-09-15T00:12:54.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020892945, 2024-09-15T00:12:54.904 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.904 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.905 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.905 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:12:54.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:12:54.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480751+0000", 2024-09-15T00:12:54.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6509999999999999e-06 2024-09-15T00:12:54.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482738+0000", 2024-09-15T00:12:54.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019867410000000002 2024-09-15T00:12:54.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482754+0000", 2024-09-15T00:12:54.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6171000000000002e-05 2024-09-15T00:12:54.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482769+0000", 2024-09-15T00:12:54.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4688000000000001e-05 2024-09-15T00:12:54.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482835+0000", 2024-09-15T00:12:54.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6526000000000006e-05 2024-09-15T00:12:54.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482858+0000", 2024-09-15T00:12:54.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2959000000000002e-05 2024-09-15T00:12:54.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501587+0000", 2024-09-15T00:12:54.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018728471 2024-09-15T00:12:54.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501642+0000", 2024-09-15T00:12:54.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5738000000000001e-05 2024-09-15T00:12:54.908 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.908 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.908 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.908 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146835 3.12 e98/68 3:4eee39d6:::rbd_data.11394501364.0000000000000000:head v 98'474, mlcod=93'472)", 2024-09-15T00:12:54.909 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:12:54.909 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.302725419, 2024-09-15T00:12:54.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020080434000000001, 2024-09-15T00:12:54.909 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.909 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.909 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.909 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:12:54.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:12:54.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589378+0000", 2024-09-15T00:12:54.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7770000000000001e-06 2024-09-15T00:12:54.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590678+0000", 2024-09-15T00:12:54.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012991750000000001 2024-09-15T00:12:54.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590699+0000", 2024-09-15T00:12:54.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1353e-05 2024-09-15T00:12:54.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590710+0000", 2024-09-15T00:12:54.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1245e-05 2024-09-15T00:12:54.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590800+0000", 2024-09-15T00:12:54.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9963000000000006e-05 2024-09-15T00:12:54.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590823+0000", 2024-09-15T00:12:54.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3289e-05 2024-09-15T00:12:54.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609368+0000", 2024-09-15T00:12:54.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018545078 2024-09-15T00:12:54.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609457+0000", 2024-09-15T00:12:54.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8554000000000006e-05 2024-09-15T00:12:54.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.913 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.913 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146837 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7201319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:12:54.913 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:12:54.913 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.291945955999999, 2024-09-15T00:12:54.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024751038, 2024-09-15T00:12:54.913 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.913 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.914 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.914 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:54.914 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:54.914 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.914 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146837 2024-09-15T00:12:54.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.914 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:12:54.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:12:54.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600157+0000", 2024-09-15T00:12:54.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:12:54.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601965+0000", 2024-09-15T00:12:54.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001807613 2024-09-15T00:12:54.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601985+0000", 2024-09-15T00:12:54.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0679e-05 2024-09-15T00:12:54.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601998+0000", 2024-09-15T00:12:54.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2955999999999999e-05 2024-09-15T00:12:54.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602070+0000", 2024-09-15T00:12:54.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1958000000000005e-05 2024-09-15T00:12:54.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602237+0000", 2024-09-15T00:12:54.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016621299999999999 2024-09-15T00:12:54.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:54.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602799+0000", 2024-09-15T00:12:54.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000562723 2024-09-15T00:12:54.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:54.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.620811+0000", 2024-09-15T00:12:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018011207000000001 2024-09-15T00:12:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624793+0000", 2024-09-15T00:12:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0039828770000000001 2024-09-15T00:12:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624855+0000", 2024-09-15T00:12:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.143e-05 2024-09-15T00:12:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624907+0000", 2024-09-15T00:12:54.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2262000000000002e-05 2024-09-15T00:12:54.919 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.919 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.919 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.919 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146936 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:12:54.919 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:12:54.919 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.138432636999999, 2024-09-15T00:12:54.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056847126999999997, 2024-09-15T00:12:54.919 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.919 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.919 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.919 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:54.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:54.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.920 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146936 2024-09-15T00:12:54.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.920 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:12:54.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:12:54.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753671+0000", 2024-09-15T00:12:54.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2839999999999999e-06 2024-09-15T00:12:54.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753677+0000", 2024-09-15T00:12:54.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0260000000000004e-06 2024-09-15T00:12:54.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753691+0000", 2024-09-15T00:12:54.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3891e-05 2024-09-15T00:12:54.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753698+0000", 2024-09-15T00:12:54.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8480000000000001e-06 2024-09-15T00:12:54.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753750+0000", 2024-09-15T00:12:54.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1338000000000002e-05 2024-09-15T00:12:54.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753814+0000", 2024-09-15T00:12:54.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4642999999999998e-05 2024-09-15T00:12:54.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:54.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.756359+0000", 2024-09-15T00:12:54.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025450619999999998 2024-09-15T00:12:54.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:54.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810435+0000", 2024-09-15T00:12:54.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.054075397999999997 2024-09-15T00:12:54.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:54.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810446+0000", 2024-09-15T00:12:54.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1005000000000001e-05 2024-09-15T00:12:54.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810482+0000", 2024-09-15T00:12:54.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6494e-05 2024-09-15T00:12:54.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810517+0000", 2024-09-15T00:12:54.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4137999999999997e-05 2024-09-15T00:12:54.924 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.924 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.925 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.925 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:146936 3.6 e98/97 ondisk, result = 0)", 2024-09-15T00:12:54.925 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:12:54.925 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.128649583, 2024-09-15T00:12:54.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046987205999999997, 2024-09-15T00:12:54.925 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.925 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.925 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:54.925 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:12:54.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:12:54.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763453+0000", 2024-09-15T00:12:54.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5199999999999996e-07 2024-09-15T00:12:54.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763457+0000", 2024-09-15T00:12:54.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8569999999999997e-06 2024-09-15T00:12:54.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763469+0000", 2024-09-15T00:12:54.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1977999999999999e-05 2024-09-15T00:12:54.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763476+0000", 2024-09-15T00:12:54.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:12:54.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810423+0000", 2024-09-15T00:12:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946993999999999 2024-09-15T00:12:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810431+0000", 2024-09-15T00:12:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0339999999999997e-06 2024-09-15T00:12:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810440+0000", 2024-09-15T00:12:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9989999999999997e-06 2024-09-15T00:12:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.929 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.929 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:151036 3.1f e103/68 3:f84a1a09:::rbd_data.113958cb0e55.000000000000000c:head v 103'452, mlcod=100'450)", 2024-09-15T00:12:54.929 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:12:54.929 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.5553033550000004, 2024-09-15T00:12:54.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020618753, 2024-09-15T00:12:54.929 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.929 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.929 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.929 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:12:54.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:12:54.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:12:54.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:12:54.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340692+0000", 2024-09-15T00:12:54.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038928890000000001 2024-09-15T00:12:54.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340704+0000", 2024-09-15T00:12:54.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1497e-05 2024-09-15T00:12:54.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340709+0000", 2024-09-15T00:12:54.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1529999999999996e-06 2024-09-15T00:12:54.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340789+0000", 2024-09-15T00:12:54.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9907999999999995e-05 2024-09-15T00:12:54.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340810+0000", 2024-09-15T00:12:54.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1837000000000001e-05 2024-09-15T00:12:54.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357368+0000", 2024-09-15T00:12:54.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016557199000000002 2024-09-15T00:12:54.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357417+0000", 2024-09-15T00:12:54.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9783000000000001e-05 2024-09-15T00:12:54.933 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.933 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.933 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.933 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:151042 3.6 3:6419cde4:::rbd_data.113958cb0e55.0000000000000031:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:12:54.933 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:12:54.933 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.5262252370000002, 2024-09-15T00:12:54.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025627375000000001, 2024-09-15T00:12:54.933 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.933 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.933 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.934 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:54.934 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:54.934 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.934 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 151042 2024-09-15T00:12:54.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.934 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:12:54.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:12:54.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:12:54.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7899999999999998e-07 2024-09-15T00:12:54.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365881+0000", 2024-09-15T00:12:54.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9770000000000002e-06 2024-09-15T00:12:54.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365890+0000", 2024-09-15T00:12:54.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5920000000000001e-06 2024-09-15T00:12:54.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365894+0000", 2024-09-15T00:12:54.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4569999999999998e-06 2024-09-15T00:12:54.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365927+0000", 2024-09-15T00:12:54.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2036000000000003e-05 2024-09-15T00:12:54.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365970+0000", 2024-09-15T00:12:54.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3197e-05 2024-09-15T00:12:54.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:54.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.367432+0000", 2024-09-15T00:12:54.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014621720000000001 2024-09-15T00:12:54.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:54.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391358+0000", 2024-09-15T00:12:54.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023925921999999999 2024-09-15T00:12:54.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:54.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391415+0000", 2024-09-15T00:12:54.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7253999999999998e-05 2024-09-15T00:12:54.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391456+0000", 2024-09-15T00:12:54.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0497999999999997e-05 2024-09-15T00:12:54.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391504+0000", 2024-09-15T00:12:54.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8591000000000003e-05 2024-09-15T00:12:54.938 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.939 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.939 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:151042 3.6 e103/97 ondisk, result = 0)", 2024-09-15T00:12:54.939 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:12:54.939 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.5204822240000002, 2024-09-15T00:12:54.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019892730000000001, 2024-09-15T00:12:54.939 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.939 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.939 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:54.939 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:12:54.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:12:54.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:12:54.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8100000000000003e-07 2024-09-15T00:12:54.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371623+0000", 2024-09-15T00:12:54.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6290000000000002e-06 2024-09-15T00:12:54.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371632+0000", 2024-09-15T00:12:54.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6379999999999993e-06 2024-09-15T00:12:54.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371636+0000", 2024-09-15T00:12:54.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:12:54.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391403+0000", 2024-09-15T00:12:54.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019766982999999998 2024-09-15T00:12:54.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391411+0000", 2024-09-15T00:12:54.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7309999999999993e-06 2024-09-15T00:12:54.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391513+0000", 2024-09-15T00:12:54.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101648 2024-09-15T00:12:54.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.944 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.945 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152025 3.3 e103/68 3:c0279197:::rbd_data.113958cb0e55.0000000000000007:head v 103'464, mlcod=102'461)", 2024-09-15T00:12:54.945 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:12:54.945 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.3759946279999999, 2024-09-15T00:12:54.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021264476000000001, 2024-09-15T00:12:54.945 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.945 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.945 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.945 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:12:54.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:12:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516109+0000", 2024-09-15T00:12:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.666e-06 2024-09-15T00:12:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.519997+0000", 2024-09-15T00:12:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003888449 2024-09-15T00:12:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520016+0000", 2024-09-15T00:12:54.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8059000000000001e-05 2024-09-15T00:12:54.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520026+0000", 2024-09-15T00:12:54.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9599999999999995e-06 2024-09-15T00:12:54.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520089+0000", 2024-09-15T00:12:54.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3074e-05 2024-09-15T00:12:54.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520108+0000", 2024-09-15T00:12:54.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8981999999999999e-05 2024-09-15T00:12:54.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537326+0000", 2024-09-15T00:12:54.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017218102999999998 2024-09-15T00:12:54.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537372+0000", 2024-09-15T00:12:54.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6183000000000002e-05 2024-09-15T00:12:54.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.949 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.949 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152284 3.f e103/68 3:f4272b5d:::rbd_data.113958cb0e55.0000000000000006:head v 103'370, mlcod=103'368)", 2024-09-15T00:12:54.949 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:12:54.949 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.3275180970000005, 2024-09-15T00:12:54.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020145026999999999, 2024-09-15T00:12:54.949 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.949 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.949 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.949 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:12:54.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:12:54.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564585+0000", 2024-09-15T00:12:54.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7000000000000003e-07 2024-09-15T00:12:54.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568352+0000", 2024-09-15T00:12:54.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037675909999999998 2024-09-15T00:12:54.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568370+0000", 2024-09-15T00:12:54.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7158000000000001e-05 2024-09-15T00:12:54.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568378+0000", 2024-09-15T00:12:54.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6449999999999994e-06 2024-09-15T00:12:54.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568454+0000", 2024-09-15T00:12:54.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5285999999999997e-05 2024-09-15T00:12:54.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568472+0000", 2024-09-15T00:12:54.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8447000000000001e-05 2024-09-15T00:12:54.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584693+0000", 2024-09-15T00:12:54.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016221467999999999 2024-09-15T00:12:54.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584729+0000", 2024-09-15T00:12:54.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5462e-05 2024-09-15T00:12:54.953 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.953 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.953 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.953 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152500 3.17 e103/68 3:edff5adf:::rbd_data.113958cb0e55.000000000000000d:head v 103'461, mlcod=103'459)", 2024-09-15T00:12:54.953 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:12:54.953 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.2505315120000002, 2024-09-15T00:12:54.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020217809, 2024-09-15T00:12:54.954 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.954 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.954 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.954 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:12:54.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:12:54.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641572+0000", 2024-09-15T00:12:54.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-15T00:12:54.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644961+0000", 2024-09-15T00:12:54.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033882610000000001 2024-09-15T00:12:54.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644979+0000", 2024-09-15T00:12:54.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8179e-05 2024-09-15T00:12:54.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644987+0000", 2024-09-15T00:12:54.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8860000000000004e-06 2024-09-15T00:12:54.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645044+0000", 2024-09-15T00:12:54.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7141000000000001e-05 2024-09-15T00:12:54.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645062+0000", 2024-09-15T00:12:54.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8258e-05 2024-09-15T00:12:54.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661749+0000", 2024-09-15T00:12:54.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016686553999999999 2024-09-15T00:12:54.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661788+0000", 2024-09-15T00:12:54.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9690000000000001e-05 2024-09-15T00:12:54.957 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.957 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.958 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.958 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153171 3.18 e103/69 3:1db186e8:::rbd_data.113958cb0e55.0000000000000002:head v 103'520, mlcod=103'518)", 2024-09-15T00:12:54.958 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:12:54.958 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.1120466059999998, 2024-09-15T00:12:54.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022266838000000001, 2024-09-15T00:12:54.958 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.958 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.958 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.958 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:12:54.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:12:54.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780057+0000", 2024-09-15T00:12:54.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5880000000000001e-06 2024-09-15T00:12:54.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781273+0000", 2024-09-15T00:12:54.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001216087 2024-09-15T00:12:54.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781311+0000", 2024-09-15T00:12:54.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7656e-05 2024-09-15T00:12:54.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781325+0000", 2024-09-15T00:12:54.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4508999999999999e-05 2024-09-15T00:12:54.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781403+0000", 2024-09-15T00:12:54.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7927000000000001e-05 2024-09-15T00:12:54.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781422+0000", 2024-09-15T00:12:54.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8575000000000001e-05 2024-09-15T00:12:54.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802249+0000", 2024-09-15T00:12:54.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020827577 2024-09-15T00:12:54.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802322+0000", 2024-09-15T00:12:54.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2918999999999998e-05 2024-09-15T00:12:54.962 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.962 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.962 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.962 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153772 3.12 e103/68 3:4911a4f6:::rbd_data.113958cb0e55.0000000000000001:head v 103'490, mlcod=103'488)", 2024-09-15T00:12:54.962 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:12:54.962 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0091759790000001, 2024-09-15T00:12:54.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021181004, 2024-09-15T00:12:54.962 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.962 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.962 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.963 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:12:54.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:12:54.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882927+0000", 2024-09-15T00:12:54.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.426e-06 2024-09-15T00:12:54.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883966+0000", 2024-09-15T00:12:54.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010384610000000001 2024-09-15T00:12:54.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883985+0000", 2024-09-15T00:12:54.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9276e-05 2024-09-15T00:12:54.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883994+0000", 2024-09-15T00:12:54.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9490000000000002e-06 2024-09-15T00:12:54.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884054+0000", 2024-09-15T00:12:54.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9419999999999997e-05 2024-09-15T00:12:54.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884073+0000", 2024-09-15T00:12:54.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9137e-05 2024-09-15T00:12:54.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904038+0000", 2024-09-15T00:12:54.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019965092 2024-09-15T00:12:54.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904107+0000", 2024-09-15T00:12:54.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9243000000000001e-05 2024-09-15T00:12:54.966 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.966 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.966 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.966 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163111 3.18 e106/69 3:18b9f527:::rbd_data.11395d771336.0000000000000000:head v 106'538, mlcod=105'536)", 2024-09-15T00:12:54.967 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.463478+0000", 2024-09-15T00:12:54.967 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.428624471, 2024-09-15T00:12:54.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015225839999999999, 2024-09-15T00:12:54.967 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.967 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.967 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.967 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.463478+0000", 2024-09-15T00:12:54.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.463478+0000", 2024-09-15T00:12:54.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.463479+0000", 2024-09-15T00:12:54.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.438e-06 2024-09-15T00:12:54.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464332+0000", 2024-09-15T00:12:54.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00085324499999999998 2024-09-15T00:12:54.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464356+0000", 2024-09-15T00:12:54.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4173e-05 2024-09-15T00:12:54.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464370+0000", 2024-09-15T00:12:54.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3539e-05 2024-09-15T00:12:54.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464439+0000", 2024-09-15T00:12:54.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8668000000000001e-05 2024-09-15T00:12:54.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464462+0000", 2024-09-15T00:12:54.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3034999999999999e-05 2024-09-15T00:12:54.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.478656+0000", 2024-09-15T00:12:54.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014194783000000001 2024-09-15T00:12:54.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.478703+0000", 2024-09-15T00:12:54.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6959000000000002e-05 2024-09-15T00:12:54.971 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.971 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.971 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.971 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163112 3.3 e106/68 3:c668f3d5:::rbd_data.11395d771336.0000000000000001:head v 106'489, mlcod=105'487)", 2024-09-15T00:12:54.971 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:12:54.971 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.42727871, 2024-09-15T00:12:54.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019317183000000002, 2024-09-15T00:12:54.971 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.971 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.971 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.971 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:12:54.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:12:54.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464824+0000", 2024-09-15T00:12:54.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1100000000000005e-07 2024-09-15T00:12:54.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466874+0000", 2024-09-15T00:12:54.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002049497 2024-09-15T00:12:54.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466884+0000", 2024-09-15T00:12:54.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0424999999999999e-05 2024-09-15T00:12:54.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466890+0000", 2024-09-15T00:12:54.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-15T00:12:54.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466944+0000", 2024-09-15T00:12:54.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4592999999999997e-05 2024-09-15T00:12:54.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466958+0000", 2024-09-15T00:12:54.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3993e-05 2024-09-15T00:12:54.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484068+0000", 2024-09-15T00:12:54.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017109994 2024-09-15T00:12:54.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484140+0000", 2024-09-15T00:12:54.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2224000000000005e-05 2024-09-15T00:12:54.975 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.975 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.975 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.975 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163117 3.1f e106/68 3:fbcc0048:::rbd_data.11395d771336.0000000000000003:head v 106'489, mlcod=106'487)", 2024-09-15T00:12:54.975 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.470675+0000", 2024-09-15T00:12:54.975 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4214266520000001, 2024-09-15T00:12:54.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013840234999999999, 2024-09-15T00:12:54.975 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.976 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.976 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.976 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.470675+0000", 2024-09-15T00:12:54.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.470675+0000", 2024-09-15T00:12:54.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.470676+0000", 2024-09-15T00:12:54.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:12:54.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.471937+0000", 2024-09-15T00:12:54.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001261362 2024-09-15T00:12:54.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.471949+0000", 2024-09-15T00:12:54.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1479e-05 2024-09-15T00:12:54.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.471955+0000", 2024-09-15T00:12:54.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7190000000000004e-06 2024-09-15T00:12:54.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.472018+0000", 2024-09-15T00:12:54.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2850999999999998e-05 2024-09-15T00:12:54.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.472035+0000", 2024-09-15T00:12:54.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6626999999999999e-05 2024-09-15T00:12:54.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484475+0000", 2024-09-15T00:12:54.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012440094 2024-09-15T00:12:54.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484516+0000", 2024-09-15T00:12:54.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0698000000000002e-05 2024-09-15T00:12:54.979 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.979 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.980 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.980 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163119 3.0 e106/94 3:06118613:::rbd_data.11395d771336.0000000000000004:head v 106'549, mlcod=105'547)", 2024-09-15T00:12:54.980 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:12:54.980 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4156635070000001, 2024-09-15T00:12:54.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017175867000000001, 2024-09-15T00:12:54.980 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.980 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.980 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.980 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:12:54.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:12:54.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:12:54.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6700000000000002e-07 2024-09-15T00:12:54.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477786+0000", 2024-09-15T00:12:54.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013468149999999999 2024-09-15T00:12:54.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477797+0000", 2024-09-15T00:12:54.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0635000000000001e-05 2024-09-15T00:12:54.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477805+0000", 2024-09-15T00:12:54.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2989999999999997e-06 2024-09-15T00:12:54.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477859+0000", 2024-09-15T00:12:54.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3699000000000002e-05 2024-09-15T00:12:54.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477878+0000", 2024-09-15T00:12:54.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9335e-05 2024-09-15T00:12:54.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493560+0000", 2024-09-15T00:12:54.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015681748999999998 2024-09-15T00:12:54.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493614+0000", 2024-09-15T00:12:54.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4367999999999999e-05 2024-09-15T00:12:54.984 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.984 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.984 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.984 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:163139 3.1 3:80ed57f9:::rbd_data.11395d771336.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:54.984 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.480656+0000", 2024-09-15T00:12:54.984 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4114460120000001, 2024-09-15T00:12:54.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013898246, 2024-09-15T00:12:54.985 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:54.985 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:54.985 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:54.985 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:54.985 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:54.985 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:54.985 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 163139 2024-09-15T00:12:54.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.985 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:54.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:54.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.480656+0000", 2024-09-15T00:12:54.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.480656+0000", 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.480658+0000", 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0140000000000001e-06 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.481554+0000", 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00089608500000000002 2024-09-15T00:12:54.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:54.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.481568+0000", 2024-09-15T00:12:54.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4331999999999999e-05 2024-09-15T00:12:54.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:54.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.481576+0000", 2024-09-15T00:12:54.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0399999999999993e-06 2024-09-15T00:12:54.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:54.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.481659+0000", 2024-09-15T00:12:54.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3004999999999996e-05 2024-09-15T00:12:54.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:54.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.481787+0000", 2024-09-15T00:12:54.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012715299999999999 2024-09-15T00:12:54.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:54.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.482096+0000", 2024-09-15T00:12:54.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00030970100000000001 2024-09-15T00:12:54.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:54.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.494197+0000", 2024-09-15T00:12:54.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012100319 2024-09-15T00:12:54.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:54.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.494462+0000", 2024-09-15T00:12:54.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026532599999999997 2024-09-15T00:12:54.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:54.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.494513+0000", 2024-09-15T00:12:54.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0739999999999999e-05 2024-09-15T00:12:54.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:54.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:54.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:54.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.494554+0000", 2024-09-15T00:12:54.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1530999999999999e-05 2024-09-15T00:12:54.990 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.990 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.990 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.990 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:54.990 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:54.990 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:54.992 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:54.992 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:54.990+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:55.092 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:55.091+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:55.146 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:55.146 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:55.146 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:55.146 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:55.155 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:55.186 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:12:55.196 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:55.194+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:55.294 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:55.292+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:55.307 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:55.308 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:55.308 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:55.308 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:55.308 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:55.308 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #48, image image_client.0-clone48 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent49 2024-09-15T00:12:55.319 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:55.341 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:55.342 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:55.367 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:55.368 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:55.386 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:12:55.386 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-15T00:12:55.395 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:55.393+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:55.403 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:55.403 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:55.422 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:55.422 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:55.437 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:55.438 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:55.439 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:55.439 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:55.447 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:55.457 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:55.458 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:55.475 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:55.476 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:55.496 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:55.494+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:55.503 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:55.503 INFO:teuthology.orchestra.run.smithi171.stdout:5822 write 0x8c1c2b6 thru 0x8c1c9ab (0x6f6 bytes) 2024-09-15T00:12:55.514 INFO:teuthology.orchestra.run.smithi171.stdout:5823 read 0x8fea589 thru 0x8fefed0 (0x5948 bytes) 2024-09-15T00:12:55.525 INFO:teuthology.orchestra.run.smithi171.stdout:5824 write 0xe7612c0 thru 0xe76c3ae (0xb0ef bytes) 2024-09-15T00:12:55.545 INFO:teuthology.orchestra.run.smithi171.stdout:5825 writesame 0x550fcd9 thru 0x551ea37 data_size 0x1(0xed5f bytes) 2024-09-15T00:12:55.571 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:55.572 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:55.583 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:55.594 INFO:teuthology.orchestra.run.smithi171.stdout:5826 trunc from 0xe76c3af to 0x4f52a80 2024-09-15T00:12:55.596 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:55.594+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:55.629 INFO:teuthology.orchestra.run.smithi171.stdout:5827 write 0x4676bb3 thru 0x467a4f9 (0x3947 bytes) 2024-09-15T00:12:55.637 INFO:teuthology.orchestra.run.smithi171.stdout:5829 punch from 0x2d9d5e1 to 0x2da4329, (0x6d48 bytes) 2024-09-15T00:12:55.637 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:55.637 INFO:teuthology.orchestra.run.smithi171.stdout:5831 read 0x28e36a6 thru 0x28f2e98 (0xf7f3 bytes) 2024-09-15T00:12:55.653 INFO:teuthology.orchestra.run.smithi171.stdout:5832 write 0xdedc664 thru 0xdee8c77 (0xc614 bytes) 2024-09-15T00:12:55.667 INFO:teuthology.orchestra.run.smithi171.stdout:5833 read 0x98336a1 thru 0x98375b7 (0x3f17 bytes) 2024-09-15T00:12:55.667 INFO:teuthology.orchestra.run.smithi171.stdout:5834 read 0x7847992 thru 0x78514a3 (0x9b12 bytes) 2024-09-15T00:12:55.676 INFO:teuthology.orchestra.run.smithi171.stdout:5835 trunc from 0xdee8c78 to 0x68b549d 2024-09-15T00:12:55.697 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:55.695+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:55.707 INFO:teuthology.orchestra.run.smithi171.stdout:5836 write 0xae9b240 thru 0xaea356a (0x832b bytes) 2024-09-15T00:12:55.711 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:55.711 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:55.720 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:55.722 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:55.730 INFO:teuthology.orchestra.run.smithi171.stdout:5839 trunc from 0xaea356b to 0x237104a 2024-09-15T00:12:55.768 INFO:teuthology.orchestra.run.smithi171.stdout:5840 read 0x18058a0 thru 0x180d3b9 (0x7b1a bytes) 2024-09-15T00:12:55.769 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:55.769 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:55.778 INFO:teuthology.orchestra.run.smithi171.stdout:5847 write 0x62d18f5 thru 0x62d1960 (0x6c bytes) 2024-09-15T00:12:55.781 INFO:teuthology.orchestra.run.smithi171.stdout:5849 write 0x48c4ee7 thru 0x48cd613 (0x872d bytes) 2024-09-15T00:12:55.798 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:55.796+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:55.798 INFO:teuthology.orchestra.run.smithi171.stdout:5851 writesame 0x53d2aec thru 0x53e0d1b data_size 0x70(0xe230 bytes) 2024-09-15T00:12:55.828 INFO:teuthology.orchestra.run.smithi171.stdout:5852 writesame 0xa611394 thru 0xa61c1e1 data_size 0x4a(0xae4e bytes) 2024-09-15T00:12:55.828 INFO:teuthology.orchestra.run.smithi171.stdout:5853 read 0x64e0550 thru 0x64e80f0 (0x7ba1 bytes) 2024-09-15T00:12:55.828 INFO:teuthology.orchestra.run.smithi171.stdout:5854 trunc from 0xa61c1e2 to 0xa7369fc 2024-09-15T00:12:55.833 INFO:teuthology.orchestra.run.smithi171.stdout:5855 writesame 0x67e2c70 thru 0x67e423c data_size 0x1(0x15cd bytes) 2024-09-15T00:12:55.849 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:55.850 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:55.851 INFO:teuthology.orchestra.run.smithi171.stdout:5856 trunc from 0xa7369fc to 0x292acaf 2024-09-15T00:12:55.861 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:55.873 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:55.873 INFO:teuthology.orchestra.run.smithi171.stdout:5861 read 0x126fc55 thru 0x127e8c0 (0xec6c bytes) 2024-09-15T00:12:55.877 INFO:teuthology.orchestra.run.smithi171.stdout:5862 punch from 0xf79ad1 to 0xf7a205, (0x734 bytes) 2024-09-15T00:12:55.877 INFO:teuthology.orchestra.run.smithi171.stdout:5863 punch from 0x544223 to 0x5456eb, (0x14c8 bytes) 2024-09-15T00:12:55.877 INFO:teuthology.orchestra.run.smithi171.stdout:5864 read 0x1c8ceee thru 0x1c8deba (0xfcd bytes) 2024-09-15T00:12:55.877 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:55.886 INFO:teuthology.orchestra.run.smithi171.stdout:5866 write 0x53ed74b thru 0x53eebfc (0x14b2 bytes) 2024-09-15T00:12:55.891 INFO:teuthology.orchestra.run.smithi171.stdout:5867 trunc from 0x53eebfd to 0x5dd584c 2024-09-15T00:12:55.897 INFO:teuthology.orchestra.run.smithi171.stdout:5868 writesame 0x2ab740d thru 0x2ac1afa data_size 0x17(0xa6ee bytes) 2024-09-15T00:12:55.899 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:55.897+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:55.911 INFO:teuthology.orchestra.run.smithi171.stdout:5869 read 0x4fa3144 thru 0x4fb1ca4 (0xeb61 bytes) 2024-09-15T00:12:55.911 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:55.911 INFO:teuthology.orchestra.run.smithi171.stdout:5871 read 0x25f68b6 thru 0x25f72e3 (0xa2e bytes) 2024-09-15T00:12:55.911 INFO:teuthology.orchestra.run.smithi171.stdout:5873 read 0x1920323 thru 0x192436e (0x404c bytes) 2024-09-15T00:12:55.911 INFO:teuthology.orchestra.run.smithi171.stdout:5874 write 0xe982be thru 0xe9d0fe (0x4e41 bytes) 2024-09-15T00:12:55.933 INFO:teuthology.orchestra.run.smithi171.stdout:5875 writesame 0x1bc4b5 thru 0x1c4df4 data_size 0x90(0x8940 bytes) 2024-09-15T00:12:55.977 INFO:teuthology.orchestra.run.smithi171.stdout:5879 write 0x67a79f9 thru 0x67b3833 (0xbe3b bytes) 2024-09-15T00:12:55.993 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:55.993 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:55.994 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:56.000 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:55.998+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:56.000 INFO:teuthology.orchestra.run.smithi171.stdout:5882 trunc from 0x67b3834 to 0x1692bc9 2024-09-15T00:12:56.002 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:56.073 INFO:teuthology.orchestra.run.smithi171.stdout:5883 write 0xc18bf04 thru 0xc19afe2 (0xf0df bytes) 2024-09-15T00:12:56.087 INFO:teuthology.orchestra.run.smithi171.stdout:5884 read 0x245c138 thru 0x245d671 (0x153a bytes) 2024-09-15T00:12:56.090 INFO:teuthology.orchestra.run.smithi171.stdout:5885 trunc from 0xc19afe3 to 0x97cb8a6 2024-09-15T00:12:56.100 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:56.099+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:56.108 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:12:56.108 INFO:teuthology.orchestra.run.smithi171.stdout:5886 read 0x8b93315 thru 0x8b94ced (0x19d9 bytes) 2024-09-15T00:12:56.109 INFO:teuthology.orchestra.run.smithi171.stdout:5887 read 0x1f01b0c thru 0x1f02f08 (0x13fd bytes) 2024-09-15T00:12:56.117 INFO:teuthology.orchestra.run.smithi171.stdout:5888 writesame 0xbf56dca thru 0xbf64e56 data_size 0x1(0xe08d bytes) 2024-09-15T00:12:56.118 INFO:teuthology.orchestra.run.smithi171.stdout:5889 writesame 0x6001f4d thru 0x6010fac data_size 0x30(0xf060 bytes) 2024-09-15T00:12:56.131 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:56.131 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:56.136 INFO:teuthology.orchestra.run.smithi171.stdout:5890 read 0x8a430e6 thru 0x8a48e6a (0x5d85 bytes) 2024-09-15T00:12:56.136 INFO:teuthology.orchestra.run.smithi171.stdout:5894 write 0x2fbaca4 thru 0x2fc2e52 (0x81af bytes) 2024-09-15T00:12:56.142 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:56.153 INFO:teuthology.orchestra.run.smithi171.stdout:5895 write 0x67b927e thru 0x67beaff (0x5882 bytes) 2024-09-15T00:12:56.171 INFO:teuthology.orchestra.run.smithi171.stdout:5896 trunc from 0xbf64e57 to 0x966411a 2024-09-15T00:12:56.188 INFO:teuthology.orchestra.run.smithi171.stdout:5897 write 0xdab2e16 thru 0xdabb76f (0x895a bytes) 2024-09-15T00:12:56.205 INFO:teuthology.orchestra.run.smithi171.stdout:5898 writesame 0x40d06ab thru 0x40d5a5a data_size 0x67(0x53b0 bytes) 2024-09-15T00:12:56.218 INFO:teuthology.orchestra.run.smithi171.stdout:5901 writesame 0xb10fc80 thru 0xb1161ed data_size 0x1(0x656e bytes) 2024-09-15T00:12:56.226 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:56.224+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:56.236 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:56.236 INFO:teuthology.orchestra.run.smithi171.stdout:5904 write 0x7b82d3a thru 0x7b8bc56 (0x8f1d bytes) 2024-09-15T00:12:56.255 INFO:teuthology.orchestra.run.smithi171.stdout:5907 writesame 0x63711d5 thru 0x637eee4 data_size 0x24(0xdd10 bytes) 2024-09-15T00:12:56.269 INFO:teuthology.orchestra.run.smithi171.stdout:5908 read 0xbfc602a thru 0xbfd43ea (0xe3c1 bytes) 2024-09-15T00:12:56.269 INFO:teuthology.orchestra.run.smithi171.stdout:5909 write 0x4da746e thru 0x4daed5d (0x78f0 bytes) 2024-09-15T00:12:56.289 INFO:teuthology.orchestra.run.smithi171.stdout:5911 read 0xa3aa667 thru 0xa3ba21f (0xfbb9 bytes) 2024-09-15T00:12:56.289 INFO:teuthology.orchestra.run.smithi171.stdout:5912 write 0xc8d50aa thru 0xc8e4627 (0xf57e bytes) 2024-09-15T00:12:56.301 INFO:teuthology.orchestra.run.smithi171.stdout:5914 punch from 0xa578498 to 0xa585fe7, (0xdb4f bytes) 2024-09-15T00:12:56.301 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:56.300+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:56.303 INFO:teuthology.orchestra.run.smithi171.stdout:5915 trunc from 0xdabb770 to 0xbf21b6d 2024-09-15T00:12:56.311 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:56.311 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:56.321 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:56.322 INFO:teuthology.orchestra.run.smithi171.stdout:5916 writesame 0x3183981 thru 0x318c9f9 data_size 0xd(0x9079 bytes) 2024-09-15T00:12:56.333 INFO:teuthology.orchestra.run.smithi171.stdout:5917 punch from 0x6e6bad5 to 0x6e7babf, (0xffea bytes) 2024-09-15T00:12:56.333 INFO:teuthology.orchestra.run.smithi171.stdout:5918 punch from 0x59360fe to 0x593c724, (0x6626 bytes) 2024-09-15T00:12:56.333 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:56.333 INFO:teuthology.orchestra.run.smithi171.stdout:5922 read 0xb3690cd thru 0xb370a4f (0x7983 bytes) 2024-09-15T00:12:56.333 INFO:teuthology.orchestra.run.smithi171.stdout:5923 read 0x21d1496 thru 0x21d8af8 (0x7663 bytes) 2024-09-15T00:12:56.333 INFO:teuthology.orchestra.run.smithi171.stdout:5924 write 0x8abcccd thru 0x8ac9cbd (0xcff1 bytes) 2024-09-15T00:12:56.347 INFO:teuthology.orchestra.run.smithi171.stdout:5925 punch from 0xa045786 to 0xa04a9b5, (0x522f bytes) 2024-09-15T00:12:56.347 INFO:teuthology.orchestra.run.smithi171.stdout:5926 read 0x88cb079 thru 0x88d7c93 (0xcc1b bytes) 2024-09-15T00:12:56.347 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:56.347 INFO:teuthology.orchestra.run.smithi171.stdout:5930 read 0xd5ef7a thru 0xd6ef0b (0xff92 bytes) 2024-09-15T00:12:56.356 INFO:teuthology.orchestra.run.smithi171.stdout:5932 trunc from 0xbf21b6d to 0x4be5eb0 2024-09-15T00:12:56.388 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:56.388 INFO:teuthology.orchestra.run.smithi171.stdout:5936 read 0x16505ce thru 0x1651dcc (0x17ff bytes) 2024-09-15T00:12:56.389 INFO:teuthology.orchestra.run.smithi171.stdout:5937 punch from 0x2f72f12 to 0x2f7c49b, (0x9589 bytes) 2024-09-15T00:12:56.389 INFO:teuthology.orchestra.run.smithi171.stdout:5938 read 0x1297d14 thru 0x12a3d53 (0xc040 bytes) 2024-09-15T00:12:56.402 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:56.401+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:56.405 INFO:teuthology.orchestra.run.smithi171.stdout:5940 writesame 0xc319be0 thru 0xc31bba4 data_size 0x1(0x1fc5 bytes) 2024-09-15T00:12:56.405 INFO:teuthology.orchestra.run.smithi171.stdout:5942 write 0x9607381 thru 0x960bbb3 (0x4833 bytes) 2024-09-15T00:12:56.412 INFO:teuthology.orchestra.run.smithi171.stdout:5944 read 0xaa688c0 thru 0xaa6c58d (0x3cce bytes) 2024-09-15T00:12:56.413 INFO:teuthology.orchestra.run.smithi171.stdout:5946 punch from 0x2fd5c to 0x37796, (0x7a3a bytes) 2024-09-15T00:12:56.415 INFO:teuthology.orchestra.run.smithi171.stdout:5948 trunc from 0xc31bba5 to 0xa05b329 2024-09-15T00:12:56.421 INFO:teuthology.orchestra.run.smithi171.stdout:5951 punch from 0x6b1db3b to 0x6b255ba, (0x7a7f bytes) 2024-09-15T00:12:56.421 INFO:teuthology.orchestra.run.smithi171.stdout:5953 writesame 0x939482e thru 0x9399f30 data_size 0x5(0x5703 bytes) 2024-09-15T00:12:56.430 INFO:teuthology.orchestra.run.smithi171.stdout:5954 clone 51 order 19 su 16384 sc 15 2024-09-15T00:12:56.475 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:56.475 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:56.484 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:56.503 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:56.502+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:56.587 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:56.587 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:56.595 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:56.639 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:56.637+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:56.698 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:56.698 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:56.704 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:56.703+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:56.707 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:12:56.805 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:56.804+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:56.850 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:56.851 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:56.859 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:12:56.907 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:56.905+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:57.003 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:57.003 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:57.007 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:57.006+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:57.011 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:12:57.108 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:57.107+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:57.117 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is set 2024-09-15T00:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:57.169 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:12:57.189 INFO:teuthology.orchestra.run.smithi171.stdout:truncating image image_client.0-clone50 from 0xa05b329 (overlap 0x1692bc9) to 0x120efd4 2024-09-15T00:12:57.210 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:57.208+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:57.310 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:57.308+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:57.325 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #49, image image_client.0-clone49 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent50 2024-09-15T00:12:57.333 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:57.333 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:57.342 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:12:57.411 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:57.409+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:57.485 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:57.485 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:57.488 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:12:57.494 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:12:57.533 INFO:teuthology.orchestra.run.smithi171.stdout:5955 read 0x3a3cb2d thru 0x3a3ecfc (0x21d0 bytes) 2024-09-15T00:12:57.534 INFO:teuthology.orchestra.run.smithi171.stdout:5956 read 0x26bafa0 thru 0x26c0525 (0x5586 bytes) 2024-09-15T00:12:57.534 INFO:teuthology.orchestra.run.smithi171.stdout:5957 punch from 0x156f4b1 to 0x1577ded, (0x893c bytes) 2024-09-15T00:12:57.537 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:57.536+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:57.538 INFO:teuthology.orchestra.run.smithi171.stdout:5958 write 0x74dee43 thru 0x74e16d8 (0x2896 bytes) 2024-09-15T00:12:57.548 INFO:teuthology.orchestra.run.smithi171.stdout:5959 writesame 0xa466ce8 thru 0xa46de95 data_size 0x1(0x71ae bytes) 2024-09-15T00:12:57.553 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:57.565 INFO:teuthology.orchestra.run.smithi171.stdout:5961 write 0xe8b630f thru 0xe8b9293 (0x2f85 bytes) 2024-09-15T00:12:57.569 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:57.569 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:57.569 INFO:teuthology.orchestra.run.smithi171.stdout:5964 write 0x7e11939 thru 0x7e119ce (0x96 bytes) 2024-09-15T00:12:57.572 INFO:teuthology.orchestra.run.smithi171.stdout:5965 write 0x73b05b9 thru 0x73b1735 (0x117d bytes) 2024-09-15T00:12:57.574 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:57.574 INFO:teuthology.orchestra.run.smithi171.stdout:5968 read 0xcc6f2cb thru 0xcc72e4d (0x3b83 bytes) 2024-09-15T00:12:57.585 INFO:teuthology.orchestra.run.smithi171.stdout:5969 trunc from 0xe8b9294 to 0x65e89d5 2024-09-15T00:12:57.613 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:12:57.613 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:12:57.614 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:57.612+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:57.619 INFO:teuthology.orchestra.run.smithi171.stdout:5970 write 0x6c27c thru 0x7449c (0x8221 bytes) 2024-09-15T00:12:57.624 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:12:57.633 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:57.633 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:57.641 INFO:teuthology.orchestra.run.smithi171.stdout:5971 read 0x4beb197 thru 0x4bf2bb3 (0x7a1d bytes) 2024-09-15T00:12:57.650 INFO:teuthology.orchestra.run.smithi171.stdout:5972 write 0x896bee8 thru 0x8972a1a (0x6b33 bytes) 2024-09-15T00:12:57.656 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:57.657 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:57.660 INFO:teuthology.orchestra.run.smithi171.stdout:5973 punch from 0xcbc49a to 0xcbe567, (0x20cd bytes) 2024-09-15T00:12:57.660 INFO:teuthology.orchestra.run.smithi171.stdout:5974 read 0x2d1696e thru 0x2d18af8 (0x218b bytes) 2024-09-15T00:12:57.660 INFO:teuthology.orchestra.run.smithi171.stdout:5975 read 0x62a5607 thru 0x62ac2ec (0x6ce6 bytes) 2024-09-15T00:12:57.662 INFO:teuthology.orchestra.run.smithi171.stdout:5977 trunc from 0x8972a1b to 0x751584c 2024-09-15T00:12:57.675 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:12:57.675 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:12:57.675 INFO:teuthology.orchestra.run.smithi171.stdout:5978 punch from 0x1437cb2 to 0x143c6bc, (0x4a0a bytes) 2024-09-15T00:12:57.688 INFO:teuthology.orchestra.run.smithi171.stdout:5979 write 0xbe93d8f thru 0xbe9f215 (0xb487 bytes) 2024-09-15T00:12:57.696 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:57.696 INFO:teuthology.orchestra.run.smithi171.stdout:5982 punch from 0x2a4306b to 0x2a4cde9, (0x9d7e bytes) 2024-09-15T00:12:57.696 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:57.696 INFO:teuthology.orchestra.run.smithi171.stdout:5986 write 0x955fc46 thru 0x95662df (0x669a bytes) 2024-09-15T00:12:57.696 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:57.697 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:57.705 INFO:teuthology.orchestra.run.smithi171.stdout:5987 trunc from 0xbe9f216 to 0xce0d742 2024-09-15T00:12:57.710 INFO:teuthology.orchestra.run.smithi171.stdout:compare and write disabled 2024-09-15T00:12:57.714 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:12:57.714 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:12:57.714 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:57.713+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:57.717 INFO:teuthology.orchestra.run.smithi171.stdout:5990 write 0xd8be74f thru 0xd8c70ff (0x89b1 bytes) 2024-09-15T00:12:57.727 INFO:teuthology.orchestra.run.smithi171.stdout:5991 read 0x5b9eca8 thru 0x5ba401c (0x5375 bytes) 2024-09-15T00:12:57.727 INFO:teuthology.orchestra.run.smithi171.stdout:5992 read 0x3a74be2 thru 0x3a7f2a6 (0xa6c5 bytes) 2024-09-15T00:12:57.727 INFO:teuthology.orchestra.run.smithi171.stdout:5993 read 0x8c32de4 thru 0x8c38dd5 (0x5ff2 bytes) 2024-09-15T00:12:57.727 INFO:teuthology.orchestra.run.smithi171.stdout:5994 writesame 0x696b2eb thru 0x696bfcf data_size 0x1(0xce5 bytes) 2024-09-15T00:12:57.730 INFO:teuthology.orchestra.run.smithi171.stdout:5995 write 0x2babfb9 thru 0x2bb73f6 (0xb43e bytes) 2024-09-15T00:12:57.732 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:12:57.732 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:12:57.741 INFO:teuthology.orchestra.run.smithi171.stdout:5997 punch from 0xd4ac83c to 0xd4ba431, (0xdbf5 bytes) 2024-09-15T00:12:57.741 INFO:teuthology.orchestra.run.smithi171.stdout:5998 read 0xa4aa5cf thru 0xa4b1a38 (0x746a bytes) 2024-09-15T00:12:57.741 INFO:teuthology.orchestra.run.smithi171.stdout:5999 write 0x2c9cbac thru 0x2c9fe3d (0x3292 bytes) 2024-09-15T00:12:57.744 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:57.744 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:57.744 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:57.744 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:57.751 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:12:57.752 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:12:57.756 INFO:teuthology.orchestra.run.smithi171.stdout:6000 trunc from 0xd8c7100 to 0x23335af 2024-09-15T00:12:57.759 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:12:57.774 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:12:57.774 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:12:57.810 INFO:teuthology.orchestra.run.smithi171.stdout:checking clone #50, image image_client.0-clone50 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent51 2024-09-15T00:12:57.815 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:57.814+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:57.916 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:57.914+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:57.918 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:57.918 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:57.918 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:57.919 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:57.919 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:57.928 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:12:58.017 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:58.015+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:58.081 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:58.081 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:58.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:58.082 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:58.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.082 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133834 3.1b e85/69 3:da9a78a3:::rbd_data.11395dfcf2b8.0000000000000000:head v 85'356, mlcod=85'354)", 2024-09-15T00:12:58.082 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:58.082 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.51549121, 2024-09-15T00:12:58.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026079054000000001, 2024-09-15T00:12:58.082 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.082 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.082 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.082 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:58.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:12:58.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563338+0000", 2024-09-15T00:12:58.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.418e-06 2024-09-15T00:12:58.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569385+0000", 2024-09-15T00:12:58.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006047335 2024-09-15T00:12:58.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569418+0000", 2024-09-15T00:12:58.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2957999999999999e-05 2024-09-15T00:12:58.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569432+0000", 2024-09-15T00:12:58.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3718e-05 2024-09-15T00:12:58.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569519+0000", 2024-09-15T00:12:58.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7022999999999997e-05 2024-09-15T00:12:58.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569540+0000", 2024-09-15T00:12:58.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1426000000000001e-05 2024-09-15T00:12:58.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589362+0000", 2024-09-15T00:12:58.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019821398 2024-09-15T00:12:58.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589415+0000", 2024-09-15T00:12:58.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3777999999999999e-05 2024-09-15T00:12:58.086 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.086 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.086 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.086 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133836 3.1 e85/67 3:849f9a98:::rbd_data.11395dfcf2b8.0000000000000002:head v 85'304, mlcod=84'301)", 2024-09-15T00:12:58.086 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:58.086 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.507796299999999, 2024-09-15T00:12:58.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023231600000000002, 2024-09-15T00:12:58.086 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.087 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.087 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.087 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:58.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:12:58.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571032+0000", 2024-09-15T00:12:58.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-15T00:12:58.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576635+0000", 2024-09-15T00:12:58.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056028700000000002 2024-09-15T00:12:58.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576650+0000", 2024-09-15T00:12:58.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.505e-05 2024-09-15T00:12:58.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576658+0000", 2024-09-15T00:12:58.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6879999999999999e-06 2024-09-15T00:12:58.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576717+0000", 2024-09-15T00:12:58.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175000000000002e-05 2024-09-15T00:12:58.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576742+0000", 2024-09-15T00:12:58.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4734000000000001e-05 2024-09-15T00:12:58.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594234+0000", 2024-09-15T00:12:58.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017492527000000001 2024-09-15T00:12:58.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594263+0000", 2024-09-15T00:12:58.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8629e-05 2024-09-15T00:12:58.090 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.090 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.090 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.090 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146609 3.b e98/69 3:d39b42ba:::rbd_data.11394501364.0000000000000001:head v 98'283, mlcod=93'281)", 2024-09-15T00:12:58.091 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:12:58.091 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.598078185, 2024-09-15T00:12:58.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020892945, 2024-09-15T00:12:58.091 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.091 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.091 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.091 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:12:58.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:12:58.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480751+0000", 2024-09-15T00:12:58.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6509999999999999e-06 2024-09-15T00:12:58.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482738+0000", 2024-09-15T00:12:58.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019867410000000002 2024-09-15T00:12:58.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482754+0000", 2024-09-15T00:12:58.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6171000000000002e-05 2024-09-15T00:12:58.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482769+0000", 2024-09-15T00:12:58.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4688000000000001e-05 2024-09-15T00:12:58.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482835+0000", 2024-09-15T00:12:58.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6526000000000006e-05 2024-09-15T00:12:58.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482858+0000", 2024-09-15T00:12:58.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2959000000000002e-05 2024-09-15T00:12:58.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501587+0000", 2024-09-15T00:12:58.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018728471 2024-09-15T00:12:58.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501642+0000", 2024-09-15T00:12:58.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5738000000000001e-05 2024-09-15T00:12:58.095 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.095 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.095 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.095 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146835 3.12 e98/68 3:4eee39d6:::rbd_data.11394501364.0000000000000000:head v 98'474, mlcod=93'472)", 2024-09-15T00:12:58.095 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:12:58.095 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.489450915000001, 2024-09-15T00:12:58.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020080434000000001, 2024-09-15T00:12:58.095 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.095 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.095 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.095 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:12:58.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:12:58.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589378+0000", 2024-09-15T00:12:58.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7770000000000001e-06 2024-09-15T00:12:58.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590678+0000", 2024-09-15T00:12:58.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012991750000000001 2024-09-15T00:12:58.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590699+0000", 2024-09-15T00:12:58.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1353e-05 2024-09-15T00:12:58.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590710+0000", 2024-09-15T00:12:58.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1245e-05 2024-09-15T00:12:58.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590800+0000", 2024-09-15T00:12:58.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9963000000000006e-05 2024-09-15T00:12:58.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590823+0000", 2024-09-15T00:12:58.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3289e-05 2024-09-15T00:12:58.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609368+0000", 2024-09-15T00:12:58.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018545078 2024-09-15T00:12:58.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609457+0000", 2024-09-15T00:12:58.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8554000000000006e-05 2024-09-15T00:12:58.099 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.099 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.099 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.099 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146837 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7201319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:12:58.099 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:12:58.099 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.478671452, 2024-09-15T00:12:58.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024751038, 2024-09-15T00:12:58.100 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.100 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.100 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.100 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.100 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.100 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.100 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146837 2024-09-15T00:12:58.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.100 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:12:58.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:12:58.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600157+0000", 2024-09-15T00:12:58.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:12:58.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601965+0000", 2024-09-15T00:12:58.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001807613 2024-09-15T00:12:58.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601985+0000", 2024-09-15T00:12:58.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0679e-05 2024-09-15T00:12:58.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601998+0000", 2024-09-15T00:12:58.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2955999999999999e-05 2024-09-15T00:12:58.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602070+0000", 2024-09-15T00:12:58.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1958000000000005e-05 2024-09-15T00:12:58.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602237+0000", 2024-09-15T00:12:58.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016621299999999999 2024-09-15T00:12:58.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:58.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602799+0000", 2024-09-15T00:12:58.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000562723 2024-09-15T00:12:58.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:58.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.620811+0000", 2024-09-15T00:12:58.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018011207000000001 2024-09-15T00:12:58.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:58.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624793+0000", 2024-09-15T00:12:58.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0039828770000000001 2024-09-15T00:12:58.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624855+0000", 2024-09-15T00:12:58.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.143e-05 2024-09-15T00:12:58.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624907+0000", 2024-09-15T00:12:58.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2262000000000002e-05 2024-09-15T00:12:58.105 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.105 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.105 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.105 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146936 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:12:58.105 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:12:58.105 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.325158133, 2024-09-15T00:12:58.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056847126999999997, 2024-09-15T00:12:58.105 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.105 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.105 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.106 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.106 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.106 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.106 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146936 2024-09-15T00:12:58.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.106 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:12:58.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:12:58.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753671+0000", 2024-09-15T00:12:58.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2839999999999999e-06 2024-09-15T00:12:58.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753677+0000", 2024-09-15T00:12:58.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0260000000000004e-06 2024-09-15T00:12:58.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753691+0000", 2024-09-15T00:12:58.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3891e-05 2024-09-15T00:12:58.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753698+0000", 2024-09-15T00:12:58.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8480000000000001e-06 2024-09-15T00:12:58.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753750+0000", 2024-09-15T00:12:58.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1338000000000002e-05 2024-09-15T00:12:58.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753814+0000", 2024-09-15T00:12:58.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4642999999999998e-05 2024-09-15T00:12:58.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:58.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.756359+0000", 2024-09-15T00:12:58.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025450619999999998 2024-09-15T00:12:58.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:58.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810435+0000", 2024-09-15T00:12:58.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.054075397999999997 2024-09-15T00:12:58.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:58.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810446+0000", 2024-09-15T00:12:58.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1005000000000001e-05 2024-09-15T00:12:58.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810482+0000", 2024-09-15T00:12:58.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6494e-05 2024-09-15T00:12:58.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810517+0000", 2024-09-15T00:12:58.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4137999999999997e-05 2024-09-15T00:12:58.110 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.110 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.111 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.111 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:146936 3.6 e98/97 ondisk, result = 0)", 2024-09-15T00:12:58.111 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:12:58.111 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.315375079000001, 2024-09-15T00:12:58.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046987205999999997, 2024-09-15T00:12:58.111 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.111 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.111 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:58.111 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:12:58.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:12:58.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763453+0000", 2024-09-15T00:12:58.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5199999999999996e-07 2024-09-15T00:12:58.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763457+0000", 2024-09-15T00:12:58.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8569999999999997e-06 2024-09-15T00:12:58.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763469+0000", 2024-09-15T00:12:58.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1977999999999999e-05 2024-09-15T00:12:58.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763476+0000", 2024-09-15T00:12:58.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:12:58.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810423+0000", 2024-09-15T00:12:58.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946993999999999 2024-09-15T00:12:58.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810431+0000", 2024-09-15T00:12:58.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0339999999999997e-06 2024-09-15T00:12:58.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810440+0000", 2024-09-15T00:12:58.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9989999999999997e-06 2024-09-15T00:12:58.114 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.114 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.114 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.115 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:151036 3.1f e103/68 3:f84a1a09:::rbd_data.113958cb0e55.000000000000000c:head v 103'452, mlcod=100'450)", 2024-09-15T00:12:58.115 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:12:58.115 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7420288510000006, 2024-09-15T00:12:58.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020618753, 2024-09-15T00:12:58.115 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.115 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.115 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.115 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:12:58.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:12:58.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:12:58.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:12:58.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340692+0000", 2024-09-15T00:12:58.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038928890000000001 2024-09-15T00:12:58.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340704+0000", 2024-09-15T00:12:58.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1497e-05 2024-09-15T00:12:58.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340709+0000", 2024-09-15T00:12:58.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1529999999999996e-06 2024-09-15T00:12:58.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340789+0000", 2024-09-15T00:12:58.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9907999999999995e-05 2024-09-15T00:12:58.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340810+0000", 2024-09-15T00:12:58.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1837000000000001e-05 2024-09-15T00:12:58.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357368+0000", 2024-09-15T00:12:58.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016557199000000002 2024-09-15T00:12:58.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357417+0000", 2024-09-15T00:12:58.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9783000000000001e-05 2024-09-15T00:12:58.119 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.119 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.119 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.119 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:151042 3.6 3:6419cde4:::rbd_data.113958cb0e55.0000000000000031:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:12:58.119 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:12:58.119 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7129507329999996, 2024-09-15T00:12:58.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025627375000000001, 2024-09-15T00:12:58.119 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.119 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.120 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.120 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.120 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.120 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.120 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 151042 2024-09-15T00:12:58.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.120 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:12:58.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:12:58.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:12:58.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7899999999999998e-07 2024-09-15T00:12:58.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365881+0000", 2024-09-15T00:12:58.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9770000000000002e-06 2024-09-15T00:12:58.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365890+0000", 2024-09-15T00:12:58.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5920000000000001e-06 2024-09-15T00:12:58.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365894+0000", 2024-09-15T00:12:58.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4569999999999998e-06 2024-09-15T00:12:58.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365927+0000", 2024-09-15T00:12:58.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2036000000000003e-05 2024-09-15T00:12:58.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365970+0000", 2024-09-15T00:12:58.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3197e-05 2024-09-15T00:12:58.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:58.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.367432+0000", 2024-09-15T00:12:58.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014621720000000001 2024-09-15T00:12:58.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:58.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391358+0000", 2024-09-15T00:12:58.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023925921999999999 2024-09-15T00:12:58.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:58.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391415+0000", 2024-09-15T00:12:58.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7253999999999998e-05 2024-09-15T00:12:58.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391456+0000", 2024-09-15T00:12:58.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0497999999999997e-05 2024-09-15T00:12:58.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391504+0000", 2024-09-15T00:12:58.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8591000000000003e-05 2024-09-15T00:12:58.125 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.125 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.125 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.125 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:151042 3.6 e103/97 ondisk, result = 0)", 2024-09-15T00:12:58.125 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:12:58.125 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7072077199999995, 2024-09-15T00:12:58.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019892730000000001, 2024-09-15T00:12:58.125 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.125 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.125 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:58.125 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:12:58.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:12:58.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:12:58.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8100000000000003e-07 2024-09-15T00:12:58.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371623+0000", 2024-09-15T00:12:58.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6290000000000002e-06 2024-09-15T00:12:58.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371632+0000", 2024-09-15T00:12:58.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6379999999999993e-06 2024-09-15T00:12:58.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371636+0000", 2024-09-15T00:12:58.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:12:58.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391403+0000", 2024-09-15T00:12:58.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019766982999999998 2024-09-15T00:12:58.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391411+0000", 2024-09-15T00:12:58.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7309999999999993e-06 2024-09-15T00:12:58.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391513+0000", 2024-09-15T00:12:58.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101648 2024-09-15T00:12:58.129 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.129 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.129 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.129 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152025 3.3 e103/68 3:c0279197:::rbd_data.113958cb0e55.0000000000000007:head v 103'464, mlcod=102'461)", 2024-09-15T00:12:58.129 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:12:58.129 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5627201240000002, 2024-09-15T00:12:58.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021264476000000001, 2024-09-15T00:12:58.129 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.129 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.129 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.129 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:12:58.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:12:58.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516109+0000", 2024-09-15T00:12:58.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.666e-06 2024-09-15T00:12:58.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.519997+0000", 2024-09-15T00:12:58.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003888449 2024-09-15T00:12:58.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520016+0000", 2024-09-15T00:12:58.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8059000000000001e-05 2024-09-15T00:12:58.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520026+0000", 2024-09-15T00:12:58.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9599999999999995e-06 2024-09-15T00:12:58.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520089+0000", 2024-09-15T00:12:58.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3074e-05 2024-09-15T00:12:58.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520108+0000", 2024-09-15T00:12:58.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8981999999999999e-05 2024-09-15T00:12:58.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537326+0000", 2024-09-15T00:12:58.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017218102999999998 2024-09-15T00:12:58.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537372+0000", 2024-09-15T00:12:58.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6183000000000002e-05 2024-09-15T00:12:58.133 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.136 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.136 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.136 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152284 3.f e103/68 3:f4272b5d:::rbd_data.113958cb0e55.0000000000000006:head v 103'370, mlcod=103'368)", 2024-09-15T00:12:58.136 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:12:58.136 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5142435929999998, 2024-09-15T00:12:58.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020145026999999999, 2024-09-15T00:12:58.136 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.136 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.137 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.137 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:12:58.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:12:58.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564585+0000", 2024-09-15T00:12:58.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7000000000000003e-07 2024-09-15T00:12:58.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568352+0000", 2024-09-15T00:12:58.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037675909999999998 2024-09-15T00:12:58.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568370+0000", 2024-09-15T00:12:58.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7158000000000001e-05 2024-09-15T00:12:58.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568378+0000", 2024-09-15T00:12:58.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6449999999999994e-06 2024-09-15T00:12:58.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568454+0000", 2024-09-15T00:12:58.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5285999999999997e-05 2024-09-15T00:12:58.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568472+0000", 2024-09-15T00:12:58.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8447000000000001e-05 2024-09-15T00:12:58.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584693+0000", 2024-09-15T00:12:58.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016221467999999999 2024-09-15T00:12:58.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584729+0000", 2024-09-15T00:12:58.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5462e-05 2024-09-15T00:12:58.140 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.140 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.140 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.141 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152500 3.17 e103/68 3:edff5adf:::rbd_data.113958cb0e55.000000000000000d:head v 103'461, mlcod=103'459)", 2024-09-15T00:12:58.141 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:12:58.141 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.4372570079999996, 2024-09-15T00:12:58.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020217809, 2024-09-15T00:12:58.141 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.141 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.141 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.141 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:12:58.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:12:58.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641572+0000", 2024-09-15T00:12:58.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-15T00:12:58.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644961+0000", 2024-09-15T00:12:58.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033882610000000001 2024-09-15T00:12:58.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644979+0000", 2024-09-15T00:12:58.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8179e-05 2024-09-15T00:12:58.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644987+0000", 2024-09-15T00:12:58.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8860000000000004e-06 2024-09-15T00:12:58.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645044+0000", 2024-09-15T00:12:58.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7141000000000001e-05 2024-09-15T00:12:58.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645062+0000", 2024-09-15T00:12:58.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8258e-05 2024-09-15T00:12:58.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661749+0000", 2024-09-15T00:12:58.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016686553999999999 2024-09-15T00:12:58.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661788+0000", 2024-09-15T00:12:58.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9690000000000001e-05 2024-09-15T00:12:58.145 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.145 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.145 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.145 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153171 3.18 e103/69 3:1db186e8:::rbd_data.113958cb0e55.0000000000000002:head v 103'520, mlcod=103'518)", 2024-09-15T00:12:58.145 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:12:58.145 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2987721019999992, 2024-09-15T00:12:58.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022266838000000001, 2024-09-15T00:12:58.145 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.145 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.145 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.145 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:12:58.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:12:58.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780057+0000", 2024-09-15T00:12:58.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5880000000000001e-06 2024-09-15T00:12:58.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781273+0000", 2024-09-15T00:12:58.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001216087 2024-09-15T00:12:58.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781311+0000", 2024-09-15T00:12:58.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7656e-05 2024-09-15T00:12:58.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781325+0000", 2024-09-15T00:12:58.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4508999999999999e-05 2024-09-15T00:12:58.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781403+0000", 2024-09-15T00:12:58.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7927000000000001e-05 2024-09-15T00:12:58.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781422+0000", 2024-09-15T00:12:58.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8575000000000001e-05 2024-09-15T00:12:58.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802249+0000", 2024-09-15T00:12:58.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020827577 2024-09-15T00:12:58.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802322+0000", 2024-09-15T00:12:58.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2918999999999998e-05 2024-09-15T00:12:58.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.149 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.149 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153772 3.12 e103/68 3:4911a4f6:::rbd_data.113958cb0e55.0000000000000001:head v 103'490, mlcod=103'488)", 2024-09-15T00:12:58.149 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:12:58.149 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1959014749999994, 2024-09-15T00:12:58.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021181004, 2024-09-15T00:12:58.149 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.150 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.150 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.150 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:12:58.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:12:58.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882927+0000", 2024-09-15T00:12:58.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.426e-06 2024-09-15T00:12:58.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883966+0000", 2024-09-15T00:12:58.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010384610000000001 2024-09-15T00:12:58.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883985+0000", 2024-09-15T00:12:58.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9276e-05 2024-09-15T00:12:58.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883994+0000", 2024-09-15T00:12:58.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9490000000000002e-06 2024-09-15T00:12:58.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884054+0000", 2024-09-15T00:12:58.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9419999999999997e-05 2024-09-15T00:12:58.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884073+0000", 2024-09-15T00:12:58.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9137e-05 2024-09-15T00:12:58.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904038+0000", 2024-09-15T00:12:58.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019965092 2024-09-15T00:12:58.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904107+0000", 2024-09-15T00:12:58.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9243000000000001e-05 2024-09-15T00:12:58.153 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.153 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.153 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.153 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163111 3.18 e106/69 3:18b9f527:::rbd_data.11395d771336.0000000000000000:head v 106'538, mlcod=105'536)", 2024-09-15T00:12:58.154 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.463478+0000", 2024-09-15T00:12:58.154 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6153499670000002, 2024-09-15T00:12:58.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015225839999999999, 2024-09-15T00:12:58.154 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.154 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.154 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.154 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.463478+0000", 2024-09-15T00:12:58.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.463478+0000", 2024-09-15T00:12:58.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.463479+0000", 2024-09-15T00:12:58.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.438e-06 2024-09-15T00:12:58.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464332+0000", 2024-09-15T00:12:58.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00085324499999999998 2024-09-15T00:12:58.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464356+0000", 2024-09-15T00:12:58.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4173e-05 2024-09-15T00:12:58.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464370+0000", 2024-09-15T00:12:58.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3539e-05 2024-09-15T00:12:58.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464439+0000", 2024-09-15T00:12:58.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8668000000000001e-05 2024-09-15T00:12:58.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464462+0000", 2024-09-15T00:12:58.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3034999999999999e-05 2024-09-15T00:12:58.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.478656+0000", 2024-09-15T00:12:58.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014194783000000001 2024-09-15T00:12:58.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.478703+0000", 2024-09-15T00:12:58.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6959000000000002e-05 2024-09-15T00:12:58.157 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.157 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.158 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.158 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163112 3.3 e106/68 3:c668f3d5:::rbd_data.11395d771336.0000000000000001:head v 106'489, mlcod=105'487)", 2024-09-15T00:12:58.158 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:12:58.158 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6140042059999997, 2024-09-15T00:12:58.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019317183000000002, 2024-09-15T00:12:58.158 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.158 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.158 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.158 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:12:58.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:12:58.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464824+0000", 2024-09-15T00:12:58.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1100000000000005e-07 2024-09-15T00:12:58.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466874+0000", 2024-09-15T00:12:58.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002049497 2024-09-15T00:12:58.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466884+0000", 2024-09-15T00:12:58.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0424999999999999e-05 2024-09-15T00:12:58.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466890+0000", 2024-09-15T00:12:58.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-15T00:12:58.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466944+0000", 2024-09-15T00:12:58.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4592999999999997e-05 2024-09-15T00:12:58.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466958+0000", 2024-09-15T00:12:58.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3993e-05 2024-09-15T00:12:58.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484068+0000", 2024-09-15T00:12:58.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017109994 2024-09-15T00:12:58.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484140+0000", 2024-09-15T00:12:58.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2224000000000005e-05 2024-09-15T00:12:58.162 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.162 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.163 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163117 3.1f e106/68 3:fbcc0048:::rbd_data.11395d771336.0000000000000003:head v 106'489, mlcod=106'487)", 2024-09-15T00:12:58.163 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.470675+0000", 2024-09-15T00:12:58.163 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6081521480000003, 2024-09-15T00:12:58.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013840234999999999, 2024-09-15T00:12:58.163 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.163 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.163 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.163 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.470675+0000", 2024-09-15T00:12:58.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.470675+0000", 2024-09-15T00:12:58.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.470676+0000", 2024-09-15T00:12:58.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:12:58.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.471937+0000", 2024-09-15T00:12:58.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001261362 2024-09-15T00:12:58.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.471949+0000", 2024-09-15T00:12:58.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1479e-05 2024-09-15T00:12:58.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.471955+0000", 2024-09-15T00:12:58.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7190000000000004e-06 2024-09-15T00:12:58.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.472018+0000", 2024-09-15T00:12:58.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2850999999999998e-05 2024-09-15T00:12:58.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.472035+0000", 2024-09-15T00:12:58.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6626999999999999e-05 2024-09-15T00:12:58.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484475+0000", 2024-09-15T00:12:58.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012440094 2024-09-15T00:12:58.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484516+0000", 2024-09-15T00:12:58.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0698000000000002e-05 2024-09-15T00:12:58.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.167 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.167 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163119 3.0 e106/94 3:06118613:::rbd_data.11395d771336.0000000000000004:head v 106'549, mlcod=105'547)", 2024-09-15T00:12:58.167 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:12:58.167 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6023890029999999, 2024-09-15T00:12:58.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017175867000000001, 2024-09-15T00:12:58.168 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.168 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.168 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.168 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:12:58.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:12:58.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:12:58.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6700000000000002e-07 2024-09-15T00:12:58.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477786+0000", 2024-09-15T00:12:58.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013468149999999999 2024-09-15T00:12:58.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477797+0000", 2024-09-15T00:12:58.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0635000000000001e-05 2024-09-15T00:12:58.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477805+0000", 2024-09-15T00:12:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2989999999999997e-06 2024-09-15T00:12:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477859+0000", 2024-09-15T00:12:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3699000000000002e-05 2024-09-15T00:12:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477878+0000", 2024-09-15T00:12:58.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9335e-05 2024-09-15T00:12:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493560+0000", 2024-09-15T00:12:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015681748999999998 2024-09-15T00:12:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493614+0000", 2024-09-15T00:12:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4367999999999999e-05 2024-09-15T00:12:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:163139 3.1 3:80ed57f9:::rbd_data.11395d771336.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:12:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.480656+0000", 2024-09-15T00:12:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5981715080000001, 2024-09-15T00:12:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013898246, 2024-09-15T00:12:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 163139 2024-09-15T00:12:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.480656+0000", 2024-09-15T00:12:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.480656+0000", 2024-09-15T00:12:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.480658+0000", 2024-09-15T00:12:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0140000000000001e-06 2024-09-15T00:12:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.481554+0000", 2024-09-15T00:12:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00089608500000000002 2024-09-15T00:12:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.481568+0000", 2024-09-15T00:12:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4331999999999999e-05 2024-09-15T00:12:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.481576+0000", 2024-09-15T00:12:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0399999999999993e-06 2024-09-15T00:12:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.481659+0000", 2024-09-15T00:12:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3004999999999996e-05 2024-09-15T00:12:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.481787+0000", 2024-09-15T00:12:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012715299999999999 2024-09-15T00:12:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.482096+0000", 2024-09-15T00:12:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00030970100000000001 2024-09-15T00:12:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.494197+0000", 2024-09-15T00:12:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012100319 2024-09-15T00:12:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.494462+0000", 2024-09-15T00:12:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026532599999999997 2024-09-15T00:12:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.494513+0000", 2024-09-15T00:12:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0739999999999999e-05 2024-09-15T00:12:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.494554+0000", 2024-09-15T00:12:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1530999999999999e-05 2024-09-15T00:12:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.177 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.177 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:58.179 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:12:58.179 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:12:58.177+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:12:58.279 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:12:58.278+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:12:58.326 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:58.326 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:58.326 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:58.326 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:58.335 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:12:58.345 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:12:58.382 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:58.381+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:58.441 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:58.441 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:58.441 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:58.441 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:58.441 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:58.450 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:12:58.482 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:58.480+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:58.488 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:58.488 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:58.489 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:58.489 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:58.557 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:58.557 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:58.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:58.557 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:58.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.557 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:12:58.557 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:58.557 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.374665970999999, 2024-09-15T00:12:58.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:12:58.557 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.557 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.558 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.558 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:12:58.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.558 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:58.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:12:58.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:12:58.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:12:58.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:12:58.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:12:58.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:12:58.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:12:58.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:12:58.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:12:58.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:12:58.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:12:58.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:12:58.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:12:58.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:58.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:12:58.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:12:58.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:58.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:12:58.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:12:58.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:58.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:12:58.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:12:58.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:12:58.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:12:58.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:12:58.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:12:58.563 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.563 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.563 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.563 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:12:58.563 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:58.563 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.644824791999994, 2024-09-15T00:12:58.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:12:58.564 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.564 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.564 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.564 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.564 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.564 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.564 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:12:58.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.564 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:58.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:12:58.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:12:58.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:12:58.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:12:58.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:12:58.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:12:58.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:12:58.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:12:58.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:12:58.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:12:58.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:12:58.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:12:58.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:12:58.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:58.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:12:58.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:12:58.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:58.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:12:58.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:12:58.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:58.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:12:58.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:12:58.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:12:58.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:12:58.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:12:58.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:12:58.569 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.569 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.569 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.569 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:12:58.569 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:58.569 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.637866428999999, 2024-09-15T00:12:58.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:12:58.569 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.569 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.570 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:58.570 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:58.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:12:58.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:12:58.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:12:58.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:12:58.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:12:58.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:12:58.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:12:58.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:12:58.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:12:58.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:12:58.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:12:58.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:12:58.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:12:58.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:12:58.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:12:58.573 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.573 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.573 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.573 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:58.573 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:58.573 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.890677592000003, 2024-09-15T00:12:58.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:12:58.574 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.574 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.574 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:58.574 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.574 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.574 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.574 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:12:58.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.574 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:58.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:12:58.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:12:58.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:12:58.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:12:58.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:12:58.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:12:58.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:12:58.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:12:58.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:12:58.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:12:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:12:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:12:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:12:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:12:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:12:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:58.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:12:58.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:12:58.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:58.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:12:58.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:12:58.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:58.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:12:58.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:12:58.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:12:58.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:12:58.579 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.579 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.579 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.579 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97493 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:58.579 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:58.579 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.701094871999999, 2024-09-15T00:12:58.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033259141999999998, 2024-09-15T00:12:58.580 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.580 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.580 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:58.580 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.580 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.580 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.580 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97493 2024-09-15T00:12:58.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.580 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:58.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:58.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:12:58.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-15T00:12:58.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853401+0000", 2024-09-15T00:12:58.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:12:58.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853407+0000", 2024-09-15T00:12:58.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-15T00:12:58.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853411+0000", 2024-09-15T00:12:58.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.481e-06 2024-09-15T00:12:58.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886544+0000", 2024-09-15T00:12:58.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033132861999999999 2024-09-15T00:12:58.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886587+0000", 2024-09-15T00:12:58.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2942000000000003e-05 2024-09-15T00:12:58.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886655+0000", 2024-09-15T00:12:58.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7711000000000003e-05 2024-09-15T00:12:58.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.583 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.584 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97494 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:58.584 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:58.584 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.701073749000003, 2024-09-15T00:12:58.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033392390000000001, 2024-09-15T00:12:58.584 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.584 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.584 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:58.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.584 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97494 2024-09-15T00:12:58.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.585 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:58.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:58.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:12:58.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-15T00:12:58.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853419+0000", 2024-09-15T00:12:58.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.917e-06 2024-09-15T00:12:58.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853425+0000", 2024-09-15T00:12:58.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:12:58.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853433+0000", 2024-09-15T00:12:58.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8120000000000008e-06 2024-09-15T00:12:58.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886680+0000", 2024-09-15T00:12:58.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033247207000000001 2024-09-15T00:12:58.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886726+0000", 2024-09-15T00:12:58.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5902000000000002e-05 2024-09-15T00:12:58.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886809+0000", 2024-09-15T00:12:58.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3475999999999998e-05 2024-09-15T00:12:58.588 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.588 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.588 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.588 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97495 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:58.588 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:58.588 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.701052795999999, 2024-09-15T00:12:58.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033640503000000002, 2024-09-15T00:12:58.589 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.589 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.589 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:58.589 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.589 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.589 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.589 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97495 2024-09-15T00:12:58.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.589 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:58.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:58.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:12:58.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:12:58.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853440+0000", 2024-09-15T00:12:58.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.804e-06 2024-09-15T00:12:58.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853446+0000", 2024-09-15T00:12:58.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5559999999999998e-06 2024-09-15T00:12:58.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853450+0000", 2024-09-15T00:12:58.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9519999999999996e-06 2024-09-15T00:12:58.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886904+0000", 2024-09-15T00:12:58.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033454207999999999 2024-09-15T00:12:58.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887007+0000", 2024-09-15T00:12:58.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102904 2024-09-15T00:12:58.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887078+0000", 2024-09-15T00:12:58.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1719999999999998e-05 2024-09-15T00:12:58.593 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.593 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.593 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.593 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97496 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:58.593 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:58.593 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.701035697999998, 2024-09-15T00:12:58.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033802431000000001, 2024-09-15T00:12:58.593 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.593 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.593 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:58.594 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.594 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.594 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.594 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97496 2024-09-15T00:12:58.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.594 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:58.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:58.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:12:58.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.53e-07 2024-09-15T00:12:58.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853462+0000", 2024-09-15T00:12:58.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-15T00:12:58.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853468+0000", 2024-09-15T00:12:58.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3609999999999997e-06 2024-09-15T00:12:58.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853473+0000", 2024-09-15T00:12:58.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-15T00:12:58.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887111+0000", 2024-09-15T00:12:58.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033638376999999997 2024-09-15T00:12:58.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887178+0000", 2024-09-15T00:12:58.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6877000000000004e-05 2024-09-15T00:12:58.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887257+0000", 2024-09-15T00:12:58.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9616999999999993e-05 2024-09-15T00:12:58.597 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.597 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.597 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.598 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97587 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:58.598 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:58.598 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.701012464000002, 2024-09-15T00:12:58.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033973642999999998, 2024-09-15T00:12:58.598 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.598 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.598 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:58.598 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.598 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.598 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.598 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97587 2024-09-15T00:12:58.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.598 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:58.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:12:58.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853479+0000", 2024-09-15T00:12:58.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:12:58.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853481+0000", 2024-09-15T00:12:58.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.136e-06 2024-09-15T00:12:58.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853487+0000", 2024-09-15T00:12:58.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2439999999999998e-06 2024-09-15T00:12:58.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853492+0000", 2024-09-15T00:12:58.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:12:58.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887284+0000", 2024-09-15T00:12:58.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033791827000000003 2024-09-15T00:12:58.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887361+0000", 2024-09-15T00:12:58.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6983000000000005e-05 2024-09-15T00:12:58.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887452+0000", 2024-09-15T00:12:58.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1236999999999994e-05 2024-09-15T00:12:58.602 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.602 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.602 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.602 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97588 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:12:58.602 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:58.602 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.700992307, 2024-09-15T00:12:58.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034134617999999999, 2024-09-15T00:12:58.602 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.602 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.602 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:58.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.603 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.603 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.603 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97588 2024-09-15T00:12:58.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.603 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:58.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:12:58.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853499+0000", 2024-09-15T00:12:58.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:12:58.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853501+0000", 2024-09-15T00:12:58.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.26e-06 2024-09-15T00:12:58.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853507+0000", 2024-09-15T00:12:58.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2899999999999999e-06 2024-09-15T00:12:58.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853512+0000", 2024-09-15T00:12:58.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8330000000000001e-06 2024-09-15T00:12:58.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887495+0000", 2024-09-15T00:12:58.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033982529999999997 2024-09-15T00:12:58.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:58.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887548+0000", 2024-09-15T00:12:58.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3445000000000001e-05 2024-09-15T00:12:58.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887633+0000", 2024-09-15T00:12:58.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4890999999999997e-05 2024-09-15T00:12:58.606 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.606 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.606 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.607 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126866 3.6 3.973c9fa6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:58.607 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:58.607 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.597670362000002, 2024-09-15T00:12:58.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153301000000003, 2024-09-15T00:12:58.607 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.607 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.607 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:58.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.607 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126866 2024-09-15T00:12:58.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.608 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:58.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:12:58.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956821+0000", 2024-09-15T00:12:58.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.91e-07 2024-09-15T00:12:58.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956822+0000", 2024-09-15T00:12:58.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.705e-06 2024-09-15T00:12:58.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956828+0000", 2024-09-15T00:12:58.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6760000000000004e-06 2024-09-15T00:12:58.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956833+0000", 2024-09-15T00:12:58.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5379999999999996e-06 2024-09-15T00:12:58.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989970+0000", 2024-09-15T00:12:58.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033137560000000003 2024-09-15T00:12:58.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989974+0000", 2024-09-15T00:12:58.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-15T00:12:58.611 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.611 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.611 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.611 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126951 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:58.611 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:58.611 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.597544679000002, 2024-09-15T00:12:58.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033152585999999998, 2024-09-15T00:12:58.611 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.611 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.611 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:58.611 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.611 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.612 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.612 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126951 2024-09-15T00:12:58.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.612 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:58.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:58.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:12:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-07 2024-09-15T00:12:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956953+0000", 2024-09-15T00:12:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9310000000000002e-06 2024-09-15T00:12:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956957+0000", 2024-09-15T00:12:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5379999999999998e-06 2024-09-15T00:12:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956959+0000", 2024-09-15T00:12:58.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6599999999999999e-06 2024-09-15T00:12:58.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990095+0000", 2024-09-15T00:12:58.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033135855999999998 2024-09-15T00:12:58.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990099+0000", 2024-09-15T00:12:58.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4010000000000001e-06 2024-09-15T00:12:58.615 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.615 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.615 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.615 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126952 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:58.615 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:58.615 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.59752769, 2024-09-15T00:12:58.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033156066999999997, 2024-09-15T00:12:58.616 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.616 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.616 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:58.616 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.616 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.616 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.616 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126952 2024-09-15T00:12:58.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.616 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:58.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:58.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:12:58.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:12:58.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956965+0000", 2024-09-15T00:12:58.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3379999999999999e-06 2024-09-15T00:12:58.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956968+0000", 2024-09-15T00:12:58.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0419999999999998e-06 2024-09-15T00:12:58.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956970+0000", 2024-09-15T00:12:58.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4899999999999999e-06 2024-09-15T00:12:58.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990113+0000", 2024-09-15T00:12:58.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033142642 2024-09-15T00:12:58.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990119+0000", 2024-09-15T00:12:58.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2979999999999997e-06 2024-09-15T00:12:58.619 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.619 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.619 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.619 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126954 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:58.620 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:58.620 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.597482194999998, 2024-09-15T00:12:58.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033158449, 2024-09-15T00:12:58.620 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.620 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.620 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:58.620 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.620 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.620 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.620 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126954 2024-09-15T00:12:58.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.620 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:58.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:58.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:12:58.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-15T00:12:58.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957010+0000", 2024-09-15T00:12:58.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3659999999999999e-06 2024-09-15T00:12:58.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957013+0000", 2024-09-15T00:12:58.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1140000000000002e-06 2024-09-15T00:12:58.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957016+0000", 2024-09-15T00:12:58.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1209999999999998e-06 2024-09-15T00:12:58.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990163+0000", 2024-09-15T00:12:58.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033146451 2024-09-15T00:12:58.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990167+0000", 2024-09-15T00:12:58.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0579999999999999e-06 2024-09-15T00:12:58.623 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.623 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.623 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.624 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126955 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:58.624 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:58.624 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.597470981000001, 2024-09-15T00:12:58.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033160617000000003, 2024-09-15T00:12:58.624 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.624 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.624 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:58.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.624 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126955 2024-09-15T00:12:58.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.624 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:58.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:58.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:12:58.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-15T00:12:58.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957021+0000", 2024-09-15T00:12:58.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-15T00:12:58.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957024+0000", 2024-09-15T00:12:58.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1599999999999998e-06 2024-09-15T00:12:58.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957027+0000", 2024-09-15T00:12:58.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000001e-06 2024-09-15T00:12:58.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990177+0000", 2024-09-15T00:12:58.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033149741000000003 2024-09-15T00:12:58.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990180+0000", 2024-09-15T00:12:58.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4910000000000001e-06 2024-09-15T00:12:58.627 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.627 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.627 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.628 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126956 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:58.628 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:58.628 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.597460859999998, 2024-09-15T00:12:58.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033163633999999997, 2024-09-15T00:12:58.628 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.628 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.628 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:58.628 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.628 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.628 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.628 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126956 2024-09-15T00:12:58.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.629 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:58.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:58.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:12:58.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:58.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957031+0000", 2024-09-15T00:12:58.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-06 2024-09-15T00:12:58.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957034+0000", 2024-09-15T00:12:58.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9960000000000002e-06 2024-09-15T00:12:58.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957037+0000", 2024-09-15T00:12:58.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6419999999999999e-06 2024-09-15T00:12:58.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990190+0000", 2024-09-15T00:12:58.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153152999999998 2024-09-15T00:12:58.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990193+0000", 2024-09-15T00:12:58.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6069999999999999e-06 2024-09-15T00:12:58.632 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.632 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.632 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.632 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126957 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:58.632 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:58.632 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.597451210000003, 2024-09-15T00:12:58.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033166442999999997, 2024-09-15T00:12:58.632 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.632 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.633 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:58.633 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.633 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.633 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.633 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126957 2024-09-15T00:12:58.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.633 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:58.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:12:58.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957040+0000", 2024-09-15T00:12:58.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:12:58.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957041+0000", 2024-09-15T00:12:58.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0440000000000001e-06 2024-09-15T00:12:58.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957044+0000", 2024-09-15T00:12:58.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0170000000000001e-06 2024-09-15T00:12:58.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957048+0000", 2024-09-15T00:12:58.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0319999999999997e-06 2024-09-15T00:12:58.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990202+0000", 2024-09-15T00:12:58.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033154640999999999 2024-09-15T00:12:58.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990206+0000", 2024-09-15T00:12:58.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.507e-06 2024-09-15T00:12:58.636 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.636 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.636 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.636 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126958 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:58.636 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:58.636 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.597439979999997, 2024-09-15T00:12:58.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033167613999999998, 2024-09-15T00:12:58.636 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.637 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.637 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:58.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.637 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126958 2024-09-15T00:12:58.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.637 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:58.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:58.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:12:58.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:12:58.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957053+0000", 2024-09-15T00:12:58.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.756e-06 2024-09-15T00:12:58.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957059+0000", 2024-09-15T00:12:58.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2770000000000002e-06 2024-09-15T00:12:58.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957063+0000", 2024-09-15T00:12:58.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2459999999999997e-06 2024-09-15T00:12:58.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990215+0000", 2024-09-15T00:12:58.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033151509000000003 2024-09-15T00:12:58.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990218+0000", 2024-09-15T00:12:58.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5099999999999999e-06 2024-09-15T00:12:58.640 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.640 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.640 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.640 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126959 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:58.640 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:58.640 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.597420976000002, 2024-09-15T00:12:58.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033161673000000003, 2024-09-15T00:12:58.641 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.641 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.641 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:58.641 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.641 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.641 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.641 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126959 2024-09-15T00:12:58.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.641 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:58.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:58.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:12:58.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:12:58.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957072+0000", 2024-09-15T00:12:58.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2239999999999998e-06 2024-09-15T00:12:58.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957079+0000", 2024-09-15T00:12:58.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.19e-06 2024-09-15T00:12:58.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957083+0000", 2024-09-15T00:12:58.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:12:58.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990228+0000", 2024-09-15T00:12:58.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033144818 2024-09-15T00:12:58.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990231+0000", 2024-09-15T00:12:58.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:12:58.644 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.644 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.644 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.645 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126960 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:58.645 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:58.645 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.597402168999999, 2024-09-15T00:12:58.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033155166, 2024-09-15T00:12:58.645 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:58.645 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:58.645 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:12:58.645 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:58.645 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:58.645 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:58.646 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126960 2024-09-15T00:12:58.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.646 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:58.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:58.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:58.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:58.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:58.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:58.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:58.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:12:58.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:12:58.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:58.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957091+0000", 2024-09-15T00:12:58.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:12:58.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:58.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957097+0000", 2024-09-15T00:12:58.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6050000000000003e-06 2024-09-15T00:12:58.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:58.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957102+0000", 2024-09-15T00:12:58.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5539999999999999e-06 2024-09-15T00:12:58.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:58.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990240+0000", 2024-09-15T00:12:58.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033138289000000001 2024-09-15T00:12:58.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:58.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:58.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:58.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990244+0000", 2024-09-15T00:12:58.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4740000000000001e-06 2024-09-15T00:12:58.649 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.649 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.649 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.649 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:58.649 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:58.649 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:58.651 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:12:58.651 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:58.649+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:58.752 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:58.750+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:58.755 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:58.755 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:58.755 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:12:58.755 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:58.764 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:12:58.852 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:12:58.850+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:12:58.908 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:58.908 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:12:58.908 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:12:58.908 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:12:58.908 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:58.916 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:12:58.953 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:58.952+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:59.020 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:12:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:12:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:12:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:12:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:12:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 70.775363845000001, 2024-09-15T00:12:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:12:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:12:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:12:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:12:59.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:12:59.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:12:59.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:12:59.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:12:59.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:12:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:12:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:12:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:12:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:12:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:12:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:12:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:12:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:12:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:12:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:12:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:12:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:12:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:12:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:12:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:12:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:12:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:12:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58878 3.8 3:12456482:::rbd_data.1139766160da.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.729058261000006, 2024-09-15T00:12:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061956199000000003, 2024-09-15T00:12:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58878 2024-09-15T00:12:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:12:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289151+0000", 2024-09-15T00:12:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:12:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289473+0000", 2024-09-15T00:12:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032183800000000002 2024-09-15T00:12:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289487+0000", 2024-09-15T00:12:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-05 2024-09-15T00:12:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289493+0000", 2024-09-15T00:12:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4489999999999996e-06 2024-09-15T00:12:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289542+0000", 2024-09-15T00:12:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8844999999999997e-05 2024-09-15T00:12:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289642+0000", 2024-09-15T00:12:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9607999999999999e-05 2024-09-15T00:12:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.293908+0000", 2024-09-15T00:12:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659350000000002 2024-09-15T00:12:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351022+0000", 2024-09-15T00:12:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057114535000000001 2024-09-15T00:12:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351033+0000", 2024-09-15T00:12:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0263e-05 2024-09-15T00:12:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351070+0000", 2024-09-15T00:12:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7666000000000002e-05 2024-09-15T00:12:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351106+0000", 2024-09-15T00:12:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999997e-05 2024-09-15T00:12:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58879 3.7 3:e169894d:::rbd_data.1139766160da.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.726706773000004, 2024-09-15T00:12:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043716294000000003, 2024-09-15T00:12:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58879 2024-09-15T00:12:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:12:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291502+0000", 2024-09-15T00:12:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-15T00:12:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291664+0000", 2024-09-15T00:12:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016216599999999999 2024-09-15T00:12:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291674+0000", 2024-09-15T00:12:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0276e-05 2024-09-15T00:12:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291680+0000", 2024-09-15T00:12:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2580000000000003e-06 2024-09-15T00:12:59.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291732+0000", 2024-09-15T00:12:59.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2114999999999998e-05 2024-09-15T00:12:59.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291866+0000", 2024-09-15T00:12:59.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013416900000000001 2024-09-15T00:12:59.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:12:59.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.296426+0000", 2024-09-15T00:12:59.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045595779999999999 2024-09-15T00:12:59.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:59.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335075+0000", 2024-09-15T00:12:59.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038649347000000001 2024-09-15T00:12:59.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:59.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335097+0000", 2024-09-15T00:12:59.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2565000000000002e-05 2024-09-15T00:12:59.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:59.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335161+0000", 2024-09-15T00:12:59.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3928000000000002e-05 2024-09-15T00:12:59.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335217+0000", 2024-09-15T00:12:59.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6042000000000003e-05 2024-09-15T00:12:59.038 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.038 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.038 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.038 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58880 3.8 3:13d91cfd:::rbd_data.1139766160da.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:12:59.038 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:59.039 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.723314665999993, 2024-09-15T00:12:59.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056331463999999998, 2024-09-15T00:12:59.039 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.039 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.039 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:59.039 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.039 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.039 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.039 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58880 2024-09-15T00:12:59.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.039 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:59.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:12:59.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294896+0000", 2024-09-15T00:12:59.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.356e-06 2024-09-15T00:12:59.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295288+0000", 2024-09-15T00:12:59.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039223200000000001 2024-09-15T00:12:59.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295304+0000", 2024-09-15T00:12:59.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5937e-05 2024-09-15T00:12:59.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295312+0000", 2024-09-15T00:12:59.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1489999999999994e-06 2024-09-15T00:12:59.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333162+0000", 2024-09-15T00:12:59.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037849840000000003 2024-09-15T00:12:59.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333269+0000", 2024-09-15T00:12:59.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010749800000000001 2024-09-15T00:12:59.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:12:59.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.336198+0000", 2024-09-15T00:12:59.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029284739999999999 2024-09-15T00:12:59.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:59.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351137+0000", 2024-09-15T00:12:59.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014939103 2024-09-15T00:12:59.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:59.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351150+0000", 2024-09-15T00:12:59.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3542e-05 2024-09-15T00:12:59.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:59.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351191+0000", 2024-09-15T00:12:59.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0791000000000003e-05 2024-09-15T00:12:59.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351225+0000", 2024-09-15T00:12:59.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3541999999999998e-05 2024-09-15T00:12:59.044 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.044 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.044 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.044 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58878 3.8 e56/23 ondisk, result = 0)", 2024-09-15T00:12:59.044 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:59.044 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.716784883000003, 2024-09-15T00:12:59.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049689096000000002, 2024-09-15T00:12:59.044 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.045 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.045 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:59.045 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:59.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:12:59.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301425+0000", 2024-09-15T00:12:59.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.782e-06 2024-09-15T00:12:59.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301433+0000", 2024-09-15T00:12:59.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689999999999994e-06 2024-09-15T00:12:59.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301449+0000", 2024-09-15T00:12:59.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5767e-05 2024-09-15T00:12:59.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301458+0000", 2024-09-15T00:12:59.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:12:59.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351009+0000", 2024-09-15T00:12:59.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049551523 2024-09-15T00:12:59.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351017+0000", 2024-09-15T00:12:59.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8359999999999992e-06 2024-09-15T00:12:59.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351112+0000", 2024-09-15T00:12:59.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5173999999999994e-05 2024-09-15T00:12:59.048 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.048 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.048 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.048 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127004 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 6881280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:59.048 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:59.048 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.049879419, 2024-09-15T00:12:59.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735720000000003, 2024-09-15T00:12:59.049 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.049 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.049 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:59.049 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.049 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.049 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.049 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127004 2024-09-15T00:12:59.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.049 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:59.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:59.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:12:59.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:12:59.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968331+0000", 2024-09-15T00:12:59.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.849e-06 2024-09-15T00:12:59.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968334+0000", 2024-09-15T00:12:59.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7689999999999998e-06 2024-09-15T00:12:59.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968338+0000", 2024-09-15T00:12:59.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2490000000000001e-06 2024-09-15T00:12:59.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008987+0000", 2024-09-15T00:12:59.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040649117999999998 2024-09-15T00:12:59.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009027+0000", 2024-09-15T00:12:59.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0357e-05 2024-09-15T00:12:59.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009064+0000", 2024-09-15T00:12:59.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7036000000000003e-05 2024-09-15T00:12:59.052 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.052 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.053 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.053 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127005 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7004160~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:59.053 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:59.053 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.049866866999999, 2024-09-15T00:12:59.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040824358999999998, 2024-09-15T00:12:59.053 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.053 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.053 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:59.053 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.053 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.053 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.053 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127005 2024-09-15T00:12:59.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:12:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-15T00:12:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968343+0000", 2024-09-15T00:12:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3430000000000001e-06 2024-09-15T00:12:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968349+0000", 2024-09-15T00:12:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3080000000000004e-06 2024-09-15T00:12:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968355+0000", 2024-09-15T00:12:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9499999999999998e-06 2024-09-15T00:12:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009091+0000", 2024-09-15T00:12:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735645000000001 2024-09-15T00:12:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009130+0000", 2024-09-15T00:12:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8951e-05 2024-09-15T00:12:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009165+0000", 2024-09-15T00:12:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5926000000000001e-05 2024-09-15T00:12:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127006 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7127040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.049849809000001, 2024-09-15T00:12:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040912783000000001, 2024-09-15T00:12:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127006 2024-09-15T00:12:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:12:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968359+0000", 2024-09-15T00:12:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0419999999999999e-06 2024-09-15T00:12:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968363+0000", 2024-09-15T00:12:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5999999999999998e-06 2024-09-15T00:12:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968366+0000", 2024-09-15T00:12:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5179999999999999e-06 2024-09-15T00:12:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009196+0000", 2024-09-15T00:12:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040830313 2024-09-15T00:12:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009234+0000", 2024-09-15T00:12:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8562000000000001e-05 2024-09-15T00:12:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009271+0000", 2024-09-15T00:12:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6538000000000002e-05 2024-09-15T00:12:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127007 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7249920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:12:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.049839609999999, 2024-09-15T00:12:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041019218000000003, 2024-09-15T00:12:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127007 2024-09-15T00:12:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:12:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:12:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968369+0000", 2024-09-15T00:12:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-15T00:12:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968370+0000", 2024-09-15T00:12:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.012e-06 2024-09-15T00:12:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968374+0000", 2024-09-15T00:12:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3839999999999999e-06 2024-09-15T00:12:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968376+0000", 2024-09-15T00:12:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5339999999999998e-06 2024-09-15T00:12:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009292+0000", 2024-09-15T00:12:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040915092 2024-09-15T00:12:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009333+0000", 2024-09-15T00:12:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1105e-05 2024-09-15T00:12:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009388+0000", 2024-09-15T00:12:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4868999999999999e-05 2024-09-15T00:12:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127008 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7364608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:12:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.049828488999999, 2024-09-15T00:12:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041115290999999998, 2024-09-15T00:12:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127008 2024-09-15T00:12:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:12:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:12:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968380+0000", 2024-09-15T00:12:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3999999999999998e-07 2024-09-15T00:12:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968381+0000", 2024-09-15T00:12:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8700000000000004e-07 2024-09-15T00:12:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968385+0000", 2024-09-15T00:12:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9260000000000002e-06 2024-09-15T00:12:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968389+0000", 2024-09-15T00:12:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.685e-06 2024-09-15T00:12:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009419+0000", 2024-09-15T00:12:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041029852999999998 2024-09-15T00:12:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009458+0000", 2024-09-15T00:12:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8538000000000003e-05 2024-09-15T00:12:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009495+0000", 2024-09-15T00:12:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7061999999999999e-05 2024-09-15T00:12:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127009 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7487488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.049794372000001, 2024-09-15T00:12:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041233063, 2024-09-15T00:12:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127009 2024-09-15T00:12:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:12:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7199999999999999e-07 2024-09-15T00:12:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968417+0000", 2024-09-15T00:12:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3810000000000002e-06 2024-09-15T00:12:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968423+0000", 2024-09-15T00:12:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8820000000000003e-06 2024-09-15T00:12:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968433+0000", 2024-09-15T00:12:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0005e-05 2024-09-15T00:12:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009527+0000", 2024-09-15T00:12:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041093750999999998 2024-09-15T00:12:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009601+0000", 2024-09-15T00:12:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3979000000000004e-05 2024-09-15T00:12:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009647+0000", 2024-09-15T00:12:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5493e-05 2024-09-15T00:12:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127010 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7610368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.049765368000003, 2024-09-15T00:12:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041325924999999999, 2024-09-15T00:12:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127010 2024-09-15T00:12:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:12:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:12:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968445+0000", 2024-09-15T00:12:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:12:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968452+0000", 2024-09-15T00:12:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-15T00:12:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968456+0000", 2024-09-15T00:12:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5210000000000004e-06 2024-09-15T00:12:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009664+0000", 2024-09-15T00:12:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041207404000000003 2024-09-15T00:12:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009723+0000", 2024-09-15T00:12:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9036000000000002e-05 2024-09-15T00:12:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009768+0000", 2024-09-15T00:12:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5578000000000003e-05 2024-09-15T00:12:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127011 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7733248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.049747132999997, 2024-09-15T00:12:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041482441000000002, 2024-09-15T00:12:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127011 2024-09-15T00:12:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:12:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-15T00:12:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968466+0000", 2024-09-15T00:12:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:12:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968470+0000", 2024-09-15T00:12:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6469999999999997e-06 2024-09-15T00:12:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968473+0000", 2024-09-15T00:12:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.977e-06 2024-09-15T00:12:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009800+0000", 2024-09-15T00:12:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041326918999999997 2024-09-15T00:12:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009869+0000", 2024-09-15T00:12:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8935000000000003e-05 2024-09-15T00:12:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009943+0000", 2024-09-15T00:12:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4188000000000002e-05 2024-09-15T00:12:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127012 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7856128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:12:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.049731631, 2024-09-15T00:12:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041640756000000001, 2024-09-15T00:12:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127012 2024-09-15T00:12:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:12:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:12:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968477+0000", 2024-09-15T00:12:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:12:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968482+0000", 2024-09-15T00:12:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6180000000000003e-06 2024-09-15T00:12:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968486+0000", 2024-09-15T00:12:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.501e-06 2024-09-15T00:12:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968489+0000", 2024-09-15T00:12:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1530000000000001e-06 2024-09-15T00:12:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009983+0000", 2024-09-15T00:12:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041493980999999999 2024-09-15T00:12:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010051+0000", 2024-09-15T00:12:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8039000000000004e-05 2024-09-15T00:12:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010117+0000", 2024-09-15T00:12:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6254000000000006e-05 2024-09-15T00:12:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127013 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7979008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:12:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.049715810999999, 2024-09-15T00:12:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041763863999999998, 2024-09-15T00:12:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127013 2024-09-15T00:12:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:12:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:12:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969767+0000", 2024-09-15T00:12:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001275006 2024-09-15T00:12:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969772+0000", 2024-09-15T00:12:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4769999999999997e-06 2024-09-15T00:12:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969781+0000", 2024-09-15T00:12:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1959999999999997e-06 2024-09-15T00:12:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969788+0000", 2024-09-15T00:12:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6359999999999994e-06 2024-09-15T00:12:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010162+0000", 2024-09-15T00:12:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040373291999999998 2024-09-15T00:12:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010212+0000", 2024-09-15T00:12:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0543e-05 2024-09-15T00:12:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010256+0000", 2024-09-15T00:12:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3714000000000002e-05 2024-09-15T00:12:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127015 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8224768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:12:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.048376875000002, 2024-09-15T00:12:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040710186000000002, 2024-09-15T00:12:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127015 2024-09-15T00:12:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:12:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:12:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969832+0000", 2024-09-15T00:12:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:12:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969834+0000", 2024-09-15T00:12:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2919999999999998e-06 2024-09-15T00:12:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969843+0000", 2024-09-15T00:12:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0480000000000004e-06 2024-09-15T00:12:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969848+0000", 2024-09-15T00:12:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-15T00:12:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010457+0000", 2024-09-15T00:12:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040609054999999998 2024-09-15T00:12:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010501+0000", 2024-09-15T00:12:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3396000000000003e-05 2024-09-15T00:12:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010541+0000", 2024-09-15T00:12:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0599000000000002e-05 2024-09-15T00:12:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127016 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:12:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.048326248999999, 2024-09-15T00:12:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040766216000000001, 2024-09-15T00:12:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127016 2024-09-15T00:12:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:12:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:12:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969886+0000", 2024-09-15T00:12:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8449999999999996e-06 2024-09-15T00:12:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969897+0000", 2024-09-15T00:12:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-15T00:12:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969904+0000", 2024-09-15T00:12:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1169999999999998e-06 2024-09-15T00:12:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010561+0000", 2024-09-15T00:12:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040657470000000001 2024-09-15T00:12:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010607+0000", 2024-09-15T00:12:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6108e-05 2024-09-15T00:12:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010648+0000", 2024-09-15T00:12:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0479000000000003e-05 2024-09-15T00:12:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127694 3.9 3:9390b205:::rbd_data.11396481e1ec.00000000000000df:head [set-alloc-hint object_size 1048576 write_size 1048576,write 819200~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T00:12:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:12:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.793607131999998, 2024-09-15T00:12:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90376419399999997, 2024-09-15T00:12:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127694 2024-09-15T00:12:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:12:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:12:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224602+0000", 2024-09-15T00:12:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5899999999999995e-07 2024-09-15T00:12:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224610+0000", 2024-09-15T00:12:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9130000000000003e-06 2024-09-15T00:12:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224620+0000", 2024-09-15T00:12:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0426e-05 2024-09-15T00:12:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224627+0000", 2024-09-15T00:12:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4390000000000003e-06 2024-09-15T00:12:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224675+0000", 2024-09-15T00:12:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7636000000000003e-05 2024-09-15T00:12:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:12:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224681+0000", 2024-09-15T00:12:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1460000000000001e-06 2024-09-15T00:12:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126388+0000", 2024-09-15T00:12:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90170724999999996 2024-09-15T00:12:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126526+0000", 2024-09-15T00:12:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013730599999999999 2024-09-15T00:12:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126684+0000", 2024-09-15T00:12:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000157935 2024-09-15T00:12:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.127932+0000", 2024-09-15T00:12:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012479240000000001 2024-09-15T00:12:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128269+0000", 2024-09-15T00:12:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033761099999999998 2024-09-15T00:12:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128324+0000", 2024-09-15T00:12:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5244999999999997e-05 2024-09-15T00:12:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128365+0000", 2024-09-15T00:12:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0503999999999997e-05 2024-09-15T00:12:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:134707 3.15 3:af1fbdb5:::rbd_data.113933a4f79a.000000000000001c:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T00:12:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:12:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.325640018000001, 2024-09-15T00:12:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057328626000000001, 2024-09-15T00:12:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:12:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:12:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 134707 2024-09-15T00:12:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:12:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:12:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692569+0000", 2024-09-15T00:12:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.339e-06 2024-09-15T00:12:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692574+0000", 2024-09-15T00:12:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-15T00:12:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692584+0000", 2024-09-15T00:12:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0363000000000001e-05 2024-09-15T00:12:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692590+0000", 2024-09-15T00:12:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.818e-06 2024-09-15T00:12:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692633+0000", 2024-09-15T00:12:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.265e-05 2024-09-15T00:12:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692718+0000", 2024-09-15T00:12:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5279999999999997e-05 2024-09-15T00:12:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:12:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.695940+0000", 2024-09-15T00:12:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003222052 2024-09-15T00:12:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749714+0000", 2024-09-15T00:12:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.053774126999999998 2024-09-15T00:12:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:12:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749788+0000", 2024-09-15T00:12:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3693000000000001e-05 2024-09-15T00:12:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:12:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749847+0000", 2024-09-15T00:12:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175999999999997e-05 2024-09-15T00:12:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749897+0000", 2024-09-15T00:12:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9311999999999998e-05 2024-09-15T00:12:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:134707 3.15 e86/55 ondisk, result = 0)", 2024-09-15T00:12:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:12:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.317078669000001, 2024-09-15T00:12:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048596066, 2024-09-15T00:12:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:12:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:12:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:12:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:12:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:12:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:12:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:12:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:12:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:12:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:12:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701131+0000", 2024-09-15T00:12:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3549999999999999e-06 2024-09-15T00:12:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:12:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701139+0000", 2024-09-15T00:12:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1650000000000006e-06 2024-09-15T00:12:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:12:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701156+0000", 2024-09-15T00:12:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6730000000000001e-05 2024-09-15T00:12:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701167+0000", 2024-09-15T00:12:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1504e-05 2024-09-15T00:12:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:12:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749629+0000", 2024-09-15T00:12:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048461952000000003 2024-09-15T00:12:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:12:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749640+0000", 2024-09-15T00:12:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0874e-05 2024-09-15T00:12:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:12:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:12:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:12:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749725+0000", 2024-09-15T00:12:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5486000000000001e-05 2024-09-15T00:12:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:12:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:12:59.118 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:12:59.119 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:12:59.119 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:59.117+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:59.134 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:12:59.220 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:12:59.218+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:12:59.237 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:59.237 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:59.237 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:59.237 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:59.246 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:12:59.320 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:59.319+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:59.389 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:59.389 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:59.389 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:59.390 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:59.390 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:59.397 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:12:59.421 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:12:59.419+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:12:59.502 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:59.503 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:59.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:59.503 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:59.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.503 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:59.503 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:59.503 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.928333890000005, 2024-09-15T00:12:59.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:12:59.503 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.503 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.503 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:59.504 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.504 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.504 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.504 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:12:59.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.504 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:59.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:12:59.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:12:59.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:12:59.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:12:59.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:12:59.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:12:59.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:12:59.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:12:59.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:12:59.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:12:59.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:12:59.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:12:59.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:12:59.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:59.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:12:59.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:12:59.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:59.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:12:59.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:12:59.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:59.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:12:59.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:12:59.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:59.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:12:59.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:12:59.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:12:59.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:12:59.508 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.509 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.509 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.509 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:12:59.509 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:59.509 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.928299198000005, 2024-09-15T00:12:59.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:12:59.509 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.509 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.509 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:59.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.510 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:12:59.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.510 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:59.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:59.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:12:59.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:12:59.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:12:59.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:12:59.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:12:59.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:12:59.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:12:59.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:12:59.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:12:59.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:12:59.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:12:59.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:12:59.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:12:59.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:12:59.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:12:59.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:59.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:12:59.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:12:59.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:59.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:12:59.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:12:59.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:59.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:12:59.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:12:59.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:12:59.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:12:59.514 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.515 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.515 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.515 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.515 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:59.515 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.887625510000007, 2024-09-15T00:12:59.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:12:59.515 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.515 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.515 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.515 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.515 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.515 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.516 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:12:59.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.516 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:59.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:59.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:12:59.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:12:59.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:12:59.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:12:59.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:12:59.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:12:59.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:12:59.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:12:59.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:12:59.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:12:59.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:12:59.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:12:59.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:12:59.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:12:59.519 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.519 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.519 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.519 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.519 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:59.519 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.887608979000007, 2024-09-15T00:12:59.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:12:59.519 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.519 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.520 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.520 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:12:59.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.520 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:59.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:12:59.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:12:59.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:12:59.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:12:59.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:12:59.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:12:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:12:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:12:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:12:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:12:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:12:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:12:59.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:12:59.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:12:59.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:12:59.523 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.523 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.523 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.523 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.523 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:59.524 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.8875922, 2024-09-15T00:12:59.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:12:59.524 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.524 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.524 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.524 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.524 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.524 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.524 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:12:59.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.524 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:59.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:59.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:12:59.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:12:59.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:12:59.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:12:59.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:12:59.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:12:59.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:12:59.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:12:59.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:12:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:12:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:12:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:12:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:12:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:12:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.887574470000004, 2024-09-15T00:12:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:12:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:12:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:59.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:59.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:12:59.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:12:59.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:12:59.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:12:59.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:12:59.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:12:59.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:12:59.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:12:59.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:12:59.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:12:59.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:12:59.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:12:59.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:12:59.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:12:59.532 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.532 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.532 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.532 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.532 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:59.532 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.887557306000005, 2024-09-15T00:12:59.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:12:59.532 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.532 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.532 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.532 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.532 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.533 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.533 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:12:59.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.533 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:59.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:59.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:12:59.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:12:59.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:12:59.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:12:59.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:12:59.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:12:59.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:12:59.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:12:59.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:12:59.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:12:59.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:12:59.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:12:59.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:12:59.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:12:59.536 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.536 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.536 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.536 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.536 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:59.536 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.887535630000002, 2024-09-15T00:12:59.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:12:59.536 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.536 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.537 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.537 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.537 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.537 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.537 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:12:59.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.537 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:59.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:59.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:12:59.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:12:59.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:12:59.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:12:59.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:12:59.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:12:59.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:12:59.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:12:59.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:12:59.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:12:59.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:12:59.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:12:59.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:12:59.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:12:59.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.540 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.540 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.540 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:59.541 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.887489505999994, 2024-09-15T00:12:59.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:12:59.541 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.541 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.541 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.541 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:12:59.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.541 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:12:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:12:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:12:59.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:12:59.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:12:59.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:12:59.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:12:59.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:12:59.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:12:59.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:12:59.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:12:59.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:12:59.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:12:59.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:12:59.544 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.544 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.545 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.545 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.545 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:59.545 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.887459749000001, 2024-09-15T00:12:59.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:12:59.545 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.545 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.545 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.545 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.545 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.545 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.545 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:12:59.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.546 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:59.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:12:59.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:12:59.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:12:59.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:12:59.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:12:59.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:12:59.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:12:59.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:12:59.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:12:59.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:12:59.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:12:59.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:12:59.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:12:59.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:12:59.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:12:59.549 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.549 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.549 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.549 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.549 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:59.549 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.887433430000002, 2024-09-15T00:12:59.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:12:59.549 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.549 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.549 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.549 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.550 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:12:59.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.550 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:59.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:59.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:12:59.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:12:59.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:12:59.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:12:59.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:12:59.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:12:59.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:12:59.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:12:59.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:12:59.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:12:59.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.887397567999997, 2024-09-15T00:12:59.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:12:59.554 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.554 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.554 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.554 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.554 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.554 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.554 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:12:59.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.554 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:59.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:59.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:12:59.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:12:59.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:12:59.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:12:59.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:12:59.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:12:59.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:12:59.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:12:59.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:12:59.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:12:59.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:12:59.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:12:59.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:12:59.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:12:59.557 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.557 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.557 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.887375849999998, 2024-09-15T00:12:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:12:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:12:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:59.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:12:59.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:12:59.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:12:59.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:12:59.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:12:59.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:12:59.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:12:59.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:12:59.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:12:59.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:12:59.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:12:59.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:12:59.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:12:59.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:12:59.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:12:59.565 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.565 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.565 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.565 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.565 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:59.565 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.887179383000003, 2024-09-15T00:12:59.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:12:59.565 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.565 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.565 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.566 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.566 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.566 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.566 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:12:59.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.566 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:59.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:59.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:12:59.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:12:59.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:12:59.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:12:59.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:12:59.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:12:59.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:12:59.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:12:59.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:12:59.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:12:59.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:12:59.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:12:59.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:12:59.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:12:59.569 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.569 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.569 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.569 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.569 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:59.570 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.887153544, 2024-09-15T00:12:59.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:12:59.570 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.570 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.570 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.570 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.570 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.570 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.570 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:12:59.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.570 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:59.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:59.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:12:59.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:12:59.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:12:59.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:12:59.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:12:59.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:12:59.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:12:59.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:12:59.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:12:59.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:12:59.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:12:59.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:12:59.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:12:59.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:12:59.573 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.573 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.573 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.574 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.574 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:59.574 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.887091897999994, 2024-09-15T00:12:59.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:12:59.574 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.574 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.574 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.574 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.574 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.574 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.574 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:12:59.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.574 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:59.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:12:59.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:12:59.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:12:59.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:12:59.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:12:59.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:12:59.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:12:59.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:12:59.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:12:59.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:12:59.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:12:59.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:12:59.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:12:59.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:12:59.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:12:59.578 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.578 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.578 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.578 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.578 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:59.578 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.887052569999994, 2024-09-15T00:12:59.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:12:59.578 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.578 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.578 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.578 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.579 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.579 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.579 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:12:59.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.579 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:59.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:12:59.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:12:59.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:12:59.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:12:59.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:12:59.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:12:59.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:12:59.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:12:59.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:12:59.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:12:59.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:12:59.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:12:59.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:12:59.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:12:59.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:12:59.582 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.582 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.582 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.582 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.582 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:59.582 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.885327169999996, 2024-09-15T00:12:59.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:12:59.583 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.583 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.583 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.583 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.583 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.583 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.583 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:12:59.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.583 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:59.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:12:59.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:12:59.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:12:59.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:12:59.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:12:59.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:12:59.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:12:59.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:12:59.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:12:59.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:12:59.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:12:59.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:12:59.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:12:59.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:12:59.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:12:59.586 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.586 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.586 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.587 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.587 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:59.587 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.885305009000007, 2024-09-15T00:12:59.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:12:59.587 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.587 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.587 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.587 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.587 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.587 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.587 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:12:59.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.587 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:59.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:59.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:12:59.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:12:59.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:12:59.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:12:59.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:12:59.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:12:59.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:12:59.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:12:59.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:12:59.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:12:59.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:12:59.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:12:59.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:12:59.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:12:59.590 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.591 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.591 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:145034 3.6 3:618673df:::rbd_object_map.1139646a7d4e.000000000000002b:head [writefull 0~48] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T00:12:59.591 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:12:59.591 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.483051405000001, 2024-09-15T00:12:59.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0081079740000001, 2024-09-15T00:12:59.591 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.591 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.591 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:59.591 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.591 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.591 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.592 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 145034 2024-09-15T00:12:59.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.592 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:12:59.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:12:59.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017198+0000", 2024-09-15T00:12:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0100000000000004e-07 2024-09-15T00:12:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017202+0000", 2024-09-15T00:12:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:12:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017212+0000", 2024-09-15T00:12:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7189999999999996e-06 2024-09-15T00:12:59.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017219+0000", 2024-09-15T00:12:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7839999999999998e-06 2024-09-15T00:12:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017268+0000", 2024-09-15T00:12:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:12:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:12:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017277+0000", 2024-09-15T00:12:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1899999999999995e-06 2024-09-15T00:12:59.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019139+0000", 2024-09-15T00:12:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0018626129999999 2024-09-15T00:12:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019236+0000", 2024-09-15T00:12:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6513000000000002e-05 2024-09-15T00:12:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:12:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019318+0000", 2024-09-15T00:12:59.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1848000000000006e-05 2024-09-15T00:12:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:12:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.023151+0000", 2024-09-15T00:12:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038332169999999999 2024-09-15T00:12:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:12:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025217+0000", 2024-09-15T00:12:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020657610000000002 2024-09-15T00:12:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:12:59.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025267+0000", 2024-09-15T00:12:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0198000000000003e-05 2024-09-15T00:12:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025305+0000", 2024-09-15T00:12:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8065999999999998e-05 2024-09-15T00:12:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.597 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.597 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:59.599 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:12:59.599 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:59.597+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:59.700 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:12:59.698+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:12:59.701 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:59.702 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:59.702 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:12:59.702 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:59.710 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:12:59.788 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:12:59.800 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:12:59.799+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:12:59.811 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:59.812 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:12:59.812 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:12:59.812 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:12:59.812 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:12:59.820 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:12:59.901 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:12:59.900+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:12:59.910 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:12:59.911 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:12:59.927 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:12:59.927 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:12:59.943 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:12:59.943 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:12:59.958 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:12:59.959 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:12:59.964 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:12:59.964 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:12:59.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:12:59.964 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:12:59.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.964 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.964 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:59.965 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 75.352295209000005, 2024-09-15T00:12:59.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:12:59.965 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.965 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.965 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.965 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.965 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.965 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.965 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:12:59.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.965 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:59.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:59.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:12:59.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:12:59.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:12:59.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:12:59.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:12:59.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:12:59.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:12:59.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:12:59.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:12:59.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:12:59.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:12:59.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:12:59.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:12:59.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:12:59.968 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.969 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.969 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.969 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.969 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:59.969 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 75.352273980000007, 2024-09-15T00:12:59.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:12:59.969 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:12:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:12:59.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:12:59.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:12:59.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:12:59.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:12:59.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:12:59.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:12:59.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:12:59.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:12:59.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:12:59.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:12:59.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:12:59.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:12:59.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:12:59.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:12:59.974 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.974 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.974 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.974 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.975 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:59.975 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 75.352248438999993, 2024-09-15T00:12:59.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:12:59.975 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.975 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.975 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.975 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.975 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.975 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.975 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:12:59.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.975 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:59.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:59.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:12:59.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:12:59.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:12:59.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:12:59.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:12:59.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:12:59.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:12:59.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:12:59.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:12:59.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:12:59.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:12:59.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:12:59.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:12:59.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:12:59.978 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.978 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.979 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.979 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.979 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:59.979 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 75.352225963999999, 2024-09-15T00:12:59.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:12:59.979 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.979 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.979 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.979 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.979 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.979 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.979 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:12:59.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.980 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:59.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:12:59.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:12:59.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:12:59.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:12:59.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:12:59.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:12:59.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:12:59.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:12:59.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:12:59.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:12:59.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:12:59.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:12:59.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:12:59.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:12:59.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:12:59.983 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.983 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.983 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.983 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.983 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:59.983 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 75.352205454, 2024-09-15T00:12:59.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:12:59.983 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.983 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.984 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.984 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.984 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.984 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.984 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:12:59.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.984 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:59.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:59.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:12:59.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:12:59.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:12:59.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:12:59.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:12:59.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:12:59.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:12:59.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:12:59.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:12:59.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:12:59.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:12:59.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:12:59.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:12:59.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:12:59.987 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.987 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.987 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.987 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.987 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:59.988 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 75.352185138999999, 2024-09-15T00:12:59.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:12:59.988 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.988 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.988 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.988 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.988 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.988 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.988 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:12:59.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.988 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:59.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:12:59.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:12:59.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:12:59.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:12:59.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:12:59.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:12:59.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:12:59.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:12:59.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:12:59.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:12:59.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:12:59.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:12:59.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:12:59.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:12:59.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:12:59.991 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.991 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.991 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.992 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.992 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:59.992 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 75.352160362999996, 2024-09-15T00:12:59.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:12:59.992 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.992 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.992 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:12:59.992 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.992 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.992 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.992 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:12:59.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.993 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:59.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:59.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:12:59.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:12:59.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:12:59.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:12:59.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:12:59.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:12:59.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:12:59.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:12:59.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:12:59.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:12:59.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:12:59.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:12:59.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:12:59.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:12:59.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:12:59.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.996 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:12:59.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:12:59.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.996 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:12:59.996 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:59.996 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 75.172818976000002, 2024-09-15T00:12:59.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:12:59.996 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:12:59.996 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:12:59.996 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:12:59.996 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:12:59.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:12:59.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:12:59.997 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:12:59.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.997 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:12:59.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:12:59.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:59.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:12:59.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:12:59.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:12:59.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:12:59.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:12:59.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:12:59.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:12:59.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:12:59.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:12:59.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:12:59.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:12:59.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:12:59.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:12:59.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:12:59.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:12:59.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:12:59.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:12:59.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:12:59.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:12:59.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:12:59.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:12:59.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:13:00.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:13:00.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:00.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:13:00.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:13:00.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:00.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:13:00.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:13:00.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:00.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:13:00.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:13:00.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:00.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:13:00.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:13:00.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:00.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:13:00.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:13:00.001 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.001 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:00.002 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.002 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:13:00.002 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:00.002 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 75.155865524000006, 2024-09-15T00:13:00.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:13:00.002 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:00.002 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:00.002 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:00.002 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:00.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:00.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:00.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:00.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:00.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:00.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:13:00.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:13:00.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:00.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:13:00.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:13:00.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:00.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:13:00.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:13:00.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:00.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:13:00.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:13:00.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:00.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:13:00.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:13:00.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:00.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:13:00.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:13:00.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:00.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:13:00.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:13:00.005 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.005 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:00.005 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.006 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:13:00.006 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:00.006 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.861434617, 2024-09-15T00:13:00.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:13:00.006 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:00.006 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:00.006 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:00.006 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:00.006 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:00.006 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:00.006 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:13:00.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.007 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:00.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:00.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:00.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:00.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:00.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:00.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:13:00.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:13:00.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:00.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:13:00.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:13:00.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:00.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:13:00.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:13:00.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:00.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:13:00.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:13:00.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:00.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:13:00.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:13:00.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:00.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:13:00.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:13:00.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:00.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:13:00.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:13:00.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:00.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:13:00.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:13:00.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:00.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:13:00.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:13:00.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:00.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:13:00.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:13:00.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:00.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:13:00.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:13:00.011 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.011 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:00.011 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.011 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:13:00.012 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:00.012 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.848978803000001, 2024-09-15T00:13:00.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:13:00.012 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:00.012 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:00.012 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:00.012 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:00.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:00.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:00.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:00.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:00.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:00.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:13:00.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:13:00.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:00.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:13:00.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:13:00.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:00.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:13:00.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:13:00.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:00.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:13:00.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:13:00.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:00.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:13:00.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:13:00.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:13:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:13:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:13:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:13:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:146935 3.16 3:6c79e092:::rbd_data.11394501364.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.208199284999999, 2024-09-15T00:13:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.058987488999999997, 2024-09-15T00:13:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 146935 2024-09-15T00:13:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:00.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:00.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:00.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:00.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:00.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753644+0000", 2024-09-15T00:13:00.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6199999999999997e-07 2024-09-15T00:13:00.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:00.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753648+0000", 2024-09-15T00:13:00.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-15T00:13:00.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:00.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753659+0000", 2024-09-15T00:13:00.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0703000000000001e-05 2024-09-15T00:13:00.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:00.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753664+0000", 2024-09-15T00:13:00.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.04e-06 2024-09-15T00:13:00.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753709+0000", 2024-09-15T00:13:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4728000000000003e-05 2024-09-15T00:13:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753756+0000", 2024-09-15T00:13:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7808e-05 2024-09-15T00:13:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.757051+0000", 2024-09-15T00:13:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294249 2024-09-15T00:13:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:00.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812553+0000", 2024-09-15T00:13:00.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055501807 2024-09-15T00:13:00.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:00.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812564+0000", 2024-09-15T00:13:00.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1467999999999999e-05 2024-09-15T00:13:00.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:00.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812598+0000", 2024-09-15T00:13:00.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3553000000000002e-05 2024-09-15T00:13:00.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:00.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812630+0000", 2024-09-15T00:13:00.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2895000000000003e-05 2024-09-15T00:13:00.021 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.021 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:00.021 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.021 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:146935 3.16 e98/95 ondisk, result = 0)", 2024-09-15T00:13:00.021 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:00.021 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.197396589, 2024-09-15T00:13:00.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048111830000000001, 2024-09-15T00:13:00.021 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:00.022 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:00.022 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:00.022 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:00.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:00.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:00.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:00.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:00.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:00.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764447+0000", 2024-09-15T00:13:00.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0500000000000002e-07 2024-09-15T00:13:00.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:00.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764450+0000", 2024-09-15T00:13:00.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4400000000000001e-06 2024-09-15T00:13:00.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:00.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764462+0000", 2024-09-15T00:13:00.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1603000000000001e-05 2024-09-15T00:13:00.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:00.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764469+0000", 2024-09-15T00:13:00.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5089999999999996e-06 2024-09-15T00:13:00.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:00.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812541+0000", 2024-09-15T00:13:00.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048071585 2024-09-15T00:13:00.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:00.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812548+0000", 2024-09-15T00:13:00.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9979999999999997e-06 2024-09-15T00:13:00.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:00.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812557+0000", 2024-09-15T00:13:00.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7899999999999994e-06 2024-09-15T00:13:00.025 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.025 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:00.025 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.025 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:152505 3.17 3:ea3f268c:::rbd_data.113958cb0e55.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:00.025 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:00.025 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.295290627, 2024-09-15T00:13:00.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.060817959999999997, 2024-09-15T00:13:00.026 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:00.026 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:00.026 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:00.026 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:00.026 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:00.026 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:00.026 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 152505 2024-09-15T00:13:00.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.026 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:00.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:00.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:00.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:00.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:00.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:00.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666553+0000", 2024-09-15T00:13:00.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-15T00:13:00.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:00.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666562+0000", 2024-09-15T00:13:00.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0909999999999999e-06 2024-09-15T00:13:00.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:00.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666572+0000", 2024-09-15T00:13:00.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0214000000000001e-05 2024-09-15T00:13:00.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:00.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666577+0000", 2024-09-15T00:13:00.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3800000000000002e-06 2024-09-15T00:13:00.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:00.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666617+0000", 2024-09-15T00:13:00.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9304000000000002e-05 2024-09-15T00:13:00.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:00.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666672+0000", 2024-09-15T00:13:00.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5206e-05 2024-09-15T00:13:00.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:00.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.670821+0000", 2024-09-15T00:13:00.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041493169999999996 2024-09-15T00:13:00.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:00.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727290+0000", 2024-09-15T00:13:00.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.056468580999999997 2024-09-15T00:13:00.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:00.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727302+0000", 2024-09-15T00:13:00.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2605e-05 2024-09-15T00:13:00.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:00.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727337+0000", 2024-09-15T00:13:00.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4783999999999998e-05 2024-09-15T00:13:00.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:00.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727370+0000", 2024-09-15T00:13:00.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2366000000000001e-05 2024-09-15T00:13:00.031 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.031 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:00.031 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.031 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:152505 3.17 e103/68 ondisk, result = 0)", 2024-09-15T00:13:00.031 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:00.031 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.283212191000001, 2024-09-15T00:13:00.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048665938999999998, 2024-09-15T00:13:00.031 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:00.031 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:00.031 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:00.031 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:00.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:00.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:00.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:00.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:00.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:00.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678631+0000", 2024-09-15T00:13:00.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5099999999999998e-07 2024-09-15T00:13:00.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:00.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678634+0000", 2024-09-15T00:13:00.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.478e-06 2024-09-15T00:13:00.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:00.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678647+0000", 2024-09-15T00:13:00.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2085e-05 2024-09-15T00:13:00.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:00.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678653+0000", 2024-09-15T00:13:00.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2099999999999998e-06 2024-09-15T00:13:00.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:00.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727277+0000", 2024-09-15T00:13:00.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048624405000000002 2024-09-15T00:13:00.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:00.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727285+0000", 2024-09-15T00:13:00.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2060000000000007e-06 2024-09-15T00:13:00.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:00.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727296+0000", 2024-09-15T00:13:00.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0604e-05 2024-09-15T00:13:00.035 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.035 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:00.035 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.035 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153959 3.c 3:326b8454:::rbd_data.113958cb0e55.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:00.035 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:00.035 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.8719842450000002, 2024-09-15T00:13:00.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.068102041000000002, 2024-09-15T00:13:00.035 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:00.035 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:00.035 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:00.035 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:00.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:00.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:00.036 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153959 2024-09-15T00:13:00.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.036 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:00.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:00.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:00.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:00.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:00.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:00.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089859+0000", 2024-09-15T00:13:00.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1800000000000004e-07 2024-09-15T00:13:00.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:00.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089864+0000", 2024-09-15T00:13:00.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2079999999999999e-06 2024-09-15T00:13:00.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:00.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089880+0000", 2024-09-15T00:13:00.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5695000000000001e-05 2024-09-15T00:13:00.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:00.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089889+0000", 2024-09-15T00:13:00.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.935e-06 2024-09-15T00:13:00.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:00.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089942+0000", 2024-09-15T00:13:00.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3219e-05 2024-09-15T00:13:00.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:00.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.090066+0000", 2024-09-15T00:13:00.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012385099999999999 2024-09-15T00:13:00.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:00.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.095262+0000", 2024-09-15T00:13:00.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051963950000000004 2024-09-15T00:13:00.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:00.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157865+0000", 2024-09-15T00:13:00.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.062602865999999993 2024-09-15T00:13:00.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:00.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157891+0000", 2024-09-15T00:13:00.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5778e-05 2024-09-15T00:13:00.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:00.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157924+0000", 2024-09-15T00:13:00.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3133000000000003e-05 2024-09-15T00:13:00.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:00.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157960+0000", 2024-09-15T00:13:00.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6043e-05 2024-09-15T00:13:00.040 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.040 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:00.041 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.041 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:153959 3.c e103/23 ondisk, result = 0)", 2024-09-15T00:13:00.041 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:00.041 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.8593285870000003, 2024-09-15T00:13:00.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055370211000000003, 2024-09-15T00:13:00.041 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:00.041 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:00.041 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:00.041 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:00.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:00.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:00.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:00.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:00.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:00.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102515+0000", 2024-09-15T00:13:00.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.206e-06 2024-09-15T00:13:00.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:00.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102520+0000", 2024-09-15T00:13:00.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1209999999999998e-06 2024-09-15T00:13:00.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:00.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102537+0000", 2024-09-15T00:13:00.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:00.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:00.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102546+0000", 2024-09-15T00:13:00.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2329999999999995e-06 2024-09-15T00:13:00.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:00.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157847+0000", 2024-09-15T00:13:00.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055300798999999998 2024-09-15T00:13:00.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:00.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157857+0000", 2024-09-15T00:13:00.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.046e-05 2024-09-15T00:13:00.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:00.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157884+0000", 2024-09-15T00:13:00.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.671e-05 2024-09-15T00:13:00.044 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.044 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:00.044 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.045 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163346 3.1d 3:baa43c12:::rbd_data.11395d771336.0000000000000065:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:13:00.045 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:00.045 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.1907140140000001, 2024-09-15T00:13:00.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047558544000000001, 2024-09-15T00:13:00.045 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:00.045 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:00.045 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:00.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:00.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:00.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:00.045 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163346 2024-09-15T00:13:00.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.045 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:00.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:00.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:00.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:00.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:00.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:00.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771130+0000", 2024-09-15T00:13:00.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5939999999999999e-06 2024-09-15T00:13:00.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:00.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771141+0000", 2024-09-15T00:13:00.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1001e-05 2024-09-15T00:13:00.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:00.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771154+0000", 2024-09-15T00:13:00.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2921e-05 2024-09-15T00:13:00.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:00.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771160+0000", 2024-09-15T00:13:00.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3849999999999999e-06 2024-09-15T00:13:00.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:00.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771207+0000", 2024-09-15T00:13:00.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6888999999999998e-05 2024-09-15T00:13:00.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:00.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771306+0000", 2024-09-15T00:13:00.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9259000000000006e-05 2024-09-15T00:13:00.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:00.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.774692+0000", 2024-09-15T00:13:00.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033854850000000001 2024-09-15T00:13:00.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:00.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818600+0000", 2024-09-15T00:13:00.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043908006999999999 2024-09-15T00:13:00.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:00.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818616+0000", 2024-09-15T00:13:00.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6688e-05 2024-09-15T00:13:00.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:00.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818651+0000", 2024-09-15T00:13:00.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4885000000000002e-05 2024-09-15T00:13:00.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:00.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818687+0000", 2024-09-15T00:13:00.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5429999999999998e-05 2024-09-15T00:13:00.050 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.050 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:00.050 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.050 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163578 3.1c 3:3efd71ab:::rbd_data.11395d771336.0000000000000060:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-15T00:13:00.050 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:00.051 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.7912912649999999, 2024-09-15T00:13:00.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.085154081000000006, 2024-09-15T00:13:00.051 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:00.051 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:00.051 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:00.051 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:00.051 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:00.051 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:00.051 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163578 2024-09-15T00:13:00.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.051 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:00.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:00.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:00.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:00.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:00.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:00.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170552+0000", 2024-09-15T00:13:00.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8599999999999997e-07 2024-09-15T00:13:00.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:00.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170557+0000", 2024-09-15T00:13:00.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-15T00:13:00.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:00.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170567+0000", 2024-09-15T00:13:00.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0628999999999999e-05 2024-09-15T00:13:00.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:00.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170573+0000", 2024-09-15T00:13:00.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1649999999999997e-06 2024-09-15T00:13:00.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:00.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170619+0000", 2024-09-15T00:13:00.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6405e-05 2024-09-15T00:13:00.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:00.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170666+0000", 2024-09-15T00:13:00.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7259000000000003e-05 2024-09-15T00:13:00.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:00.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.175060+0000", 2024-09-15T00:13:00.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043940170000000001 2024-09-15T00:13:00.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:00.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255559+0000", 2024-09-15T00:13:00.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080499065999999994 2024-09-15T00:13:00.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:00.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255616+0000", 2024-09-15T00:13:00.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6422000000000003e-05 2024-09-15T00:13:00.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:00.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255655+0000", 2024-09-15T00:13:00.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9632999999999998e-05 2024-09-15T00:13:00.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:00.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255705+0000", 2024-09-15T00:13:00.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.973e-05 2024-09-15T00:13:00.056 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.056 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:00.056 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.056 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:163578 3.1c e107/23 ondisk, result = 0)", 2024-09-15T00:13:00.056 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:00.056 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.7760691739999999, 2024-09-15T00:13:00.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069939984999999996, 2024-09-15T00:13:00.056 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:00.056 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:00.057 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:00.057 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:00.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:00.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:00.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:00.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:00.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:00.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:00.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185774+0000", 2024-09-15T00:13:00.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.093e-06 2024-09-15T00:13:00.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:00.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185779+0000", 2024-09-15T00:13:00.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9470000000000002e-06 2024-09-15T00:13:00.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:00.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185794+0000", 2024-09-15T00:13:00.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4450000000000001e-05 2024-09-15T00:13:00.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:00.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185801+0000", 2024-09-15T00:13:00.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9189999999999999e-06 2024-09-15T00:13:00.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:00.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255604+0000", 2024-09-15T00:13:00.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069802201999999994 2024-09-15T00:13:00.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:00.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255611+0000", 2024-09-15T00:13:00.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7740000000000004e-06 2024-09-15T00:13:00.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:00.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:00.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:00.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255713+0000", 2024-09-15T00:13:00.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0001016 2024-09-15T00:13:00.060 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.060 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:00.060 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.060 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:00.060 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:00.060 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:00.061 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:00.061 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:00.062 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:00.062 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:00.063 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:00.063 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:00.060+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:00.063 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:00.064 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:00.064 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:00.065 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:00.133 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is unset 2024-09-15T00:13:00.161 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:00.160+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:00.220 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:00.221 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:00.230 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:00.262 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:00.261+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:00.334 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:00.334 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:00.343 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:00.387 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:00.385+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:00.445 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:00.445 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:00.453 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:00.495 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:00.493+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:00.564 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:00.563+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:00.597 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:00.597 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:00.606 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:00.690 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:00.688+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:00.748 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:00.748 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:00.757 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:00.766 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:00.765+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:00.867 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:00.865+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:00.899 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:00.899 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:00.909 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:00.968 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:00.966+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:01.019 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:01.019 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:01.028 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:01.069 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:01.068+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:01.169 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:01.168+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:01.182 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:01.182 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:01.191 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:01.275 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:01.274+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:01.336 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:01.336 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:01.345 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:01.371 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:01.369+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:01.472 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:01.471+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:01.489 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:01.489 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:01.498 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:01.582 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:01.581+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:01.642 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:01.642 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:01.651 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:01.674 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:01.673+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:01.775 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:01.774+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:01.794 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:01.795 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:01.804 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:01.887 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:01.886+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:01.948 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:01.948 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:01.957 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:01.977 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:01.976+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:02.066 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:13:02.078 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:02.076+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:02.101 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:02.101 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:02.111 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:02.149 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:13:02.179 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:02.177+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:02.203 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:02.203 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:02.222 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:02.222 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:02.239 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:02.240 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:02.259 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:02.259 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:02.280 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:02.280 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:02.281 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:02.279+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:02.299 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:02.299 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:02.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:02.299 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:02.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.299 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:02.299 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:02.299 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.115735892000004, 2024-09-15T00:13:02.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:13:02.299 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.299 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.300 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:02.300 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.300 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.300 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.300 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:13:02.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.300 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:02.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:02.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:13:02.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:13:02.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:13:02.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:13:02.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:13:02.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:13:02.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:13:02.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:13:02.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:13:02.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:13:02.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:13:02.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:13:02.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:02.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:13:02.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:13:02.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:02.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:13:02.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:13:02.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:02.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:13:02.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:13:02.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:02.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:13:02.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:13:02.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:13:02.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:13:02.305 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.305 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.305 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.305 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:13:02.305 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:02.305 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.385894712999999, 2024-09-15T00:13:02.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:13:02.305 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.305 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.305 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:02.305 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.306 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:13:02.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.306 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:02.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:02.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:13:02.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:13:02.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:13:02.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:13:02.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:13:02.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:13:02.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:13:02.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:13:02.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:13:02.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:13:02.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:13:02.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:13:02.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:02.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:13:02.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:13:02.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:02.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:13:02.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:13:02.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:02.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:13:02.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:13:02.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:02.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:13:02.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:13:02.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:13:02.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:13:02.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.310 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.311 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.311 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:13:02.311 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:02.311 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.378936350000004, 2024-09-15T00:13:02.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:13:02.311 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.311 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.311 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.311 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:02.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:02.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:13:02.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:13:02.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:13:02.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:13:02.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:13:02.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:13:02.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:13:02.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:13:02.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:13:02.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:13:02.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:13:02.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:13:02.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:13:02.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:13:02.314 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.314 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.314 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.315 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:02.315 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:02.315 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.631747512999993, 2024-09-15T00:13:02.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:13:02.315 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.315 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.315 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:02.315 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.315 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.315 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.315 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:13:02.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.316 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:02.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:02.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:13:02.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:13:02.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:13:02.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:13:02.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:13:02.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:13:02.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:13:02.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:13:02.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:13:02.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:13:02.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:13:02.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:13:02.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:13:02.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:13:02.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:13:02.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:02.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:13:02.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:13:02.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:02.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:13:02.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:13:02.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:02.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:13:02.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:13:02.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:13:02.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:13:02.320 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.320 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.320 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.321 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97493 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:02.321 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:02.321 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.442164793000003, 2024-09-15T00:13:02.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033259141999999998, 2024-09-15T00:13:02.321 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.321 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.321 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.321 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.321 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.321 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.321 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97493 2024-09-15T00:13:02.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.321 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:02.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:02.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:02.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-15T00:13:02.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853401+0000", 2024-09-15T00:13:02.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:13:02.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853407+0000", 2024-09-15T00:13:02.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-15T00:13:02.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853411+0000", 2024-09-15T00:13:02.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.481e-06 2024-09-15T00:13:02.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886544+0000", 2024-09-15T00:13:02.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033132861999999999 2024-09-15T00:13:02.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886587+0000", 2024-09-15T00:13:02.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2942000000000003e-05 2024-09-15T00:13:02.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886655+0000", 2024-09-15T00:13:02.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7711000000000003e-05 2024-09-15T00:13:02.325 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.325 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.325 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.325 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97494 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:02.325 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:02.325 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.44214367, 2024-09-15T00:13:02.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033392390000000001, 2024-09-15T00:13:02.325 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.325 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.325 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.325 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.325 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.325 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.326 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97494 2024-09-15T00:13:02.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.326 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:02.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:02.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:02.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-15T00:13:02.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853419+0000", 2024-09-15T00:13:02.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.917e-06 2024-09-15T00:13:02.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853425+0000", 2024-09-15T00:13:02.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:13:02.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853433+0000", 2024-09-15T00:13:02.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8120000000000008e-06 2024-09-15T00:13:02.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886680+0000", 2024-09-15T00:13:02.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033247207000000001 2024-09-15T00:13:02.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886726+0000", 2024-09-15T00:13:02.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5902000000000002e-05 2024-09-15T00:13:02.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886809+0000", 2024-09-15T00:13:02.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3475999999999998e-05 2024-09-15T00:13:02.329 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.329 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.329 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.329 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97495 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:02.329 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:02.329 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.442122716999997, 2024-09-15T00:13:02.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033640503000000002, 2024-09-15T00:13:02.330 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.330 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.330 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.330 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.330 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.330 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.330 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97495 2024-09-15T00:13:02.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.330 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:02.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:02.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:02.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:13:02.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853440+0000", 2024-09-15T00:13:02.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.804e-06 2024-09-15T00:13:02.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853446+0000", 2024-09-15T00:13:02.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5559999999999998e-06 2024-09-15T00:13:02.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853450+0000", 2024-09-15T00:13:02.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9519999999999996e-06 2024-09-15T00:13:02.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886904+0000", 2024-09-15T00:13:02.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033454207999999999 2024-09-15T00:13:02.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887007+0000", 2024-09-15T00:13:02.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102904 2024-09-15T00:13:02.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887078+0000", 2024-09-15T00:13:02.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1719999999999998e-05 2024-09-15T00:13:02.333 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.333 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.333 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.333 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97496 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:02.334 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:02.334 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.442105619000003, 2024-09-15T00:13:02.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033802431000000001, 2024-09-15T00:13:02.334 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.334 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.334 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.334 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97496 2024-09-15T00:13:02.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.334 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:02.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:02.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:02.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.53e-07 2024-09-15T00:13:02.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853462+0000", 2024-09-15T00:13:02.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-15T00:13:02.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853468+0000", 2024-09-15T00:13:02.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3609999999999997e-06 2024-09-15T00:13:02.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853473+0000", 2024-09-15T00:13:02.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-15T00:13:02.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887111+0000", 2024-09-15T00:13:02.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033638376999999997 2024-09-15T00:13:02.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887178+0000", 2024-09-15T00:13:02.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6877000000000004e-05 2024-09-15T00:13:02.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887257+0000", 2024-09-15T00:13:02.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9616999999999993e-05 2024-09-15T00:13:02.337 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97587 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.442082384999999, 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033973642999999998, 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97587 2024-09-15T00:13:02.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.339 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:02.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:02.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853479+0000", 2024-09-15T00:13:02.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:13:02.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853481+0000", 2024-09-15T00:13:02.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.136e-06 2024-09-15T00:13:02.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853487+0000", 2024-09-15T00:13:02.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2439999999999998e-06 2024-09-15T00:13:02.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853492+0000", 2024-09-15T00:13:02.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:13:02.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887284+0000", 2024-09-15T00:13:02.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033791827000000003 2024-09-15T00:13:02.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887361+0000", 2024-09-15T00:13:02.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6983000000000005e-05 2024-09-15T00:13:02.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887452+0000", 2024-09-15T00:13:02.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1236999999999994e-05 2024-09-15T00:13:02.342 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.342 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.342 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.342 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97588 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:02.342 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:02.342 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.442062227999998, 2024-09-15T00:13:02.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034134617999999999, 2024-09-15T00:13:02.342 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.342 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.342 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.343 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.343 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.343 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.343 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97588 2024-09-15T00:13:02.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.343 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:02.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:02.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853499+0000", 2024-09-15T00:13:02.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:13:02.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853501+0000", 2024-09-15T00:13:02.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.26e-06 2024-09-15T00:13:02.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853507+0000", 2024-09-15T00:13:02.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2899999999999999e-06 2024-09-15T00:13:02.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853512+0000", 2024-09-15T00:13:02.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8330000000000001e-06 2024-09-15T00:13:02.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887495+0000", 2024-09-15T00:13:02.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033982529999999997 2024-09-15T00:13:02.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887548+0000", 2024-09-15T00:13:02.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3445000000000001e-05 2024-09-15T00:13:02.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887633+0000", 2024-09-15T00:13:02.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4890999999999997e-05 2024-09-15T00:13:02.346 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.346 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.346 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.346 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126866 3.6 3.973c9fa6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.347 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:02.347 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.338740283, 2024-09-15T00:13:02.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153301000000003, 2024-09-15T00:13:02.347 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.347 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.347 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:02.347 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.347 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.347 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.347 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126866 2024-09-15T00:13:02.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.347 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:02.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:02.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956821+0000", 2024-09-15T00:13:02.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.91e-07 2024-09-15T00:13:02.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956822+0000", 2024-09-15T00:13:02.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.705e-06 2024-09-15T00:13:02.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956828+0000", 2024-09-15T00:13:02.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6760000000000004e-06 2024-09-15T00:13:02.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956833+0000", 2024-09-15T00:13:02.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5379999999999996e-06 2024-09-15T00:13:02.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989970+0000", 2024-09-15T00:13:02.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033137560000000003 2024-09-15T00:13:02.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989974+0000", 2024-09-15T00:13:02.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-15T00:13:02.350 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.350 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.350 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.350 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126951 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.351 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:13:02.351 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.3386146, 2024-09-15T00:13:02.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033152585999999998, 2024-09-15T00:13:02.351 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.351 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.351 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:02.351 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.351 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.351 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.351 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126951 2024-09-15T00:13:02.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.351 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:13:02.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:13:02.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:13:02.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-07 2024-09-15T00:13:02.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956953+0000", 2024-09-15T00:13:02.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9310000000000002e-06 2024-09-15T00:13:02.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956957+0000", 2024-09-15T00:13:02.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5379999999999998e-06 2024-09-15T00:13:02.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956959+0000", 2024-09-15T00:13:02.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6599999999999999e-06 2024-09-15T00:13:02.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990095+0000", 2024-09-15T00:13:02.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033135855999999998 2024-09-15T00:13:02.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990099+0000", 2024-09-15T00:13:02.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4010000000000001e-06 2024-09-15T00:13:02.354 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.354 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.354 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.355 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126952 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.355 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:02.355 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.338597610999997, 2024-09-15T00:13:02.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033156066999999997, 2024-09-15T00:13:02.355 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.355 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.355 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:02.355 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.355 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.355 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.355 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126952 2024-09-15T00:13:02.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.355 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:02.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:02.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:02.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:13:02.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956965+0000", 2024-09-15T00:13:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3379999999999999e-06 2024-09-15T00:13:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956968+0000", 2024-09-15T00:13:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0419999999999998e-06 2024-09-15T00:13:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956970+0000", 2024-09-15T00:13:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4899999999999999e-06 2024-09-15T00:13:02.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990113+0000", 2024-09-15T00:13:02.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033142642 2024-09-15T00:13:02.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990119+0000", 2024-09-15T00:13:02.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2979999999999997e-06 2024-09-15T00:13:02.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.358 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.358 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.359 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126954 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.359 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:02.359 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.338552116000002, 2024-09-15T00:13:02.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033158449, 2024-09-15T00:13:02.359 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.359 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.359 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:02.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.359 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126954 2024-09-15T00:13:02.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.359 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:02.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:02.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:02.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-15T00:13:02.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957010+0000", 2024-09-15T00:13:02.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3659999999999999e-06 2024-09-15T00:13:02.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957013+0000", 2024-09-15T00:13:02.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1140000000000002e-06 2024-09-15T00:13:02.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957016+0000", 2024-09-15T00:13:02.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1209999999999998e-06 2024-09-15T00:13:02.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990163+0000", 2024-09-15T00:13:02.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033146451 2024-09-15T00:13:02.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990167+0000", 2024-09-15T00:13:02.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0579999999999999e-06 2024-09-15T00:13:02.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.362 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.363 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126955 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.363 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:02.363 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.338540901999998, 2024-09-15T00:13:02.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033160617000000003, 2024-09-15T00:13:02.363 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.363 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.363 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:02.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.363 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126955 2024-09-15T00:13:02.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.363 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:02.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:02.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:02.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-15T00:13:02.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957021+0000", 2024-09-15T00:13:02.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-15T00:13:02.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957024+0000", 2024-09-15T00:13:02.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1599999999999998e-06 2024-09-15T00:13:02.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957027+0000", 2024-09-15T00:13:02.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000001e-06 2024-09-15T00:13:02.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990177+0000", 2024-09-15T00:13:02.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033149741000000003 2024-09-15T00:13:02.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990180+0000", 2024-09-15T00:13:02.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4910000000000001e-06 2024-09-15T00:13:02.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.366 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.366 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126956 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.367 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:02.367 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.338530781000003, 2024-09-15T00:13:02.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033163633999999997, 2024-09-15T00:13:02.367 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.367 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.367 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:02.367 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.367 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.367 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.367 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126956 2024-09-15T00:13:02.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.367 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:02.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:02.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:02.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:02.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957031+0000", 2024-09-15T00:13:02.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-06 2024-09-15T00:13:02.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957034+0000", 2024-09-15T00:13:02.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9960000000000002e-06 2024-09-15T00:13:02.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957037+0000", 2024-09-15T00:13:02.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6419999999999999e-06 2024-09-15T00:13:02.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990190+0000", 2024-09-15T00:13:02.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153152999999998 2024-09-15T00:13:02.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990193+0000", 2024-09-15T00:13:02.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6069999999999999e-06 2024-09-15T00:13:02.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.370 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.371 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126957 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.371 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:02.371 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.338521131, 2024-09-15T00:13:02.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033166442999999997, 2024-09-15T00:13:02.371 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.371 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.371 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:02.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.371 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126957 2024-09-15T00:13:02.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.371 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:02.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:02.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957040+0000", 2024-09-15T00:13:02.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:13:02.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957041+0000", 2024-09-15T00:13:02.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0440000000000001e-06 2024-09-15T00:13:02.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957044+0000", 2024-09-15T00:13:02.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0170000000000001e-06 2024-09-15T00:13:02.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957048+0000", 2024-09-15T00:13:02.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0319999999999997e-06 2024-09-15T00:13:02.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990202+0000", 2024-09-15T00:13:02.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033154640999999999 2024-09-15T00:13:02.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990206+0000", 2024-09-15T00:13:02.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.507e-06 2024-09-15T00:13:02.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.374 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.375 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126958 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.375 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:02.375 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.338509901000002, 2024-09-15T00:13:02.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033167613999999998, 2024-09-15T00:13:02.375 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.375 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.375 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:02.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.375 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126958 2024-09-15T00:13:02.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.375 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:02.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:02.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:02.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:13:02.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957053+0000", 2024-09-15T00:13:02.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.756e-06 2024-09-15T00:13:02.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957059+0000", 2024-09-15T00:13:02.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2770000000000002e-06 2024-09-15T00:13:02.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957063+0000", 2024-09-15T00:13:02.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2459999999999997e-06 2024-09-15T00:13:02.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990215+0000", 2024-09-15T00:13:02.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033151509000000003 2024-09-15T00:13:02.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990218+0000", 2024-09-15T00:13:02.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5099999999999999e-06 2024-09-15T00:13:02.378 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.379 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.379 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126959 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.379 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:02.379 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.338490897, 2024-09-15T00:13:02.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033161673000000003, 2024-09-15T00:13:02.379 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.379 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.379 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:02.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.379 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126959 2024-09-15T00:13:02.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.380 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:02.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:02.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:02.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:13:02.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957072+0000", 2024-09-15T00:13:02.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2239999999999998e-06 2024-09-15T00:13:02.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957079+0000", 2024-09-15T00:13:02.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.19e-06 2024-09-15T00:13:02.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957083+0000", 2024-09-15T00:13:02.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:13:02.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990228+0000", 2024-09-15T00:13:02.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033144818 2024-09-15T00:13:02.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990231+0000", 2024-09-15T00:13:02.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:13:02.382 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.382 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.383 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.383 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126960 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.383 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:02.383 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.338472090000003, 2024-09-15T00:13:02.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033155166, 2024-09-15T00:13:02.383 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.383 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.383 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:02.383 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.383 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.383 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.383 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126960 2024-09-15T00:13:02.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.384 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:02.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:02.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:02.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:13:02.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957091+0000", 2024-09-15T00:13:02.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:13:02.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957097+0000", 2024-09-15T00:13:02.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6050000000000003e-06 2024-09-15T00:13:02.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957102+0000", 2024-09-15T00:13:02.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5539999999999999e-06 2024-09-15T00:13:02.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990240+0000", 2024-09-15T00:13:02.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033138289000000001 2024-09-15T00:13:02.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990244+0000", 2024-09-15T00:13:02.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4740000000000001e-06 2024-09-15T00:13:02.387 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.387 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.387 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.387 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.387 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.387 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:02.388 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:02.388 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:02.389 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:02.389 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:02.390 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:02.390 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:02.391 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:02.391 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:02.387+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:02.489 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:02.488+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:02.534 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:02.534 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:02.535 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:02.535 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:02.543 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:02.590 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:02.589+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:02.686 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:02.687 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:02.687 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:02.687 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:02.687 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:02.691 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:02.690+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:02.695 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:02.792 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:02.790+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:02.800 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:02.800 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:02.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:02.800 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:02.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.800 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:13:02.800 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:02.800 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 74.554701335000004, 2024-09-15T00:13:02.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:13:02.800 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.800 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.801 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:02.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.801 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:13:02.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.801 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:02.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:02.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:13:02.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:13:02.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:13:02.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:13:02.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:13:02.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:13:02.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:13:02.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:02.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:13:02.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:13:02.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:13:02.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:13:02.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:02.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:13:02.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:13:02.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:02.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:13:02.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:13:02.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:02.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:13:02.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:13:02.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:02.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:13:02.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:13:02.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:13:02.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:13:02.806 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.806 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.806 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.806 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58878 3.8 3:12456482:::rbd_data.1139766160da.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:02.806 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:02.806 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.508395750999995, 2024-09-15T00:13:02.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061956199000000003, 2024-09-15T00:13:02.806 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.807 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.807 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:02.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.807 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58878 2024-09-15T00:13:02.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.807 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:02.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:02.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289151+0000", 2024-09-15T00:13:02.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:13:02.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289473+0000", 2024-09-15T00:13:02.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032183800000000002 2024-09-15T00:13:02.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289487+0000", 2024-09-15T00:13:02.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-05 2024-09-15T00:13:02.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289493+0000", 2024-09-15T00:13:02.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4489999999999996e-06 2024-09-15T00:13:02.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289542+0000", 2024-09-15T00:13:02.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8844999999999997e-05 2024-09-15T00:13:02.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289642+0000", 2024-09-15T00:13:02.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9607999999999999e-05 2024-09-15T00:13:02.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:13:02.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.293908+0000", 2024-09-15T00:13:02.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659350000000002 2024-09-15T00:13:02.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:02.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351022+0000", 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057114535000000001 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351033+0000", 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0263e-05 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351070+0000", 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7666000000000002e-05 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351106+0000", 2024-09-15T00:13:02.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999997e-05 2024-09-15T00:13:02.812 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.812 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.812 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.812 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58879 3.7 3:e169894d:::rbd_data.1139766160da.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:02.812 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:02.812 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.506044263000007, 2024-09-15T00:13:02.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043716294000000003, 2024-09-15T00:13:02.812 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.812 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.812 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:02.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.813 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.813 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58879 2024-09-15T00:13:02.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.813 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:02.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:02.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291502+0000", 2024-09-15T00:13:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-15T00:13:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291664+0000", 2024-09-15T00:13:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016216599999999999 2024-09-15T00:13:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291674+0000", 2024-09-15T00:13:02.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0276e-05 2024-09-15T00:13:02.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291680+0000", 2024-09-15T00:13:02.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2580000000000003e-06 2024-09-15T00:13:02.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291732+0000", 2024-09-15T00:13:02.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2114999999999998e-05 2024-09-15T00:13:02.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291866+0000", 2024-09-15T00:13:02.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013416900000000001 2024-09-15T00:13:02.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:13:02.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.296426+0000", 2024-09-15T00:13:02.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045595779999999999 2024-09-15T00:13:02.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:02.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335075+0000", 2024-09-15T00:13:02.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038649347000000001 2024-09-15T00:13:02.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:02.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335097+0000", 2024-09-15T00:13:02.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2565000000000002e-05 2024-09-15T00:13:02.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:02.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335161+0000", 2024-09-15T00:13:02.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3928000000000002e-05 2024-09-15T00:13:02.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335217+0000", 2024-09-15T00:13:02.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6042000000000003e-05 2024-09-15T00:13:02.818 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.818 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.818 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.818 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58880 3.8 3:13d91cfd:::rbd_data.1139766160da.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:02.818 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:02.818 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.502652155999996, 2024-09-15T00:13:02.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056331463999999998, 2024-09-15T00:13:02.819 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.819 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.819 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:02.819 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.819 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.819 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.819 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58880 2024-09-15T00:13:02.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.819 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:02.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:02.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294896+0000", 2024-09-15T00:13:02.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.356e-06 2024-09-15T00:13:02.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295288+0000", 2024-09-15T00:13:02.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039223200000000001 2024-09-15T00:13:02.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295304+0000", 2024-09-15T00:13:02.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5937e-05 2024-09-15T00:13:02.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295312+0000", 2024-09-15T00:13:02.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1489999999999994e-06 2024-09-15T00:13:02.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333162+0000", 2024-09-15T00:13:02.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037849840000000003 2024-09-15T00:13:02.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333269+0000", 2024-09-15T00:13:02.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010749800000000001 2024-09-15T00:13:02.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:13:02.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.336198+0000", 2024-09-15T00:13:02.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029284739999999999 2024-09-15T00:13:02.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:02.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351137+0000", 2024-09-15T00:13:02.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014939103 2024-09-15T00:13:02.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:02.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351150+0000", 2024-09-15T00:13:02.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3542e-05 2024-09-15T00:13:02.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:02.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351191+0000", 2024-09-15T00:13:02.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0791000000000003e-05 2024-09-15T00:13:02.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351225+0000", 2024-09-15T00:13:02.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3541999999999998e-05 2024-09-15T00:13:02.824 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.824 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.824 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.824 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58878 3.8 e56/23 ondisk, result = 0)", 2024-09-15T00:13:02.825 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:02.825 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.496122373000006, 2024-09-15T00:13:02.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049689096000000002, 2024-09-15T00:13:02.825 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.825 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.825 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.825 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:02.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:02.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301425+0000", 2024-09-15T00:13:02.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.782e-06 2024-09-15T00:13:02.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301433+0000", 2024-09-15T00:13:02.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689999999999994e-06 2024-09-15T00:13:02.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301449+0000", 2024-09-15T00:13:02.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5767e-05 2024-09-15T00:13:02.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301458+0000", 2024-09-15T00:13:02.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:13:02.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351009+0000", 2024-09-15T00:13:02.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049551523 2024-09-15T00:13:02.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351017+0000", 2024-09-15T00:13:02.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8359999999999992e-06 2024-09-15T00:13:02.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351112+0000", 2024-09-15T00:13:02.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5173999999999994e-05 2024-09-15T00:13:02.829 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.829 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.829 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.829 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127004 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 6881280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.829 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:02.829 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.829216909000003, 2024-09-15T00:13:02.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735720000000003, 2024-09-15T00:13:02.829 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.829 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.830 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.830 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.830 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.830 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.830 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127004 2024-09-15T00:13:02.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.830 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:02.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:02.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:02.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:13:02.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968331+0000", 2024-09-15T00:13:02.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.849e-06 2024-09-15T00:13:02.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968334+0000", 2024-09-15T00:13:02.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7689999999999998e-06 2024-09-15T00:13:02.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968338+0000", 2024-09-15T00:13:02.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2490000000000001e-06 2024-09-15T00:13:02.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008987+0000", 2024-09-15T00:13:02.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040649117999999998 2024-09-15T00:13:02.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009027+0000", 2024-09-15T00:13:02.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0357e-05 2024-09-15T00:13:02.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009064+0000", 2024-09-15T00:13:02.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7036000000000003e-05 2024-09-15T00:13:02.833 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.833 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.833 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.836 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127005 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7004160~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.836 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:02.836 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.829204357000002, 2024-09-15T00:13:02.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040824358999999998, 2024-09-15T00:13:02.837 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.837 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.837 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.837 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.837 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.837 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.837 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127005 2024-09-15T00:13:02.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.837 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:02.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:02.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:02.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-15T00:13:02.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968343+0000", 2024-09-15T00:13:02.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3430000000000001e-06 2024-09-15T00:13:02.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968349+0000", 2024-09-15T00:13:02.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3080000000000004e-06 2024-09-15T00:13:02.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968355+0000", 2024-09-15T00:13:02.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9499999999999998e-06 2024-09-15T00:13:02.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009091+0000", 2024-09-15T00:13:02.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735645000000001 2024-09-15T00:13:02.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009130+0000", 2024-09-15T00:13:02.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8951e-05 2024-09-15T00:13:02.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009165+0000", 2024-09-15T00:13:02.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5926000000000001e-05 2024-09-15T00:13:02.841 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.841 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.841 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.841 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127006 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7127040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.841 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:02.841 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.829187298999997, 2024-09-15T00:13:02.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040912783000000001, 2024-09-15T00:13:02.841 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.842 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.842 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.842 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.842 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.842 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.842 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127006 2024-09-15T00:13:02.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.842 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:02.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:02.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:02.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:02.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968359+0000", 2024-09-15T00:13:02.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0419999999999999e-06 2024-09-15T00:13:02.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968363+0000", 2024-09-15T00:13:02.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5999999999999998e-06 2024-09-15T00:13:02.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968366+0000", 2024-09-15T00:13:02.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5179999999999999e-06 2024-09-15T00:13:02.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009196+0000", 2024-09-15T00:13:02.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040830313 2024-09-15T00:13:02.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009234+0000", 2024-09-15T00:13:02.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8562000000000001e-05 2024-09-15T00:13:02.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009271+0000", 2024-09-15T00:13:02.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6538000000000002e-05 2024-09-15T00:13:02.847 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.847 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.847 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.847 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127007 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7249920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.847 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:02.847 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.829177100000003, 2024-09-15T00:13:02.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041019218000000003, 2024-09-15T00:13:02.847 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.847 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.847 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.847 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.847 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.848 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.848 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127007 2024-09-15T00:13:02.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.848 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:02.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:02.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968369+0000", 2024-09-15T00:13:02.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-15T00:13:02.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968370+0000", 2024-09-15T00:13:02.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.012e-06 2024-09-15T00:13:02.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968374+0000", 2024-09-15T00:13:02.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3839999999999999e-06 2024-09-15T00:13:02.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968376+0000", 2024-09-15T00:13:02.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5339999999999998e-06 2024-09-15T00:13:02.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009292+0000", 2024-09-15T00:13:02.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040915092 2024-09-15T00:13:02.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009333+0000", 2024-09-15T00:13:02.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1105e-05 2024-09-15T00:13:02.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009388+0000", 2024-09-15T00:13:02.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4868999999999999e-05 2024-09-15T00:13:02.851 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.851 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.851 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.851 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127008 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7364608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.852 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:02.852 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.829165979000003, 2024-09-15T00:13:02.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041115290999999998, 2024-09-15T00:13:02.852 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.852 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.852 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.852 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.852 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.852 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.852 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127008 2024-09-15T00:13:02.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.852 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:02.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:02.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968380+0000", 2024-09-15T00:13:02.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3999999999999998e-07 2024-09-15T00:13:02.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968381+0000", 2024-09-15T00:13:02.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8700000000000004e-07 2024-09-15T00:13:02.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968385+0000", 2024-09-15T00:13:02.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9260000000000002e-06 2024-09-15T00:13:02.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968389+0000", 2024-09-15T00:13:02.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.685e-06 2024-09-15T00:13:02.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009419+0000", 2024-09-15T00:13:02.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041029852999999998 2024-09-15T00:13:02.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009458+0000", 2024-09-15T00:13:02.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8538000000000003e-05 2024-09-15T00:13:02.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009495+0000", 2024-09-15T00:13:02.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7061999999999999e-05 2024-09-15T00:13:02.856 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.856 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.856 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.856 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127009 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7487488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.856 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:02.856 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.829131861999997, 2024-09-15T00:13:02.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041233063, 2024-09-15T00:13:02.856 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.856 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.856 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.857 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.857 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.857 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.857 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127009 2024-09-15T00:13:02.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.857 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:02.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7199999999999999e-07 2024-09-15T00:13:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968417+0000", 2024-09-15T00:13:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3810000000000002e-06 2024-09-15T00:13:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968423+0000", 2024-09-15T00:13:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8820000000000003e-06 2024-09-15T00:13:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968433+0000", 2024-09-15T00:13:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0005e-05 2024-09-15T00:13:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009527+0000", 2024-09-15T00:13:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041093750999999998 2024-09-15T00:13:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009601+0000", 2024-09-15T00:13:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3979000000000004e-05 2024-09-15T00:13:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009647+0000", 2024-09-15T00:13:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5493e-05 2024-09-15T00:13:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127010 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7610368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.829102857999999, 2024-09-15T00:13:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041325924999999999, 2024-09-15T00:13:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127010 2024-09-15T00:13:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:13:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968445+0000", 2024-09-15T00:13:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:13:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968452+0000", 2024-09-15T00:13:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-15T00:13:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968456+0000", 2024-09-15T00:13:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5210000000000004e-06 2024-09-15T00:13:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009664+0000", 2024-09-15T00:13:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041207404000000003 2024-09-15T00:13:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009723+0000", 2024-09-15T00:13:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9036000000000002e-05 2024-09-15T00:13:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009768+0000", 2024-09-15T00:13:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5578000000000003e-05 2024-09-15T00:13:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127011 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7733248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.829084623, 2024-09-15T00:13:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041482441000000002, 2024-09-15T00:13:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127011 2024-09-15T00:13:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-15T00:13:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968466+0000", 2024-09-15T00:13:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:13:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968470+0000", 2024-09-15T00:13:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6469999999999997e-06 2024-09-15T00:13:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968473+0000", 2024-09-15T00:13:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.977e-06 2024-09-15T00:13:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009800+0000", 2024-09-15T00:13:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041326918999999997 2024-09-15T00:13:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009869+0000", 2024-09-15T00:13:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8935000000000003e-05 2024-09-15T00:13:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009943+0000", 2024-09-15T00:13:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4188000000000002e-05 2024-09-15T00:13:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127012 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7856128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.829069121000003, 2024-09-15T00:13:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041640756000000001, 2024-09-15T00:13:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127012 2024-09-15T00:13:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968477+0000", 2024-09-15T00:13:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968482+0000", 2024-09-15T00:13:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6180000000000003e-06 2024-09-15T00:13:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968486+0000", 2024-09-15T00:13:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.501e-06 2024-09-15T00:13:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968489+0000", 2024-09-15T00:13:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1530000000000001e-06 2024-09-15T00:13:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009983+0000", 2024-09-15T00:13:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041493980999999999 2024-09-15T00:13:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010051+0000", 2024-09-15T00:13:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8039000000000004e-05 2024-09-15T00:13:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010117+0000", 2024-09-15T00:13:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6254000000000006e-05 2024-09-15T00:13:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127013 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7979008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.829053301000002, 2024-09-15T00:13:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041763863999999998, 2024-09-15T00:13:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127013 2024-09-15T00:13:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969767+0000", 2024-09-15T00:13:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001275006 2024-09-15T00:13:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969772+0000", 2024-09-15T00:13:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4769999999999997e-06 2024-09-15T00:13:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969781+0000", 2024-09-15T00:13:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1959999999999997e-06 2024-09-15T00:13:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969788+0000", 2024-09-15T00:13:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6359999999999994e-06 2024-09-15T00:13:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010162+0000", 2024-09-15T00:13:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040373291999999998 2024-09-15T00:13:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010212+0000", 2024-09-15T00:13:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0543e-05 2024-09-15T00:13:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010256+0000", 2024-09-15T00:13:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3714000000000002e-05 2024-09-15T00:13:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127015 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8224768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.827714364999999, 2024-09-15T00:13:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040710186000000002, 2024-09-15T00:13:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127015 2024-09-15T00:13:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969832+0000", 2024-09-15T00:13:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:13:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969834+0000", 2024-09-15T00:13:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2919999999999998e-06 2024-09-15T00:13:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969843+0000", 2024-09-15T00:13:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0480000000000004e-06 2024-09-15T00:13:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969848+0000", 2024-09-15T00:13:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-15T00:13:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010457+0000", 2024-09-15T00:13:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040609054999999998 2024-09-15T00:13:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010501+0000", 2024-09-15T00:13:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3396000000000003e-05 2024-09-15T00:13:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010541+0000", 2024-09-15T00:13:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0599000000000002e-05 2024-09-15T00:13:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127016 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.827663739000002, 2024-09-15T00:13:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040766216000000001, 2024-09-15T00:13:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127016 2024-09-15T00:13:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:13:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969886+0000", 2024-09-15T00:13:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8449999999999996e-06 2024-09-15T00:13:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969897+0000", 2024-09-15T00:13:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-15T00:13:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969904+0000", 2024-09-15T00:13:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1169999999999998e-06 2024-09-15T00:13:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010561+0000", 2024-09-15T00:13:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040657470000000001 2024-09-15T00:13:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010607+0000", 2024-09-15T00:13:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6108e-05 2024-09-15T00:13:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010648+0000", 2024-09-15T00:13:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0479000000000003e-05 2024-09-15T00:13:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127694 3.9 3:9390b205:::rbd_data.11396481e1ec.00000000000000df:head [set-alloc-hint object_size 1048576 write_size 1048576,write 819200~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T00:13:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.572944622000001, 2024-09-15T00:13:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90376419399999997, 2024-09-15T00:13:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127694 2024-09-15T00:13:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224602+0000", 2024-09-15T00:13:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5899999999999995e-07 2024-09-15T00:13:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224610+0000", 2024-09-15T00:13:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9130000000000003e-06 2024-09-15T00:13:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224620+0000", 2024-09-15T00:13:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0426e-05 2024-09-15T00:13:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224627+0000", 2024-09-15T00:13:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4390000000000003e-06 2024-09-15T00:13:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224675+0000", 2024-09-15T00:13:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7636000000000003e-05 2024-09-15T00:13:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224681+0000", 2024-09-15T00:13:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1460000000000001e-06 2024-09-15T00:13:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126388+0000", 2024-09-15T00:13:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90170724999999996 2024-09-15T00:13:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126526+0000", 2024-09-15T00:13:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013730599999999999 2024-09-15T00:13:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126684+0000", 2024-09-15T00:13:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000157935 2024-09-15T00:13:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.127932+0000", 2024-09-15T00:13:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012479240000000001 2024-09-15T00:13:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128269+0000", 2024-09-15T00:13:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033761099999999998 2024-09-15T00:13:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128324+0000", 2024-09-15T00:13:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5244999999999997e-05 2024-09-15T00:13:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128365+0000", 2024-09-15T00:13:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0503999999999997e-05 2024-09-15T00:13:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:134707 3.15 3:af1fbdb5:::rbd_data.113933a4f79a.000000000000001c:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T00:13:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.104977508000001, 2024-09-15T00:13:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057328626000000001, 2024-09-15T00:13:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 134707 2024-09-15T00:13:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692569+0000", 2024-09-15T00:13:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.339e-06 2024-09-15T00:13:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692574+0000", 2024-09-15T00:13:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-15T00:13:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692584+0000", 2024-09-15T00:13:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0363000000000001e-05 2024-09-15T00:13:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692590+0000", 2024-09-15T00:13:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.818e-06 2024-09-15T00:13:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692633+0000", 2024-09-15T00:13:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.265e-05 2024-09-15T00:13:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692718+0000", 2024-09-15T00:13:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5279999999999997e-05 2024-09-15T00:13:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.695940+0000", 2024-09-15T00:13:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003222052 2024-09-15T00:13:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749714+0000", 2024-09-15T00:13:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.053774126999999998 2024-09-15T00:13:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749788+0000", 2024-09-15T00:13:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3693000000000001e-05 2024-09-15T00:13:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749847+0000", 2024-09-15T00:13:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175999999999997e-05 2024-09-15T00:13:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749897+0000", 2024-09-15T00:13:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9311999999999998e-05 2024-09-15T00:13:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:134707 3.15 e86/55 ondisk, result = 0)", 2024-09-15T00:13:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.096416159, 2024-09-15T00:13:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048596066, 2024-09-15T00:13:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701131+0000", 2024-09-15T00:13:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3549999999999999e-06 2024-09-15T00:13:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701139+0000", 2024-09-15T00:13:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1650000000000006e-06 2024-09-15T00:13:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701156+0000", 2024-09-15T00:13:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6730000000000001e-05 2024-09-15T00:13:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701167+0000", 2024-09-15T00:13:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1504e-05 2024-09-15T00:13:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749629+0000", 2024-09-15T00:13:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048461952000000003 2024-09-15T00:13:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749640+0000", 2024-09-15T00:13:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0874e-05 2024-09-15T00:13:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749725+0000", 2024-09-15T00:13:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5486000000000001e-05 2024-09-15T00:13:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:02.906 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:02.907 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:02.908 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:02.906+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:03.008 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:03.006+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:03.028 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:03.028 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:03.028 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:03.028 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:03.038 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:03.109 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:03.107+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:03.144 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:03.144 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:03.145 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:03.145 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:03.145 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:03.153 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:03.155 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is unset 2024-09-15T00:13:03.238 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:03.236+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:03.308 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:03.308 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:03.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:03.308 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:03.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.308 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:03.308 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:03.308 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 97.733378525999996, 2024-09-15T00:13:03.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:13:03.308 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.308 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.309 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:03.309 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.309 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.309 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.309 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:13:03.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.309 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:03.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:03.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:13:03.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:13:03.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:13:03.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:13:03.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:13:03.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:13:03.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:13:03.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:13:03.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:13:03.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:13:03.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:13:03.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:13:03.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:03.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:13:03.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:13:03.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:03.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:13:03.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:13:03.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:03.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:13:03.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:13:03.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:03.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:13:03.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:13:03.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:13:03.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:13:03.314 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.314 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.314 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.314 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:03.314 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:03.314 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 97.733343833999996, 2024-09-15T00:13:03.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:13:03.314 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.315 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.315 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:03.315 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.315 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.315 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.315 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:13:03.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.315 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:03.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:03.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:03.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:13:03.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:13:03.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:13:03.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:13:03.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:13:03.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:13:03.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:13:03.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:13:03.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:13:03.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:13:03.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:13:03.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:03.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:13:03.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:13:03.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:03.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:13:03.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:13:03.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:03.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:13:03.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:13:03.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:03.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:13:03.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:13:03.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:13:03.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:13:03.320 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.320 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.320 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.320 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.320 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:03.320 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.692670145999998, 2024-09-15T00:13:03.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:13:03.321 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.321 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.321 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.321 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.321 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.321 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.321 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:13:03.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.321 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:03.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:03.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:03.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:13:03.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:13:03.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:13:03.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:13:03.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:13:03.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:13:03.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:13:03.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:13:03.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:13:03.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:13:03.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:13:03.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:13:03.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:13:03.325 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.325 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.325 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.325 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.325 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:03.325 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.692653614999998, 2024-09-15T00:13:03.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:13:03.325 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.325 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.325 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.325 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.325 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.326 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.326 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:13:03.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.326 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:03.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:03.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:13:03.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:03.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:13:03.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:13:03.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:13:03.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:13:03.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:13:03.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:13:03.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:13:03.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:13:03.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:13:03.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:13:03.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:13:03.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:13:03.329 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.329 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.329 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.329 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.329 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:03.329 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.692636836000005, 2024-09-15T00:13:03.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:13:03.330 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.330 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.330 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.330 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.330 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.330 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.330 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:13:03.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.330 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:03.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:03.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:03.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:13:03.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:13:03.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:13:03.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:13:03.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:13:03.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:13:03.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:13:03.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:13:03.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:13:03.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:13:03.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:13:03.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:13:03.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:13:03.333 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.333 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.333 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.334 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.334 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:03.334 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.692619105999995, 2024-09-15T00:13:03.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:13:03.334 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.334 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.334 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.334 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.334 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.334 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.334 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:13:03.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.334 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:03.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:03.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:03.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:13:03.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:13:03.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:13:03.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:13:03.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:13:03.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:13:03.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:13:03.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:13:03.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:13:03.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:13:03.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:13:03.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:13:03.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:13:03.338 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.338 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.338 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.338 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.338 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:03.338 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.692601941999996, 2024-09-15T00:13:03.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:13:03.338 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.338 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.338 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.338 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.339 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.339 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.339 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:13:03.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.339 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:03.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:03.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:03.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:13:03.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:13:03.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:13:03.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:13:03.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:13:03.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:13:03.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:13:03.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:13:03.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:13:03.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:13:03.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:13:03.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:13:03.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:13:03.342 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.342 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.342 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.342 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.343 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:03.343 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.692580265999993, 2024-09-15T00:13:03.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:13:03.343 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.343 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.343 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.343 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.343 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.343 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.343 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:13:03.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.343 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:03.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:03.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:03.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:13:03.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:13:03.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:13:03.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:13:03.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:13:03.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:13:03.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:13:03.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:13:03.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:13:03.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:13:03.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:13:03.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:13:03.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:13:03.347 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.347 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.347 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.347 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.347 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:03.347 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.692534142, 2024-09-15T00:13:03.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:13:03.347 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.347 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.347 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.348 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.348 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:13:03.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.348 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:03.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:03.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:03.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:13:03.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:13:03.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:13:03.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:13:03.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:13:03.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:13:03.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:13:03.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:13:03.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:13:03.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:13:03.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:13:03.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:13:03.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:13:03.351 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.351 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.351 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.351 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.352 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:03.352 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.692504385000007, 2024-09-15T00:13:03.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:13:03.352 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.352 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.352 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.352 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:13:03.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.352 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:03.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:03.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:13:03.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:13:03.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:13:03.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:13:03.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:13:03.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:03.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:13:03.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:13:03.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:13:03.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:13:03.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:13:03.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:13:03.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:13:03.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:13:03.356 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.356 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.356 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.356 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.356 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:03.356 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.692478066000007, 2024-09-15T00:13:03.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:13:03.356 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.356 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.356 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.356 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.356 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.357 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.357 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:13:03.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.357 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:03.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:03.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:03.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:03.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:13:03.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:13:03.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:13:03.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:13:03.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:13:03.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:13:03.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:13:03.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:13:03.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:13:03.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:13:03.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:13:03.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:13:03.360 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.360 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.360 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.360 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.361 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:03.361 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.692442204000002, 2024-09-15T00:13:03.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:13:03.361 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.361 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.361 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.361 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:13:03.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.361 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:03.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:03.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:03.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:13:03.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:13:03.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:13:03.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:13:03.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:13:03.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:13:03.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:13:03.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:13:03.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:13:03.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:13:03.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:13:03.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:13:03.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:13:03.365 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.365 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.365 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.365 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.365 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:03.365 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.692420486000003, 2024-09-15T00:13:03.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:13:03.365 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.365 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.365 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.365 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.366 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.366 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.366 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:13:03.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.366 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:03.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:03.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:13:03.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:13:03.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:13:03.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:13:03.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:13:03.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:13:03.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:13:03.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:13:03.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:13:03.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:13:03.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:13:03.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:13:03.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:13:03.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:13:03.369 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.369 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.369 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.369 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.369 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:03.369 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.692224018999994, 2024-09-15T00:13:03.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:13:03.370 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.370 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.370 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.370 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:13:03.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.370 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:03.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:03.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:03.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:13:03.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:13:03.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:13:03.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:13:03.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:13:03.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:13:03.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:13:03.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:13:03.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:13:03.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:13:03.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:13:03.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:13:03.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:13:03.373 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.373 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.374 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.374 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.374 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:03.374 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.692198180000005, 2024-09-15T00:13:03.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:13:03.374 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.374 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.374 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.375 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:13:03.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.375 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:03.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:03.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:03.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:13:03.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:13:03.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:13:03.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:13:03.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:13:03.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:13:03.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:13:03.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:13:03.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:13:03.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:13:03.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:13:03.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:13:03.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:13:03.378 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.378 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.378 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.378 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.378 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:03.378 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.692136533999999, 2024-09-15T00:13:03.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:13:03.379 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.379 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.379 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.379 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:13:03.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.379 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:03.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:03.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:13:03.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:13:03.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:13:03.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:13:03.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:13:03.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:13:03.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:13:03.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:13:03.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:13:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:13:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:13:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:13:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:13:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:13:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.692097206, 2024-09-15T00:13:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:13:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:13:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.384 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:03.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:03.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:13:03.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:13:03.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:13:03.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:13:03.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:13:03.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:13:03.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:13:03.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:13:03.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:13:03.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:13:03.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:13:03.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:03.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:13:03.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:13:03.387 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.387 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.387 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.387 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.387 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:03.387 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.690371806000002, 2024-09-15T00:13:03.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:13:03.387 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.388 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.388 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.388 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:13:03.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.388 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:03.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:03.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:13:03.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:13:03.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:13:03.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:13:03.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:13:03.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:13:03.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:13:03.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:13:03.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:13:03.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:13:03.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:13:03.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:13:03.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:13:03.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:13:03.392 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.392 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.392 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.392 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.392 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:03.392 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.690349644999998, 2024-09-15T00:13:03.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:13:03.392 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.392 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.393 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.393 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:13:03.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.393 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:03.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:03.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:03.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:13:03.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:13:03.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:13:03.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:13:03.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:13:03.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:13:03.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:13:03.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:13:03.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:13:03.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:13:03.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:13:03.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:13:03.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:13:03.397 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.397 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.397 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.397 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:145034 3.6 3:618673df:::rbd_object_map.1139646a7d4e.000000000000002b:head [writefull 0~48] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T00:13:03.397 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:03.397 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.288096040999999, 2024-09-15T00:13:03.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0081079740000001, 2024-09-15T00:13:03.397 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.397 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.397 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:03.397 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.398 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.398 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.398 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 145034 2024-09-15T00:13:03.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.398 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:03.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:03.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017198+0000", 2024-09-15T00:13:03.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0100000000000004e-07 2024-09-15T00:13:03.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017202+0000", 2024-09-15T00:13:03.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:13:03.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017212+0000", 2024-09-15T00:13:03.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7189999999999996e-06 2024-09-15T00:13:03.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017219+0000", 2024-09-15T00:13:03.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7839999999999998e-06 2024-09-15T00:13:03.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017268+0000", 2024-09-15T00:13:03.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:13:03.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:13:03.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017277+0000", 2024-09-15T00:13:03.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1899999999999995e-06 2024-09-15T00:13:03.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019139+0000", 2024-09-15T00:13:03.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0018626129999999 2024-09-15T00:13:03.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019236+0000", 2024-09-15T00:13:03.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6513000000000002e-05 2024-09-15T00:13:03.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:03.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019318+0000", 2024-09-15T00:13:03.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1848000000000006e-05 2024-09-15T00:13:03.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:03.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.023151+0000", 2024-09-15T00:13:03.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038332169999999999 2024-09-15T00:13:03.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:03.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025217+0000", 2024-09-15T00:13:03.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020657610000000002 2024-09-15T00:13:03.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:03.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025267+0000", 2024-09-15T00:13:03.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0198000000000003e-05 2024-09-15T00:13:03.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025305+0000", 2024-09-15T00:13:03.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8065999999999998e-05 2024-09-15T00:13:03.404 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.404 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.404 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.404 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.404 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.404 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:03.406 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:03.407 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:03.405+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:03.498 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 0] out_osds: [2, 7, 1] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-15T00:13:03.499 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:13:03.499 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-15T00:13:03.499 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-15T00:13:03.506 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:03.504+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:03.552 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:03.552 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:03.552 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:03.552 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:03.561 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:03.606 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:03.605+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:03.703 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:03.704 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:03.704 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:03.704 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:03.704 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:03.707 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:03.705+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:03.712 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:03.785 INFO:tasks.ceph.mon.a.smithi171.stderr:2024-09-15T00:13:03.783+0000 7f0425d35640 -1 mon.a@0(leader).osd e115 definitely_dead 0 2024-09-15T00:13:03.807 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:03.806+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:03.857 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:03.857 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:03.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:03.857 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:03.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.857 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.857 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:03.857 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 79.244726654000004, 2024-09-15T00:13:03.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:13:03.857 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.858 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.858 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.858 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:13:03.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.858 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:03.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:03.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:03.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:13:03.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:13:03.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:13:03.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:13:03.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:13:03.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:13:03.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:13:03.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:13:03.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:13:03.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:13:03.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:13:03.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:13:03.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:13:03.862 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.864 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.864 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.865 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.865 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:03.865 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 79.244705425000006, 2024-09-15T00:13:03.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:13:03.865 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.865 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.865 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.866 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:13:03.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.866 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:03.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:03.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:13:03.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:13:03.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:13:03.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:03.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:13:03.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:13:03.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:13:03.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:13:03.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:13:03.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:13:03.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:13:03.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:13:03.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:13:03.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:13:03.870 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.870 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.870 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.870 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.870 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:03.870 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 79.244679884000007, 2024-09-15T00:13:03.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:13:03.870 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.870 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.870 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.871 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:13:03.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.871 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:03.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:03.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:03.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:03.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:13:03.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:13:03.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:13:03.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:13:03.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:13:03.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:13:03.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:13:03.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:13:03.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:13:03.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:13:03.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:13:03.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:13:03.874 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.874 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.874 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.875 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.875 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:03.875 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 79.244657408999998, 2024-09-15T00:13:03.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:13:03.875 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.875 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.875 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.875 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.875 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.875 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.875 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:13:03.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.875 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:03.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:03.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:13:03.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:13:03.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:13:03.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:13:03.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:13:03.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:13:03.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:13:03.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:13:03.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:13:03.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:13:03.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:13:03.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:13:03.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:13:03.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:13:03.879 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.879 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.879 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.879 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.879 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:03.879 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 79.244636899, 2024-09-15T00:13:03.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:13:03.880 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.880 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.880 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.880 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.880 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.880 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.880 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:13:03.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.880 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:03.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:03.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:03.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:13:03.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:13:03.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:13:03.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:13:03.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:13:03.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:13:03.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:13:03.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:13:03.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:13:03.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:13:03.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:13:03.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:13:03.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:13:03.883 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.884 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.884 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.884 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.884 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:03.884 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 79.244616583999999, 2024-09-15T00:13:03.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:13:03.884 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.884 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.884 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.884 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.884 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.885 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.885 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:13:03.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.885 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:03.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:03.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:13:03.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:13:03.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:13:03.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:13:03.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:13:03.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:13:03.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:13:03.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:13:03.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:13:03.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:13:03.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:13:03.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:13:03.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:13:03.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:13:03.888 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.888 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.888 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.888 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.888 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:03.888 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 79.244591807999996, 2024-09-15T00:13:03.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:13:03.889 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.889 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.889 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.889 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.889 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.889 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.889 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:13:03.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.889 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:03.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:03.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:03.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:13:03.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:13:03.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:13:03.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:13:03.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:13:03.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:13:03.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:13:03.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:13:03.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:13:03.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:13:03.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:13:03.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:13:03.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:13:03.893 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.893 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.893 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.893 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:03.893 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:03.894 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 79.065250421000002, 2024-09-15T00:13:03.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:13:03.894 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.894 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.894 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:03.894 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.894 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.894 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.894 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:13:03.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.894 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:03.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:03.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:13:03.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:13:03.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:13:03.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:13:03.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:13:03.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:13:03.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:13:03.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:13:03.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:13:03.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:13:03.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:13:03.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:13:03.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:03.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:13:03.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:13:03.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:03.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:13:03.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:13:03.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:03.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:13:03.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:13:03.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:03.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:13:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:13:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:13:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:13:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:13:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 79.048296969000006, 2024-09-15T00:13:03.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:13:03.900 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.900 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.900 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.900 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:03.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:03.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:13:03.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:13:03.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:13:03.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:13:03.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:13:03.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:13:03.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:13:03.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:13:03.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:13:03.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:13:03.903 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.903 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.903 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.904 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:13:03.904 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:03.904 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.753866062, 2024-09-15T00:13:03.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:13:03.904 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.904 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.904 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:03.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.904 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:13:03.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.904 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:03.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:03.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:13:03.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:13:03.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:13:03.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:13:03.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:13:03.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:13:03.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:13:03.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:13:03.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:13:03.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:13:03.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:13:03.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:13:03.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:03.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:13:03.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:13:03.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:03.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:13:03.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:13:03.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:03.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:13:03.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:13:03.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:03.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:13:03.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:13:03.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:13:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:13:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:13:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.741410247999994, 2024-09-15T00:13:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:13:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:03.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:03.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:13:03.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:13:03.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:13:03.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:13:03.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:13:03.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:13:03.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:13:03.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:13:03.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:13:03.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:13:03.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:13:03.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:13:03.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:13:03.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:13:03.914 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.914 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.914 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.914 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:146935 3.16 3:6c79e092:::rbd_data.11394501364.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:03.914 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:03.914 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.100630729999999, 2024-09-15T00:13:03.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.058987488999999997, 2024-09-15T00:13:03.914 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.914 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.914 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:03.915 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.915 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.915 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.915 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 146935 2024-09-15T00:13:03.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.915 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:03.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:03.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753644+0000", 2024-09-15T00:13:03.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6199999999999997e-07 2024-09-15T00:13:03.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753648+0000", 2024-09-15T00:13:03.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-15T00:13:03.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753659+0000", 2024-09-15T00:13:03.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0703000000000001e-05 2024-09-15T00:13:03.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753664+0000", 2024-09-15T00:13:03.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.04e-06 2024-09-15T00:13:03.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753709+0000", 2024-09-15T00:13:03.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4728000000000003e-05 2024-09-15T00:13:03.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753756+0000", 2024-09-15T00:13:03.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7808e-05 2024-09-15T00:13:03.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:03.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.757051+0000", 2024-09-15T00:13:03.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294249 2024-09-15T00:13:03.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:03.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812553+0000", 2024-09-15T00:13:03.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055501807 2024-09-15T00:13:03.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:03.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812564+0000", 2024-09-15T00:13:03.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1467999999999999e-05 2024-09-15T00:13:03.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:03.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812598+0000", 2024-09-15T00:13:03.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3553000000000002e-05 2024-09-15T00:13:03.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812630+0000", 2024-09-15T00:13:03.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2895000000000003e-05 2024-09-15T00:13:03.920 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.920 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.920 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.920 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:146935 3.16 e98/95 ondisk, result = 0)", 2024-09-15T00:13:03.920 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:03.920 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.089828034, 2024-09-15T00:13:03.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048111830000000001, 2024-09-15T00:13:03.920 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.920 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.920 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.920 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:03.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:03.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764447+0000", 2024-09-15T00:13:03.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0500000000000002e-07 2024-09-15T00:13:03.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764450+0000", 2024-09-15T00:13:03.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4400000000000001e-06 2024-09-15T00:13:03.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764462+0000", 2024-09-15T00:13:03.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1603000000000001e-05 2024-09-15T00:13:03.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764469+0000", 2024-09-15T00:13:03.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5089999999999996e-06 2024-09-15T00:13:03.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812541+0000", 2024-09-15T00:13:03.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048071585 2024-09-15T00:13:03.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812548+0000", 2024-09-15T00:13:03.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9979999999999997e-06 2024-09-15T00:13:03.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812557+0000", 2024-09-15T00:13:03.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7899999999999994e-06 2024-09-15T00:13:03.924 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.924 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.924 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.924 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:152505 3.17 3:ea3f268c:::rbd_data.113958cb0e55.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:03.924 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:03.924 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.187722072, 2024-09-15T00:13:03.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.060817959999999997, 2024-09-15T00:13:03.924 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.924 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.925 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:03.925 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.925 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.925 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.925 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 152505 2024-09-15T00:13:03.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.925 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:03.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:03.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666553+0000", 2024-09-15T00:13:03.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-15T00:13:03.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666562+0000", 2024-09-15T00:13:03.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0909999999999999e-06 2024-09-15T00:13:03.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666572+0000", 2024-09-15T00:13:03.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0214000000000001e-05 2024-09-15T00:13:03.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666577+0000", 2024-09-15T00:13:03.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3800000000000002e-06 2024-09-15T00:13:03.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666617+0000", 2024-09-15T00:13:03.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9304000000000002e-05 2024-09-15T00:13:03.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666672+0000", 2024-09-15T00:13:03.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5206e-05 2024-09-15T00:13:03.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:03.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.670821+0000", 2024-09-15T00:13:03.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041493169999999996 2024-09-15T00:13:03.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:03.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727290+0000", 2024-09-15T00:13:03.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.056468580999999997 2024-09-15T00:13:03.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:03.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727302+0000", 2024-09-15T00:13:03.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2605e-05 2024-09-15T00:13:03.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:03.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727337+0000", 2024-09-15T00:13:03.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4783999999999998e-05 2024-09-15T00:13:03.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727370+0000", 2024-09-15T00:13:03.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2366000000000001e-05 2024-09-15T00:13:03.931 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.931 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.931 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.931 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:152505 3.17 e103/68 ondisk, result = 0)", 2024-09-15T00:13:03.931 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:03.931 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.175643636, 2024-09-15T00:13:03.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048665938999999998, 2024-09-15T00:13:03.931 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.931 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.932 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.932 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:03.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:03.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678631+0000", 2024-09-15T00:13:03.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5099999999999998e-07 2024-09-15T00:13:03.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678634+0000", 2024-09-15T00:13:03.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.478e-06 2024-09-15T00:13:03.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678647+0000", 2024-09-15T00:13:03.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2085e-05 2024-09-15T00:13:03.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678653+0000", 2024-09-15T00:13:03.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2099999999999998e-06 2024-09-15T00:13:03.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727277+0000", 2024-09-15T00:13:03.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048624405000000002 2024-09-15T00:13:03.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727285+0000", 2024-09-15T00:13:03.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2060000000000007e-06 2024-09-15T00:13:03.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727296+0000", 2024-09-15T00:13:03.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0604e-05 2024-09-15T00:13:03.935 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.935 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.935 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.935 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153959 3.c 3:326b8454:::rbd_data.113958cb0e55.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:03.935 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:03.936 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.76441569, 2024-09-15T00:13:03.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.068102041000000002, 2024-09-15T00:13:03.936 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.936 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.936 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:03.936 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.936 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.936 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.936 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153959 2024-09-15T00:13:03.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.936 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:03.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:03.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089859+0000", 2024-09-15T00:13:03.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1800000000000004e-07 2024-09-15T00:13:03.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089864+0000", 2024-09-15T00:13:03.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2079999999999999e-06 2024-09-15T00:13:03.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089880+0000", 2024-09-15T00:13:03.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5695000000000001e-05 2024-09-15T00:13:03.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089889+0000", 2024-09-15T00:13:03.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.935e-06 2024-09-15T00:13:03.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089942+0000", 2024-09-15T00:13:03.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3219e-05 2024-09-15T00:13:03.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.090066+0000", 2024-09-15T00:13:03.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012385099999999999 2024-09-15T00:13:03.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:03.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.095262+0000", 2024-09-15T00:13:03.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051963950000000004 2024-09-15T00:13:03.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:03.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157865+0000", 2024-09-15T00:13:03.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.062602865999999993 2024-09-15T00:13:03.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:03.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157891+0000", 2024-09-15T00:13:03.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5778e-05 2024-09-15T00:13:03.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:03.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157924+0000", 2024-09-15T00:13:03.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3133000000000003e-05 2024-09-15T00:13:03.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157960+0000", 2024-09-15T00:13:03.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6043e-05 2024-09-15T00:13:03.941 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.941 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.941 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.941 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:153959 3.c e103/23 ondisk, result = 0)", 2024-09-15T00:13:03.941 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:03.941 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.751760032, 2024-09-15T00:13:03.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055370211000000003, 2024-09-15T00:13:03.942 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.942 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.942 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.942 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:03.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:03.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102515+0000", 2024-09-15T00:13:03.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.206e-06 2024-09-15T00:13:03.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102520+0000", 2024-09-15T00:13:03.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1209999999999998e-06 2024-09-15T00:13:03.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102537+0000", 2024-09-15T00:13:03.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:03.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102546+0000", 2024-09-15T00:13:03.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2329999999999995e-06 2024-09-15T00:13:03.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157847+0000", 2024-09-15T00:13:03.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055300798999999998 2024-09-15T00:13:03.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157857+0000", 2024-09-15T00:13:03.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.046e-05 2024-09-15T00:13:03.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157884+0000", 2024-09-15T00:13:03.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.671e-05 2024-09-15T00:13:03.945 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.945 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.945 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.945 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163346 3.1d 3:baa43c12:::rbd_data.11395d771336.0000000000000065:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:13:03.946 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:03.946 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.083145459000001, 2024-09-15T00:13:03.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047558544000000001, 2024-09-15T00:13:03.946 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.946 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.946 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:03.946 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.946 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.946 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.946 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163346 2024-09-15T00:13:03.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.946 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:03.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:03.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771130+0000", 2024-09-15T00:13:03.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5939999999999999e-06 2024-09-15T00:13:03.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771141+0000", 2024-09-15T00:13:03.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1001e-05 2024-09-15T00:13:03.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771154+0000", 2024-09-15T00:13:03.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2921e-05 2024-09-15T00:13:03.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771160+0000", 2024-09-15T00:13:03.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3849999999999999e-06 2024-09-15T00:13:03.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771207+0000", 2024-09-15T00:13:03.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6888999999999998e-05 2024-09-15T00:13:03.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771306+0000", 2024-09-15T00:13:03.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9259000000000006e-05 2024-09-15T00:13:03.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:03.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.774692+0000", 2024-09-15T00:13:03.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033854850000000001 2024-09-15T00:13:03.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:03.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818600+0000", 2024-09-15T00:13:03.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043908006999999999 2024-09-15T00:13:03.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:03.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818616+0000", 2024-09-15T00:13:03.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6688e-05 2024-09-15T00:13:03.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:03.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818651+0000", 2024-09-15T00:13:03.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4885000000000002e-05 2024-09-15T00:13:03.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818687+0000", 2024-09-15T00:13:03.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5429999999999998e-05 2024-09-15T00:13:03.951 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.951 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.951 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.952 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163578 3.1c 3:3efd71ab:::rbd_data.11395d771336.0000000000000060:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-15T00:13:03.952 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:03.952 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.6837227099999996, 2024-09-15T00:13:03.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.085154081000000006, 2024-09-15T00:13:03.952 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.952 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.952 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:03.952 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:03.952 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:03.952 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:03.952 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163578 2024-09-15T00:13:03.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.952 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:03.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:03.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170552+0000", 2024-09-15T00:13:03.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8599999999999997e-07 2024-09-15T00:13:03.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170557+0000", 2024-09-15T00:13:03.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-15T00:13:03.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170567+0000", 2024-09-15T00:13:03.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0628999999999999e-05 2024-09-15T00:13:03.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170573+0000", 2024-09-15T00:13:03.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1649999999999997e-06 2024-09-15T00:13:03.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170619+0000", 2024-09-15T00:13:03.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6405e-05 2024-09-15T00:13:03.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170666+0000", 2024-09-15T00:13:03.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7259000000000003e-05 2024-09-15T00:13:03.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:03.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.175060+0000", 2024-09-15T00:13:03.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043940170000000001 2024-09-15T00:13:03.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:03.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255559+0000", 2024-09-15T00:13:03.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080499065999999994 2024-09-15T00:13:03.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:03.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255616+0000", 2024-09-15T00:13:03.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6422000000000003e-05 2024-09-15T00:13:03.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:03.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255655+0000", 2024-09-15T00:13:03.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9632999999999998e-05 2024-09-15T00:13:03.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255705+0000", 2024-09-15T00:13:03.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.973e-05 2024-09-15T00:13:03.958 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.958 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.958 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.958 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:163578 3.1c e107/23 ondisk, result = 0)", 2024-09-15T00:13:03.959 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:03.959 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.6685006189999996, 2024-09-15T00:13:03.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069939984999999996, 2024-09-15T00:13:03.959 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:03.959 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:03.959 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:03.959 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:03.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:03.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:03.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:03.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:03.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:03.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:03.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185774+0000", 2024-09-15T00:13:03.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.093e-06 2024-09-15T00:13:03.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:03.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185779+0000", 2024-09-15T00:13:03.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9470000000000002e-06 2024-09-15T00:13:03.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:03.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185794+0000", 2024-09-15T00:13:03.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4450000000000001e-05 2024-09-15T00:13:03.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:03.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185801+0000", 2024-09-15T00:13:03.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9189999999999999e-06 2024-09-15T00:13:03.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:03.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255604+0000", 2024-09-15T00:13:03.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069802201999999994 2024-09-15T00:13:03.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:03.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255611+0000", 2024-09-15T00:13:03.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7740000000000004e-06 2024-09-15T00:13:03.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:03.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:03.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:03.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255713+0000", 2024-09-15T00:13:03.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0001016 2024-09-15T00:13:03.962 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.962 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.962 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.963 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:03.963 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:03.963 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:03.964 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:03.965 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:03.962+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:04.064 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:04.063+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:04.108 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:04.108 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:04.109 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:04.109 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:04.117 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:04.156 INFO:teuthology.orchestra.run.smithi171.stderr:marked in osd.2. 2024-09-15T00:13:04.175 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-15T00:13:04.202 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:04.200+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:04.266 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:04.266 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:04.266 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:04.266 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:04.267 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:04.267 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:04.265+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:04.275 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:04.368 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:04.366+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:04.391 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:13:04.421 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:04.421 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:04.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:04.421 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:04.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.422 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:04.422 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:04.422 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 79.751446279000007, 2024-09-15T00:13:04.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:13:04.422 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.422 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.422 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:04.422 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.422 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.422 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.422 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:13:04.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.423 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:04.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:04.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:13:04.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:13:04.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:13:04.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:13:04.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:13:04.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:13:04.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:13:04.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:13:04.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:13:04.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:13:04.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:13:04.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:13:04.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:04.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:13:04.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:13:04.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:04.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:13:04.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:13:04.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:04.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:13:04.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:13:04.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:04.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:13:04.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:13:04.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:13:04.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:13:04.427 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.427 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.427 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.428 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125618 3.10 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:04.428 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:04.428 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.580307197000003, 2024-09-15T00:13:04.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030860942999999998, 2024-09-15T00:13:04.428 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.428 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.428 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:04.428 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.428 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.428 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.428 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125618 2024-09-15T00:13:04.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.429 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:04.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:04.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838499+0000", 2024-09-15T00:13:04.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6189999999999998e-06 2024-09-15T00:13:04.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839599+0000", 2024-09-15T00:13:04.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001099764 2024-09-15T00:13:04.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839620+0000", 2024-09-15T00:13:04.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0764e-05 2024-09-15T00:13:04.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839631+0000", 2024-09-15T00:13:04.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1335000000000001e-05 2024-09-15T00:13:04.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839725+0000", 2024-09-15T00:13:04.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4217999999999998e-05 2024-09-15T00:13:04.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839910+0000", 2024-09-15T00:13:04.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000184298 2024-09-15T00:13:04.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:04.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843171+0000", 2024-09-15T00:13:04.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032613540000000002 2024-09-15T00:13:04.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:04.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869187+0000", 2024-09-15T00:13:04.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026015467 2024-09-15T00:13:04.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:04.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869240+0000", 2024-09-15T00:13:04.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3733000000000002e-05 2024-09-15T00:13:04.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:04.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869309+0000", 2024-09-15T00:13:04.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8381999999999997e-05 2024-09-15T00:13:04.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869358+0000", 2024-09-15T00:13:04.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-15T00:13:04.433 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.433 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.434 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.434 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125619 3.a 3:507b29bf:::rbd_data.11396481e1ec.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:04.434 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:04.434 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.579145068000003, 2024-09-15T00:13:04.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045768870000000003, 2024-09-15T00:13:04.434 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.434 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.434 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:04.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.434 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125619 2024-09-15T00:13:04.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.435 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:04.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:04.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839660+0000", 2024-09-15T00:13:04.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.22e-06 2024-09-15T00:13:04.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840511+0000", 2024-09-15T00:13:04.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000851284 2024-09-15T00:13:04.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840522+0000", 2024-09-15T00:13:04.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0569e-05 2024-09-15T00:13:04.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840527+0000", 2024-09-15T00:13:04.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.412e-06 2024-09-15T00:13:04.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840573+0000", 2024-09-15T00:13:04.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5247000000000002e-05 2024-09-15T00:13:04.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840682+0000", 2024-09-15T00:13:04.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109368 2024-09-15T00:13:04.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:04.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843976+0000", 2024-09-15T00:13:04.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294445 2024-09-15T00:13:04.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:04.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870833+0000", 2024-09-15T00:13:04.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026856944000000001 2024-09-15T00:13:04.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:04.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885344+0000", 2024-09-15T00:13:04.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014510744000000001 2024-09-15T00:13:04.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:04.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885382+0000", 2024-09-15T00:13:04.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7821000000000003e-05 2024-09-15T00:13:04.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885428+0000", 2024-09-15T00:13:04.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5816000000000003e-05 2024-09-15T00:13:04.439 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.439 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.439 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.439 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125621 3.a 3:53594db9:::rbd_data.11396481e1ec.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:04.440 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:04.440 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.565621688999997, 2024-09-15T00:13:04.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032481797, 2024-09-15T00:13:04.440 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.440 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.440 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:04.440 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.440 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.440 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.440 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125621 2024-09-15T00:13:04.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.440 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:04.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:04.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853185+0000", 2024-09-15T00:13:04.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.43e-06 2024-09-15T00:13:04.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854125+0000", 2024-09-15T00:13:04.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000940539 2024-09-15T00:13:04.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854141+0000", 2024-09-15T00:13:04.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5568000000000001e-05 2024-09-15T00:13:04.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854148+0000", 2024-09-15T00:13:04.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6529999999999994e-06 2024-09-15T00:13:04.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870872+0000", 2024-09-15T00:13:04.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016723135 2024-09-15T00:13:04.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870975+0000", 2024-09-15T00:13:04.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010314099999999999 2024-09-15T00:13:04.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:04.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.873082+0000", 2024-09-15T00:13:04.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021069859999999999 2024-09-15T00:13:04.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885439+0000", 2024-09-15T00:13:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012357343999999999 2024-09-15T00:13:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885569+0000", 2024-09-15T00:13:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012995199999999999 2024-09-15T00:13:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885624+0000", 2024-09-15T00:13:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5281999999999997e-05 2024-09-15T00:13:04.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885664+0000", 2024-09-15T00:13:04.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9767000000000001e-05 2024-09-15T00:13:04.445 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.445 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.445 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.445 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:147075 3.0 3:06ab39f2:::rbd_data.11394501364.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:04.445 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:04.445 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.469723307999999, 2024-09-15T00:13:04.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046402906000000001, 2024-09-15T00:13:04.446 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.446 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.446 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:04.446 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.446 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.446 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.446 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 147075 2024-09-15T00:13:04.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.446 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:04.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:04.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:04.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4399999999999999e-07 2024-09-15T00:13:04.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949086+0000", 2024-09-15T00:13:04.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.138e-06 2024-09-15T00:13:04.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949096+0000", 2024-09-15T00:13:04.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0438999999999999e-05 2024-09-15T00:13:04.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949101+0000", 2024-09-15T00:13:04.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8999999999999997e-06 2024-09-15T00:13:04.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949145+0000", 2024-09-15T00:13:04.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4141000000000003e-05 2024-09-15T00:13:04.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949195+0000", 2024-09-15T00:13:04.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9657999999999998e-05 2024-09-15T00:13:04.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:04.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.950923+0000", 2024-09-15T00:13:04.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001728753 2024-09-15T00:13:04.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:04.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995390+0000", 2024-09-15T00:13:04.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044466335000000003 2024-09-15T00:13:04.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:04.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995401+0000", 2024-09-15T00:13:04.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1671000000000001e-05 2024-09-15T00:13:04.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:04.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995432+0000", 2024-09-15T00:13:04.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0704999999999998e-05 2024-09-15T00:13:04.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995484+0000", 2024-09-15T00:13:04.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1422000000000003e-05 2024-09-15T00:13:04.451 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.451 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.451 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.451 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:147075 3.0 e98/94 ondisk, result = 0)", 2024-09-15T00:13:04.451 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:04.451 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.461923587000001, 2024-09-15T00:13:04.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038514459000000001, 2024-09-15T00:13:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956881+0000", 2024-09-15T00:13:04.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0599999999999999e-07 2024-09-15T00:13:04.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956885+0000", 2024-09-15T00:13:04.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3500000000000001e-06 2024-09-15T00:13:04.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956897+0000", 2024-09-15T00:13:04.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2303999999999999e-05 2024-09-15T00:13:04.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956902+0000", 2024-09-15T00:13:04.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5169999999999999e-06 2024-09-15T00:13:04.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995377+0000", 2024-09-15T00:13:04.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038474544999999999 2024-09-15T00:13:04.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995385+0000", 2024-09-15T00:13:04.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6240000000000002e-06 2024-09-15T00:13:04.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995395+0000", 2024-09-15T00:13:04.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0213e-05 2024-09-15T00:13:04.455 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.455 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.455 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.455 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153172 3.14 3:2a7843e3:::rbd_data.113958cb0e55.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 983040~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:04.455 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:04.455 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.641034523, 2024-09-15T00:13:04.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.033232668, 2024-09-15T00:13:04.456 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.456 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.456 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:04.456 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.456 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.456 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.456 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153172 2024-09-15T00:13:04.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.456 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:04.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:04.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777770+0000", 2024-09-15T00:13:04.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.017e-06 2024-09-15T00:13:04.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788009+0000", 2024-09-15T00:13:04.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010238146 2024-09-15T00:13:04.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788024+0000", 2024-09-15T00:13:04.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5582000000000001e-05 2024-09-15T00:13:04.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788048+0000", 2024-09-15T00:13:04.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4173e-05 2024-09-15T00:13:04.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788097+0000", 2024-09-15T00:13:04.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8498999999999998e-05 2024-09-15T00:13:04.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788201+0000", 2024-09-15T00:13:04.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010398799999999999 2024-09-15T00:13:04.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:04.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788393+0000", 2024-09-15T00:13:04.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000192337 2024-09-15T00:13:04.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:04.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810637+0000", 2024-09-15T00:13:04.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.022244202000000001 2024-09-15T00:13:04.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:04.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810903+0000", 2024-09-15T00:13:04.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00026574600000000003 2024-09-15T00:13:04.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:04.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810953+0000", 2024-09-15T00:13:04.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9509000000000003e-05 2024-09-15T00:13:04.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.811002+0000", 2024-09-15T00:13:04.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9468999999999997e-05 2024-09-15T00:13:04.461 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.461 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.461 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.461 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170014 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:04.461 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:04.461 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.5717394169999999, 2024-09-15T00:13:04.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030291116999999999, 2024-09-15T00:13:04.462 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.462 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.462 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:04.462 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.462 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.462 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.462 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170014 2024-09-15T00:13:04.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.462 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:04.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:04.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:04.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:04.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847073+0000", 2024-09-15T00:13:04.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9589999999999995e-06 2024-09-15T00:13:04.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847080+0000", 2024-09-15T00:13:04.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2930000000000003e-06 2024-09-15T00:13:04.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847086+0000", 2024-09-15T00:13:04.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.631e-06 2024-09-15T00:13:04.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877213+0000", 2024-09-15T00:13:04.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030127017999999998 2024-09-15T00:13:04.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877281+0000", 2024-09-15T00:13:04.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7916999999999994e-05 2024-09-15T00:13:04.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877356+0000", 2024-09-15T00:13:04.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4796999999999996e-05 2024-09-15T00:13:04.465 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.465 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.466 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.466 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170017 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1310720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:04.466 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:04.466 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.5717107659999998, 2024-09-15T00:13:04.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030437623, 2024-09-15T00:13:04.466 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.466 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.466 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:04.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.466 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170017 2024-09-15T00:13:04.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.467 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:04.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:04.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847094+0000", 2024-09-15T00:13:04.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:13:04.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847096+0000", 2024-09-15T00:13:04.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.136e-06 2024-09-15T00:13:04.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847104+0000", 2024-09-15T00:13:04.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7190000000000001e-06 2024-09-15T00:13:04.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847109+0000", 2024-09-15T00:13:04.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3279999999999996e-06 2024-09-15T00:13:04.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877384+0000", 2024-09-15T00:13:04.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030275348000000001 2024-09-15T00:13:04.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877451+0000", 2024-09-15T00:13:04.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6509999999999998e-05 2024-09-15T00:13:04.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877531+0000", 2024-09-15T00:13:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9993000000000005e-05 2024-09-15T00:13:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170018 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1351680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.5716884210000002, 2024-09-15T00:13:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030591884999999999, 2024-09-15T00:13:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.471 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:04.471 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.471 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.471 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.471 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170018 2024-09-15T00:13:04.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.471 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:04.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:04.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:04.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:13:04.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847118+0000", 2024-09-15T00:13:04.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.03e-06 2024-09-15T00:13:04.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847124+0000", 2024-09-15T00:13:04.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:04.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847129+0000", 2024-09-15T00:13:04.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8600000000000001e-06 2024-09-15T00:13:04.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877560+0000", 2024-09-15T00:13:04.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030431145999999999 2024-09-15T00:13:04.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877629+0000", 2024-09-15T00:13:04.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8793000000000004e-05 2024-09-15T00:13:04.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877707+0000", 2024-09-15T00:13:04.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8139999999999994e-05 2024-09-15T00:13:04.478 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.478 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.478 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.478 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170019 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1392640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:04.478 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:04.478 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.5716676730000003, 2024-09-15T00:13:04.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030769023, 2024-09-15T00:13:04.479 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.479 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.479 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:04.479 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.479 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.479 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.479 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170019 2024-09-15T00:13:04.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.479 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:04.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:04.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847137+0000", 2024-09-15T00:13:04.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:13:04.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847139+0000", 2024-09-15T00:13:04.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.142e-06 2024-09-15T00:13:04.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847146+0000", 2024-09-15T00:13:04.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8120000000000002e-06 2024-09-15T00:13:04.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847157+0000", 2024-09-15T00:13:04.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1093e-05 2024-09-15T00:13:04.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877735+0000", 2024-09-15T00:13:04.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578195999999998 2024-09-15T00:13:04.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877810+0000", 2024-09-15T00:13:04.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5563999999999997e-05 2024-09-15T00:13:04.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877905+0000", 2024-09-15T00:13:04.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4809000000000007e-05 2024-09-15T00:13:04.482 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.483 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.483 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.483 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170020 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:04.483 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:04.483 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.5716404329999998, 2024-09-15T00:13:04.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030917150000000001, 2024-09-15T00:13:04.483 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.483 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.483 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:04.483 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.483 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.483 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.484 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170020 2024-09-15T00:13:04.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.484 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:04.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:04.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:04.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:13:04.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847166+0000", 2024-09-15T00:13:04.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2790000000000001e-06 2024-09-15T00:13:04.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847173+0000", 2024-09-15T00:13:04.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3969999999999999e-06 2024-09-15T00:13:04.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847177+0000", 2024-09-15T00:13:04.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8620000000000002e-06 2024-09-15T00:13:04.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877934+0000", 2024-09-15T00:13:04.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030756143999999999 2024-09-15T00:13:04.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878002+0000", 2024-09-15T00:13:04.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7951e-05 2024-09-15T00:13:04.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878081+0000", 2024-09-15T00:13:04.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9087999999999999e-05 2024-09-15T00:13:04.487 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.487 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.487 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.487 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170094 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:04.487 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:04.487 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.5713831840000001, 2024-09-15T00:13:04.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030836142, 2024-09-15T00:13:04.488 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.488 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.488 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:04.488 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.488 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.488 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.488 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170094 2024-09-15T00:13:04.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.488 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:04.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:04.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:04.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3499999999999996e-07 2024-09-15T00:13:04.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847424+0000", 2024-09-15T00:13:04.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2850000000000001e-06 2024-09-15T00:13:04.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847430+0000", 2024-09-15T00:13:04.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3260000000000001e-06 2024-09-15T00:13:04.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847435+0000", 2024-09-15T00:13:04.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.356e-06 2024-09-15T00:13:04.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878110+0000", 2024-09-15T00:13:04.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030675021 2024-09-15T00:13:04.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878177+0000", 2024-09-15T00:13:04.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6506000000000003e-05 2024-09-15T00:13:04.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878257+0000", 2024-09-15T00:13:04.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0012999999999994e-05 2024-09-15T00:13:04.491 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.491 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.491 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.492 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170096 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 458752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:04.492 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:04.492 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.5713412279999996, 2024-09-15T00:13:04.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030977030999999999, 2024-09-15T00:13:04.492 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.492 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.492 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:04.492 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.492 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.492 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.492 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170096 2024-09-15T00:13:04.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.493 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:04.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:04.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:04.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:04.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847465+0000", 2024-09-15T00:13:04.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0559999999999999e-06 2024-09-15T00:13:04.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847472+0000", 2024-09-15T00:13:04.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3130000000000004e-06 2024-09-15T00:13:04.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847477+0000", 2024-09-15T00:13:04.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9930000000000003e-06 2024-09-15T00:13:04.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878288+0000", 2024-09-15T00:13:04.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030811609 2024-09-15T00:13:04.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878358+0000", 2024-09-15T00:13:04.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0215000000000005e-05 2024-09-15T00:13:04.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878440+0000", 2024-09-15T00:13:04.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1366999999999995e-05 2024-09-15T00:13:04.496 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.496 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.496 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.496 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170097 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:04.496 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:04.496 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.5713211430000005, 2024-09-15T00:13:04.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031134757999999998, 2024-09-15T00:13:04.497 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.497 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.497 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:04.497 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.497 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.497 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.497 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170097 2024-09-15T00:13:04.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.497 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:04.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:04.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:04.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:13:04.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847485+0000", 2024-09-15T00:13:04.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9190000000000002e-06 2024-09-15T00:13:04.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847492+0000", 2024-09-15T00:13:04.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7109999999999998e-06 2024-09-15T00:13:04.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847497+0000", 2024-09-15T00:13:04.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0259999999999998e-06 2024-09-15T00:13:04.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878469+0000", 2024-09-15T00:13:04.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030972115000000001 2024-09-15T00:13:04.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878538+0000", 2024-09-15T00:13:04.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8669000000000003e-05 2024-09-15T00:13:04.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878618+0000", 2024-09-15T00:13:04.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9874000000000001e-05 2024-09-15T00:13:04.501 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.501 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.501 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.501 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170099 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 532480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:04.502 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:04.502 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.5712695810000001, 2024-09-15T00:13:04.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031262183999999998, 2024-09-15T00:13:04.502 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.502 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.502 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:04.502 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.502 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.503 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170099 2024-09-15T00:13:04.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.503 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:04.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:04.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847535+0000", 2024-09-15T00:13:04.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:13:04.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847537+0000", 2024-09-15T00:13:04.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:04.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847544+0000", 2024-09-15T00:13:04.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9630000000000001e-06 2024-09-15T00:13:04.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847549+0000", 2024-09-15T00:13:04.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8910000000000003e-06 2024-09-15T00:13:04.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878646+0000", 2024-09-15T00:13:04.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031097400000000001 2024-09-15T00:13:04.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878717+0000", 2024-09-15T00:13:04.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0993000000000003e-05 2024-09-15T00:13:04.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878797+0000", 2024-09-15T00:13:04.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9425000000000006e-05 2024-09-15T00:13:04.506 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.506 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.506 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.507 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170101 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:04.507 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:04.507 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.5712282020000004, 2024-09-15T00:13:04.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031674568, 2024-09-15T00:13:04.507 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.507 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.507 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:04.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.507 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170101 2024-09-15T00:13:04.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.508 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:04.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:04.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:04.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:13:04.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847578+0000", 2024-09-15T00:13:04.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0389999999999999e-06 2024-09-15T00:13:04.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847585+0000", 2024-09-15T00:13:04.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.336e-06 2024-09-15T00:13:04.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847596+0000", 2024-09-15T00:13:04.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015999999999999e-05 2024-09-15T00:13:04.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878825+0000", 2024-09-15T00:13:04.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031229848000000001 2024-09-15T00:13:04.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878899+0000", 2024-09-15T00:13:04.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:04.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879250+0000", 2024-09-15T00:13:04.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035110000000000002 2024-09-15T00:13:04.511 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.511 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.511 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.511 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170102 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 614400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:04.511 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:04.511 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.5712009049999995, 2024-09-15T00:13:04.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031577371999999999, 2024-09-15T00:13:04.511 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.511 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.512 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:04.512 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.512 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.512 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.512 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170102 2024-09-15T00:13:04.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.512 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:04.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:04.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:04.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:13:04.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847610+0000", 2024-09-15T00:13:04.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6329999999999999e-06 2024-09-15T00:13:04.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847618+0000", 2024-09-15T00:13:04.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4599999999999997e-06 2024-09-15T00:13:04.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847623+0000", 2024-09-15T00:13:04.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2909999999999998e-06 2024-09-15T00:13:04.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879016+0000", 2024-09-15T00:13:04.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031393227000000003 2024-09-15T00:13:04.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879091+0000", 2024-09-15T00:13:04.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5384999999999997e-05 2024-09-15T00:13:04.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879180+0000", 2024-09-15T00:13:04.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8941000000000001e-05 2024-09-15T00:13:04.515 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.515 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.515 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170132 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.5711535249999997, 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031722579000000001, 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170132 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:04.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:04.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847651+0000", 2024-09-15T00:13:04.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1900000000000003e-07 2024-09-15T00:13:04.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847653+0000", 2024-09-15T00:13:04.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0329999999999998e-06 2024-09-15T00:13:04.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847659+0000", 2024-09-15T00:13:04.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4169999999999997e-06 2024-09-15T00:13:04.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847670+0000", 2024-09-15T00:13:04.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1031e-05 2024-09-15T00:13:04.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879205+0000", 2024-09-15T00:13:04.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031534982000000003 2024-09-15T00:13:04.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879286+0000", 2024-09-15T00:13:04.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0675999999999998e-05 2024-09-15T00:13:04.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879373+0000", 2024-09-15T00:13:04.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6921000000000003e-05 2024-09-15T00:13:04.520 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.520 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.520 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.520 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170133 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:04.520 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:04.520 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.5711263720000002, 2024-09-15T00:13:04.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031894817999999998, 2024-09-15T00:13:04.520 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170133 2024-09-15T00:13:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853540+0000", 2024-09-15T00:13:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058619409999999999 2024-09-15T00:13:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853545+0000", 2024-09-15T00:13:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3430000000000003e-06 2024-09-15T00:13:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:04.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853554+0000", 2024-09-15T00:13:04.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4069999999999999e-06 2024-09-15T00:13:04.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:04.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853561+0000", 2024-09-15T00:13:04.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5529999999999998e-06 2024-09-15T00:13:04.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:04.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879399+0000", 2024-09-15T00:13:04.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025837941 2024-09-15T00:13:04.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:04.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879477+0000", 2024-09-15T00:13:04.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8367000000000004e-05 2024-09-15T00:13:04.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:04.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:04.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:04.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879572+0000", 2024-09-15T00:13:04.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5266000000000006e-05 2024-09-15T00:13:04.524 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.524 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.524 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.524 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:04.524 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:04.524 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:04.525 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:04.526 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:04.527 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:04.527 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:04.525+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:04.529 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:04.529 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:04.545 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:04.545 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:04.561 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:04.561 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:04.577 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:04.578 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:04.594 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:04.594 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:04.610 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:04.611 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:04.627 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:04.627 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:04.627 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:04.626+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:04.671 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:04.671 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:04.680 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:04.728 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:04.727+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:04.782 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:04.782 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:04.791 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:04.829 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:04.828+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:04.895 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:04.895 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:04.903 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:04.930 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:04.929+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:05.009 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:05.009 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:05.021 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:05.067 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:05.065+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:05.124 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:05.125 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:05.132 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:05.130+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:05.133 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:05.174 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:13:05.233 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:05.231+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:05.282 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:05.282 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:05.292 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:05.334 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:05.332+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:05.435 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:05.434+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:05.437 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:05.437 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:05.447 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:05.536 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:05.535+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:05.550 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:05.550 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:05.559 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:05.637 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:05.636+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:05.664 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:05.664 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:05.673 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:05.756 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:05.755+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:05.817 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:05.817 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:05.826 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:05.868 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:05.866+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:05.939 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:05.938+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:05.971 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:05.972 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:05.980 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:06.014 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is set 2024-09-15T00:13:06.073 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:06.071+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:06.120 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:06.120 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:06.129 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:06.171 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:06.169+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:06.241 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:06.240+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:06.274 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:06.275 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:06.283 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:06.343 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:06.340+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:06.444 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:06.444 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:06.444 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:06.442+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:06.454 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:06.545 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:06.543+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:06.606 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:06.606 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:06.616 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:06.639 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:13:06.660 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:06.657+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:06.727 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:06.728 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:06.736 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:06.746 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:06.745+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:06.767 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:06.767 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:06.787 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:06.787 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:06.803 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:06.803 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:06.824 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:06.824 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:06.841 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:06.841 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:06.847 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:06.845+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:06.859 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:06.875 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:06.875 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:06.882 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:06.883 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:06.883 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:06.883 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:06.883 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:06.892 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:06.893 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:06.893 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:06.948 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:06.947+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:07.003 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:07.003 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:07.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:07.003 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:07.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.004 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:07.004 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:07.004 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 101.428745384, 2024-09-15T00:13:07.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:13:07.004 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.004 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.004 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:07.004 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.004 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.004 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.004 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:13:07.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.005 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:07.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:07.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:13:07.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:13:07.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:13:07.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:13:07.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:13:07.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:13:07.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:13:07.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:13:07.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:13:07.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:13:07.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:13:07.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:13:07.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:07.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:13:07.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:13:07.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:07.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:13:07.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:13:07.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:07.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:13:07.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:13:07.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:07.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:13:07.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:13:07.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:13:07.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:13:07.009 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.009 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.009 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.010 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:07.010 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:07.010 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 101.428710692, 2024-09-15T00:13:07.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:13:07.010 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.010 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.010 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:07.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.010 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:13:07.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.011 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:07.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:07.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:07.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:13:07.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:13:07.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:13:07.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:13:07.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:13:07.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:13:07.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:13:07.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:13:07.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:13:07.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:13:07.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:13:07.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:07.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:13:07.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:13:07.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:07.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:13:07.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:13:07.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:07.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:13:07.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:13:07.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:07.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:13:07.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:13:07.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:13:07.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:13:07.015 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.015 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.015 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.016 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.016 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:07.016 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.388037003999997, 2024-09-15T00:13:07.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:13:07.016 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.016 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.016 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.016 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.016 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.016 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.017 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:13:07.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.017 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:07.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:07.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:07.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:13:07.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:13:07.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:13:07.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:13:07.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:13:07.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:13:07.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:13:07.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:13:07.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:13:07.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:13:07.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:13:07.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:13:07.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:13:07.020 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.020 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.020 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.021 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.021 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:07.021 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.388020472999997, 2024-09-15T00:13:07.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:13:07.021 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.021 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.021 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.021 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.021 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.021 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.021 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:13:07.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.021 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:07.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:07.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:13:07.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:07.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:13:07.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:13:07.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:13:07.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:13:07.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:13:07.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:13:07.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:13:07.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:13:07.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:13:07.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:13:07.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:13:07.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:13:07.025 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.025 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.025 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.025 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.025 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:07.025 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.388003694000005, 2024-09-15T00:13:07.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:13:07.025 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.026 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.026 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.026 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.026 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.026 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.026 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:13:07.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.026 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:07.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:07.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:07.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:13:07.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:13:07.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:13:07.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:13:07.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:13:07.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:13:07.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:13:07.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:13:07.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:13:07.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:13:07.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:13:07.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:13:07.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:13:07.029 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.029 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.029 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.030 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.030 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:07.030 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.387985963999995, 2024-09-15T00:13:07.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:13:07.030 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.030 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.030 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.030 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.030 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.030 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.030 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:13:07.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.031 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:07.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:07.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:07.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:13:07.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:13:07.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:13:07.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:13:07.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:13:07.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:13:07.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:13:07.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:13:07.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:13:07.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:13:07.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:13:07.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:13:07.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:13:07.034 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.034 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.034 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.034 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.034 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:07.034 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.387968799999996, 2024-09-15T00:13:07.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:13:07.035 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.035 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.035 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.035 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.035 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.035 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.035 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:13:07.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.035 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:07.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:07.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:07.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:13:07.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:13:07.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:13:07.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:13:07.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:13:07.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:13:07.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:13:07.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:13:07.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:13:07.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:13:07.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:13:07.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:13:07.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:13:07.038 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.039 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.039 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.039 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.039 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:07.039 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.387947123999993, 2024-09-15T00:13:07.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:13:07.039 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.039 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.039 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.039 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.039 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.039 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.039 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:13:07.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.040 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:07.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:07.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:07.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:13:07.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:13:07.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:13:07.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:13:07.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:13:07.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:13:07.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:13:07.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:13:07.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:13:07.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:13:07.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:13:07.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:13:07.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:13:07.043 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.043 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.043 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.043 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.044 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:07.044 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.387900999999999, 2024-09-15T00:13:07.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:13:07.044 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.044 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.044 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.044 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.044 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.044 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.045 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:13:07.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.045 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:07.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:07.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:07.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:13:07.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:13:07.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:13:07.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:13:07.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:13:07.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:13:07.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:13:07.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:13:07.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:13:07.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:13:07.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:13:07.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:13:07.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:13:07.048 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.048 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.048 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.048 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.048 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:07.049 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.387871243000006, 2024-09-15T00:13:07.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:13:07.049 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.049 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.049 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.049 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.049 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.049 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.049 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:13:07.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.049 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:07.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:07.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:13:07.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:13:07.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:13:07.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:13:07.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:13:07.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:07.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:13:07.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:13:07.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:13:07.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:13:07.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:13:07.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:13:07.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:13:07.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:13:07.053 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.053 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.053 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.053 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.053 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:07.053 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.387844924000007, 2024-09-15T00:13:07.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:13:07.053 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.053 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.053 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.053 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.053 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.054 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.054 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:13:07.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.054 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:07.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:07.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:07.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:07.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:13:07.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:13:07.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:13:07.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:13:07.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:13:07.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:13:07.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:13:07.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:13:07.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:13:07.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:13:07.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:13:07.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:13:07.057 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.057 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.058 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.058 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.058 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:07.058 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.387809062000002, 2024-09-15T00:13:07.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:13:07.058 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.058 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.058 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.058 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.058 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.058 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.058 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:13:07.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.059 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:07.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:07.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:07.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:13:07.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:13:07.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:13:07.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:13:07.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:13:07.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:13:07.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:13:07.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:13:07.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:13:07.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:13:07.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:13:07.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:13:07.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:13:07.062 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.062 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.062 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.062 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.062 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:07.062 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.387787344000003, 2024-09-15T00:13:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:13:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:13:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:07.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:13:07.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:13:07.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:13:07.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:13:07.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:13:07.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:13:07.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:13:07.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:13:07.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:13:07.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:13:07.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:13:07.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:13:07.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:13:07.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:13:07.066 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.067 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.067 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.067 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.067 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:07.067 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.387590876999994, 2024-09-15T00:13:07.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:13:07.067 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.067 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.067 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.067 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.067 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.067 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.068 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:13:07.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.068 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:07.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:07.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:07.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:13:07.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:13:07.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:13:07.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:13:07.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:13:07.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:13:07.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:13:07.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:13:07.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:13:07.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:13:07.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:13:07.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:13:07.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:13:07.071 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.071 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.071 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.072 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.072 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:07.072 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.387565038000005, 2024-09-15T00:13:07.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:13:07.072 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.072 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.072 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.072 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.072 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.072 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.072 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:13:07.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.072 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:07.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:07.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:07.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:13:07.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:13:07.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:13:07.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:13:07.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:13:07.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:13:07.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:13:07.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:13:07.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:13:07.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:13:07.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:13:07.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:13:07.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:13:07.076 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.076 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.076 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.076 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.076 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:07.076 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.387503391999999, 2024-09-15T00:13:07.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:13:07.077 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.077 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.077 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.077 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.077 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.077 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.077 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:13:07.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.077 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:07.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:07.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:13:07.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:13:07.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:13:07.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:13:07.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:13:07.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:13:07.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:13:07.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:13:07.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:13:07.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:13:07.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:13:07.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:13:07.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:13:07.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:13:07.080 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.080 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.081 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.081 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.081 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:07.081 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.387464064, 2024-09-15T00:13:07.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:13:07.081 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.081 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.081 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.081 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.081 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.081 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.081 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:13:07.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.082 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:07.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:07.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:13:07.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:13:07.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:13:07.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:13:07.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:13:07.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:13:07.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:13:07.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:13:07.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:13:07.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:13:07.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:13:07.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:07.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:13:07.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:13:07.085 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.085 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.085 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.085 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.085 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:07.085 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.385738664000002, 2024-09-15T00:13:07.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:13:07.085 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.085 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.086 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.086 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.086 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.086 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.086 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:13:07.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.086 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:07.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:07.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:13:07.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:13:07.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:13:07.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:13:07.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:13:07.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:13:07.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:13:07.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:13:07.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:13:07.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:13:07.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:13:07.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:13:07.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:13:07.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:13:07.089 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.089 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.089 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.090 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.090 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:07.090 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.385716502999998, 2024-09-15T00:13:07.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:13:07.090 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.090 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.090 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.090 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.090 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.090 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.090 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:13:07.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.091 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:07.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:07.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:07.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:13:07.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:13:07.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:13:07.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:13:07.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:13:07.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:13:07.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:13:07.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:13:07.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:13:07.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:13:07.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:13:07.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:13:07.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:13:07.094 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.094 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.094 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.094 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:145034 3.6 3:618673df:::rbd_object_map.1139646a7d4e.000000000000002b:head [writefull 0~48] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T00:13:07.094 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:07.094 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 26.983462898999999, 2024-09-15T00:13:07.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0081079740000001, 2024-09-15T00:13:07.094 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.094 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.095 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:07.095 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.095 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.095 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.095 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 145034 2024-09-15T00:13:07.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.095 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:07.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:07.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017198+0000", 2024-09-15T00:13:07.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0100000000000004e-07 2024-09-15T00:13:07.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017202+0000", 2024-09-15T00:13:07.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:13:07.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017212+0000", 2024-09-15T00:13:07.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7189999999999996e-06 2024-09-15T00:13:07.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017219+0000", 2024-09-15T00:13:07.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7839999999999998e-06 2024-09-15T00:13:07.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017268+0000", 2024-09-15T00:13:07.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:13:07.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:13:07.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017277+0000", 2024-09-15T00:13:07.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1899999999999995e-06 2024-09-15T00:13:07.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019139+0000", 2024-09-15T00:13:07.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0018626129999999 2024-09-15T00:13:07.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019236+0000", 2024-09-15T00:13:07.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6513000000000002e-05 2024-09-15T00:13:07.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:07.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019318+0000", 2024-09-15T00:13:07.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1848000000000006e-05 2024-09-15T00:13:07.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:07.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.023151+0000", 2024-09-15T00:13:07.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038332169999999999 2024-09-15T00:13:07.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:07.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025217+0000", 2024-09-15T00:13:07.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020657610000000002 2024-09-15T00:13:07.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:07.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025267+0000", 2024-09-15T00:13:07.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0198000000000003e-05 2024-09-15T00:13:07.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025305+0000", 2024-09-15T00:13:07.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8065999999999998e-05 2024-09-15T00:13:07.100 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.100 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.101 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.101 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.101 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.101 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:07.103 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:07.103 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:07.099+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:07.203 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:07.201+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:07.253 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:07.253 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:07.253 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:07.253 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:07.262 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:07.304 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:07.301+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:07.404 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:07.404 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:07.405 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:07.405 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:07.405 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:07.405 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:07.403+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:07.413 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:07.505 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:07.503+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:07.518 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:07.518 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:07.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:07.519 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:07.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.519 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.519 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:07.519 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.905772669000001, 2024-09-15T00:13:07.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:13:07.520 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.520 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.520 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.521 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:13:07.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.521 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:07.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:07.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:07.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:13:07.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:13:07.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:13:07.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:13:07.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:13:07.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:13:07.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:13:07.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:13:07.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:13:07.529 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.529 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.529 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.530 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.530 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:07.530 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.905751440000003, 2024-09-15T00:13:07.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:13:07.531 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.531 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.531 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.531 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.531 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.532 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.532 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:13:07.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.532 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:07.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:07.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:13:07.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:13:07.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:13:07.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:07.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:13:07.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:13:07.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:13:07.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:13:07.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:13:07.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:13:07.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:13:07.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:13:07.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:13:07.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:13:07.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.540 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.540 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.540 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:07.540 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.905725899000004, 2024-09-15T00:13:07.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:13:07.540 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.540 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.540 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.541 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:13:07.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.541 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:07.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:07.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:07.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:07.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:13:07.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:13:07.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:13:07.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:13:07.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:13:07.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:13:07.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:13:07.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:13:07.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:13:07.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:13:07.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:13:07.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:13:07.547 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.547 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.547 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.547 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.547 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:07.547 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.905703423999995, 2024-09-15T00:13:07.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:13:07.547 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.547 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.547 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.548 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.548 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.548 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.548 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:13:07.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.548 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:07.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:07.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:13:07.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:13:07.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:13:07.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:13:07.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:13:07.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:13:07.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:13:07.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:13:07.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:13:07.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:13:07.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:13:07.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:13:07.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:13:07.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:13:07.553 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.553 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.553 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.554 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.554 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:07.554 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.905682913999996, 2024-09-15T00:13:07.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:13:07.554 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.554 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.554 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.554 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.554 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.554 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.554 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:13:07.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.555 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:07.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:07.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:07.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:13:07.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:13:07.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:13:07.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:13:07.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:13:07.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:13:07.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:13:07.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:13:07.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:13:07.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:13:07.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:13:07.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:13:07.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:13:07.560 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.560 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.560 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.560 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.560 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:07.560 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.905662598999996, 2024-09-15T00:13:07.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:13:07.560 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.560 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.560 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.560 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.560 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.561 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.561 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:13:07.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.561 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:07.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:07.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:13:07.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:13:07.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:13:07.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:13:07.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:13:07.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:13:07.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:13:07.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:13:07.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:13:07.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:13:07.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:13:07.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:13:07.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:13:07.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:13:07.566 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.566 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.566 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.567 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.567 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:07.567 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.905637823000006, 2024-09-15T00:13:07.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:13:07.567 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.567 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.567 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.567 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.567 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.567 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.567 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:13:07.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.568 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:07.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:07.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:07.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:13:07.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:13:07.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:13:07.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:13:07.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:13:07.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:13:07.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:13:07.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:13:07.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:13:07.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:13:07.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:13:07.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:13:07.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:13:07.571 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.571 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.571 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.571 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:07.571 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:07.571 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.726296435999998, 2024-09-15T00:13:07.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:13:07.571 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.572 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.572 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:07.572 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.572 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.572 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.572 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:13:07.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.572 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:07.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:07.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:13:07.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:13:07.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:13:07.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:13:07.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:13:07.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:13:07.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:13:07.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:13:07.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:13:07.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:13:07.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:13:07.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:13:07.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:07.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:13:07.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:13:07.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:07.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:13:07.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:13:07.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:07.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:13:07.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:13:07.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:07.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:13:07.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:13:07.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:13:07.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:13:07.581 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.581 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.581 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.581 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:13:07.581 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:07.581 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.709342984000003, 2024-09-15T00:13:07.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:13:07.581 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.581 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.581 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.581 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:07.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:07.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:13:07.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:13:07.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:13:07.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:13:07.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:13:07.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:13:07.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:13:07.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:13:07.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:13:07.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:13:07.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:13:07.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:13:07.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:13:07.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:13:07.585 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.585 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.585 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.585 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:13:07.585 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:07.585 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.414912076999997, 2024-09-15T00:13:07.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:13:07.585 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.585 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.586 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:07.586 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.586 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.586 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.586 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:13:07.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.586 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:07.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:07.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:13:07.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:13:07.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:13:07.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:13:07.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:13:07.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:13:07.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:13:07.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:13:07.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:13:07.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:13:07.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:13:07.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:13:07.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:07.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:13:07.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:13:07.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:07.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:13:07.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:13:07.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:07.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:13:07.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:13:07.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:07.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:13:07.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:13:07.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:13:07.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:13:07.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.591 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.591 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:13:07.591 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:07.591 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.402456263000005, 2024-09-15T00:13:07.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:13:07.591 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.591 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.592 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.592 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:07.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:07.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:13:07.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:13:07.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:13:07.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:13:07.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:13:07.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:13:07.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:13:07.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:13:07.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:13:07.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:13:07.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:13:07.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:13:07.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:13:07.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:13:07.595 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.595 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.595 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.595 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:146935 3.16 3:6c79e092:::rbd_data.11394501364.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:07.595 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:07.595 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.761676744999999, 2024-09-15T00:13:07.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.058987488999999997, 2024-09-15T00:13:07.596 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.596 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.596 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:07.596 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.596 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.596 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.596 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 146935 2024-09-15T00:13:07.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.596 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:07.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:07.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753644+0000", 2024-09-15T00:13:07.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6199999999999997e-07 2024-09-15T00:13:07.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753648+0000", 2024-09-15T00:13:07.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-15T00:13:07.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753659+0000", 2024-09-15T00:13:07.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0703000000000001e-05 2024-09-15T00:13:07.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753664+0000", 2024-09-15T00:13:07.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.04e-06 2024-09-15T00:13:07.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753709+0000", 2024-09-15T00:13:07.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4728000000000003e-05 2024-09-15T00:13:07.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753756+0000", 2024-09-15T00:13:07.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7808e-05 2024-09-15T00:13:07.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:07.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.757051+0000", 2024-09-15T00:13:07.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294249 2024-09-15T00:13:07.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:07.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812553+0000", 2024-09-15T00:13:07.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055501807 2024-09-15T00:13:07.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:07.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812564+0000", 2024-09-15T00:13:07.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1467999999999999e-05 2024-09-15T00:13:07.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:07.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812598+0000", 2024-09-15T00:13:07.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3553000000000002e-05 2024-09-15T00:13:07.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812630+0000", 2024-09-15T00:13:07.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2895000000000003e-05 2024-09-15T00:13:07.601 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.601 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.601 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.601 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:146935 3.16 e98/95 ondisk, result = 0)", 2024-09-15T00:13:07.601 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:07.601 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.750874049, 2024-09-15T00:13:07.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048111830000000001, 2024-09-15T00:13:07.601 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.601 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.602 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.602 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:07.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:07.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764447+0000", 2024-09-15T00:13:07.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0500000000000002e-07 2024-09-15T00:13:07.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764450+0000", 2024-09-15T00:13:07.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4400000000000001e-06 2024-09-15T00:13:07.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764462+0000", 2024-09-15T00:13:07.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1603000000000001e-05 2024-09-15T00:13:07.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764469+0000", 2024-09-15T00:13:07.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5089999999999996e-06 2024-09-15T00:13:07.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812541+0000", 2024-09-15T00:13:07.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048071585 2024-09-15T00:13:07.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812548+0000", 2024-09-15T00:13:07.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9979999999999997e-06 2024-09-15T00:13:07.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812557+0000", 2024-09-15T00:13:07.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7899999999999994e-06 2024-09-15T00:13:07.605 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.605 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.605 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.606 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:152505 3.17 3:ea3f268c:::rbd_data.113958cb0e55.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:07.606 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:07.606 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.848768087, 2024-09-15T00:13:07.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.060817959999999997, 2024-09-15T00:13:07.606 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.606 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.606 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:07.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.606 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 152505 2024-09-15T00:13:07.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.606 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:07.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:07.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666553+0000", 2024-09-15T00:13:07.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-15T00:13:07.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666562+0000", 2024-09-15T00:13:07.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0909999999999999e-06 2024-09-15T00:13:07.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666572+0000", 2024-09-15T00:13:07.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0214000000000001e-05 2024-09-15T00:13:07.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666577+0000", 2024-09-15T00:13:07.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3800000000000002e-06 2024-09-15T00:13:07.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666617+0000", 2024-09-15T00:13:07.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9304000000000002e-05 2024-09-15T00:13:07.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666672+0000", 2024-09-15T00:13:07.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5206e-05 2024-09-15T00:13:07.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:07.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.670821+0000", 2024-09-15T00:13:07.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041493169999999996 2024-09-15T00:13:07.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:07.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727290+0000", 2024-09-15T00:13:07.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.056468580999999997 2024-09-15T00:13:07.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:07.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727302+0000", 2024-09-15T00:13:07.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2605e-05 2024-09-15T00:13:07.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:07.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727337+0000", 2024-09-15T00:13:07.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4783999999999998e-05 2024-09-15T00:13:07.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727370+0000", 2024-09-15T00:13:07.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2366000000000001e-05 2024-09-15T00:13:07.611 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.611 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.611 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.611 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:152505 3.17 e103/68 ondisk, result = 0)", 2024-09-15T00:13:07.611 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:07.611 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.836689651, 2024-09-15T00:13:07.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048665938999999998, 2024-09-15T00:13:07.612 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.612 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.612 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.612 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:07.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:07.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678631+0000", 2024-09-15T00:13:07.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5099999999999998e-07 2024-09-15T00:13:07.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678634+0000", 2024-09-15T00:13:07.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.478e-06 2024-09-15T00:13:07.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678647+0000", 2024-09-15T00:13:07.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2085e-05 2024-09-15T00:13:07.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678653+0000", 2024-09-15T00:13:07.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2099999999999998e-06 2024-09-15T00:13:07.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727277+0000", 2024-09-15T00:13:07.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048624405000000002 2024-09-15T00:13:07.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727285+0000", 2024-09-15T00:13:07.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2060000000000007e-06 2024-09-15T00:13:07.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727296+0000", 2024-09-15T00:13:07.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0604e-05 2024-09-15T00:13:07.616 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.616 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.616 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.616 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153959 3.c 3:326b8454:::rbd_data.113958cb0e55.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:07.616 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:07.616 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.425461705, 2024-09-15T00:13:07.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.068102041000000002, 2024-09-15T00:13:07.616 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.616 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.616 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:07.617 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.617 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.617 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.617 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153959 2024-09-15T00:13:07.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.617 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:07.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:07.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089859+0000", 2024-09-15T00:13:07.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1800000000000004e-07 2024-09-15T00:13:07.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089864+0000", 2024-09-15T00:13:07.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2079999999999999e-06 2024-09-15T00:13:07.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089880+0000", 2024-09-15T00:13:07.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5695000000000001e-05 2024-09-15T00:13:07.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089889+0000", 2024-09-15T00:13:07.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.935e-06 2024-09-15T00:13:07.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089942+0000", 2024-09-15T00:13:07.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3219e-05 2024-09-15T00:13:07.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.090066+0000", 2024-09-15T00:13:07.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012385099999999999 2024-09-15T00:13:07.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:07.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.095262+0000", 2024-09-15T00:13:07.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051963950000000004 2024-09-15T00:13:07.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:07.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157865+0000", 2024-09-15T00:13:07.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.062602865999999993 2024-09-15T00:13:07.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:07.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157891+0000", 2024-09-15T00:13:07.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5778e-05 2024-09-15T00:13:07.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:07.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157924+0000", 2024-09-15T00:13:07.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3133000000000003e-05 2024-09-15T00:13:07.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157960+0000", 2024-09-15T00:13:07.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6043e-05 2024-09-15T00:13:07.622 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.622 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.622 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.622 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:153959 3.c e103/23 ondisk, result = 0)", 2024-09-15T00:13:07.622 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:07.622 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.412806047, 2024-09-15T00:13:07.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055370211000000003, 2024-09-15T00:13:07.622 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.622 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.622 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.622 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:07.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:07.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102515+0000", 2024-09-15T00:13:07.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.206e-06 2024-09-15T00:13:07.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102520+0000", 2024-09-15T00:13:07.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1209999999999998e-06 2024-09-15T00:13:07.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102537+0000", 2024-09-15T00:13:07.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:07.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102546+0000", 2024-09-15T00:13:07.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2329999999999995e-06 2024-09-15T00:13:07.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157847+0000", 2024-09-15T00:13:07.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055300798999999998 2024-09-15T00:13:07.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157857+0000", 2024-09-15T00:13:07.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.046e-05 2024-09-15T00:13:07.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157884+0000", 2024-09-15T00:13:07.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.671e-05 2024-09-15T00:13:07.626 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.626 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.626 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.626 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163346 3.1d 3:baa43c12:::rbd_data.11395d771336.0000000000000065:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:13:07.626 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:07.626 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.744191474000001, 2024-09-15T00:13:07.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047558544000000001, 2024-09-15T00:13:07.626 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.626 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.626 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:07.627 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.627 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.627 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.627 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163346 2024-09-15T00:13:07.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.627 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:07.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:07.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771130+0000", 2024-09-15T00:13:07.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5939999999999999e-06 2024-09-15T00:13:07.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771141+0000", 2024-09-15T00:13:07.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1001e-05 2024-09-15T00:13:07.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771154+0000", 2024-09-15T00:13:07.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2921e-05 2024-09-15T00:13:07.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771160+0000", 2024-09-15T00:13:07.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3849999999999999e-06 2024-09-15T00:13:07.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771207+0000", 2024-09-15T00:13:07.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6888999999999998e-05 2024-09-15T00:13:07.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771306+0000", 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9259000000000006e-05 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.774692+0000", 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033854850000000001 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818600+0000", 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043908006999999999 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:07.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818616+0000", 2024-09-15T00:13:07.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6688e-05 2024-09-15T00:13:07.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:07.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818651+0000", 2024-09-15T00:13:07.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4885000000000002e-05 2024-09-15T00:13:07.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818687+0000", 2024-09-15T00:13:07.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5429999999999998e-05 2024-09-15T00:13:07.631 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.631 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.631 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.632 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163578 3.1c 3:3efd71ab:::rbd_data.11395d771336.0000000000000060:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-15T00:13:07.632 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:07.632 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.344768725, 2024-09-15T00:13:07.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.085154081000000006, 2024-09-15T00:13:07.632 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.632 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.632 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:07.632 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:07.632 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:07.632 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:07.632 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163578 2024-09-15T00:13:07.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.632 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:07.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:07.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170552+0000", 2024-09-15T00:13:07.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8599999999999997e-07 2024-09-15T00:13:07.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170557+0000", 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170567+0000", 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0628999999999999e-05 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170573+0000", 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1649999999999997e-06 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170619+0000", 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6405e-05 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170666+0000", 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7259000000000003e-05 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.175060+0000", 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043940170000000001 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255559+0000", 2024-09-15T00:13:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080499065999999994 2024-09-15T00:13:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255616+0000", 2024-09-15T00:13:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6422000000000003e-05 2024-09-15T00:13:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255655+0000", 2024-09-15T00:13:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9632999999999998e-05 2024-09-15T00:13:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255705+0000", 2024-09-15T00:13:07.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.973e-05 2024-09-15T00:13:07.637 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.637 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.637 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.637 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:163578 3.1c e107/23 ondisk, result = 0)", 2024-09-15T00:13:07.637 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:07.637 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.329546634, 2024-09-15T00:13:07.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069939984999999996, 2024-09-15T00:13:07.637 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:07.637 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:07.637 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:07.638 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:07.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:07.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:07.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:07.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:07.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:07.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:07.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185774+0000", 2024-09-15T00:13:07.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.093e-06 2024-09-15T00:13:07.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:07.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185779+0000", 2024-09-15T00:13:07.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9470000000000002e-06 2024-09-15T00:13:07.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:07.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185794+0000", 2024-09-15T00:13:07.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4450000000000001e-05 2024-09-15T00:13:07.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:07.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185801+0000", 2024-09-15T00:13:07.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9189999999999999e-06 2024-09-15T00:13:07.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:07.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255604+0000", 2024-09-15T00:13:07.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069802201999999994 2024-09-15T00:13:07.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:07.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255611+0000", 2024-09-15T00:13:07.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7740000000000004e-06 2024-09-15T00:13:07.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:07.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:07.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:07.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255713+0000", 2024-09-15T00:13:07.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0001016 2024-09-15T00:13:07.641 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.641 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.641 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.641 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:07.641 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:07.641 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:07.642 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:07.641+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:07.643 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:07.744 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:07.742+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:07.787 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:07.787 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:07.787 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:07.787 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:07.795 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:07.844 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:07.842+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:07.898 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:07.898 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:07.898 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:07.899 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:07.899 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:07.907 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:07.950 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:07.948+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:08.011 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:08.011 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:08.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:08.011 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:08.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.012 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:08.012 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:08.012 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.341574976999993, 2024-09-15T00:13:08.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:13:08.012 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.012 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.012 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:08.012 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.012 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.012 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.012 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:13:08.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.013 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:08.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:08.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:13:08.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:13:08.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:13:08.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:13:08.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:13:08.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:13:08.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:13:08.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:13:08.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:13:08.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:13:08.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:13:08.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:13:08.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:08.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:13:08.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:13:08.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:08.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:13:08.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:13:08.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:08.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:13:08.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:13:08.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:08.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:13:08.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:13:08.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:13:08.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:13:08.017 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.017 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.018 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.018 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125618 3.10 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:08.018 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:08.018 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.170435894999997, 2024-09-15T00:13:08.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030860942999999998, 2024-09-15T00:13:08.018 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.018 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.018 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:08.018 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.018 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.018 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.018 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125618 2024-09-15T00:13:08.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.019 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:08.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:08.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838499+0000", 2024-09-15T00:13:08.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6189999999999998e-06 2024-09-15T00:13:08.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839599+0000", 2024-09-15T00:13:08.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001099764 2024-09-15T00:13:08.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839620+0000", 2024-09-15T00:13:08.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0764e-05 2024-09-15T00:13:08.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839631+0000", 2024-09-15T00:13:08.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1335000000000001e-05 2024-09-15T00:13:08.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839725+0000", 2024-09-15T00:13:08.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4217999999999998e-05 2024-09-15T00:13:08.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839910+0000", 2024-09-15T00:13:08.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000184298 2024-09-15T00:13:08.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:08.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843171+0000", 2024-09-15T00:13:08.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032613540000000002 2024-09-15T00:13:08.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:08.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869187+0000", 2024-09-15T00:13:08.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026015467 2024-09-15T00:13:08.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:08.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869240+0000", 2024-09-15T00:13:08.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3733000000000002e-05 2024-09-15T00:13:08.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869309+0000", 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8381999999999997e-05 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869358+0000", 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125619 3.a 3:507b29bf:::rbd_data.11396481e1ec.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:08.023 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.169273766000003, 2024-09-15T00:13:08.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045768870000000003, 2024-09-15T00:13:08.024 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.024 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.024 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:08.024 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.024 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.024 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.024 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125619 2024-09-15T00:13:08.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.024 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:08.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:08.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839660+0000", 2024-09-15T00:13:08.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.22e-06 2024-09-15T00:13:08.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840511+0000", 2024-09-15T00:13:08.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000851284 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840522+0000", 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0569e-05 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840527+0000", 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.412e-06 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840573+0000", 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5247000000000002e-05 2024-09-15T00:13:08.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840682+0000", 2024-09-15T00:13:08.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109368 2024-09-15T00:13:08.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:08.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843976+0000", 2024-09-15T00:13:08.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294445 2024-09-15T00:13:08.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:08.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870833+0000", 2024-09-15T00:13:08.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026856944000000001 2024-09-15T00:13:08.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:08.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885344+0000", 2024-09-15T00:13:08.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014510744000000001 2024-09-15T00:13:08.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:08.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885382+0000", 2024-09-15T00:13:08.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7821000000000003e-05 2024-09-15T00:13:08.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885428+0000", 2024-09-15T00:13:08.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5816000000000003e-05 2024-09-15T00:13:08.029 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.029 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.029 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.029 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125621 3.a 3:53594db9:::rbd_data.11396481e1ec.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:08.029 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:08.029 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.155750386999998, 2024-09-15T00:13:08.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032481797, 2024-09-15T00:13:08.029 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.029 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.029 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:08.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.030 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.030 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.030 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125621 2024-09-15T00:13:08.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.030 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:08.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:08.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853185+0000", 2024-09-15T00:13:08.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.43e-06 2024-09-15T00:13:08.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854125+0000", 2024-09-15T00:13:08.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000940539 2024-09-15T00:13:08.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854141+0000", 2024-09-15T00:13:08.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5568000000000001e-05 2024-09-15T00:13:08.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854148+0000", 2024-09-15T00:13:08.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6529999999999994e-06 2024-09-15T00:13:08.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870872+0000", 2024-09-15T00:13:08.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016723135 2024-09-15T00:13:08.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870975+0000", 2024-09-15T00:13:08.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010314099999999999 2024-09-15T00:13:08.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:08.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.873082+0000", 2024-09-15T00:13:08.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021069859999999999 2024-09-15T00:13:08.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:08.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885439+0000", 2024-09-15T00:13:08.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012357343999999999 2024-09-15T00:13:08.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:08.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885569+0000", 2024-09-15T00:13:08.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012995199999999999 2024-09-15T00:13:08.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:08.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885624+0000", 2024-09-15T00:13:08.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5281999999999997e-05 2024-09-15T00:13:08.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885664+0000", 2024-09-15T00:13:08.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9767000000000001e-05 2024-09-15T00:13:08.035 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.035 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.035 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.035 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:147075 3.0 3:06ab39f2:::rbd_data.11394501364.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:08.035 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:08.035 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.059852006, 2024-09-15T00:13:08.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046402906000000001, 2024-09-15T00:13:08.035 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.035 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.036 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:08.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.036 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 147075 2024-09-15T00:13:08.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.036 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:08.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:08.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:08.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4399999999999999e-07 2024-09-15T00:13:08.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949086+0000", 2024-09-15T00:13:08.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.138e-06 2024-09-15T00:13:08.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949096+0000", 2024-09-15T00:13:08.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0438999999999999e-05 2024-09-15T00:13:08.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949101+0000", 2024-09-15T00:13:08.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8999999999999997e-06 2024-09-15T00:13:08.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949145+0000", 2024-09-15T00:13:08.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4141000000000003e-05 2024-09-15T00:13:08.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949195+0000", 2024-09-15T00:13:08.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9657999999999998e-05 2024-09-15T00:13:08.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:08.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.950923+0000", 2024-09-15T00:13:08.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001728753 2024-09-15T00:13:08.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:08.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995390+0000", 2024-09-15T00:13:08.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044466335000000003 2024-09-15T00:13:08.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:08.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995401+0000", 2024-09-15T00:13:08.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1671000000000001e-05 2024-09-15T00:13:08.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:08.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995432+0000", 2024-09-15T00:13:08.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0704999999999998e-05 2024-09-15T00:13:08.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995484+0000", 2024-09-15T00:13:08.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1422000000000003e-05 2024-09-15T00:13:08.041 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.041 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.041 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.041 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:147075 3.0 e98/94 ondisk, result = 0)", 2024-09-15T00:13:08.041 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:08.042 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.052052284999998, 2024-09-15T00:13:08.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038514459000000001, 2024-09-15T00:13:08.042 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.042 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.042 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.042 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:08.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:08.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956881+0000", 2024-09-15T00:13:08.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0599999999999999e-07 2024-09-15T00:13:08.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956885+0000", 2024-09-15T00:13:08.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3500000000000001e-06 2024-09-15T00:13:08.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956897+0000", 2024-09-15T00:13:08.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2303999999999999e-05 2024-09-15T00:13:08.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956902+0000", 2024-09-15T00:13:08.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5169999999999999e-06 2024-09-15T00:13:08.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995377+0000", 2024-09-15T00:13:08.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038474544999999999 2024-09-15T00:13:08.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995385+0000", 2024-09-15T00:13:08.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6240000000000002e-06 2024-09-15T00:13:08.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995395+0000", 2024-09-15T00:13:08.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0213e-05 2024-09-15T00:13:08.045 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.045 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.045 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.045 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153172 3.14 3:2a7843e3:::rbd_data.113958cb0e55.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 983040~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:08.046 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:08.046 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.231163220999999, 2024-09-15T00:13:08.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.033232668, 2024-09-15T00:13:08.046 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.046 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.046 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:08.046 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.046 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.046 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.046 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153172 2024-09-15T00:13:08.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.046 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777770+0000", 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.017e-06 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788009+0000", 2024-09-15T00:13:08.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010238146 2024-09-15T00:13:08.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788024+0000", 2024-09-15T00:13:08.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5582000000000001e-05 2024-09-15T00:13:08.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788048+0000", 2024-09-15T00:13:08.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4173e-05 2024-09-15T00:13:08.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788097+0000", 2024-09-15T00:13:08.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8498999999999998e-05 2024-09-15T00:13:08.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788201+0000", 2024-09-15T00:13:08.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010398799999999999 2024-09-15T00:13:08.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:08.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788393+0000", 2024-09-15T00:13:08.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000192337 2024-09-15T00:13:08.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:08.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810637+0000", 2024-09-15T00:13:08.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.022244202000000001 2024-09-15T00:13:08.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:08.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810903+0000", 2024-09-15T00:13:08.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00026574600000000003 2024-09-15T00:13:08.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:08.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810953+0000", 2024-09-15T00:13:08.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9509000000000003e-05 2024-09-15T00:13:08.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.811002+0000", 2024-09-15T00:13:08.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9468999999999997e-05 2024-09-15T00:13:08.051 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.051 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.051 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.051 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170014 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:08.051 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:08.051 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.161868115000001, 2024-09-15T00:13:08.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030291116999999999, 2024-09-15T00:13:08.051 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.051 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.051 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.051 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.052 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.052 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.052 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170014 2024-09-15T00:13:08.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.052 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:08.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:08.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:08.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:08.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847073+0000", 2024-09-15T00:13:08.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9589999999999995e-06 2024-09-15T00:13:08.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847080+0000", 2024-09-15T00:13:08.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2930000000000003e-06 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847086+0000", 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.631e-06 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877213+0000", 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030127017999999998 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877281+0000", 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7916999999999994e-05 2024-09-15T00:13:08.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877356+0000", 2024-09-15T00:13:08.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4796999999999996e-05 2024-09-15T00:13:08.055 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.055 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.055 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.055 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170017 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1310720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:08.055 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:08.055 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.161839464, 2024-09-15T00:13:08.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030437623, 2024-09-15T00:13:08.055 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.056 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.056 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.056 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.056 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.056 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.056 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170017 2024-09-15T00:13:08.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.056 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:08.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:08.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847094+0000", 2024-09-15T00:13:08.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:13:08.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847096+0000", 2024-09-15T00:13:08.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.136e-06 2024-09-15T00:13:08.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847104+0000", 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7190000000000001e-06 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847109+0000", 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3279999999999996e-06 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877384+0000", 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030275348000000001 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877451+0000", 2024-09-15T00:13:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6509999999999998e-05 2024-09-15T00:13:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877531+0000", 2024-09-15T00:13:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9993000000000005e-05 2024-09-15T00:13:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170018 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1351680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.161817119, 2024-09-15T00:13:08.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030591884999999999, 2024-09-15T00:13:08.060 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.060 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.060 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.060 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.060 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.060 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.060 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170018 2024-09-15T00:13:08.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.060 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:08.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847118+0000", 2024-09-15T00:13:08.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.03e-06 2024-09-15T00:13:08.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847124+0000", 2024-09-15T00:13:08.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:08.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847129+0000", 2024-09-15T00:13:08.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8600000000000001e-06 2024-09-15T00:13:08.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877560+0000", 2024-09-15T00:13:08.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030431145999999999 2024-09-15T00:13:08.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877629+0000", 2024-09-15T00:13:08.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8793000000000004e-05 2024-09-15T00:13:08.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877707+0000", 2024-09-15T00:13:08.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8139999999999994e-05 2024-09-15T00:13:08.063 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.063 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.063 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.064 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170019 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1392640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:08.064 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:08.064 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.161796370999999, 2024-09-15T00:13:08.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030769023, 2024-09-15T00:13:08.064 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.064 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.064 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.064 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.064 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.064 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.064 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170019 2024-09-15T00:13:08.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.065 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:08.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:08.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847137+0000", 2024-09-15T00:13:08.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:13:08.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847139+0000", 2024-09-15T00:13:08.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.142e-06 2024-09-15T00:13:08.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847146+0000", 2024-09-15T00:13:08.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8120000000000002e-06 2024-09-15T00:13:08.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847157+0000", 2024-09-15T00:13:08.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1093e-05 2024-09-15T00:13:08.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877735+0000", 2024-09-15T00:13:08.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578195999999998 2024-09-15T00:13:08.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877810+0000", 2024-09-15T00:13:08.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5563999999999997e-05 2024-09-15T00:13:08.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877905+0000", 2024-09-15T00:13:08.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4809000000000007e-05 2024-09-15T00:13:08.068 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.068 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.068 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.068 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170020 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:08.068 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:08.068 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.161769131, 2024-09-15T00:13:08.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030917150000000001, 2024-09-15T00:13:08.069 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.069 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.069 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.069 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.069 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.069 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.069 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170020 2024-09-15T00:13:08.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.069 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:08.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847166+0000", 2024-09-15T00:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2790000000000001e-06 2024-09-15T00:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847173+0000", 2024-09-15T00:13:08.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3969999999999999e-06 2024-09-15T00:13:08.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847177+0000", 2024-09-15T00:13:08.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8620000000000002e-06 2024-09-15T00:13:08.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877934+0000", 2024-09-15T00:13:08.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030756143999999999 2024-09-15T00:13:08.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878002+0000", 2024-09-15T00:13:08.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7951e-05 2024-09-15T00:13:08.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878081+0000", 2024-09-15T00:13:08.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9087999999999999e-05 2024-09-15T00:13:08.072 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.072 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.072 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.073 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170094 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:08.073 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:08.073 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.161511881999999, 2024-09-15T00:13:08.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030836142, 2024-09-15T00:13:08.073 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.073 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.073 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.073 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.073 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.073 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.073 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170094 2024-09-15T00:13:08.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.073 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:08.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:08.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:08.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3499999999999996e-07 2024-09-15T00:13:08.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847424+0000", 2024-09-15T00:13:08.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2850000000000001e-06 2024-09-15T00:13:08.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847430+0000", 2024-09-15T00:13:08.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3260000000000001e-06 2024-09-15T00:13:08.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847435+0000", 2024-09-15T00:13:08.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.356e-06 2024-09-15T00:13:08.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878110+0000", 2024-09-15T00:13:08.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030675021 2024-09-15T00:13:08.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878177+0000", 2024-09-15T00:13:08.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6506000000000003e-05 2024-09-15T00:13:08.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878257+0000", 2024-09-15T00:13:08.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0012999999999994e-05 2024-09-15T00:13:08.077 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.077 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.077 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.077 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170096 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 458752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:08.077 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:08.077 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.161469926000001, 2024-09-15T00:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030977030999999999, 2024-09-15T00:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170096 2024-09-15T00:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847465+0000", 2024-09-15T00:13:08.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0559999999999999e-06 2024-09-15T00:13:08.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847472+0000", 2024-09-15T00:13:08.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3130000000000004e-06 2024-09-15T00:13:08.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847477+0000", 2024-09-15T00:13:08.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9930000000000003e-06 2024-09-15T00:13:08.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878288+0000", 2024-09-15T00:13:08.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030811609 2024-09-15T00:13:08.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878358+0000", 2024-09-15T00:13:08.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0215000000000005e-05 2024-09-15T00:13:08.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878440+0000", 2024-09-15T00:13:08.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1366999999999995e-05 2024-09-15T00:13:08.082 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.082 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.082 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.082 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170097 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:08.082 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:08.082 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.161449841, 2024-09-15T00:13:08.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031134757999999998, 2024-09-15T00:13:08.082 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.082 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.082 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.082 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.083 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.083 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.083 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170097 2024-09-15T00:13:08.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.083 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:08.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:08.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:08.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:13:08.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847485+0000", 2024-09-15T00:13:08.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9190000000000002e-06 2024-09-15T00:13:08.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847492+0000", 2024-09-15T00:13:08.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7109999999999998e-06 2024-09-15T00:13:08.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847497+0000", 2024-09-15T00:13:08.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0259999999999998e-06 2024-09-15T00:13:08.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878469+0000", 2024-09-15T00:13:08.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030972115000000001 2024-09-15T00:13:08.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878538+0000", 2024-09-15T00:13:08.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8669000000000003e-05 2024-09-15T00:13:08.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878618+0000", 2024-09-15T00:13:08.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9874000000000001e-05 2024-09-15T00:13:08.086 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.086 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.087 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.087 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170099 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 532480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:08.087 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:08.087 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.161398279, 2024-09-15T00:13:08.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031262183999999998, 2024-09-15T00:13:08.087 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.087 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.087 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.087 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170099 2024-09-15T00:13:08.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.088 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:08.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:08.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847535+0000", 2024-09-15T00:13:08.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:13:08.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847537+0000", 2024-09-15T00:13:08.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:08.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847544+0000", 2024-09-15T00:13:08.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9630000000000001e-06 2024-09-15T00:13:08.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847549+0000", 2024-09-15T00:13:08.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8910000000000003e-06 2024-09-15T00:13:08.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878646+0000", 2024-09-15T00:13:08.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031097400000000001 2024-09-15T00:13:08.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878717+0000", 2024-09-15T00:13:08.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0993000000000003e-05 2024-09-15T00:13:08.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878797+0000", 2024-09-15T00:13:08.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9425000000000006e-05 2024-09-15T00:13:08.091 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.091 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.091 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.091 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170101 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:08.091 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:08.091 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.161356899999999, 2024-09-15T00:13:08.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031674568, 2024-09-15T00:13:08.091 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.091 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.092 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.092 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.092 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.092 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.092 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170101 2024-09-15T00:13:08.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.092 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:08.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:08.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:08.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:13:08.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847578+0000", 2024-09-15T00:13:08.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0389999999999999e-06 2024-09-15T00:13:08.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847585+0000", 2024-09-15T00:13:08.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.336e-06 2024-09-15T00:13:08.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847596+0000", 2024-09-15T00:13:08.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015999999999999e-05 2024-09-15T00:13:08.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878825+0000", 2024-09-15T00:13:08.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031229848000000001 2024-09-15T00:13:08.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878899+0000", 2024-09-15T00:13:08.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:08.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879250+0000", 2024-09-15T00:13:08.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035110000000000002 2024-09-15T00:13:08.096 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.096 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.096 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.096 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170102 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 614400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:08.096 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:08.096 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.161329603, 2024-09-15T00:13:08.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031577371999999999, 2024-09-15T00:13:08.096 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.096 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.096 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.097 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170102 2024-09-15T00:13:08.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.097 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:08.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:08.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:08.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:13:08.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847610+0000", 2024-09-15T00:13:08.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6329999999999999e-06 2024-09-15T00:13:08.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847618+0000", 2024-09-15T00:13:08.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4599999999999997e-06 2024-09-15T00:13:08.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847623+0000", 2024-09-15T00:13:08.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2909999999999998e-06 2024-09-15T00:13:08.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879016+0000", 2024-09-15T00:13:08.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031393227000000003 2024-09-15T00:13:08.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879091+0000", 2024-09-15T00:13:08.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5384999999999997e-05 2024-09-15T00:13:08.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879180+0000", 2024-09-15T00:13:08.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8941000000000001e-05 2024-09-15T00:13:08.100 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.100 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.100 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.101 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170132 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:08.101 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:08.101 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.161282223000001, 2024-09-15T00:13:08.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031722579000000001, 2024-09-15T00:13:08.101 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.101 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.101 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.101 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170132 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847651+0000", 2024-09-15T00:13:08.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1900000000000003e-07 2024-09-15T00:13:08.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847653+0000", 2024-09-15T00:13:08.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0329999999999998e-06 2024-09-15T00:13:08.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847659+0000", 2024-09-15T00:13:08.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4169999999999997e-06 2024-09-15T00:13:08.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847670+0000", 2024-09-15T00:13:08.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1031e-05 2024-09-15T00:13:08.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879205+0000", 2024-09-15T00:13:08.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031534982000000003 2024-09-15T00:13:08.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879286+0000", 2024-09-15T00:13:08.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0675999999999998e-05 2024-09-15T00:13:08.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879373+0000", 2024-09-15T00:13:08.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6921000000000003e-05 2024-09-15T00:13:08.105 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.105 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.105 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.105 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170133 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:08.105 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:08.105 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.161255069999999, 2024-09-15T00:13:08.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031894817999999998, 2024-09-15T00:13:08.105 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.106 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.106 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.106 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.106 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.106 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.106 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170133 2024-09-15T00:13:08.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.106 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:08.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:08.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853540+0000", 2024-09-15T00:13:08.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058619409999999999 2024-09-15T00:13:08.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853545+0000", 2024-09-15T00:13:08.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3430000000000003e-06 2024-09-15T00:13:08.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853554+0000", 2024-09-15T00:13:08.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4069999999999999e-06 2024-09-15T00:13:08.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853561+0000", 2024-09-15T00:13:08.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5529999999999998e-06 2024-09-15T00:13:08.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879399+0000", 2024-09-15T00:13:08.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025837941 2024-09-15T00:13:08.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879477+0000", 2024-09-15T00:13:08.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8367000000000004e-05 2024-09-15T00:13:08.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879572+0000", 2024-09-15T00:13:08.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5266000000000006e-05 2024-09-15T00:13:08.109 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.109 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.110 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.110 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.110 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.110 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:08.110 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:13:08.112 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:08.112 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:08.110+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:08.212 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:08.209+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:08.261 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:08.261 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:08.261 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:08.261 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:08.270 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:08.313 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:08.311+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:08.413 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:08.410+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:08.414 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:08.414 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:08.414 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:08.423 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:08.514 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:08.513+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:08.566 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:08.567 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:08.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:08.567 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:08.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.567 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:08.567 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:08.567 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.485745185, 2024-09-15T00:13:08.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:13:08.567 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.567 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.567 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:08.567 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.568 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.568 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.568 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:13:08.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.568 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:08.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:08.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:13:08.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:13:08.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:13:08.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:13:08.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:13:08.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:13:08.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:13:08.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:13:08.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:13:08.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:13:08.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:13:08.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:13:08.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:08.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:13:08.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:13:08.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:08.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:13:08.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:13:08.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:08.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:13:08.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:13:08.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:08.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:13:08.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:13:08.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:13:08.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:13:08.573 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.573 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.573 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.573 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:13:08.573 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:08.573 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.473004643, 2024-09-15T00:13:08.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:13:08.573 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.573 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.573 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.573 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:13:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:13:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:13:08.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:13:08.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:13:08.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:13:08.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:13:08.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:13:08.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:13:08.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:13:08.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:13:08.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:13:08.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:13:08.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:13:08.577 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.577 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.577 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.577 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:08.577 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:08.577 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.340648884, 2024-09-15T00:13:08.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:13:08.577 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.577 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.577 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:08.577 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.577 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.578 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.578 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:13:08.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.578 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:08.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:08.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:13:08.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:13:08.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:13:08.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:13:08.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:13:08.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:13:08.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:13:08.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:13:08.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:13:08.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:13:08.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:13:08.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:13:08.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:08.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:13:08.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:13:08.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:08.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:13:08.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:13:08.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:08.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:13:08.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:13:08.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:08.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:13:08.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:13:08.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:13:08.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:13:08.583 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.583 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.583 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.583 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:13:08.583 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:08.583 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.327187239, 2024-09-15T00:13:08.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:13:08.583 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.583 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.583 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.583 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:08.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:08.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:13:08.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:13:08.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:13:08.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:13:08.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:13:08.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:13:08.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:13:08.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:13:08.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:13:08.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:13:08.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:13:08.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:13:08.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:13:08.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:13:08.587 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.587 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.587 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.587 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:13:08.587 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:08.588 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.142185756000003, 2024-09-15T00:13:08.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:13:08.588 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.588 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.588 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:08.588 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.588 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.588 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.588 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:13:08.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.588 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:08.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:08.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:13:08.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:13:08.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:13:08.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:13:08.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:13:08.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:13:08.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:13:08.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:13:08.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:13:08.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:13:08.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:13:08.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:13:08.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:08.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:13:08.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:13:08.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:08.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:13:08.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:13:08.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:08.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:13:08.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:13:08.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:08.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:13:08.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:13:08.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:13:08.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:13:08.593 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.593 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.593 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.593 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:13:08.593 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:08.593 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.129309488999994, 2024-09-15T00:13:08.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:13:08.593 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.593 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.594 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.594 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:08.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:08.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:13:08.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:13:08.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:13:08.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:13:08.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:13:08.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:13:08.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:13:08.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:13:08.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:13:08.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:13:08.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:13:08.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:13:08.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:13:08.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:13:08.597 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.597 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.597 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.597 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:13:08.597 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:08.597 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.670794345000004, 2024-09-15T00:13:08.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:13:08.597 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.597 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.598 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:08.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.598 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:13:08.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.598 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:08.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:08.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:13:08.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:13:08.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:13:08.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:13:08.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:13:08.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:13:08.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:13:08.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:13:08.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:13:08.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:13:08.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:13:08.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:13:08.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:08.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:13:08.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:13:08.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:08.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:13:08.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:13:08.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:08.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:13:08.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:13:08.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:08.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:13:08.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:13:08.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:13:08.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:13:08.603 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.603 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.603 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.603 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:08.603 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:08.603 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.953283784999996, 2024-09-15T00:13:08.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:13:08.604 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.604 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.604 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.604 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.604 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.604 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.604 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:13:08.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.604 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:08.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:08.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:13:08.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:13:08.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:13:08.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:13:08.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:13:08.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:13:08.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:13:08.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:13:08.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:13:08.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:13:08.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:13:08.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:13:08.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:13:08.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:13:08.607 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.953265176000002, 2024-09-15T00:13:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:13:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.609 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:13:08.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.609 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:08.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:08.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:08.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:13:08.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:13:08.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:13:08.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:13:08.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:13:08.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:13:08.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:13:08.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:13:08.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:13:08.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:13:08.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:13:08.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:13:08.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:13:08.612 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.612 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.612 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.612 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:08.612 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:08.612 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.953239851999996, 2024-09-15T00:13:08.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:13:08.613 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.613 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.613 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.613 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.613 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.613 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.613 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:13:08.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.613 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:08.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:08.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:08.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:13:08.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:13:08.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:13:08.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:13:08.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:13:08.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:13:08.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:13:08.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:13:08.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:13:08.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:13:08.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:13:08.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:13:08.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:13:08.616 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.617 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.617 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.617 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:08.617 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:08.617 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.953221528, 2024-09-15T00:13:08.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:13:08.617 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.617 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.617 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.617 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.617 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.617 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.618 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:13:08.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.618 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:08.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:08.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:13:08.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:13:08.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:13:08.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:13:08.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:13:08.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:08.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:13:08.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:13:08.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:13:08.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:13:08.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:13:08.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:13:08.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:13:08.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:13:08.621 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.621 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.621 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.622 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:08.622 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:08.622 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.953038954999997, 2024-09-15T00:13:08.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:13:08.622 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.622 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.622 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.622 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.622 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.622 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.622 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:13:08.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.622 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:08.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:08.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:08.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:13:08.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:13:08.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:13:08.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:13:08.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:13:08.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:13:08.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:13:08.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:13:08.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:13:08.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:13:08.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:13:08.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:13:08.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:13:08.626 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.626 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.626 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.626 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:08.626 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:08.626 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.953010113999994, 2024-09-15T00:13:08.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:13:08.626 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.626 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.626 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.627 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.627 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.627 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.627 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:13:08.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.627 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:08.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:08.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:08.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:13:08.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:13:08.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:13:08.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:13:08.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:13:08.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:13:08.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:13:08.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:13:08.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:13:08.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:13:08.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:13:08.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:13:08.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:13:08.630 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.630 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.630 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.631 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:08.631 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:08.631 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.952987902999993, 2024-09-15T00:13:08.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:13:08.631 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.631 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.631 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.631 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.631 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.631 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.631 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:13:08.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.631 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:08.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:08.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:08.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:13:08.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:13:08.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:13:08.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:13:08.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:13:08.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:13:08.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:13:08.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:13:08.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:13:08.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:13:08.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:13:08.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:13:08.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:13:08.635 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.635 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.635 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.635 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:08.635 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:08.635 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.952966743999994, 2024-09-15T00:13:08.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:13:08.635 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.635 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.635 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:13:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:08.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:08.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:13:08.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:13:08.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:13:08.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:13:08.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:13:08.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:13:08.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:13:08.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:13:08.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:13:08.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:13:08.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:13:08.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:13:08.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:13:08.639 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.639 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.639 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.639 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:08.639 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:08.639 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.952936796000003, 2024-09-15T00:13:08.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:13:08.640 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.640 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.640 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.640 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.640 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.640 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.640 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:13:08.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.640 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:08.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:08.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:08.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:13:08.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:13:08.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:13:08.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:13:08.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:13:08.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:13:08.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:13:08.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:13:08.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:13:08.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:13:08.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:13:08.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:13:08.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:13:08.644 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.644 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.644 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.644 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:08.644 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:08.644 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.952914832000005, 2024-09-15T00:13:08.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:13:08.644 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.644 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.644 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.644 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.645 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.645 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.645 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:13:08.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.645 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:08.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:08.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:08.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:13:08.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:13:08.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:13:08.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:13:08.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:13:08.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:13:08.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:13:08.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:13:08.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:13:08.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:13:08.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:13:08.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:13:08.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:13:08.648 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.648 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.648 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.648 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:08.649 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:08.649 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.952897405000002, 2024-09-15T00:13:08.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:13:08.649 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.649 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.649 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.649 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.649 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.649 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.649 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:13:08.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.649 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:08.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:08.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:13:08.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:13:08.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:13:08.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:13:08.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:13:08.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:08.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:13:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:13:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:13:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:13:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:13:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:13:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:13:08.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:13:08.653 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.653 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.653 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.653 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:08.653 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:08.653 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.952875606000006, 2024-09-15T00:13:08.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:13:08.653 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.653 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.654 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:08.654 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.654 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.654 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.654 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:13:08.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.654 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:08.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:08.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:13:08.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:13:08.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:13:08.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:13:08.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:13:08.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:13:08.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:13:08.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:13:08.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:13:08.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:13:08.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:13:08.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:13:08.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:13:08.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:13:08.657 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.657 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.657 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.657 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:08.657 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:08.658 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.903647246999995, 2024-09-15T00:13:08.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:13:08.658 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:08.658 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:08.658 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:08.658 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:08.658 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:08.658 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.658 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:13:08.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.658 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:08.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:08.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:13:08.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:13:08.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:08.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:13:08.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:13:08.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:13:08.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:13:08.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:08.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:13:08.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:13:08.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:08.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:13:08.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:13:08.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:08.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:13:08.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:13:08.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:08.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:13:08.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:13:08.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:08.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:13:08.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:13:08.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:08.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:13:08.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:13:08.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:08.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:08.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:08.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:13:08.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:13:08.663 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.663 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.663 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.663 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:08.663 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:08.663 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:08.664 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:08.665 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:08.663+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:08.768 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:08.762+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:08.769 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:08.769 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:08.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:08.769 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:172604 3.4 3.1196fbc4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-15T00:13:08.769 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:08.221334+0000", 2024-09-15T00:13:08.769 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.54591323800000002, 2024-09-15T00:13:08.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.54593499499999998, 2024-09-15T00:13:08.769 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:08.769 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:08.769 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for peered", 2024-09-15T00:13:08.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:08.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:08.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:08.770 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 172604 2024-09-15T00:13:08.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:08.770 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:08.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:08.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:08.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.221334+0000", 2024-09-15T00:13:08.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:08.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:08.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:08.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:08.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.221334+0000", 2024-09-15T00:13:08.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:08.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:08.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:08.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:08.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.221336+0000", 2024-09-15T00:13:08.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5570000000000001e-06 2024-09-15T00:13:08.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:08.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:08.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:08.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.221343+0000", 2024-09-15T00:13:08.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9090000000000003e-06 2024-09-15T00:13:08.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:08.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:08.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:08.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.221358+0000", 2024-09-15T00:13:08.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5481e-05 2024-09-15T00:13:08.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:08.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:08.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:08.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.221366+0000", 2024-09-15T00:13:08.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.34e-06 2024-09-15T00:13:08.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:08.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:08.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:08.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.221421+0000", 2024-09-15T00:13:08.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5549e-05 2024-09-15T00:13:08.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:08.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:08.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:08.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.221429+0000", 2024-09-15T00:13:08.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9209999999999992e-06 2024-09-15T00:13:08.773 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:08.773 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:08.773 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:08.773 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:08.773 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:13:08.773 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 1 2024-09-15T00:13:08.773 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:08.777 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:08.866 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:08.865+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:08.879 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:08.879 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:08.887 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:08.904 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:13:08.972 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:08.970+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:09.036 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:09.036 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:09.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:09.037 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:09.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.037 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9, version: 37'47, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9@37'47, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:09.037 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:09.037 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.1795758599999999, 2024-09-15T00:13:09.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054905559999999997, 2024-09-15T00:13:09.037 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.037 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.037 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.037 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:09.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:09.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853793+0000", 2024-09-15T00:13:09.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9599999999999999e-07 2024-09-15T00:13:09.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854798+0000", 2024-09-15T00:13:09.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010050770000000001 2024-09-15T00:13:09.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854814+0000", 2024-09-15T00:13:09.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6478999999999999e-05 2024-09-15T00:13:09.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854825+0000", 2024-09-15T00:13:09.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0492999999999999e-05 2024-09-15T00:13:09.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854889+0000", 2024-09-15T00:13:09.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4134999999999995e-05 2024-09-15T00:13:09.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854898+0000", 2024-09-15T00:13:09.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6260000000000001e-06 2024-09-15T00:13:09.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.859283+0000", 2024-09-15T00:13:09.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043852500000000003 2024-09-15T00:13:09.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.041 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.041 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15, version: 59'205, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15@59'205, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:09.041 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:09.041 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.053355405, 2024-09-15T00:13:09.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059047939999999997, 2024-09-15T00:13:09.041 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.042 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.042 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.042 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:09.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:09.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:09.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.44e-07 2024-09-15T00:13:09.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980977+0000", 2024-09-15T00:13:09.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00096402300000000005 2024-09-15T00:13:09.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980995+0000", 2024-09-15T00:13:09.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7214000000000002e-05 2024-09-15T00:13:09.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981007+0000", 2024-09-15T00:13:09.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2028000000000001e-05 2024-09-15T00:13:09.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981069+0000", 2024-09-15T00:13:09.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1977000000000006e-05 2024-09-15T00:13:09.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981077+0000", 2024-09-15T00:13:09.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2120000000000003e-06 2024-09-15T00:13:09.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.985918+0000", 2024-09-15T00:13:09.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048406960000000002 2024-09-15T00:13:09.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.045 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.045 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head, version: 62'221, data_included: [0~262144,393216~786432], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head@62'221, size: 1572864, copy_subset: [(0, 1572864)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:09.045 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:09.045 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.0310871399999999, 2024-09-15T00:13:09.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054170770000000002, 2024-09-15T00:13:09.046 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.046 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.046 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.046 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:09.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:09.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002282+0000", 2024-09-15T00:13:09.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4499999999999997e-07 2024-09-15T00:13:09.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003196+0000", 2024-09-15T00:13:09.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091476399999999999 2024-09-15T00:13:09.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003213+0000", 2024-09-15T00:13:09.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6554000000000001e-05 2024-09-15T00:13:09.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003230+0000", 2024-09-15T00:13:09.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6617999999999999e-05 2024-09-15T00:13:09.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003292+0000", 2024-09-15T00:13:09.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2193999999999994e-05 2024-09-15T00:13:09.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003300+0000", 2024-09-15T00:13:09.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1429999999999998e-06 2024-09-15T00:13:09.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.007698+0000", 2024-09-15T00:13:09.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043981589999999996 2024-09-15T00:13:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.98035706, 2024-09-15T00:13:09.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067633270000000004, 2024-09-15T00:13:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053012+0000", 2024-09-15T00:13:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1659999999999999e-06 2024-09-15T00:13:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054086+0000", 2024-09-15T00:13:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010732789999999999 2024-09-15T00:13:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054100+0000", 2024-09-15T00:13:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4123e-05 2024-09-15T00:13:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054114+0000", 2024-09-15T00:13:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4442e-05 2024-09-15T00:13:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054192+0000", 2024-09-15T00:13:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8288000000000007e-05 2024-09-15T00:13:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054200+0000", 2024-09-15T00:13:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-15T00:13:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.059774+0000", 2024-09-15T00:13:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055741230000000003 2024-09-15T00:13:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.054 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.972039732, 2024-09-15T00:13:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049777950000000001, 2024-09-15T00:13:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061330+0000", 2024-09-15T00:13:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.11e-06 2024-09-15T00:13:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062350+0000", 2024-09-15T00:13:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010207969999999999 2024-09-15T00:13:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062365+0000", 2024-09-15T00:13:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4157e-05 2024-09-15T00:13:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062380+0000", 2024-09-15T00:13:09.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5129e-05 2024-09-15T00:13:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062467+0000", 2024-09-15T00:13:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7247e-05 2024-09-15T00:13:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062475+0000", 2024-09-15T00:13:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4300000000000006e-06 2024-09-15T00:13:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.066306+0000", 2024-09-15T00:13:09.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038309250000000002 2024-09-15T00:13:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.9661567739999999, 2024-09-15T00:13:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052454889999999999, 2024-09-15T00:13:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067212+0000", 2024-09-15T00:13:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-07 2024-09-15T00:13:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068119+0000", 2024-09-15T00:13:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090671199999999995 2024-09-15T00:13:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068135+0000", 2024-09-15T00:13:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5739e-05 2024-09-15T00:13:09.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068150+0000", 2024-09-15T00:13:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5693e-05 2024-09-15T00:13:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068236+0000", 2024-09-15T00:13:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6100000000000006e-05 2024-09-15T00:13:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068245+0000", 2024-09-15T00:13:09.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1960000000000008e-06 2024-09-15T00:13:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.072457+0000", 2024-09-15T00:13:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042123719999999998 2024-09-15T00:13:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.9601784250000001, 2024-09-15T00:13:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049532020000000003, 2024-09-15T00:13:09.063 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:13:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074164+0000", 2024-09-15T00:13:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097354400000000002 2024-09-15T00:13:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074178+0000", 2024-09-15T00:13:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4134999999999999e-05 2024-09-15T00:13:09.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074192+0000", 2024-09-15T00:13:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.451e-05 2024-09-15T00:13:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074272+0000", 2024-09-15T00:13:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9989999999999998e-05 2024-09-15T00:13:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074280+0000", 2024-09-15T00:13:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9990000000000008e-06 2024-09-15T00:13:09.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078143+0000", 2024-09-15T00:13:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00386258 2024-09-15T00:13:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [4194304~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.954467613, 2024-09-15T00:13:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049408730000000001, 2024-09-15T00:13:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.067 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2399999999999999e-07 2024-09-15T00:13:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079875+0000", 2024-09-15T00:13:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097349100000000005 2024-09-15T00:13:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079889+0000", 2024-09-15T00:13:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4148000000000001e-05 2024-09-15T00:13:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079904+0000", 2024-09-15T00:13:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5159e-05 2024-09-15T00:13:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079964+0000", 2024-09-15T00:13:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0204999999999997e-05 2024-09-15T00:13:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079971+0000", 2024-09-15T00:13:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2490000000000004e-06 2024-09-15T00:13:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.083841+0000", 2024-09-15T00:13:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038701970000000001 2024-09-15T00:13:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.071 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [5242880~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 6291456, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.9487626329999999, 2024-09-15T00:13:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052525740000000003, 2024-09-15T00:13:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:13:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085579+0000", 2024-09-15T00:13:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097307600000000002 2024-09-15T00:13:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085592+0000", 2024-09-15T00:13:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3277e-05 2024-09-15T00:13:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085608+0000", 2024-09-15T00:13:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5773e-05 2024-09-15T00:13:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085687+0000", 2024-09-15T00:13:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8956999999999996e-05 2024-09-15T00:13:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085694+0000", 2024-09-15T00:13:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.956e-06 2024-09-15T00:13:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.089858+0000", 2024-09-15T00:13:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041641159999999998 2024-09-15T00:13:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.075 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:172552 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-15T00:13:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.8443053250000001, 2024-09-15T00:13:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.997031471, 2024-09-15T00:13:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 172552 2024-09-15T00:13:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189064+0000", 2024-09-15T00:13:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.198e-06 2024-09-15T00:13:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189069+0000", 2024-09-15T00:13:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9069999999999998e-06 2024-09-15T00:13:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189080+0000", 2024-09-15T00:13:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:13:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189086+0000", 2024-09-15T00:13:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5429999999999999e-06 2024-09-15T00:13:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189128+0000", 2024-09-15T00:13:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2255000000000001e-05 2024-09-15T00:13:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189135+0000", 2024-09-15T00:13:09.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6200000000000001e-06 2024-09-15T00:13:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185787+0000", 2024-09-15T00:13:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99665157199999999 2024-09-15T00:13:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185926+0000", 2024-09-15T00:13:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013970399999999999 2024-09-15T00:13:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.186094+0000", 2024-09-15T00:13:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016813099999999999 2024-09-15T00:13:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9, version: 37'60, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9@37'60, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:09.083 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.785444966, 2024-09-15T00:13:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049454060000000003, 2024-09-15T00:13:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:09.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247924+0000", 2024-09-15T00:13:09.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:13:09.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248394+0000", 2024-09-15T00:13:09.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047063399999999999 2024-09-15T00:13:09.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248411+0000", 2024-09-15T00:13:09.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6384000000000001e-05 2024-09-15T00:13:09.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248441+0000", 2024-09-15T00:13:09.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0559999999999999e-05 2024-09-15T00:13:09.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248499+0000", 2024-09-15T00:13:09.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7274000000000001e-05 2024-09-15T00:13:09.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248507+0000", 2024-09-15T00:13:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2390000000000002e-06 2024-09-15T00:13:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.252869+0000", 2024-09-15T00:13:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043617200000000004 2024-09-15T00:13:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head, version: 59'393, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head@59'393, size: 5701632, copy_subset: [(0, 5701632)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.6559966990000001, 2024-09-15T00:13:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004807469, 2024-09-15T00:13:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4799999999999998e-07 2024-09-15T00:13:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377830+0000", 2024-09-15T00:13:09.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000457954 2024-09-15T00:13:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377847+0000", 2024-09-15T00:13:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6741000000000001e-05 2024-09-15T00:13:09.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377858+0000", 2024-09-15T00:13:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0951999999999999e-05 2024-09-15T00:13:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377929+0000", 2024-09-15T00:13:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0697000000000004e-05 2024-09-15T00:13:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377941+0000", 2024-09-15T00:13:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2529e-05 2024-09-15T00:13:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.382179+0000", 2024-09-15T00:13:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042379480000000001 2024-09-15T00:13:09.094 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head, version: 88'473, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head@88'473, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.573877266, 2024-09-15T00:13:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005823272, 2024-09-15T00:13:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459492+0000", 2024-09-15T00:13:09.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9800000000000003e-07 2024-09-15T00:13:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459940+0000", 2024-09-15T00:13:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044854699999999999 2024-09-15T00:13:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459957+0000", 2024-09-15T00:13:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6586e-05 2024-09-15T00:13:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459968+0000", 2024-09-15T00:13:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.145e-05 2024-09-15T00:13:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460043+0000", 2024-09-15T00:13:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4451999999999998e-05 2024-09-15T00:13:09.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460057+0000", 2024-09-15T00:13:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.417e-05 2024-09-15T00:13:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.465314+0000", 2024-09-15T00:13:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052574689999999999 2024-09-15T00:13:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.3387010850000001, 2024-09-15T00:13:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051811399999999999, 2024-09-15T00:13:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694668+0000", 2024-09-15T00:13:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4600000000000005e-07 2024-09-15T00:13:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695136+0000", 2024-09-15T00:13:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00046864399999999999 2024-09-15T00:13:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695153+0000", 2024-09-15T00:13:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6218e-05 2024-09-15T00:13:09.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695189+0000", 2024-09-15T00:13:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6393000000000002e-05 2024-09-15T00:13:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695263+0000", 2024-09-15T00:13:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4175e-05 2024-09-15T00:13:09.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695271+0000", 2024-09-15T00:13:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2800000000000003e-06 2024-09-15T00:13:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.699848+0000", 2024-09-15T00:13:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045768839999999998 2024-09-15T00:13:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.330602877, 2024-09-15T00:13:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048716510000000003, 2024-09-15T00:13:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:09.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702783+0000", 2024-09-15T00:13:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7268000000000001e-05 2024-09-15T00:13:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703221+0000", 2024-09-15T00:13:09.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00043866899999999998 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703237+0000", 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5553000000000002e-05 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703247+0000", 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0392e-05 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703358+0000", 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110676 2024-09-15T00:13:09.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703369+0000", 2024-09-15T00:13:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0716e-05 2024-09-15T00:13:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.707637+0000", 2024-09-15T00:13:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042683770000000003 2024-09-15T00:13:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18, version: 62'274, data_included: [0~786432], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18@62'274, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.2845217950000001, 2024-09-15T00:13:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005453014, 2024-09-15T00:13:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748847+0000", 2024-09-15T00:13:09.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8299999999999997e-07 2024-09-15T00:13:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749010+0000", 2024-09-15T00:13:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000163144 2024-09-15T00:13:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749027+0000", 2024-09-15T00:13:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6518999999999999e-05 2024-09-15T00:13:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749037+0000", 2024-09-15T00:13:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0438000000000001e-05 2024-09-15T00:13:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749112+0000", 2024-09-15T00:13:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4946999999999999e-05 2024-09-15T00:13:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749123+0000", 2024-09-15T00:13:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1014e-05 2024-09-15T00:13:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.754299+0000", 2024-09-15T00:13:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051763690000000001 2024-09-15T00:13:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20, version: 80'334, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20@80'334, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 20=[]:{20: [20]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.2298847509999999, 2024-09-15T00:13:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048933570000000001, 2024-09-15T00:13:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.113 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803484+0000", 2024-09-15T00:13:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:13:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803938+0000", 2024-09-15T00:13:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045364599999999999 2024-09-15T00:13:09.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803954+0000", 2024-09-15T00:13:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6456999999999998e-05 2024-09-15T00:13:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803965+0000", 2024-09-15T00:13:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.076e-05 2024-09-15T00:13:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804023+0000", 2024-09-15T00:13:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7787000000000001e-05 2024-09-15T00:13:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804031+0000", 2024-09-15T00:13:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2160000000000006e-06 2024-09-15T00:13:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.808377+0000", 2024-09-15T00:13:09.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043458969999999996 2024-09-15T00:13:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [0~131072,196608~786432,1048576~131072], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.1930981549999999, 2024-09-15T00:13:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047737029999999998, 2024-09-15T00:13:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840271+0000", 2024-09-15T00:13:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:13:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840721+0000", 2024-09-15T00:13:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045029800000000002 2024-09-15T00:13:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840738+0000", 2024-09-15T00:13:09.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6671999999999999e-05 2024-09-15T00:13:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840749+0000", 2024-09-15T00:13:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1251e-05 2024-09-15T00:13:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840817+0000", 2024-09-15T00:13:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8046000000000005e-05 2024-09-15T00:13:09.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840825+0000", 2024-09-15T00:13:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2519999999999999e-06 2024-09-15T00:13:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.845044+0000", 2024-09-15T00:13:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042185900000000004 2024-09-15T00:13:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [1179648~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2228224, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.185074108, 2024-09-15T00:13:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053056819999999999, 2024-09-15T00:13:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848295+0000", 2024-09-15T00:13:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4000000000000001e-07 2024-09-15T00:13:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848728+0000", 2024-09-15T00:13:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000433709 2024-09-15T00:13:09.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848745+0000", 2024-09-15T00:13:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6167999999999998e-05 2024-09-15T00:13:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848788+0000", 2024-09-15T00:13:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3269000000000002e-05 2024-09-15T00:13:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848857+0000", 2024-09-15T00:13:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9453000000000001e-05 2024-09-15T00:13:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848866+0000", 2024-09-15T00:13:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2779999999999993e-06 2024-09-15T00:13:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.853600+0000", 2024-09-15T00:13:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047341650000000002 2024-09-15T00:13:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head, version: 93'455, data_included: [0~98304,114688~16384,147456~589824,753664~65536,835584~16384,868352~262144], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head@93'455, size: 2342912, copy_subset: [(0, 2342912)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1130496, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.1259400820000001, 2024-09-15T00:13:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063250900000000002, 2024-09-15T00:13:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:09.127 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907429+0000", 2024-09-15T00:13:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.85e-07 2024-09-15T00:13:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908008+0000", 2024-09-15T00:13:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00057949599999999998 2024-09-15T00:13:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908026+0000", 2024-09-15T00:13:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7700999999999999e-05 2024-09-15T00:13:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908038+0000", 2024-09-15T00:13:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2131e-05 2024-09-15T00:13:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908101+0000", 2024-09-15T00:13:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2596999999999997e-05 2024-09-15T00:13:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908110+0000", 2024-09-15T00:13:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8540000000000003e-06 2024-09-15T00:13:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.913753+0000", 2024-09-15T00:13:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056435260000000003 2024-09-15T00:13:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:09.131 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:09.131 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:09.133 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:09.133 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:09.134 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:09.134 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:09.134 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:09.135 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:09.135 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:09.135 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:09.135 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:09.136 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:09.136 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:09.136 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:09.137 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:09.137 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:09.130+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:09.147 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:09.148 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:09.164 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:09.165 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:09.176 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 0, 2] out_osds: [7, 1] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-15T00:13:09.176 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:13:09.176 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [3, 4, 5, 6, 0, 2] 2024-09-15T00:13:09.176 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-15T00:13:09.190 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is set 2024-09-15T00:13:09.253 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:09.252+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:09.274 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:09.274 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:09.284 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:09.334 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:09.332+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:09.431 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:09.431 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:09.435 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:09.433+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:09.439 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:09.498 INFO:tasks.ceph.mon.a.smithi171.stderr:2024-09-15T00:13:09.497+0000 7f0425d35640 -1 mon.a@0(leader).osd e121 definitely_dead 0 2024-09-15T00:13:09.536 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:09.534+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:09.584 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:09.584 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:09.591 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:09.675 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:09.674+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:09.736 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:09.736 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:09.738 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:09.736+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:09.744 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:09.839 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:09.838+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:09.888 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:09.889 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:09.897 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:09.941 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:09.938+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:10.041 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:10.041 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:10.041 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:10.038+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:10.050 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:10.142 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:10.140+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:10.154 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:10.154 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:10.163 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:10.186 INFO:teuthology.orchestra.run.smithi171.stderr:marked out osd.2. 2024-09-15T00:13:10.242 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:10.240+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:10.294 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:10.295 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:10.303 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:10.346 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:10.345+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:10.406 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:10.407 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:10.415 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:10.456 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:10.455+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:10.523 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:10.523 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:10.532 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:10.545 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:10.544+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:10.634 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:10.634 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:10.643 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:10.646 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:10.643+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:10.746 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:10.746 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:10.746 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:10.745+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:10.755 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:10.847 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:10.845+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:10.867 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:10.948 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:10.946+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:11.010 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:11.010 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:11.019 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:11.049 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:11.047+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:11.122 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:11.122 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:11.131 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:11.150 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:11.148+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:11.179 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:13:11.212 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:13:11.237 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:11.238 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:11.246 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:11.290 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:11.287+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:11.316 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:11.317 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:11.335 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:11.335 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:11.351 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:11.351 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:11.351 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:11.349+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:11.352 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:11.352 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:11.359 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:11.367 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:11.367 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:11.384 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:11.384 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:11.401 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:11.401 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:11.417 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:11.418 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:11.433 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:11.434 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:11.451 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:11.450+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:11.505 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:11.506 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:11.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:11.506 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:11.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.506 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:11.506 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:11.506 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 86.835904588999995, 2024-09-15T00:13:11.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:13:11.506 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.506 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.506 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:11.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.507 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:13:11.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.507 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:11.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:11.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:13:11.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:13:11.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:13:11.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:13:11.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:13:11.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:13:11.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:13:11.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:13:11.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:13:11.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:13:11.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:13:11.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:13:11.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:11.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:13:11.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:13:11.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:11.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:13:11.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:13:11.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:11.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:13:11.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:13:11.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:11.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:13:11.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:13:11.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:13:11.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:13:11.512 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.512 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.512 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.512 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125618 3.10 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:11.512 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:11.512 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 49.664765506999998, 2024-09-15T00:13:11.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030860942999999998, 2024-09-15T00:13:11.512 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.512 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.512 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:11.512 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.512 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.513 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.513 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125618 2024-09-15T00:13:11.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.513 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:11.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:11.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838499+0000", 2024-09-15T00:13:11.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6189999999999998e-06 2024-09-15T00:13:11.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839599+0000", 2024-09-15T00:13:11.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001099764 2024-09-15T00:13:11.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839620+0000", 2024-09-15T00:13:11.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0764e-05 2024-09-15T00:13:11.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839631+0000", 2024-09-15T00:13:11.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1335000000000001e-05 2024-09-15T00:13:11.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839725+0000", 2024-09-15T00:13:11.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4217999999999998e-05 2024-09-15T00:13:11.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839910+0000", 2024-09-15T00:13:11.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000184298 2024-09-15T00:13:11.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:11.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843171+0000", 2024-09-15T00:13:11.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032613540000000002 2024-09-15T00:13:11.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:11.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869187+0000", 2024-09-15T00:13:11.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026015467 2024-09-15T00:13:11.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:11.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869240+0000", 2024-09-15T00:13:11.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3733000000000002e-05 2024-09-15T00:13:11.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:11.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869309+0000", 2024-09-15T00:13:11.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8381999999999997e-05 2024-09-15T00:13:11.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869358+0000", 2024-09-15T00:13:11.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-15T00:13:11.517 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.517 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.517 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.518 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125619 3.a 3:507b29bf:::rbd_data.11396481e1ec.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:11.518 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:11.518 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 49.663603377999998, 2024-09-15T00:13:11.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045768870000000003, 2024-09-15T00:13:11.518 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.518 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.518 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:11.518 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.518 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.518 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.518 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125619 2024-09-15T00:13:11.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.518 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:11.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:11.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839660+0000", 2024-09-15T00:13:11.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.22e-06 2024-09-15T00:13:11.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840511+0000", 2024-09-15T00:13:11.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000851284 2024-09-15T00:13:11.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840522+0000", 2024-09-15T00:13:11.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0569e-05 2024-09-15T00:13:11.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840527+0000", 2024-09-15T00:13:11.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.412e-06 2024-09-15T00:13:11.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840573+0000", 2024-09-15T00:13:11.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5247000000000002e-05 2024-09-15T00:13:11.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840682+0000", 2024-09-15T00:13:11.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109368 2024-09-15T00:13:11.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:11.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843976+0000", 2024-09-15T00:13:11.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294445 2024-09-15T00:13:11.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:11.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870833+0000", 2024-09-15T00:13:11.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026856944000000001 2024-09-15T00:13:11.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:11.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885344+0000", 2024-09-15T00:13:11.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014510744000000001 2024-09-15T00:13:11.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:11.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885382+0000", 2024-09-15T00:13:11.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7821000000000003e-05 2024-09-15T00:13:11.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885428+0000", 2024-09-15T00:13:11.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5816000000000003e-05 2024-09-15T00:13:11.523 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.524 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.524 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.524 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125621 3.a 3:53594db9:::rbd_data.11396481e1ec.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:11.524 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:11.524 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 49.650079998999999, 2024-09-15T00:13:11.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032481797, 2024-09-15T00:13:11.524 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.524 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.524 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:11.524 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.525 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.525 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.525 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125621 2024-09-15T00:13:11.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.525 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:11.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:11.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853185+0000", 2024-09-15T00:13:11.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.43e-06 2024-09-15T00:13:11.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854125+0000", 2024-09-15T00:13:11.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000940539 2024-09-15T00:13:11.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854141+0000", 2024-09-15T00:13:11.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5568000000000001e-05 2024-09-15T00:13:11.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854148+0000", 2024-09-15T00:13:11.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6529999999999994e-06 2024-09-15T00:13:11.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870872+0000", 2024-09-15T00:13:11.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016723135 2024-09-15T00:13:11.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870975+0000", 2024-09-15T00:13:11.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010314099999999999 2024-09-15T00:13:11.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:11.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.873082+0000", 2024-09-15T00:13:11.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021069859999999999 2024-09-15T00:13:11.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:11.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885439+0000", 2024-09-15T00:13:11.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012357343999999999 2024-09-15T00:13:11.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:11.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885569+0000", 2024-09-15T00:13:11.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012995199999999999 2024-09-15T00:13:11.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:11.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885624+0000", 2024-09-15T00:13:11.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5281999999999997e-05 2024-09-15T00:13:11.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885664+0000", 2024-09-15T00:13:11.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9767000000000001e-05 2024-09-15T00:13:11.529 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.529 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.530 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.530 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:147075 3.0 3:06ab39f2:::rbd_data.11394501364.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:11.530 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:11.530 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 26.554181618000001, 2024-09-15T00:13:11.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046402906000000001, 2024-09-15T00:13:11.530 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.530 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.530 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:11.530 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.530 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.530 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.530 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 147075 2024-09-15T00:13:11.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.531 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:11.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:11.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:11.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4399999999999999e-07 2024-09-15T00:13:11.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949086+0000", 2024-09-15T00:13:11.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.138e-06 2024-09-15T00:13:11.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949096+0000", 2024-09-15T00:13:11.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0438999999999999e-05 2024-09-15T00:13:11.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949101+0000", 2024-09-15T00:13:11.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8999999999999997e-06 2024-09-15T00:13:11.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949145+0000", 2024-09-15T00:13:11.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4141000000000003e-05 2024-09-15T00:13:11.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949195+0000", 2024-09-15T00:13:11.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9657999999999998e-05 2024-09-15T00:13:11.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:11.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.950923+0000", 2024-09-15T00:13:11.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001728753 2024-09-15T00:13:11.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:11.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995390+0000", 2024-09-15T00:13:11.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044466335000000003 2024-09-15T00:13:11.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:11.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995401+0000", 2024-09-15T00:13:11.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1671000000000001e-05 2024-09-15T00:13:11.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:11.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995432+0000", 2024-09-15T00:13:11.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0704999999999998e-05 2024-09-15T00:13:11.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995484+0000", 2024-09-15T00:13:11.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1422000000000003e-05 2024-09-15T00:13:11.535 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.535 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.535 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.536 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:147075 3.0 e98/94 ondisk, result = 0)", 2024-09-15T00:13:11.536 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:11.536 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 26.546381897, 2024-09-15T00:13:11.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038514459000000001, 2024-09-15T00:13:11.536 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.536 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.536 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:11.536 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:11.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:11.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956881+0000", 2024-09-15T00:13:11.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0599999999999999e-07 2024-09-15T00:13:11.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956885+0000", 2024-09-15T00:13:11.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3500000000000001e-06 2024-09-15T00:13:11.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956897+0000", 2024-09-15T00:13:11.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2303999999999999e-05 2024-09-15T00:13:11.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956902+0000", 2024-09-15T00:13:11.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5169999999999999e-06 2024-09-15T00:13:11.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995377+0000", 2024-09-15T00:13:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038474544999999999 2024-09-15T00:13:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995385+0000", 2024-09-15T00:13:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6240000000000002e-06 2024-09-15T00:13:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995395+0000", 2024-09-15T00:13:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0213e-05 2024-09-15T00:13:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.540 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153172 3.14 3:2a7843e3:::rbd_data.113958cb0e55.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 983040~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:11.540 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:11.540 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 21.725492833000001, 2024-09-15T00:13:11.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.033232668, 2024-09-15T00:13:11.540 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.540 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.540 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:11.540 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.540 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.540 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.540 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153172 2024-09-15T00:13:11.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.541 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:11.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:11.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777770+0000", 2024-09-15T00:13:11.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.017e-06 2024-09-15T00:13:11.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788009+0000", 2024-09-15T00:13:11.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010238146 2024-09-15T00:13:11.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788024+0000", 2024-09-15T00:13:11.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5582000000000001e-05 2024-09-15T00:13:11.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788048+0000", 2024-09-15T00:13:11.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4173e-05 2024-09-15T00:13:11.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788097+0000", 2024-09-15T00:13:11.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8498999999999998e-05 2024-09-15T00:13:11.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788201+0000", 2024-09-15T00:13:11.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010398799999999999 2024-09-15T00:13:11.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:11.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788393+0000", 2024-09-15T00:13:11.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000192337 2024-09-15T00:13:11.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:11.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810637+0000", 2024-09-15T00:13:11.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.022244202000000001 2024-09-15T00:13:11.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:11.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810903+0000", 2024-09-15T00:13:11.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00026574600000000003 2024-09-15T00:13:11.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:11.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810953+0000", 2024-09-15T00:13:11.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9509000000000003e-05 2024-09-15T00:13:11.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.811002+0000", 2024-09-15T00:13:11.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9468999999999997e-05 2024-09-15T00:13:11.546 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.546 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.546 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.546 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170014 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:11.546 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:11.546 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.656197727, 2024-09-15T00:13:11.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030291116999999999, 2024-09-15T00:13:11.546 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.546 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.546 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:11.546 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.546 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.547 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.547 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170014 2024-09-15T00:13:11.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.547 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:11.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:11.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:11.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:11.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847073+0000", 2024-09-15T00:13:11.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9589999999999995e-06 2024-09-15T00:13:11.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847080+0000", 2024-09-15T00:13:11.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2930000000000003e-06 2024-09-15T00:13:11.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847086+0000", 2024-09-15T00:13:11.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.631e-06 2024-09-15T00:13:11.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877213+0000", 2024-09-15T00:13:11.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030127017999999998 2024-09-15T00:13:11.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877281+0000", 2024-09-15T00:13:11.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7916999999999994e-05 2024-09-15T00:13:11.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877356+0000", 2024-09-15T00:13:11.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4796999999999996e-05 2024-09-15T00:13:11.550 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.550 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.550 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.550 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170017 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1310720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:11.551 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:11.551 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.656169075999999, 2024-09-15T00:13:11.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030437623, 2024-09-15T00:13:11.551 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.551 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.551 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:11.551 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.551 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.551 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.551 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170017 2024-09-15T00:13:11.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.551 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:11.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:11.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847094+0000", 2024-09-15T00:13:11.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:13:11.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847096+0000", 2024-09-15T00:13:11.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.136e-06 2024-09-15T00:13:11.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847104+0000", 2024-09-15T00:13:11.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7190000000000001e-06 2024-09-15T00:13:11.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847109+0000", 2024-09-15T00:13:11.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3279999999999996e-06 2024-09-15T00:13:11.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877384+0000", 2024-09-15T00:13:11.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030275348000000001 2024-09-15T00:13:11.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877451+0000", 2024-09-15T00:13:11.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6509999999999998e-05 2024-09-15T00:13:11.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877531+0000", 2024-09-15T00:13:11.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9993000000000005e-05 2024-09-15T00:13:11.555 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.555 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.555 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.555 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170018 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1351680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:11.555 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:11.555 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.656146731, 2024-09-15T00:13:11.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030591884999999999, 2024-09-15T00:13:11.555 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.555 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.555 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:11.556 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.556 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.556 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.556 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170018 2024-09-15T00:13:11.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.556 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:11.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:11.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:11.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:13:11.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847118+0000", 2024-09-15T00:13:11.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.03e-06 2024-09-15T00:13:11.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847124+0000", 2024-09-15T00:13:11.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:11.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847129+0000", 2024-09-15T00:13:11.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8600000000000001e-06 2024-09-15T00:13:11.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877560+0000", 2024-09-15T00:13:11.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030431145999999999 2024-09-15T00:13:11.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877629+0000", 2024-09-15T00:13:11.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8793000000000004e-05 2024-09-15T00:13:11.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877707+0000", 2024-09-15T00:13:11.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8139999999999994e-05 2024-09-15T00:13:11.559 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.559 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.559 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.559 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170019 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1392640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:11.559 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:11.560 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.656125983000001, 2024-09-15T00:13:11.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030769023, 2024-09-15T00:13:11.560 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.560 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.560 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:11.560 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.560 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.560 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.560 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170019 2024-09-15T00:13:11.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.560 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:11.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:11.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847137+0000", 2024-09-15T00:13:11.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:13:11.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847139+0000", 2024-09-15T00:13:11.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.142e-06 2024-09-15T00:13:11.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847146+0000", 2024-09-15T00:13:11.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8120000000000002e-06 2024-09-15T00:13:11.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847157+0000", 2024-09-15T00:13:11.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1093e-05 2024-09-15T00:13:11.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877735+0000", 2024-09-15T00:13:11.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578195999999998 2024-09-15T00:13:11.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877810+0000", 2024-09-15T00:13:11.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5563999999999997e-05 2024-09-15T00:13:11.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877905+0000", 2024-09-15T00:13:11.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4809000000000007e-05 2024-09-15T00:13:11.563 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.563 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.564 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.564 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170020 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:11.564 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:11.564 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.656098742999999, 2024-09-15T00:13:11.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030917150000000001, 2024-09-15T00:13:11.564 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.564 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.564 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:11.564 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.564 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.564 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.565 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170020 2024-09-15T00:13:11.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.565 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:11.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:11.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:11.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:13:11.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847166+0000", 2024-09-15T00:13:11.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2790000000000001e-06 2024-09-15T00:13:11.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847173+0000", 2024-09-15T00:13:11.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3969999999999999e-06 2024-09-15T00:13:11.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847177+0000", 2024-09-15T00:13:11.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8620000000000002e-06 2024-09-15T00:13:11.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877934+0000", 2024-09-15T00:13:11.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030756143999999999 2024-09-15T00:13:11.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878002+0000", 2024-09-15T00:13:11.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7951e-05 2024-09-15T00:13:11.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878081+0000", 2024-09-15T00:13:11.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9087999999999999e-05 2024-09-15T00:13:11.568 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.568 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.568 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.568 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170094 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:11.568 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:11.568 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.655841494000001, 2024-09-15T00:13:11.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030836142, 2024-09-15T00:13:11.568 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.569 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.569 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:11.569 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.569 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.569 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.569 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170094 2024-09-15T00:13:11.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.569 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:11.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:11.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:11.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3499999999999996e-07 2024-09-15T00:13:11.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847424+0000", 2024-09-15T00:13:11.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2850000000000001e-06 2024-09-15T00:13:11.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847430+0000", 2024-09-15T00:13:11.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3260000000000001e-06 2024-09-15T00:13:11.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847435+0000", 2024-09-15T00:13:11.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.356e-06 2024-09-15T00:13:11.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878110+0000", 2024-09-15T00:13:11.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030675021 2024-09-15T00:13:11.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878177+0000", 2024-09-15T00:13:11.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6506000000000003e-05 2024-09-15T00:13:11.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878257+0000", 2024-09-15T00:13:11.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0012999999999994e-05 2024-09-15T00:13:11.572 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.572 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.572 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.573 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170096 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 458752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:11.573 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:11.573 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.655799538, 2024-09-15T00:13:11.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030977030999999999, 2024-09-15T00:13:11.573 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.573 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.573 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:11.573 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.573 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.573 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.573 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170096 2024-09-15T00:13:11.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.573 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:11.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:11.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:11.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:11.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847465+0000", 2024-09-15T00:13:11.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0559999999999999e-06 2024-09-15T00:13:11.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847472+0000", 2024-09-15T00:13:11.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3130000000000004e-06 2024-09-15T00:13:11.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847477+0000", 2024-09-15T00:13:11.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9930000000000003e-06 2024-09-15T00:13:11.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878288+0000", 2024-09-15T00:13:11.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030811609 2024-09-15T00:13:11.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878358+0000", 2024-09-15T00:13:11.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0215000000000005e-05 2024-09-15T00:13:11.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878440+0000", 2024-09-15T00:13:11.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1366999999999995e-05 2024-09-15T00:13:11.577 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.577 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.577 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.577 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170097 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:11.577 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:11.577 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.655779452999999, 2024-09-15T00:13:11.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031134757999999998, 2024-09-15T00:13:11.577 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.577 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.577 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:11.577 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.578 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.578 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.578 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170097 2024-09-15T00:13:11.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.578 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:11.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:11.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:11.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:13:11.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847485+0000", 2024-09-15T00:13:11.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9190000000000002e-06 2024-09-15T00:13:11.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847492+0000", 2024-09-15T00:13:11.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7109999999999998e-06 2024-09-15T00:13:11.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847497+0000", 2024-09-15T00:13:11.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0259999999999998e-06 2024-09-15T00:13:11.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878469+0000", 2024-09-15T00:13:11.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030972115000000001 2024-09-15T00:13:11.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878538+0000", 2024-09-15T00:13:11.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8669000000000003e-05 2024-09-15T00:13:11.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878618+0000", 2024-09-15T00:13:11.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9874000000000001e-05 2024-09-15T00:13:11.581 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.581 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.581 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.581 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170099 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 532480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:11.581 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:11.581 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.655727891, 2024-09-15T00:13:11.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031262183999999998, 2024-09-15T00:13:11.582 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.582 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.582 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:11.582 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.582 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.582 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.582 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170099 2024-09-15T00:13:11.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.582 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847535+0000", 2024-09-15T00:13:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:13:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847537+0000", 2024-09-15T00:13:11.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:11.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847544+0000", 2024-09-15T00:13:11.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9630000000000001e-06 2024-09-15T00:13:11.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847549+0000", 2024-09-15T00:13:11.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8910000000000003e-06 2024-09-15T00:13:11.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878646+0000", 2024-09-15T00:13:11.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031097400000000001 2024-09-15T00:13:11.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878717+0000", 2024-09-15T00:13:11.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0993000000000003e-05 2024-09-15T00:13:11.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878797+0000", 2024-09-15T00:13:11.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9425000000000006e-05 2024-09-15T00:13:11.585 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.586 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.586 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.586 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170101 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:11.586 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:11.586 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.655686512000001, 2024-09-15T00:13:11.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031674568, 2024-09-15T00:13:11.586 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.586 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.586 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:11.586 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.586 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.586 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.586 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170101 2024-09-15T00:13:11.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.587 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:11.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:11.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:11.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:13:11.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847578+0000", 2024-09-15T00:13:11.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0389999999999999e-06 2024-09-15T00:13:11.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847585+0000", 2024-09-15T00:13:11.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.336e-06 2024-09-15T00:13:11.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847596+0000", 2024-09-15T00:13:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015999999999999e-05 2024-09-15T00:13:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878825+0000", 2024-09-15T00:13:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031229848000000001 2024-09-15T00:13:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878899+0000", 2024-09-15T00:13:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879250+0000", 2024-09-15T00:13:11.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035110000000000002 2024-09-15T00:13:11.590 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.590 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.590 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.590 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170102 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 614400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:11.590 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:11.590 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.655659215, 2024-09-15T00:13:11.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031577371999999999, 2024-09-15T00:13:11.590 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.590 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.590 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:11.590 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.590 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.591 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.591 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170102 2024-09-15T00:13:11.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.591 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:11.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:11.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:11.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:13:11.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847610+0000", 2024-09-15T00:13:11.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6329999999999999e-06 2024-09-15T00:13:11.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847618+0000", 2024-09-15T00:13:11.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4599999999999997e-06 2024-09-15T00:13:11.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847623+0000", 2024-09-15T00:13:11.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2909999999999998e-06 2024-09-15T00:13:11.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879016+0000", 2024-09-15T00:13:11.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031393227000000003 2024-09-15T00:13:11.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879091+0000", 2024-09-15T00:13:11.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5384999999999997e-05 2024-09-15T00:13:11.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879180+0000", 2024-09-15T00:13:11.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8941000000000001e-05 2024-09-15T00:13:11.594 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.594 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.594 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.595 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170132 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:11.595 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:11.595 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.655611835, 2024-09-15T00:13:11.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031722579000000001, 2024-09-15T00:13:11.595 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.595 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.595 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:11.595 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.595 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.595 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.595 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170132 2024-09-15T00:13:11.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.595 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:11.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:11.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847651+0000", 2024-09-15T00:13:11.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1900000000000003e-07 2024-09-15T00:13:11.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847653+0000", 2024-09-15T00:13:11.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0329999999999998e-06 2024-09-15T00:13:11.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847659+0000", 2024-09-15T00:13:11.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4169999999999997e-06 2024-09-15T00:13:11.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847670+0000", 2024-09-15T00:13:11.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1031e-05 2024-09-15T00:13:11.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879205+0000", 2024-09-15T00:13:11.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031534982000000003 2024-09-15T00:13:11.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879286+0000", 2024-09-15T00:13:11.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0675999999999998e-05 2024-09-15T00:13:11.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879373+0000", 2024-09-15T00:13:11.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6921000000000003e-05 2024-09-15T00:13:11.599 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.599 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.599 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.599 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170133 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:11.599 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:11.599 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.655584682000001, 2024-09-15T00:13:11.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031894817999999998, 2024-09-15T00:13:11.599 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:11.599 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:11.599 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:11.600 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:11.600 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:11.600 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:11.600 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170133 2024-09-15T00:13:11.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.600 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:11.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:11.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:11.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:11.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:11.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:11.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:11.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853540+0000", 2024-09-15T00:13:11.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058619409999999999 2024-09-15T00:13:11.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:11.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853545+0000", 2024-09-15T00:13:11.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3430000000000003e-06 2024-09-15T00:13:11.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:11.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853554+0000", 2024-09-15T00:13:11.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4069999999999999e-06 2024-09-15T00:13:11.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:11.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853561+0000", 2024-09-15T00:13:11.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5529999999999998e-06 2024-09-15T00:13:11.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:11.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879399+0000", 2024-09-15T00:13:11.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025837941 2024-09-15T00:13:11.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:11.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879477+0000", 2024-09-15T00:13:11.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8367000000000004e-05 2024-09-15T00:13:11.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:11.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:11.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:11.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879572+0000", 2024-09-15T00:13:11.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5266000000000006e-05 2024-09-15T00:13:11.603 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.603 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.603 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.603 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:11.603 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:11.603 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:11.605 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:11.605 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:11.602+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:11.706 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:11.703+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:11.748 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:11.748 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:11.748 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:11.749 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:11.757 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:11.806 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:11.804+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:11.899 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:11.899 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:11.899 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:11.899 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:11.899 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:11.907 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:11.908 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:11.905+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:12.008 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:12.005+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:12.011 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:12.011 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:12.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:12.012 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:12.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.012 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:12.012 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:12.012 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.930736872, 2024-09-15T00:13:12.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:13:12.012 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.012 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.012 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:12.012 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.012 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.012 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.012 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:13:12.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.013 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:12.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:12.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:13:12.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:13:12.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:13:12.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:13:12.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:13:12.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:13:12.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:13:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:13:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:13:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:13:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:13:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:13:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:12.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:13:12.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:13:12.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:12.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:13:12.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:13:12.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:12.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:13:12.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:13:12.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:12.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:13:12.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:13:12.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:13:12.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:13:12.017 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.017 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.017 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.018 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:13:12.018 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:12.018 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.91799632999999, 2024-09-15T00:13:12.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:13:12.018 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.018 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.018 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:12.018 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:12.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:12.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:13:12.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:13:12.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:13:12.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:13:12.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:13:12.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:13:12.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:13:12.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:13:12.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:13:12.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:13:12.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:13:12.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:13:12.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:13:12.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:13:12.021 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.021 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.021 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.022 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:12.022 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:12.022 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.785640571, 2024-09-15T00:13:12.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:13:12.022 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.022 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.022 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:12.022 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.022 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.022 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.022 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:13:12.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.022 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:12.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:12.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:13:12.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:13:12.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:13:12.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:13:12.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:13:12.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:13:12.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:13:12.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:13:12.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:13:12.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:13:12.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:13:12.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:13:12.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:12.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:13:12.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:13:12.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:12.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:13:12.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:13:12.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:12.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:13:12.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:13:12.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:12.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:13:12.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:13:12.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:13:12.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:13:12.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.027 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.027 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:13:12.028 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:12.028 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.772178926, 2024-09-15T00:13:12.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:13:12.028 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.028 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.028 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:12.028 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:12.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:12.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:13:12.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:13:12.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:13:12.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:13:12.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:13:12.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:13:12.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:13:12.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:13:12.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:13:12.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:13:12.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:13:12.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:13:12.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:13:12.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:13:12.031 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.031 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.031 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.032 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:13:12.032 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:12.032 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.587177443000002, 2024-09-15T00:13:12.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:13:12.032 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.032 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.032 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:12.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.032 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:13:12.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.032 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:12.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:12.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:13:12.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:13:12.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:13:12.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:13:12.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:13:12.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:13:12.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:13:12.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:13:12.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:13:12.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:13:12.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:13:12.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:13:12.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:12.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:13:12.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:13:12.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:12.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:13:12.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:13:12.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:12.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:13:12.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:13:12.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:12.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:13:12.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:13:12.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:13:12.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:13:12.037 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.037 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.037 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.038 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:13:12.038 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:12.038 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.574301176000006, 2024-09-15T00:13:12.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:13:12.038 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.038 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.038 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:12.038 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:12.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:12.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:13:12.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:13:12.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:13:12.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:13:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:13:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:13:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:13:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:13:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:13:12.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:13:12.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:13:12.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:13:12.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:13:12.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:13:12.041 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.041 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.041 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.042 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:13:12.042 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:12.042 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 91.115786032000003, 2024-09-15T00:13:12.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:13:12.042 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.042 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.042 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:12.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.042 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:13:12.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.042 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:12.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:12.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:13:12.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:13:12.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:13:12.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:13:12.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:13:12.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:13:12.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:13:12.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:13:12.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:13:12.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:13:12.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:13:12.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:13:12.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:12.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:13:12.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:13:12.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:12.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:13:12.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:13:12.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:12.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:13:12.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:13:12.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:12.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:13:12.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:13:12.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:13:12.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:13:12.047 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.047 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.047 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.047 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:12.047 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:12.047 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.398275471999995, 2024-09-15T00:13:12.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:13:12.048 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.048 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.048 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:12.048 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.048 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.048 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.048 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:13:12.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.048 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:12.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:12.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:13:12.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:13:12.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:13:12.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:13:12.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:13:12.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:13:12.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:13:12.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:13:12.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:13:12.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:13:12.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:13:12.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:13:12.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:13:12.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:13:12.051 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.051 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.051 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.052 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:12.052 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:12.052 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.398256863, 2024-09-15T00:13:12.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:13:12.052 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.052 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.052 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:12.052 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.052 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.052 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.052 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:13:12.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.052 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:12.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:12.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:12.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:13:12.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:13:12.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:13:12.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:13:12.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:13:12.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:13:12.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:13:12.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:13:12.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:13:12.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:13:12.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:13:12.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:13:12.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:13:12.056 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.056 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.056 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.056 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:12.056 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:12.056 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.398231538999994, 2024-09-15T00:13:12.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:13:12.056 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.056 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.056 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:12.056 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.056 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.057 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.057 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:13:12.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.057 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:12.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:12.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:12.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:13:12.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:13:12.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:13:12.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:13:12.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:13:12.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:13:12.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:13:12.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:13:12.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:13:12.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:13:12.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:13:12.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:13:12.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:13:12.060 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.060 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.060 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.060 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:12.060 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:12.060 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.398213214999998, 2024-09-15T00:13:12.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:13:12.061 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.061 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.061 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:12.061 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.061 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.061 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.061 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:13:12.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.061 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:12.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:12.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:13:12.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:13:12.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:13:12.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:13:12.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:13:12.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:12.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:13:12.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:13:12.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:13:12.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:13:12.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:13:12.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:13:12.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:13:12.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:13:12.064 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.064 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.065 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.065 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:12.065 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:12.065 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.398030641999995, 2024-09-15T00:13:12.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:13:12.065 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.065 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.065 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:12.065 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.065 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.065 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.065 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:13:12.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.066 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:12.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:12.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:12.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:13:12.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:13:12.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:13:12.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:13:12.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:13:12.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:13:12.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:13:12.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:13:12.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:13:12.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:13:12.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:13:12.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:13:12.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:13:12.069 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.069 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.069 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.069 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:12.069 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:12.069 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.398001801000007, 2024-09-15T00:13:12.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:13:12.069 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.069 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.070 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:12.070 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.070 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.070 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.070 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:13:12.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.070 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:12.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:13:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:13:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:13:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:13:12.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:13:12.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:13:12.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:13:12.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:13:12.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:13:12.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:13:12.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:13:12.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:13:12.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:13:12.073 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.073 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.073 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.073 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:12.074 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:12.074 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.397979590000006, 2024-09-15T00:13:12.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:13:12.074 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.074 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.074 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:12.074 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.074 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.074 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.074 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:13:12.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.074 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:12.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:12.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:12.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:13:12.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:13:12.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:13:12.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:13:12.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:13:12.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:13:12.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:13:12.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:13:12.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:13:12.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:13:12.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:13:12.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:13:12.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:13:12.077 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.078 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.078 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.078 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:12.078 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:12.078 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.397958431000006, 2024-09-15T00:13:12.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:13:12.078 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.078 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.078 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:12.078 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.078 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.078 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.078 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:13:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:12.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:13:12.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:13:12.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:13:12.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:13:12.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:13:12.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:13:12.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:13:12.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:13:12.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:13:12.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:13:12.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:13:12.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:13:12.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:13:12.082 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.082 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.082 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.082 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:12.082 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:12.082 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.397928483000001, 2024-09-15T00:13:12.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:13:12.082 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.082 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.082 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:12.083 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.083 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.083 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.083 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:13:12.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.083 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:12.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:12.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:12.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:13:12.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:13:12.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:13:12.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:13:12.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:13:12.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:13:12.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:13:12.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:13:12.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:13:12.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:13:12.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:13:12.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:13:12.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:13:12.086 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.086 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.086 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.086 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:12.086 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:12.087 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.397906519000003, 2024-09-15T00:13:12.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:13:12.087 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.087 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.087 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:12.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.087 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:13:12.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.087 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:12.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:12.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:12.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:13:12.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:13:12.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:13:12.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:13:12.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:13:12.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:13:12.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:13:12.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:13:12.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:13:12.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:13:12.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:13:12.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:13:12.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:13:12.091 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.091 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.091 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.091 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:12.091 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:12.091 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.397889092, 2024-09-15T00:13:12.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:13:12.091 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.091 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.091 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:12.091 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.092 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.092 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.092 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:13:12.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.092 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:12.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:12.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:13:12.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:13:12.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:13:12.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:13:12.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:13:12.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:12.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:13:12.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:13:12.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:13:12.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:13:12.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:13:12.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:13:12.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:13:12.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:13:12.096 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.096 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.096 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.096 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:12.096 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:12.096 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.397867293000004, 2024-09-15T00:13:12.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:13:12.096 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.096 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.096 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:12.096 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.096 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.097 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:13:12.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.097 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:12.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:12.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:13:12.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:13:12.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:13:12.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:13:12.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:13:12.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:13:12.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:13:12.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:13:12.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:13:12.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:13:12.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:13:12.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:13:12.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:13:12.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:13:12.101 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.101 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.101 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.101 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:12.101 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:12.101 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.348638933999993, 2024-09-15T00:13:12.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:13:12.101 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:12.101 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:12.102 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:12.102 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:12.102 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:12.102 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.102 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:13:12.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.102 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:12.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:12.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:12.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:12.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:12.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:12.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:12.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:13:12.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:13:12.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:12.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:13:12.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:13:12.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:12.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:13:12.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:13:12.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:13:12.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:13:12.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:12.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:13:12.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:13:12.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:12.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:13:12.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:13:12.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:12.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:13:12.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:13:12.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:12.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:13:12.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:13:12.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:12.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:13:12.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:13:12.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:12.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:13:12.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:13:12.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:12.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:12.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:12.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:13:12.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:13:12.107 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.107 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.107 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.107 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:12.107 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:12.108 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:12.109 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:12.109 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:12.107+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:12.195 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is unset 2024-09-15T00:13:12.209 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:12.207+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:12.225 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:12.225 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:12.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGRecoveryDeleteReply(3.13 e124,123 3:cec5c353:::rbd_data.1139b531b717.0000000000000027:head,33'32)", 2024-09-15T00:13:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:12.224466+0000", 2024-09-15T00:13:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00011075, 2024-09-15T00:13:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012626200000000001, 2024-09-15T00:13:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:12.224466+0000", 2024-09-15T00:13:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:12.224466+0000", 2024-09-15T00:13:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:12.224466+0000", 2024-09-15T00:13:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:13:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:12.224469+0000", 2024-09-15T00:13:12.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1329999999999999e-06 2024-09-15T00:13:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:12.224476+0000", 2024-09-15T00:13:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1319999999999999e-06 2024-09-15T00:13:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:12.224479+0000", 2024-09-15T00:13:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7069999999999998e-06 2024-09-15T00:13:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:12.224507+0000", 2024-09-15T00:13:12.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8198999999999999e-05 2024-09-15T00:13:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:13:12.229 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 1 2024-09-15T00:13:12.229 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:12.235 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:12.319 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:12.317+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:12.399 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:12.399 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:12.399 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:12.399 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:12.399 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:12.410 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:12.452 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:12.449+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:12.512 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:12.510+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:12.571 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:12.571 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:12.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:12.571 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:12.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.571 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9, version: 37'47, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9@37'47, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:12.571 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:12.572 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.7146871150000003, 2024-09-15T00:13:12.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054905559999999997, 2024-09-15T00:13:12.572 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.572 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.572 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.572 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:12.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:12.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853793+0000", 2024-09-15T00:13:12.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9599999999999999e-07 2024-09-15T00:13:12.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854798+0000", 2024-09-15T00:13:12.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010050770000000001 2024-09-15T00:13:12.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854814+0000", 2024-09-15T00:13:12.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6478999999999999e-05 2024-09-15T00:13:12.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854825+0000", 2024-09-15T00:13:12.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0492999999999999e-05 2024-09-15T00:13:12.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854889+0000", 2024-09-15T00:13:12.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4134999999999995e-05 2024-09-15T00:13:12.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854898+0000", 2024-09-15T00:13:12.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6260000000000001e-06 2024-09-15T00:13:12.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.859283+0000", 2024-09-15T00:13:12.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043852500000000003 2024-09-15T00:13:12.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.575 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.576 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15, version: 59'205, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15@59'205, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:12.576 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:12.576 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.5884666599999999, 2024-09-15T00:13:12.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059047939999999997, 2024-09-15T00:13:12.576 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.576 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.576 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.576 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:12.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:12.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:12.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.44e-07 2024-09-15T00:13:12.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980977+0000", 2024-09-15T00:13:12.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00096402300000000005 2024-09-15T00:13:12.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980995+0000", 2024-09-15T00:13:12.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7214000000000002e-05 2024-09-15T00:13:12.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981007+0000", 2024-09-15T00:13:12.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2028000000000001e-05 2024-09-15T00:13:12.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981069+0000", 2024-09-15T00:13:12.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1977000000000006e-05 2024-09-15T00:13:12.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981077+0000", 2024-09-15T00:13:12.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2120000000000003e-06 2024-09-15T00:13:12.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.985918+0000", 2024-09-15T00:13:12.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048406960000000002 2024-09-15T00:13:12.579 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.580 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.580 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.580 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head, version: 62'221, data_included: [0~262144,393216~786432], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head@62'221, size: 1572864, copy_subset: [(0, 1572864)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:12.580 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:12.580 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.5661983949999998, 2024-09-15T00:13:12.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054170770000000002, 2024-09-15T00:13:12.580 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.580 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.580 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.580 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:12.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:12.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002282+0000", 2024-09-15T00:13:12.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4499999999999997e-07 2024-09-15T00:13:12.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003196+0000", 2024-09-15T00:13:12.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091476399999999999 2024-09-15T00:13:12.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003213+0000", 2024-09-15T00:13:12.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6554000000000001e-05 2024-09-15T00:13:12.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003230+0000", 2024-09-15T00:13:12.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6617999999999999e-05 2024-09-15T00:13:12.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003292+0000", 2024-09-15T00:13:12.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2193999999999994e-05 2024-09-15T00:13:12.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003300+0000", 2024-09-15T00:13:12.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1429999999999998e-06 2024-09-15T00:13:12.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.007698+0000", 2024-09-15T00:13:12.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043981589999999996 2024-09-15T00:13:12.584 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.584 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.584 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.584 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:12.584 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:12.584 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.5154683149999997, 2024-09-15T00:13:12.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067633270000000004, 2024-09-15T00:13:12.584 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.584 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.584 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.584 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:12.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:12.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053012+0000", 2024-09-15T00:13:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1659999999999999e-06 2024-09-15T00:13:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054086+0000", 2024-09-15T00:13:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010732789999999999 2024-09-15T00:13:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054100+0000", 2024-09-15T00:13:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4123e-05 2024-09-15T00:13:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054114+0000", 2024-09-15T00:13:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4442e-05 2024-09-15T00:13:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054192+0000", 2024-09-15T00:13:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8288000000000007e-05 2024-09-15T00:13:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054200+0000", 2024-09-15T00:13:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-15T00:13:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.059774+0000", 2024-09-15T00:13:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055741230000000003 2024-09-15T00:13:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.5071509870000002, 2024-09-15T00:13:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049777950000000001, 2024-09-15T00:13:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.588 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061330+0000", 2024-09-15T00:13:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.11e-06 2024-09-15T00:13:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062350+0000", 2024-09-15T00:13:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010207969999999999 2024-09-15T00:13:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062365+0000", 2024-09-15T00:13:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4157e-05 2024-09-15T00:13:12.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062380+0000", 2024-09-15T00:13:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5129e-05 2024-09-15T00:13:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062467+0000", 2024-09-15T00:13:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7247e-05 2024-09-15T00:13:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062475+0000", 2024-09-15T00:13:12.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4300000000000006e-06 2024-09-15T00:13:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.066306+0000", 2024-09-15T00:13:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038309250000000002 2024-09-15T00:13:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:12.592 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.5012680290000002, 2024-09-15T00:13:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052454889999999999, 2024-09-15T00:13:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:12.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067212+0000", 2024-09-15T00:13:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-07 2024-09-15T00:13:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068119+0000", 2024-09-15T00:13:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090671199999999995 2024-09-15T00:13:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068135+0000", 2024-09-15T00:13:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5739e-05 2024-09-15T00:13:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068150+0000", 2024-09-15T00:13:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5693e-05 2024-09-15T00:13:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068236+0000", 2024-09-15T00:13:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6100000000000006e-05 2024-09-15T00:13:12.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068245+0000", 2024-09-15T00:13:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1960000000000008e-06 2024-09-15T00:13:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.072457+0000", 2024-09-15T00:13:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042123719999999998 2024-09-15T00:13:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.49528968, 2024-09-15T00:13:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049532020000000003, 2024-09-15T00:13:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:13:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074164+0000", 2024-09-15T00:13:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097354400000000002 2024-09-15T00:13:12.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074178+0000", 2024-09-15T00:13:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4134999999999999e-05 2024-09-15T00:13:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074192+0000", 2024-09-15T00:13:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.451e-05 2024-09-15T00:13:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074272+0000", 2024-09-15T00:13:12.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9989999999999998e-05 2024-09-15T00:13:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074280+0000", 2024-09-15T00:13:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9990000000000008e-06 2024-09-15T00:13:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078143+0000", 2024-09-15T00:13:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00386258 2024-09-15T00:13:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [4194304~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.4895788679999997, 2024-09-15T00:13:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049408730000000001, 2024-09-15T00:13:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2399999999999999e-07 2024-09-15T00:13:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079875+0000", 2024-09-15T00:13:12.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097349100000000005 2024-09-15T00:13:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079889+0000", 2024-09-15T00:13:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4148000000000001e-05 2024-09-15T00:13:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079904+0000", 2024-09-15T00:13:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5159e-05 2024-09-15T00:13:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079964+0000", 2024-09-15T00:13:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0204999999999997e-05 2024-09-15T00:13:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079971+0000", 2024-09-15T00:13:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2490000000000004e-06 2024-09-15T00:13:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.083841+0000", 2024-09-15T00:13:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038701970000000001 2024-09-15T00:13:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.604 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [5242880~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 6291456, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.4838738879999998, 2024-09-15T00:13:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052525740000000003, 2024-09-15T00:13:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:13:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085579+0000", 2024-09-15T00:13:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097307600000000002 2024-09-15T00:13:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085592+0000", 2024-09-15T00:13:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3277e-05 2024-09-15T00:13:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085608+0000", 2024-09-15T00:13:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5773e-05 2024-09-15T00:13:12.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085687+0000", 2024-09-15T00:13:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8956999999999996e-05 2024-09-15T00:13:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085694+0000", 2024-09-15T00:13:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.956e-06 2024-09-15T00:13:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.089858+0000", 2024-09-15T00:13:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041641159999999998 2024-09-15T00:13:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:172552 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-15T00:13:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.37941658, 2024-09-15T00:13:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.997031471, 2024-09-15T00:13:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.609 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 172552 2024-09-15T00:13:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189064+0000", 2024-09-15T00:13:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.198e-06 2024-09-15T00:13:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189069+0000", 2024-09-15T00:13:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9069999999999998e-06 2024-09-15T00:13:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189080+0000", 2024-09-15T00:13:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:13:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189086+0000", 2024-09-15T00:13:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5429999999999999e-06 2024-09-15T00:13:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189128+0000", 2024-09-15T00:13:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2255000000000001e-05 2024-09-15T00:13:12.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189135+0000", 2024-09-15T00:13:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6200000000000001e-06 2024-09-15T00:13:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185787+0000", 2024-09-15T00:13:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99665157199999999 2024-09-15T00:13:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185926+0000", 2024-09-15T00:13:12.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013970399999999999 2024-09-15T00:13:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.186094+0000", 2024-09-15T00:13:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016813099999999999 2024-09-15T00:13:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9, version: 37'60, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9@37'60, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.3205562210000004, 2024-09-15T00:13:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049454060000000003, 2024-09-15T00:13:12.614 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247924+0000", 2024-09-15T00:13:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:13:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248394+0000", 2024-09-15T00:13:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047063399999999999 2024-09-15T00:13:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248411+0000", 2024-09-15T00:13:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6384000000000001e-05 2024-09-15T00:13:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248441+0000", 2024-09-15T00:13:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0559999999999999e-05 2024-09-15T00:13:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248499+0000", 2024-09-15T00:13:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7274000000000001e-05 2024-09-15T00:13:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248507+0000", 2024-09-15T00:13:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2390000000000002e-06 2024-09-15T00:13:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.252869+0000", 2024-09-15T00:13:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043617200000000004 2024-09-15T00:13:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head, version: 59'393, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head@59'393, size: 5701632, copy_subset: [(0, 5701632)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.1911079539999996, 2024-09-15T00:13:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004807469, 2024-09-15T00:13:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.618 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:12.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4799999999999998e-07 2024-09-15T00:13:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377830+0000", 2024-09-15T00:13:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000457954 2024-09-15T00:13:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377847+0000", 2024-09-15T00:13:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6741000000000001e-05 2024-09-15T00:13:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377858+0000", 2024-09-15T00:13:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0951999999999999e-05 2024-09-15T00:13:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377929+0000", 2024-09-15T00:13:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0697000000000004e-05 2024-09-15T00:13:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377941+0000", 2024-09-15T00:13:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2529e-05 2024-09-15T00:13:12.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.382179+0000", 2024-09-15T00:13:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042379480000000001 2024-09-15T00:13:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head, version: 88'473, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head@88'473, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.1089885209999997, 2024-09-15T00:13:12.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005823272, 2024-09-15T00:13:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459492+0000", 2024-09-15T00:13:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9800000000000003e-07 2024-09-15T00:13:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459940+0000", 2024-09-15T00:13:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044854699999999999 2024-09-15T00:13:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459957+0000", 2024-09-15T00:13:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6586e-05 2024-09-15T00:13:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459968+0000", 2024-09-15T00:13:12.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.145e-05 2024-09-15T00:13:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460043+0000", 2024-09-15T00:13:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4451999999999998e-05 2024-09-15T00:13:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460057+0000", 2024-09-15T00:13:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.417e-05 2024-09-15T00:13:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.465314+0000", 2024-09-15T00:13:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052574689999999999 2024-09-15T00:13:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8738123399999997, 2024-09-15T00:13:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051811399999999999, 2024-09-15T00:13:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.627 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694668+0000", 2024-09-15T00:13:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4600000000000005e-07 2024-09-15T00:13:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695136+0000", 2024-09-15T00:13:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00046864399999999999 2024-09-15T00:13:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695153+0000", 2024-09-15T00:13:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6218e-05 2024-09-15T00:13:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695189+0000", 2024-09-15T00:13:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6393000000000002e-05 2024-09-15T00:13:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695263+0000", 2024-09-15T00:13:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4175e-05 2024-09-15T00:13:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695271+0000", 2024-09-15T00:13:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2800000000000003e-06 2024-09-15T00:13:12.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.699848+0000", 2024-09-15T00:13:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045768839999999998 2024-09-15T00:13:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8657141319999999, 2024-09-15T00:13:12.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048716510000000003, 2024-09-15T00:13:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:12.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702783+0000", 2024-09-15T00:13:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7268000000000001e-05 2024-09-15T00:13:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703221+0000", 2024-09-15T00:13:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00043866899999999998 2024-09-15T00:13:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703237+0000", 2024-09-15T00:13:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5553000000000002e-05 2024-09-15T00:13:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703247+0000", 2024-09-15T00:13:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0392e-05 2024-09-15T00:13:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703358+0000", 2024-09-15T00:13:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110676 2024-09-15T00:13:12.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703369+0000", 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0716e-05 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.707637+0000", 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042683770000000003 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18, version: 62'274, data_included: [0~786432], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18@62'274, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:12.635 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8196330500000002, 2024-09-15T00:13:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005453014, 2024-09-15T00:13:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748847+0000", 2024-09-15T00:13:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8299999999999997e-07 2024-09-15T00:13:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749010+0000", 2024-09-15T00:13:12.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000163144 2024-09-15T00:13:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749027+0000", 2024-09-15T00:13:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6518999999999999e-05 2024-09-15T00:13:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749037+0000", 2024-09-15T00:13:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0438000000000001e-05 2024-09-15T00:13:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749112+0000", 2024-09-15T00:13:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4946999999999999e-05 2024-09-15T00:13:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749123+0000", 2024-09-15T00:13:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1014e-05 2024-09-15T00:13:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.754299+0000", 2024-09-15T00:13:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051763690000000001 2024-09-15T00:13:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.639 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20, version: 80'334, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20@80'334, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 20=[]:{20: [20]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.7649960059999996, 2024-09-15T00:13:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048933570000000001, 2024-09-15T00:13:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803484+0000", 2024-09-15T00:13:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:13:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803938+0000", 2024-09-15T00:13:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045364599999999999 2024-09-15T00:13:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803954+0000", 2024-09-15T00:13:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6456999999999998e-05 2024-09-15T00:13:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803965+0000", 2024-09-15T00:13:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.076e-05 2024-09-15T00:13:12.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804023+0000", 2024-09-15T00:13:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7787000000000001e-05 2024-09-15T00:13:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804031+0000", 2024-09-15T00:13:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2160000000000006e-06 2024-09-15T00:13:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.808377+0000", 2024-09-15T00:13:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043458969999999996 2024-09-15T00:13:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [0~131072,196608~786432,1048576~131072], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.7282094099999998, 2024-09-15T00:13:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047737029999999998, 2024-09-15T00:13:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.644 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840271+0000", 2024-09-15T00:13:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:13:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840721+0000", 2024-09-15T00:13:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045029800000000002 2024-09-15T00:13:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840738+0000", 2024-09-15T00:13:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6671999999999999e-05 2024-09-15T00:13:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840749+0000", 2024-09-15T00:13:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1251e-05 2024-09-15T00:13:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840817+0000", 2024-09-15T00:13:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8046000000000005e-05 2024-09-15T00:13:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840825+0000", 2024-09-15T00:13:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2519999999999999e-06 2024-09-15T00:13:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.845044+0000", 2024-09-15T00:13:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042185900000000004 2024-09-15T00:13:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [1179648~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2228224, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.7201853629999997, 2024-09-15T00:13:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053056819999999999, 2024-09-15T00:13:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848295+0000", 2024-09-15T00:13:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4000000000000001e-07 2024-09-15T00:13:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848728+0000", 2024-09-15T00:13:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000433709 2024-09-15T00:13:12.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848745+0000", 2024-09-15T00:13:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6167999999999998e-05 2024-09-15T00:13:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848788+0000", 2024-09-15T00:13:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3269000000000002e-05 2024-09-15T00:13:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848857+0000", 2024-09-15T00:13:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9453000000000001e-05 2024-09-15T00:13:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848866+0000", 2024-09-15T00:13:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2779999999999993e-06 2024-09-15T00:13:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.853600+0000", 2024-09-15T00:13:12.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047341650000000002 2024-09-15T00:13:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head, version: 93'455, data_included: [0~98304,114688~16384,147456~589824,753664~65536,835584~16384,868352~262144], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head@93'455, size: 2342912, copy_subset: [(0, 2342912)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1130496, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.661051337, 2024-09-15T00:13:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063250900000000002, 2024-09-15T00:13:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.653 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907429+0000", 2024-09-15T00:13:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.85e-07 2024-09-15T00:13:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908008+0000", 2024-09-15T00:13:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00057949599999999998 2024-09-15T00:13:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908026+0000", 2024-09-15T00:13:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7700999999999999e-05 2024-09-15T00:13:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908038+0000", 2024-09-15T00:13:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2131e-05 2024-09-15T00:13:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908101+0000", 2024-09-15T00:13:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2596999999999997e-05 2024-09-15T00:13:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908110+0000", 2024-09-15T00:13:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8540000000000003e-06 2024-09-15T00:13:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.913753+0000", 2024-09-15T00:13:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056435260000000003 2024-09-15T00:13:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.657 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.657 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:12.659 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:12.659 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:12.657+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:12.759 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:12.758+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:12.767 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:12.767 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:12.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.767 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1a 124/123 [PushOp(3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e, version: 47'185, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e@47'185, size: 15204352, copy_subset: [(0, 15204352)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:12.767 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:12.762255+0000", 2024-09-15T00:13:12.767 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.003670048, 2024-09-15T00:13:12.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037023680000000002, 2024-09-15T00:13:12.767 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:12.767 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:12.767 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:12.767 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:12.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:12.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:12.762255+0000", 2024-09-15T00:13:12.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:12.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:12.762255+0000", 2024-09-15T00:13:12.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:12.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:12.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:12.762256+0000", 2024-09-15T00:13:12.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0800000000000004e-07 2024-09-15T00:13:12.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:12.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:12.763170+0000", 2024-09-15T00:13:12.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091429399999999996 2024-09-15T00:13:12.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:12.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:12.763187+0000", 2024-09-15T00:13:12.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6710000000000001e-05 2024-09-15T00:13:12.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:12.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:12.763197+0000", 2024-09-15T00:13:12.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0696000000000001e-05 2024-09-15T00:13:12.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:12.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:12.763258+0000", 2024-09-15T00:13:12.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0921999999999997e-05 2024-09-15T00:13:12.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:12.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:12.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:12.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:12.763267+0000", 2024-09-15T00:13:12.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0629999999999995e-06 2024-09-15T00:13:12.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.770 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:12.771 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.771 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:12.771 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:13:12.771 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 1 2024-09-15T00:13:12.771 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:12.776 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:12.861 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:12.858+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:12.885 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:12.885 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:12.885 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:12.885 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:12.886 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:12.895 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:12.961 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:12.959+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:13.047 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:13.047 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:13.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:13.048 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:13.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.048 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133834 3.1b e85/69 3:da9a78a3:::rbd_data.11395dfcf2b8.0000000000000000:head v 85'356, mlcod=85'354)", 2024-09-15T00:13:13.048 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:13.048 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.481551701999997, 2024-09-15T00:13:13.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026079054000000001, 2024-09-15T00:13:13.048 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.048 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.048 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.048 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:13.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:13.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563338+0000", 2024-09-15T00:13:13.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.418e-06 2024-09-15T00:13:13.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569385+0000", 2024-09-15T00:13:13.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006047335 2024-09-15T00:13:13.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569418+0000", 2024-09-15T00:13:13.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2957999999999999e-05 2024-09-15T00:13:13.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569432+0000", 2024-09-15T00:13:13.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3718e-05 2024-09-15T00:13:13.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569519+0000", 2024-09-15T00:13:13.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7022999999999997e-05 2024-09-15T00:13:13.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569540+0000", 2024-09-15T00:13:13.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1426000000000001e-05 2024-09-15T00:13:13.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589362+0000", 2024-09-15T00:13:13.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019821398 2024-09-15T00:13:13.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589415+0000", 2024-09-15T00:13:13.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3777999999999999e-05 2024-09-15T00:13:13.052 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.052 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.052 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.052 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133836 3.1 e85/67 3:849f9a98:::rbd_data.11395dfcf2b8.0000000000000002:head v 85'304, mlcod=84'301)", 2024-09-15T00:13:13.053 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:13.053 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.473856791999999, 2024-09-15T00:13:13.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023231600000000002, 2024-09-15T00:13:13.053 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.053 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.053 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.053 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:13.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:13.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571032+0000", 2024-09-15T00:13:13.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-15T00:13:13.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576635+0000", 2024-09-15T00:13:13.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056028700000000002 2024-09-15T00:13:13.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576650+0000", 2024-09-15T00:13:13.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.505e-05 2024-09-15T00:13:13.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576658+0000", 2024-09-15T00:13:13.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6879999999999999e-06 2024-09-15T00:13:13.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576717+0000", 2024-09-15T00:13:13.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175000000000002e-05 2024-09-15T00:13:13.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576742+0000", 2024-09-15T00:13:13.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4734000000000001e-05 2024-09-15T00:13:13.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594234+0000", 2024-09-15T00:13:13.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017492527000000001 2024-09-15T00:13:13.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594263+0000", 2024-09-15T00:13:13.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8629e-05 2024-09-15T00:13:13.056 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.057 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.057 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.057 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146609 3.b e98/69 3:d39b42ba:::rbd_data.11394501364.0000000000000001:head v 98'283, mlcod=93'281)", 2024-09-15T00:13:13.057 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:13.057 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.564138676999999, 2024-09-15T00:13:13.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020892945, 2024-09-15T00:13:13.057 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.057 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.057 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.057 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:13.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:13.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480751+0000", 2024-09-15T00:13:13.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6509999999999999e-06 2024-09-15T00:13:13.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482738+0000", 2024-09-15T00:13:13.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019867410000000002 2024-09-15T00:13:13.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482754+0000", 2024-09-15T00:13:13.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6171000000000002e-05 2024-09-15T00:13:13.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482769+0000", 2024-09-15T00:13:13.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4688000000000001e-05 2024-09-15T00:13:13.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482835+0000", 2024-09-15T00:13:13.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6526000000000006e-05 2024-09-15T00:13:13.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482858+0000", 2024-09-15T00:13:13.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2959000000000002e-05 2024-09-15T00:13:13.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501587+0000", 2024-09-15T00:13:13.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018728471 2024-09-15T00:13:13.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501642+0000", 2024-09-15T00:13:13.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5738000000000001e-05 2024-09-15T00:13:13.062 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.063 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.063 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.063 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146835 3.12 e98/68 3:4eee39d6:::rbd_data.11394501364.0000000000000000:head v 98'474, mlcod=93'472)", 2024-09-15T00:13:13.063 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:13.063 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.455511406999999, 2024-09-15T00:13:13.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020080434000000001, 2024-09-15T00:13:13.063 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.063 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.063 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.063 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:13.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:13.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589378+0000", 2024-09-15T00:13:13.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7770000000000001e-06 2024-09-15T00:13:13.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590678+0000", 2024-09-15T00:13:13.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012991750000000001 2024-09-15T00:13:13.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590699+0000", 2024-09-15T00:13:13.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1353e-05 2024-09-15T00:13:13.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590710+0000", 2024-09-15T00:13:13.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1245e-05 2024-09-15T00:13:13.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590800+0000", 2024-09-15T00:13:13.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9963000000000006e-05 2024-09-15T00:13:13.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590823+0000", 2024-09-15T00:13:13.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3289e-05 2024-09-15T00:13:13.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609368+0000", 2024-09-15T00:13:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018545078 2024-09-15T00:13:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609457+0000", 2024-09-15T00:13:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8554000000000006e-05 2024-09-15T00:13:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.068 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146837 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7201319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:13.068 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:13.068 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.444731944000001, 2024-09-15T00:13:13.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024751038, 2024-09-15T00:13:13.068 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.068 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.068 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.068 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.068 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.068 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.068 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146837 2024-09-15T00:13:13.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.068 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:13.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:13.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600157+0000", 2024-09-15T00:13:13.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:13:13.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601965+0000", 2024-09-15T00:13:13.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001807613 2024-09-15T00:13:13.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601985+0000", 2024-09-15T00:13:13.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0679e-05 2024-09-15T00:13:13.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601998+0000", 2024-09-15T00:13:13.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2955999999999999e-05 2024-09-15T00:13:13.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602070+0000", 2024-09-15T00:13:13.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1958000000000005e-05 2024-09-15T00:13:13.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602237+0000", 2024-09-15T00:13:13.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016621299999999999 2024-09-15T00:13:13.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:13.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602799+0000", 2024-09-15T00:13:13.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000562723 2024-09-15T00:13:13.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:13.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.620811+0000", 2024-09-15T00:13:13.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018011207000000001 2024-09-15T00:13:13.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:13.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624793+0000", 2024-09-15T00:13:13.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0039828770000000001 2024-09-15T00:13:13.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624855+0000", 2024-09-15T00:13:13.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.143e-05 2024-09-15T00:13:13.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624907+0000", 2024-09-15T00:13:13.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2262000000000002e-05 2024-09-15T00:13:13.073 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.073 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.073 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.073 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146936 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:13.073 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:13.074 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.291218624999999, 2024-09-15T00:13:13.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056847126999999997, 2024-09-15T00:13:13.074 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.074 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.074 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.074 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.074 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.074 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.074 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146936 2024-09-15T00:13:13.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.074 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:13.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:13.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753671+0000", 2024-09-15T00:13:13.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2839999999999999e-06 2024-09-15T00:13:13.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753677+0000", 2024-09-15T00:13:13.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0260000000000004e-06 2024-09-15T00:13:13.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753691+0000", 2024-09-15T00:13:13.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3891e-05 2024-09-15T00:13:13.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753698+0000", 2024-09-15T00:13:13.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8480000000000001e-06 2024-09-15T00:13:13.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753750+0000", 2024-09-15T00:13:13.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1338000000000002e-05 2024-09-15T00:13:13.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753814+0000", 2024-09-15T00:13:13.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4642999999999998e-05 2024-09-15T00:13:13.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:13.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.756359+0000", 2024-09-15T00:13:13.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025450619999999998 2024-09-15T00:13:13.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:13.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810435+0000", 2024-09-15T00:13:13.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.054075397999999997 2024-09-15T00:13:13.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:13.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810446+0000", 2024-09-15T00:13:13.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1005000000000001e-05 2024-09-15T00:13:13.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810482+0000", 2024-09-15T00:13:13.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6494e-05 2024-09-15T00:13:13.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810517+0000", 2024-09-15T00:13:13.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4137999999999997e-05 2024-09-15T00:13:13.079 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.079 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.079 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.079 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:146936 3.6 e98/97 ondisk, result = 0)", 2024-09-15T00:13:13.079 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:13.080 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.281435570999999, 2024-09-15T00:13:13.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046987205999999997, 2024-09-15T00:13:13.080 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.080 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.080 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:13.080 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:13.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:13.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763453+0000", 2024-09-15T00:13:13.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5199999999999996e-07 2024-09-15T00:13:13.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763457+0000", 2024-09-15T00:13:13.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8569999999999997e-06 2024-09-15T00:13:13.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763469+0000", 2024-09-15T00:13:13.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1977999999999999e-05 2024-09-15T00:13:13.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763476+0000", 2024-09-15T00:13:13.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:13:13.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810423+0000", 2024-09-15T00:13:13.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946993999999999 2024-09-15T00:13:13.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810431+0000", 2024-09-15T00:13:13.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0339999999999997e-06 2024-09-15T00:13:13.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810440+0000", 2024-09-15T00:13:13.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9989999999999997e-06 2024-09-15T00:13:13.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.083 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.084 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:151036 3.1f e103/68 3:f84a1a09:::rbd_data.113958cb0e55.000000000000000c:head v 103'452, mlcod=100'450)", 2024-09-15T00:13:13.084 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:13.084 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.708089343000001, 2024-09-15T00:13:13.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020618753, 2024-09-15T00:13:13.084 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.084 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.084 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.084 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:13.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:13.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:13.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:13:13.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340692+0000", 2024-09-15T00:13:13.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038928890000000001 2024-09-15T00:13:13.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340704+0000", 2024-09-15T00:13:13.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1497e-05 2024-09-15T00:13:13.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340709+0000", 2024-09-15T00:13:13.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1529999999999996e-06 2024-09-15T00:13:13.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340789+0000", 2024-09-15T00:13:13.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9907999999999995e-05 2024-09-15T00:13:13.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340810+0000", 2024-09-15T00:13:13.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1837000000000001e-05 2024-09-15T00:13:13.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357368+0000", 2024-09-15T00:13:13.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016557199000000002 2024-09-15T00:13:13.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357417+0000", 2024-09-15T00:13:13.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9783000000000001e-05 2024-09-15T00:13:13.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.088 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.088 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:151042 3.6 3:6419cde4:::rbd_data.113958cb0e55.0000000000000031:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:13.088 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:13.088 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.679011225, 2024-09-15T00:13:13.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025627375000000001, 2024-09-15T00:13:13.088 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.088 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.088 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.089 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 151042 2024-09-15T00:13:13.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.089 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:13.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:13.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:13.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7899999999999998e-07 2024-09-15T00:13:13.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365881+0000", 2024-09-15T00:13:13.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9770000000000002e-06 2024-09-15T00:13:13.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365890+0000", 2024-09-15T00:13:13.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5920000000000001e-06 2024-09-15T00:13:13.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365894+0000", 2024-09-15T00:13:13.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4569999999999998e-06 2024-09-15T00:13:13.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365927+0000", 2024-09-15T00:13:13.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2036000000000003e-05 2024-09-15T00:13:13.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365970+0000", 2024-09-15T00:13:13.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3197e-05 2024-09-15T00:13:13.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:13.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.367432+0000", 2024-09-15T00:13:13.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014621720000000001 2024-09-15T00:13:13.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:13.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391358+0000", 2024-09-15T00:13:13.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023925921999999999 2024-09-15T00:13:13.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:13.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391415+0000", 2024-09-15T00:13:13.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7253999999999998e-05 2024-09-15T00:13:13.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391456+0000", 2024-09-15T00:13:13.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0497999999999997e-05 2024-09-15T00:13:13.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391504+0000", 2024-09-15T00:13:13.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8591000000000003e-05 2024-09-15T00:13:13.093 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.093 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.094 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.094 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:151042 3.6 e103/97 ondisk, result = 0)", 2024-09-15T00:13:13.094 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:13.094 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.673268212, 2024-09-15T00:13:13.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019892730000000001, 2024-09-15T00:13:13.094 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.094 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.094 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:13.094 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:13.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:13.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:13.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8100000000000003e-07 2024-09-15T00:13:13.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371623+0000", 2024-09-15T00:13:13.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6290000000000002e-06 2024-09-15T00:13:13.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371632+0000", 2024-09-15T00:13:13.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6379999999999993e-06 2024-09-15T00:13:13.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371636+0000", 2024-09-15T00:13:13.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:13:13.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391403+0000", 2024-09-15T00:13:13.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019766982999999998 2024-09-15T00:13:13.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391411+0000", 2024-09-15T00:13:13.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7309999999999993e-06 2024-09-15T00:13:13.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391513+0000", 2024-09-15T00:13:13.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101648 2024-09-15T00:13:13.097 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.097 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.098 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.098 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152025 3.3 e103/68 3:c0279197:::rbd_data.113958cb0e55.0000000000000007:head v 103'464, mlcod=102'461)", 2024-09-15T00:13:13.098 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:13.098 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.528780615999999, 2024-09-15T00:13:13.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021264476000000001, 2024-09-15T00:13:13.098 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.098 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.098 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.098 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:13.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:13.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516109+0000", 2024-09-15T00:13:13.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.666e-06 2024-09-15T00:13:13.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.519997+0000", 2024-09-15T00:13:13.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003888449 2024-09-15T00:13:13.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520016+0000", 2024-09-15T00:13:13.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8059000000000001e-05 2024-09-15T00:13:13.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520026+0000", 2024-09-15T00:13:13.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9599999999999995e-06 2024-09-15T00:13:13.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520089+0000", 2024-09-15T00:13:13.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3074e-05 2024-09-15T00:13:13.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520108+0000", 2024-09-15T00:13:13.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8981999999999999e-05 2024-09-15T00:13:13.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537326+0000", 2024-09-15T00:13:13.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017218102999999998 2024-09-15T00:13:13.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537372+0000", 2024-09-15T00:13:13.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6183000000000002e-05 2024-09-15T00:13:13.102 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.102 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.102 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.102 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152284 3.f e103/68 3:f4272b5d:::rbd_data.113958cb0e55.0000000000000006:head v 103'370, mlcod=103'368)", 2024-09-15T00:13:13.102 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:13.102 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.480304085, 2024-09-15T00:13:13.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020145026999999999, 2024-09-15T00:13:13.102 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.102 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.103 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.103 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:13.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:13.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564585+0000", 2024-09-15T00:13:13.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7000000000000003e-07 2024-09-15T00:13:13.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568352+0000", 2024-09-15T00:13:13.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037675909999999998 2024-09-15T00:13:13.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568370+0000", 2024-09-15T00:13:13.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7158000000000001e-05 2024-09-15T00:13:13.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568378+0000", 2024-09-15T00:13:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6449999999999994e-06 2024-09-15T00:13:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568454+0000", 2024-09-15T00:13:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5285999999999997e-05 2024-09-15T00:13:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568472+0000", 2024-09-15T00:13:13.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8447000000000001e-05 2024-09-15T00:13:13.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584693+0000", 2024-09-15T00:13:13.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016221467999999999 2024-09-15T00:13:13.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584729+0000", 2024-09-15T00:13:13.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5462e-05 2024-09-15T00:13:13.106 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.106 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.106 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.107 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152500 3.17 e103/68 3:edff5adf:::rbd_data.113958cb0e55.000000000000000d:head v 103'461, mlcod=103'459)", 2024-09-15T00:13:13.107 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:13.107 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.4033175, 2024-09-15T00:13:13.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020217809, 2024-09-15T00:13:13.107 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.107 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.107 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.107 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:13.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:13.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641572+0000", 2024-09-15T00:13:13.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-15T00:13:13.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644961+0000", 2024-09-15T00:13:13.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033882610000000001 2024-09-15T00:13:13.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644979+0000", 2024-09-15T00:13:13.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8179e-05 2024-09-15T00:13:13.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644987+0000", 2024-09-15T00:13:13.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8860000000000004e-06 2024-09-15T00:13:13.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645044+0000", 2024-09-15T00:13:13.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7141000000000001e-05 2024-09-15T00:13:13.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645062+0000", 2024-09-15T00:13:13.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8258e-05 2024-09-15T00:13:13.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661749+0000", 2024-09-15T00:13:13.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016686553999999999 2024-09-15T00:13:13.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661788+0000", 2024-09-15T00:13:13.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9690000000000001e-05 2024-09-15T00:13:13.111 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.111 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.111 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.111 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153171 3.18 e103/69 3:1db186e8:::rbd_data.113958cb0e55.0000000000000002:head v 103'520, mlcod=103'518)", 2024-09-15T00:13:13.111 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:13.111 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.264832594000001, 2024-09-15T00:13:13.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022266838000000001, 2024-09-15T00:13:13.111 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.111 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.112 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.112 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:13.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:13.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780057+0000", 2024-09-15T00:13:13.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5880000000000001e-06 2024-09-15T00:13:13.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781273+0000", 2024-09-15T00:13:13.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001216087 2024-09-15T00:13:13.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781311+0000", 2024-09-15T00:13:13.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7656e-05 2024-09-15T00:13:13.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781325+0000", 2024-09-15T00:13:13.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4508999999999999e-05 2024-09-15T00:13:13.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781403+0000", 2024-09-15T00:13:13.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7927000000000001e-05 2024-09-15T00:13:13.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781422+0000", 2024-09-15T00:13:13.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8575000000000001e-05 2024-09-15T00:13:13.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802249+0000", 2024-09-15T00:13:13.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020827577 2024-09-15T00:13:13.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802322+0000", 2024-09-15T00:13:13.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2918999999999998e-05 2024-09-15T00:13:13.115 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.115 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.115 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.116 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153772 3.12 e103/68 3:4911a4f6:::rbd_data.113958cb0e55.0000000000000001:head v 103'490, mlcod=103'488)", 2024-09-15T00:13:13.116 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:13.116 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.161961967, 2024-09-15T00:13:13.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021181004, 2024-09-15T00:13:13.116 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.116 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.116 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.116 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:13.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:13.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882927+0000", 2024-09-15T00:13:13.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.426e-06 2024-09-15T00:13:13.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883966+0000", 2024-09-15T00:13:13.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010384610000000001 2024-09-15T00:13:13.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883985+0000", 2024-09-15T00:13:13.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9276e-05 2024-09-15T00:13:13.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883994+0000", 2024-09-15T00:13:13.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9490000000000002e-06 2024-09-15T00:13:13.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884054+0000", 2024-09-15T00:13:13.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9419999999999997e-05 2024-09-15T00:13:13.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884073+0000", 2024-09-15T00:13:13.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9137e-05 2024-09-15T00:13:13.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904038+0000", 2024-09-15T00:13:13.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019965092 2024-09-15T00:13:13.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904107+0000", 2024-09-15T00:13:13.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9243000000000001e-05 2024-09-15T00:13:13.120 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.120 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.120 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.120 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163112 3.3 e106/68 3:c668f3d5:::rbd_data.11395d771336.0000000000000001:head v 106'489, mlcod=105'487)", 2024-09-15T00:13:13.120 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:13.120 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.580064698000001, 2024-09-15T00:13:13.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019317183000000002, 2024-09-15T00:13:13.120 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.121 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.121 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.121 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:13.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:13.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464824+0000", 2024-09-15T00:13:13.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1100000000000005e-07 2024-09-15T00:13:13.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466874+0000", 2024-09-15T00:13:13.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002049497 2024-09-15T00:13:13.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466884+0000", 2024-09-15T00:13:13.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0424999999999999e-05 2024-09-15T00:13:13.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466890+0000", 2024-09-15T00:13:13.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-15T00:13:13.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466944+0000", 2024-09-15T00:13:13.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4592999999999997e-05 2024-09-15T00:13:13.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466958+0000", 2024-09-15T00:13:13.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3993e-05 2024-09-15T00:13:13.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484068+0000", 2024-09-15T00:13:13.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017109994 2024-09-15T00:13:13.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484140+0000", 2024-09-15T00:13:13.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2224000000000005e-05 2024-09-15T00:13:13.125 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.125 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.125 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.125 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163119 3.0 e106/94 3:06118613:::rbd_data.11395d771336.0000000000000004:head v 106'549, mlcod=105'547)", 2024-09-15T00:13:13.125 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:13.125 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.568449494999999, 2024-09-15T00:13:13.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017175867000000001, 2024-09-15T00:13:13.126 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.126 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.126 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.126 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:13.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:13.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:13.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6700000000000002e-07 2024-09-15T00:13:13.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477786+0000", 2024-09-15T00:13:13.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013468149999999999 2024-09-15T00:13:13.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477797+0000", 2024-09-15T00:13:13.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0635000000000001e-05 2024-09-15T00:13:13.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477805+0000", 2024-09-15T00:13:13.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2989999999999997e-06 2024-09-15T00:13:13.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477859+0000", 2024-09-15T00:13:13.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3699000000000002e-05 2024-09-15T00:13:13.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477878+0000", 2024-09-15T00:13:13.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9335e-05 2024-09-15T00:13:13.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493560+0000", 2024-09-15T00:13:13.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015681748999999998 2024-09-15T00:13:13.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493614+0000", 2024-09-15T00:13:13.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4367999999999999e-05 2024-09-15T00:13:13.129 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.130 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.130 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.130 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.5.0:5005 3.17 e110/68 3:eb0359e8:::rbd_data.113964deed84.0000000000000251:35 v 110'537, mlcod=109'534)", 2024-09-15T00:13:13.130 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:13.130 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.908599123, 2024-09-15T00:13:13.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015446896999999999, 2024-09-15T00:13:13.130 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.130 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.130 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.130 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:13.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:13.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136290+0000", 2024-09-15T00:13:13.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7400000000000003e-07 2024-09-15T00:13:13.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136294+0000", 2024-09-15T00:13:13.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4989999999999996e-06 2024-09-15T00:13:13.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136304+0000", 2024-09-15T00:13:13.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0319999999999999e-05 2024-09-15T00:13:13.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136310+0000", 2024-09-15T00:13:13.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.981e-06 2024-09-15T00:13:13.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136350+0000", 2024-09-15T00:13:13.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0088e-05 2024-09-15T00:13:13.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136364+0000", 2024-09-15T00:13:13.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3954999999999999e-05 2024-09-15T00:13:13.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151681+0000", 2024-09-15T00:13:13.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015316969999999999 2024-09-15T00:13:13.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151736+0000", 2024-09-15T00:13:13.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4410000000000003e-05 2024-09-15T00:13:13.134 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.134 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.134 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.134 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1758 3.10 e110/68 3:08149147:::rbd_data.113964deed84.0000000000000055:35 v 110'594, mlcod=109'591)", 2024-09-15T00:13:13.134 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:13.134 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.904300897000001, 2024-09-15T00:13:13.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016278652000000001, 2024-09-15T00:13:13.134 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.134 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.135 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.135 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:13.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:13.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:13.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:13.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140599+0000", 2024-09-15T00:13:13.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1109e-05 2024-09-15T00:13:13.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140605+0000", 2024-09-15T00:13:13.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-15T00:13:13.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140612+0000", 2024-09-15T00:13:13.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6089999999999997e-06 2024-09-15T00:13:13.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140656+0000", 2024-09-15T00:13:13.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3754999999999997e-05 2024-09-15T00:13:13.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140674+0000", 2024-09-15T00:13:13.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7640999999999999e-05 2024-09-15T00:13:13.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156823+0000", 2024-09-15T00:13:13.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016149047999999999 2024-09-15T00:13:13.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156866+0000", 2024-09-15T00:13:13.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3205999999999999e-05 2024-09-15T00:13:13.138 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.138 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.138 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.139 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.4.0:5722 3.e e110/95 3:74a58daf:::rbd_data.113964deed84.00000000000000c1:35 v 110'1287, mlcod=109'1284)", 2024-09-15T00:13:13.139 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:13.139 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.895226541, 2024-09-15T00:13:13.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01830555, 2024-09-15T00:13:13.139 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.139 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.139 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.139 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:13.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:13.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149662+0000", 2024-09-15T00:13:13.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:13:13.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149666+0000", 2024-09-15T00:13:13.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2220000000000004e-06 2024-09-15T00:13:13.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149672+0000", 2024-09-15T00:13:13.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3410000000000002e-06 2024-09-15T00:13:13.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149676+0000", 2024-09-15T00:13:13.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.566e-06 2024-09-15T00:13:13.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149709+0000", 2024-09-15T00:13:13.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2403000000000001e-05 2024-09-15T00:13:13.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149719+0000", 2024-09-15T00:13:13.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0608000000000001e-05 2024-09-15T00:13:13.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167944+0000", 2024-09-15T00:13:13.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018225183999999998 2024-09-15T00:13:13.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167967+0000", 2024-09-15T00:13:13.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2705e-05 2024-09-15T00:13:13.143 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.143 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.143 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.143 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.143 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.143 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:13.144 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:13.145 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:13.142+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:13.245 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:13.243+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:13.265 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:13.265 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:13.265 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:13.266 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:13.275 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:13.346 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:13.343+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:13.380 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:13.380 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:13.380 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:13.380 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:13.380 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:13.389 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:13.445 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:13:13.473 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:13.470+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:13.537 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:13.537 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:13.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:13.538 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:13.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.538 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:13.538 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:13.538 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 107.354254357, 2024-09-15T00:13:13.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:13:13.539 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.539 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.539 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.539 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.539 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.540 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.540 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:13:13.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.540 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:13.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:13.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:13:13.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:13:13.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:13:13.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:13:13.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:13:13.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:13:13.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:13:13.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:13:13.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:13:13.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:13:13.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:13:13.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:13:13.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:13.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:13:13.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:13:13.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:13.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:13:13.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:13:13.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:13.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:13:13.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:13:13.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:13:13.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:13:13.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:13:13.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:13:13.547 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.547 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.547 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.548 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:13:13.548 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:13.548 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.624413177999998, 2024-09-15T00:13:13.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:13:13.548 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.548 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.548 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.548 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:13:13.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.548 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:13.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:13.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:13:13.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:13:13.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:13:13.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:13:13.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:13:13.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:13:13.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:13:13.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:13:13.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:13:13.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:13:13.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:13:13.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:13:13.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:13.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:13:13.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:13:13.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:13.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:13:13.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:13:13.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:13.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:13:13.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:13:13.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:13:13.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:13:13.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:13:13.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:13:13.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.553 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.554 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.554 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:13:13.554 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:13.554 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.617454815000002, 2024-09-15T00:13:13.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:13:13.554 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.554 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.554 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:13.554 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:13.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:13.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:13:13.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:13:13.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:13:13.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:13:13.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:13:13.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:13:13.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:13:13.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:13:13.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:13:13.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:13:13.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:13:13.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:13:13.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:13:13.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:13:13.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.558 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.558 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:13.558 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:13.558 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 88.870265978000006, 2024-09-15T00:13:13.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:13:13.558 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.559 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.559 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:13.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.559 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:13:13.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.559 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:13.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:13.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:13:13.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:13:13.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:13:13.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:13:13.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:13:13.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:13:13.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:13:13.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:13:13.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:13:13.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:13:13.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:13:13.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:13:13.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:13:13.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:13:13.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:13:13.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:13.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:13:13.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:13:13.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:13.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:13:13.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:13:13.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:13.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:13:13.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:13:13.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:13:13.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:13:13.564 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.564 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.564 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.564 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97493 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:13.564 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:13.564 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.680683258000002, 2024-09-15T00:13:13.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033259141999999998, 2024-09-15T00:13:13.565 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.565 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.565 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:13.565 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.565 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.565 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.565 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97493 2024-09-15T00:13:13.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.565 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-15T00:13:13.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853401+0000", 2024-09-15T00:13:13.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:13:13.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853407+0000", 2024-09-15T00:13:13.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-15T00:13:13.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853411+0000", 2024-09-15T00:13:13.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.481e-06 2024-09-15T00:13:13.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886544+0000", 2024-09-15T00:13:13.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033132861999999999 2024-09-15T00:13:13.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886587+0000", 2024-09-15T00:13:13.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2942000000000003e-05 2024-09-15T00:13:13.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886655+0000", 2024-09-15T00:13:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7711000000000003e-05 2024-09-15T00:13:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97494 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.680662135000006, 2024-09-15T00:13:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033392390000000001, 2024-09-15T00:13:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.570 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:13.570 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.570 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.570 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.570 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97494 2024-09-15T00:13:13.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.570 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:13.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:13.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:13.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-15T00:13:13.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853419+0000", 2024-09-15T00:13:13.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.917e-06 2024-09-15T00:13:13.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853425+0000", 2024-09-15T00:13:13.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:13:13.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853433+0000", 2024-09-15T00:13:13.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8120000000000008e-06 2024-09-15T00:13:13.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886680+0000", 2024-09-15T00:13:13.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033247207000000001 2024-09-15T00:13:13.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886726+0000", 2024-09-15T00:13:13.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5902000000000002e-05 2024-09-15T00:13:13.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886809+0000", 2024-09-15T00:13:13.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3475999999999998e-05 2024-09-15T00:13:13.574 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.574 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.574 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.574 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97495 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:13.574 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:13.574 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.680641182000002, 2024-09-15T00:13:13.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033640503000000002, 2024-09-15T00:13:13.574 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.574 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.574 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:13.575 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.575 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.575 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.575 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97495 2024-09-15T00:13:13.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.575 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:13.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:13.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:13.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:13:13.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853440+0000", 2024-09-15T00:13:13.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.804e-06 2024-09-15T00:13:13.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853446+0000", 2024-09-15T00:13:13.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5559999999999998e-06 2024-09-15T00:13:13.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853450+0000", 2024-09-15T00:13:13.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9519999999999996e-06 2024-09-15T00:13:13.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886904+0000", 2024-09-15T00:13:13.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033454207999999999 2024-09-15T00:13:13.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887007+0000", 2024-09-15T00:13:13.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102904 2024-09-15T00:13:13.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887078+0000", 2024-09-15T00:13:13.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1719999999999998e-05 2024-09-15T00:13:13.578 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.578 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.578 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.578 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97496 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:13.579 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:13.579 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.680624084000002, 2024-09-15T00:13:13.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033802431000000001, 2024-09-15T00:13:13.579 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.579 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.579 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:13.579 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.579 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.579 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.579 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97496 2024-09-15T00:13:13.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.579 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:13.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:13.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:13.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.53e-07 2024-09-15T00:13:13.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853462+0000", 2024-09-15T00:13:13.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-15T00:13:13.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853468+0000", 2024-09-15T00:13:13.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3609999999999997e-06 2024-09-15T00:13:13.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853473+0000", 2024-09-15T00:13:13.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-15T00:13:13.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887111+0000", 2024-09-15T00:13:13.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033638376999999997 2024-09-15T00:13:13.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887178+0000", 2024-09-15T00:13:13.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6877000000000004e-05 2024-09-15T00:13:13.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887257+0000", 2024-09-15T00:13:13.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9616999999999993e-05 2024-09-15T00:13:13.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.583 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.583 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.583 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97587 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:13.583 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:13.583 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.680600850000005, 2024-09-15T00:13:13.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033973642999999998, 2024-09-15T00:13:13.583 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.583 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.583 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:13.583 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.583 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.584 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.584 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97587 2024-09-15T00:13:13.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.584 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:13.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:13.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853479+0000", 2024-09-15T00:13:13.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:13:13.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853481+0000", 2024-09-15T00:13:13.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.136e-06 2024-09-15T00:13:13.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853487+0000", 2024-09-15T00:13:13.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2439999999999998e-06 2024-09-15T00:13:13.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853492+0000", 2024-09-15T00:13:13.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:13:13.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887284+0000", 2024-09-15T00:13:13.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033791827000000003 2024-09-15T00:13:13.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887361+0000", 2024-09-15T00:13:13.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6983000000000005e-05 2024-09-15T00:13:13.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887452+0000", 2024-09-15T00:13:13.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1236999999999994e-05 2024-09-15T00:13:13.587 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.587 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.587 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.587 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97588 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:13.587 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:13.588 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.680580692999996, 2024-09-15T00:13:13.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034134617999999999, 2024-09-15T00:13:13.588 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.588 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.588 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:13.588 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.588 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.588 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.588 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97588 2024-09-15T00:13:13.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.588 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:13.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:13.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853499+0000", 2024-09-15T00:13:13.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:13:13.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853501+0000", 2024-09-15T00:13:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.26e-06 2024-09-15T00:13:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853507+0000", 2024-09-15T00:13:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2899999999999999e-06 2024-09-15T00:13:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853512+0000", 2024-09-15T00:13:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8330000000000001e-06 2024-09-15T00:13:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887495+0000", 2024-09-15T00:13:13.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033982529999999997 2024-09-15T00:13:13.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:13.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887548+0000", 2024-09-15T00:13:13.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3445000000000001e-05 2024-09-15T00:13:13.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887633+0000", 2024-09-15T00:13:13.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4890999999999997e-05 2024-09-15T00:13:13.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.592 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.592 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126866 3.6 3.973c9fa6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:13.592 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:13.592 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.577258747999998, 2024-09-15T00:13:13.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153301000000003, 2024-09-15T00:13:13.592 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.592 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.592 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:13.592 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.593 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126866 2024-09-15T00:13:13.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.593 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:13.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:13.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956821+0000", 2024-09-15T00:13:13.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.91e-07 2024-09-15T00:13:13.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956822+0000", 2024-09-15T00:13:13.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.705e-06 2024-09-15T00:13:13.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956828+0000", 2024-09-15T00:13:13.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6760000000000004e-06 2024-09-15T00:13:13.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956833+0000", 2024-09-15T00:13:13.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5379999999999996e-06 2024-09-15T00:13:13.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989970+0000", 2024-09-15T00:13:13.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033137560000000003 2024-09-15T00:13:13.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989974+0000", 2024-09-15T00:13:13.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-15T00:13:13.596 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.596 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.596 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.596 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126951 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:13.596 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:13:13.596 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.577133064999998, 2024-09-15T00:13:13.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033152585999999998, 2024-09-15T00:13:13.596 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.596 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.596 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:13.596 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.596 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.597 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.597 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126951 2024-09-15T00:13:13.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.597 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:13:13.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:13:13.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:13:13.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-07 2024-09-15T00:13:13.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956953+0000", 2024-09-15T00:13:13.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9310000000000002e-06 2024-09-15T00:13:13.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956957+0000", 2024-09-15T00:13:13.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5379999999999998e-06 2024-09-15T00:13:13.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956959+0000", 2024-09-15T00:13:13.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6599999999999999e-06 2024-09-15T00:13:13.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990095+0000", 2024-09-15T00:13:13.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033135855999999998 2024-09-15T00:13:13.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990099+0000", 2024-09-15T00:13:13.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4010000000000001e-06 2024-09-15T00:13:13.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.600 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.600 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126952 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:13.600 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:13.600 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.577116076000003, 2024-09-15T00:13:13.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033156066999999997, 2024-09-15T00:13:13.600 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.600 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.600 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:13.600 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.601 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.601 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.601 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126952 2024-09-15T00:13:13.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.601 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:13.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:13.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:13.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:13:13.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956965+0000", 2024-09-15T00:13:13.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3379999999999999e-06 2024-09-15T00:13:13.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956968+0000", 2024-09-15T00:13:13.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0419999999999998e-06 2024-09-15T00:13:13.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956970+0000", 2024-09-15T00:13:13.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4899999999999999e-06 2024-09-15T00:13:13.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990113+0000", 2024-09-15T00:13:13.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033142642 2024-09-15T00:13:13.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990119+0000", 2024-09-15T00:13:13.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2979999999999997e-06 2024-09-15T00:13:13.604 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.604 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.604 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.604 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126954 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:13.604 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:13.604 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.577070581000001, 2024-09-15T00:13:13.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033158449, 2024-09-15T00:13:13.604 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.604 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.604 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:13.605 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.605 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.605 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.605 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126954 2024-09-15T00:13:13.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.605 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:13.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:13.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:13.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-15T00:13:13.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957010+0000", 2024-09-15T00:13:13.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3659999999999999e-06 2024-09-15T00:13:13.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957013+0000", 2024-09-15T00:13:13.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1140000000000002e-06 2024-09-15T00:13:13.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957016+0000", 2024-09-15T00:13:13.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1209999999999998e-06 2024-09-15T00:13:13.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990163+0000", 2024-09-15T00:13:13.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033146451 2024-09-15T00:13:13.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990167+0000", 2024-09-15T00:13:13.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0579999999999999e-06 2024-09-15T00:13:13.608 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.608 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.608 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.608 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126955 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:13.608 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:13.608 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.577059366999997, 2024-09-15T00:13:13.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033160617000000003, 2024-09-15T00:13:13.608 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.608 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.608 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:13.609 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.609 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.609 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.609 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126955 2024-09-15T00:13:13.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.609 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:13.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:13.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:13.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-15T00:13:13.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957021+0000", 2024-09-15T00:13:13.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-15T00:13:13.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957024+0000", 2024-09-15T00:13:13.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1599999999999998e-06 2024-09-15T00:13:13.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957027+0000", 2024-09-15T00:13:13.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000001e-06 2024-09-15T00:13:13.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990177+0000", 2024-09-15T00:13:13.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033149741000000003 2024-09-15T00:13:13.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990180+0000", 2024-09-15T00:13:13.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4910000000000001e-06 2024-09-15T00:13:13.612 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.612 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.612 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.612 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126956 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:13.612 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:13.612 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.577049246000001, 2024-09-15T00:13:13.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033163633999999997, 2024-09-15T00:13:13.612 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.613 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.613 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:13.613 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.613 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.613 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.613 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126956 2024-09-15T00:13:13.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.613 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:13.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:13.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:13.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:13.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957031+0000", 2024-09-15T00:13:13.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-06 2024-09-15T00:13:13.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957034+0000", 2024-09-15T00:13:13.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9960000000000002e-06 2024-09-15T00:13:13.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957037+0000", 2024-09-15T00:13:13.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6419999999999999e-06 2024-09-15T00:13:13.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990190+0000", 2024-09-15T00:13:13.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153152999999998 2024-09-15T00:13:13.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990193+0000", 2024-09-15T00:13:13.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6069999999999999e-06 2024-09-15T00:13:13.616 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.616 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.617 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.617 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126957 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:13.617 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:13.617 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.577039595999999, 2024-09-15T00:13:13.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033166442999999997, 2024-09-15T00:13:13.617 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.617 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.617 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:13.617 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.617 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.617 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.617 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126957 2024-09-15T00:13:13.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.618 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:13.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:13.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957040+0000", 2024-09-15T00:13:13.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:13:13.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957041+0000", 2024-09-15T00:13:13.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0440000000000001e-06 2024-09-15T00:13:13.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957044+0000", 2024-09-15T00:13:13.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0170000000000001e-06 2024-09-15T00:13:13.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957048+0000", 2024-09-15T00:13:13.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0319999999999997e-06 2024-09-15T00:13:13.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990202+0000", 2024-09-15T00:13:13.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033154640999999999 2024-09-15T00:13:13.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990206+0000", 2024-09-15T00:13:13.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.507e-06 2024-09-15T00:13:13.620 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.620 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.621 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.621 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126958 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:13.621 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:13.621 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.577028366, 2024-09-15T00:13:13.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033167613999999998, 2024-09-15T00:13:13.621 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.621 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.621 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:13.621 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.621 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.621 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.622 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126958 2024-09-15T00:13:13.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.622 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:13.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:13.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:13.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:13:13.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957053+0000", 2024-09-15T00:13:13.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.756e-06 2024-09-15T00:13:13.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957059+0000", 2024-09-15T00:13:13.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2770000000000002e-06 2024-09-15T00:13:13.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957063+0000", 2024-09-15T00:13:13.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2459999999999997e-06 2024-09-15T00:13:13.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990215+0000", 2024-09-15T00:13:13.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033151509000000003 2024-09-15T00:13:13.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990218+0000", 2024-09-15T00:13:13.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5099999999999999e-06 2024-09-15T00:13:13.625 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.625 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.625 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.625 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126959 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:13.625 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:13.625 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.577009361999998, 2024-09-15T00:13:13.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033161673000000003, 2024-09-15T00:13:13.625 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.625 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.626 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:13.626 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.626 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.626 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.626 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126959 2024-09-15T00:13:13.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.626 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:13.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:13.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:13.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:13:13.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957072+0000", 2024-09-15T00:13:13.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2239999999999998e-06 2024-09-15T00:13:13.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957079+0000", 2024-09-15T00:13:13.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.19e-06 2024-09-15T00:13:13.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957083+0000", 2024-09-15T00:13:13.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:13:13.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990228+0000", 2024-09-15T00:13:13.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033144818 2024-09-15T00:13:13.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990231+0000", 2024-09-15T00:13:13.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:13:13.629 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.629 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.630 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.630 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126960 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:13.630 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:13.630 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.576990555000002, 2024-09-15T00:13:13.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033155166, 2024-09-15T00:13:13.630 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:13.630 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:13.630 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:13.630 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:13.630 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:13.630 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:13.630 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126960 2024-09-15T00:13:13.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.631 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:13.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:13.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:13.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:13.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:13.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:13.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:13.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:13.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:13:13.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:13.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957091+0000", 2024-09-15T00:13:13.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:13:13.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:13.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957097+0000", 2024-09-15T00:13:13.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6050000000000003e-06 2024-09-15T00:13:13.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:13.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957102+0000", 2024-09-15T00:13:13.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5539999999999999e-06 2024-09-15T00:13:13.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:13.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990240+0000", 2024-09-15T00:13:13.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033138289000000001 2024-09-15T00:13:13.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:13.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:13.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:13.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990244+0000", 2024-09-15T00:13:13.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4740000000000001e-06 2024-09-15T00:13:13.633 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.633 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.634 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.634 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:13.634 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:13.634 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:13.635 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:13.636 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:13.636 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:13.636 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:13.637 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:13.637 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:13.637 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:13.638 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:13.633+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:13.643 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:13.643 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:13.659 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:13.660 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:13.677 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:13.677 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:13.693 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:13.693 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:13.710 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:13.710 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:13.735 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:13.734+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:13.744 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:13.744 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:13.753 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:13.837 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:13.835+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:13.898 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:13.898 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:13.906 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:13.948 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:13.945+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:14.037 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:14.035+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:14.051 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:14.051 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:14.060 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:14.138 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:14.136+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:14.169 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:14.178 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:14.214 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:13:14.258 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:14.256+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:14.328 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:14.328 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:14.337 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:14.340 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:14.337+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:14.441 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:14.441 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:14.441 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:14.439+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:14.449 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:14.542 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:14.540+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:14.593 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:14.593 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:14.601 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:14.643 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:14.641+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:14.705 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:14.705 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:14.713 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:14.744 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:14.742+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:14.816 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:14.816 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:14.825 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:14.845 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:14.843+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:14.928 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:14.928 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:14.937 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:14.946 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:14.944+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:15.011 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is unset 2024-09-15T00:13:15.046 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:15.046 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:15.056 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:15.072 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:15.070+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:15.148 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:15.145+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:15.166 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:15.166 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:15.175 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:15.206 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 0] out_osds: [7, 1, 2] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-15T00:13:15.206 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:13:15.206 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-15T00:13:15.207 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T00:13:15.207 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T00:13:15.207 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-15T00:13:15.261 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:15.259+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:15.281 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:15.281 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:15.291 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:15.322 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:15.322 INFO:teuthology.orchestra.run.smithi184.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-15T00:13:15.323 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:15.350 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:15.348+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:15.426 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:15.426 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:15.435 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:15.479 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:15.477+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:15.538 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:15.538 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:15.547 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:15.592 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:15.589+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:15.652 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:15.651+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:15.653 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:15.653 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:15.661 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:15.722 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:13:15.789 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:15.786+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:15.807 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:15.807 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:15.817 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:15.858 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:15.856+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:15.867 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:15.867 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:15.884 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:15.884 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:15.900 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:15.901 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:15.917 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:15.917 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:15.933 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:15.934 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:15.950 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:15.950 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:15.955 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:15.954+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:15.965 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:15.965 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:15.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:15.965 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:15.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.966 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9, version: 37'47, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9@37'47, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:15.966 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:15.966 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.1088348910000008, 2024-09-15T00:13:15.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054905559999999997, 2024-09-15T00:13:15.966 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:15.966 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:15.966 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:15.966 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:15.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:15.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:15.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:15.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:15.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:15.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853793+0000", 2024-09-15T00:13:15.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9599999999999999e-07 2024-09-15T00:13:15.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:15.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854798+0000", 2024-09-15T00:13:15.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010050770000000001 2024-09-15T00:13:15.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:15.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854814+0000", 2024-09-15T00:13:15.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6478999999999999e-05 2024-09-15T00:13:15.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:15.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854825+0000", 2024-09-15T00:13:15.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0492999999999999e-05 2024-09-15T00:13:15.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:15.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854889+0000", 2024-09-15T00:13:15.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4134999999999995e-05 2024-09-15T00:13:15.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:15.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854898+0000", 2024-09-15T00:13:15.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6260000000000001e-06 2024-09-15T00:13:15.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:15.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.859283+0000", 2024-09-15T00:13:15.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043852500000000003 2024-09-15T00:13:15.969 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.970 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:15.970 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.970 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15, version: 59'205, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15@59'205, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:15.970 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:15.970 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.9826144360000004, 2024-09-15T00:13:15.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059047939999999997, 2024-09-15T00:13:15.970 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:15.970 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:15.970 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:15.970 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:15.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:15.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:15.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:15.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:15.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:15.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:15.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.44e-07 2024-09-15T00:13:15.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:15.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980977+0000", 2024-09-15T00:13:15.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00096402300000000005 2024-09-15T00:13:15.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:15.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980995+0000", 2024-09-15T00:13:15.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7214000000000002e-05 2024-09-15T00:13:15.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:15.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981007+0000", 2024-09-15T00:13:15.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2028000000000001e-05 2024-09-15T00:13:15.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:15.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981069+0000", 2024-09-15T00:13:15.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1977000000000006e-05 2024-09-15T00:13:15.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:15.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981077+0000", 2024-09-15T00:13:15.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2120000000000003e-06 2024-09-15T00:13:15.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:15.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.985918+0000", 2024-09-15T00:13:15.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048406960000000002 2024-09-15T00:13:15.974 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.974 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:15.974 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.974 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head, version: 62'221, data_included: [0~262144,393216~786432], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head@62'221, size: 1572864, copy_subset: [(0, 1572864)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:15.974 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:15.974 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.9603461709999994, 2024-09-15T00:13:15.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054170770000000002, 2024-09-15T00:13:15.974 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:15.974 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:15.974 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:15.975 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:15.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:15.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:15.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:15.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:15.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:15.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002282+0000", 2024-09-15T00:13:15.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4499999999999997e-07 2024-09-15T00:13:15.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:15.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003196+0000", 2024-09-15T00:13:15.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091476399999999999 2024-09-15T00:13:15.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:15.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003213+0000", 2024-09-15T00:13:15.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6554000000000001e-05 2024-09-15T00:13:15.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:15.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003230+0000", 2024-09-15T00:13:15.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6617999999999999e-05 2024-09-15T00:13:15.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:15.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003292+0000", 2024-09-15T00:13:15.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2193999999999994e-05 2024-09-15T00:13:15.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:15.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003300+0000", 2024-09-15T00:13:15.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1429999999999998e-06 2024-09-15T00:13:15.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:15.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.007698+0000", 2024-09-15T00:13:15.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043981589999999996 2024-09-15T00:13:15.978 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.978 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:15.978 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.978 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:15.978 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:15.978 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.9096160910000002, 2024-09-15T00:13:15.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067633270000000004, 2024-09-15T00:13:15.978 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:15.978 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:15.979 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:15.979 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:15.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:15.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:15.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:15.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:15.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:15.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053012+0000", 2024-09-15T00:13:15.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1659999999999999e-06 2024-09-15T00:13:15.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:15.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054086+0000", 2024-09-15T00:13:15.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010732789999999999 2024-09-15T00:13:15.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:15.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054100+0000", 2024-09-15T00:13:15.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4123e-05 2024-09-15T00:13:15.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:15.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054114+0000", 2024-09-15T00:13:15.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4442e-05 2024-09-15T00:13:15.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:15.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054192+0000", 2024-09-15T00:13:15.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8288000000000007e-05 2024-09-15T00:13:15.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:15.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054200+0000", 2024-09-15T00:13:15.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-15T00:13:15.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.059774+0000", 2024-09-15T00:13:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055741230000000003 2024-09-15T00:13:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.9012987629999998, 2024-09-15T00:13:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049777950000000001, 2024-09-15T00:13:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061330+0000", 2024-09-15T00:13:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.11e-06 2024-09-15T00:13:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062350+0000", 2024-09-15T00:13:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010207969999999999 2024-09-15T00:13:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062365+0000", 2024-09-15T00:13:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4157e-05 2024-09-15T00:13:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062380+0000", 2024-09-15T00:13:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5129e-05 2024-09-15T00:13:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062467+0000", 2024-09-15T00:13:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7247e-05 2024-09-15T00:13:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062475+0000", 2024-09-15T00:13:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4300000000000006e-06 2024-09-15T00:13:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.066306+0000", 2024-09-15T00:13:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038309250000000002 2024-09-15T00:13:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.8954158050000007, 2024-09-15T00:13:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052454889999999999, 2024-09-15T00:13:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067212+0000", 2024-09-15T00:13:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-07 2024-09-15T00:13:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068119+0000", 2024-09-15T00:13:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090671199999999995 2024-09-15T00:13:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068135+0000", 2024-09-15T00:13:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5739e-05 2024-09-15T00:13:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068150+0000", 2024-09-15T00:13:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5693e-05 2024-09-15T00:13:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068236+0000", 2024-09-15T00:13:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6100000000000006e-05 2024-09-15T00:13:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068245+0000", 2024-09-15T00:13:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1960000000000008e-06 2024-09-15T00:13:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.072457+0000", 2024-09-15T00:13:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042123719999999998 2024-09-15T00:13:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.8894374559999996, 2024-09-15T00:13:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049532020000000003, 2024-09-15T00:13:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:13:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074164+0000", 2024-09-15T00:13:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097354400000000002 2024-09-15T00:13:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074178+0000", 2024-09-15T00:13:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4134999999999999e-05 2024-09-15T00:13:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074192+0000", 2024-09-15T00:13:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.451e-05 2024-09-15T00:13:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074272+0000", 2024-09-15T00:13:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9989999999999998e-05 2024-09-15T00:13:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074280+0000", 2024-09-15T00:13:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9990000000000008e-06 2024-09-15T00:13:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078143+0000", 2024-09-15T00:13:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00386258 2024-09-15T00:13:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [4194304~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.8837266439999993, 2024-09-15T00:13:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049408730000000001, 2024-09-15T00:13:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2399999999999999e-07 2024-09-15T00:13:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079875+0000", 2024-09-15T00:13:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097349100000000005 2024-09-15T00:13:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079889+0000", 2024-09-15T00:13:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4148000000000001e-05 2024-09-15T00:13:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079904+0000", 2024-09-15T00:13:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5159e-05 2024-09-15T00:13:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079964+0000", 2024-09-15T00:13:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0204999999999997e-05 2024-09-15T00:13:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079971+0000", 2024-09-15T00:13:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2490000000000004e-06 2024-09-15T00:13:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.083841+0000", 2024-09-15T00:13:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038701970000000001 2024-09-15T00:13:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [5242880~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 6291456, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.8780216640000003, 2024-09-15T00:13:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052525740000000003, 2024-09-15T00:13:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:13:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085579+0000", 2024-09-15T00:13:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097307600000000002 2024-09-15T00:13:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085592+0000", 2024-09-15T00:13:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3277e-05 2024-09-15T00:13:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085608+0000", 2024-09-15T00:13:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5773e-05 2024-09-15T00:13:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085687+0000", 2024-09-15T00:13:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8956999999999996e-05 2024-09-15T00:13:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085694+0000", 2024-09-15T00:13:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.956e-06 2024-09-15T00:13:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.089858+0000", 2024-09-15T00:13:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041641159999999998 2024-09-15T00:13:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:172552 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-15T00:13:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7735643559999996, 2024-09-15T00:13:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.997031471, 2024-09-15T00:13:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 172552 2024-09-15T00:13:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189064+0000", 2024-09-15T00:13:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.198e-06 2024-09-15T00:13:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189069+0000", 2024-09-15T00:13:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9069999999999998e-06 2024-09-15T00:13:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189080+0000", 2024-09-15T00:13:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:13:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189086+0000", 2024-09-15T00:13:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5429999999999999e-06 2024-09-15T00:13:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189128+0000", 2024-09-15T00:13:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2255000000000001e-05 2024-09-15T00:13:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189135+0000", 2024-09-15T00:13:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6200000000000001e-06 2024-09-15T00:13:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185787+0000", 2024-09-15T00:13:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99665157199999999 2024-09-15T00:13:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185926+0000", 2024-09-15T00:13:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013970399999999999 2024-09-15T00:13:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.186094+0000", 2024-09-15T00:13:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016813099999999999 2024-09-15T00:13:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9, version: 37'60, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9@37'60, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.7147039970000009, 2024-09-15T00:13:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049454060000000003, 2024-09-15T00:13:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247924+0000", 2024-09-15T00:13:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:13:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248394+0000", 2024-09-15T00:13:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047063399999999999 2024-09-15T00:13:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248411+0000", 2024-09-15T00:13:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6384000000000001e-05 2024-09-15T00:13:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248441+0000", 2024-09-15T00:13:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0559999999999999e-05 2024-09-15T00:13:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248499+0000", 2024-09-15T00:13:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7274000000000001e-05 2024-09-15T00:13:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248507+0000", 2024-09-15T00:13:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2390000000000002e-06 2024-09-15T00:13:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.252869+0000", 2024-09-15T00:13:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043617200000000004 2024-09-15T00:13:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head, version: 59'393, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head@59'393, size: 5701632, copy_subset: [(0, 5701632)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5852557300000001, 2024-09-15T00:13:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004807469, 2024-09-15T00:13:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4799999999999998e-07 2024-09-15T00:13:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377830+0000", 2024-09-15T00:13:16.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000457954 2024-09-15T00:13:16.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377847+0000", 2024-09-15T00:13:16.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6741000000000001e-05 2024-09-15T00:13:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377858+0000", 2024-09-15T00:13:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0951999999999999e-05 2024-09-15T00:13:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377929+0000", 2024-09-15T00:13:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0697000000000004e-05 2024-09-15T00:13:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377941+0000", 2024-09-15T00:13:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2529e-05 2024-09-15T00:13:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.382179+0000", 2024-09-15T00:13:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042379480000000001 2024-09-15T00:13:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head, version: 88'473, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head@88'473, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5031362969999993, 2024-09-15T00:13:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005823272, 2024-09-15T00:13:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459492+0000", 2024-09-15T00:13:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9800000000000003e-07 2024-09-15T00:13:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459940+0000", 2024-09-15T00:13:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044854699999999999 2024-09-15T00:13:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459957+0000", 2024-09-15T00:13:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6586e-05 2024-09-15T00:13:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459968+0000", 2024-09-15T00:13:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.145e-05 2024-09-15T00:13:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460043+0000", 2024-09-15T00:13:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4451999999999998e-05 2024-09-15T00:13:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460057+0000", 2024-09-15T00:13:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.417e-05 2024-09-15T00:13:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.465314+0000", 2024-09-15T00:13:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052574689999999999 2024-09-15T00:13:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2679601159999994, 2024-09-15T00:13:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051811399999999999, 2024-09-15T00:13:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694668+0000", 2024-09-15T00:13:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4600000000000005e-07 2024-09-15T00:13:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695136+0000", 2024-09-15T00:13:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00046864399999999999 2024-09-15T00:13:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695153+0000", 2024-09-15T00:13:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6218e-05 2024-09-15T00:13:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695189+0000", 2024-09-15T00:13:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6393000000000002e-05 2024-09-15T00:13:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695263+0000", 2024-09-15T00:13:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4175e-05 2024-09-15T00:13:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695271+0000", 2024-09-15T00:13:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2800000000000003e-06 2024-09-15T00:13:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.699848+0000", 2024-09-15T00:13:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045768839999999998 2024-09-15T00:13:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2598619079999995, 2024-09-15T00:13:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048716510000000003, 2024-09-15T00:13:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702783+0000", 2024-09-15T00:13:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7268000000000001e-05 2024-09-15T00:13:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703221+0000", 2024-09-15T00:13:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00043866899999999998 2024-09-15T00:13:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703237+0000", 2024-09-15T00:13:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5553000000000002e-05 2024-09-15T00:13:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703247+0000", 2024-09-15T00:13:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0392e-05 2024-09-15T00:13:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703358+0000", 2024-09-15T00:13:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110676 2024-09-15T00:13:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703369+0000", 2024-09-15T00:13:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0716e-05 2024-09-15T00:13:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.707637+0000", 2024-09-15T00:13:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042683770000000003 2024-09-15T00:13:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18, version: 62'274, data_included: [0~786432], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18@62'274, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.2137808260000007, 2024-09-15T00:13:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005453014, 2024-09-15T00:13:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748847+0000", 2024-09-15T00:13:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8299999999999997e-07 2024-09-15T00:13:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749010+0000", 2024-09-15T00:13:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000163144 2024-09-15T00:13:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749027+0000", 2024-09-15T00:13:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6518999999999999e-05 2024-09-15T00:13:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749037+0000", 2024-09-15T00:13:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0438000000000001e-05 2024-09-15T00:13:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749112+0000", 2024-09-15T00:13:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4946999999999999e-05 2024-09-15T00:13:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749123+0000", 2024-09-15T00:13:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1014e-05 2024-09-15T00:13:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.754299+0000", 2024-09-15T00:13:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051763690000000001 2024-09-15T00:13:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20, version: 80'334, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20@80'334, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 20=[]:{20: [20]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1591437819999992, 2024-09-15T00:13:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048933570000000001, 2024-09-15T00:13:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803484+0000", 2024-09-15T00:13:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:13:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803938+0000", 2024-09-15T00:13:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045364599999999999 2024-09-15T00:13:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803954+0000", 2024-09-15T00:13:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6456999999999998e-05 2024-09-15T00:13:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803965+0000", 2024-09-15T00:13:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.076e-05 2024-09-15T00:13:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804023+0000", 2024-09-15T00:13:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7787000000000001e-05 2024-09-15T00:13:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804031+0000", 2024-09-15T00:13:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2160000000000006e-06 2024-09-15T00:13:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.808377+0000", 2024-09-15T00:13:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043458969999999996 2024-09-15T00:13:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [0~131072,196608~786432,1048576~131072], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1223571860000003, 2024-09-15T00:13:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047737029999999998, 2024-09-15T00:13:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840271+0000", 2024-09-15T00:13:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:13:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840721+0000", 2024-09-15T00:13:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045029800000000002 2024-09-15T00:13:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840738+0000", 2024-09-15T00:13:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6671999999999999e-05 2024-09-15T00:13:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840749+0000", 2024-09-15T00:13:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1251e-05 2024-09-15T00:13:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840817+0000", 2024-09-15T00:13:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8046000000000005e-05 2024-09-15T00:13:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840825+0000", 2024-09-15T00:13:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2519999999999999e-06 2024-09-15T00:13:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.845044+0000", 2024-09-15T00:13:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042185900000000004 2024-09-15T00:13:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [1179648~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2228224, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1143331389999993, 2024-09-15T00:13:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053056819999999999, 2024-09-15T00:13:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848295+0000", 2024-09-15T00:13:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4000000000000001e-07 2024-09-15T00:13:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848728+0000", 2024-09-15T00:13:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000433709 2024-09-15T00:13:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848745+0000", 2024-09-15T00:13:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6167999999999998e-05 2024-09-15T00:13:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848788+0000", 2024-09-15T00:13:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3269000000000002e-05 2024-09-15T00:13:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848857+0000", 2024-09-15T00:13:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9453000000000001e-05 2024-09-15T00:13:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848866+0000", 2024-09-15T00:13:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2779999999999993e-06 2024-09-15T00:13:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.853600+0000", 2024-09-15T00:13:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047341650000000002 2024-09-15T00:13:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head, version: 93'455, data_included: [0~98304,114688~16384,147456~589824,753664~65536,835584~16384,868352~262144], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head@93'455, size: 2342912, copy_subset: [(0, 2342912)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1130496, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.0551991130000005, 2024-09-15T00:13:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063250900000000002, 2024-09-15T00:13:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907429+0000", 2024-09-15T00:13:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.85e-07 2024-09-15T00:13:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908008+0000", 2024-09-15T00:13:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00057949599999999998 2024-09-15T00:13:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908026+0000", 2024-09-15T00:13:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7700999999999999e-05 2024-09-15T00:13:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908038+0000", 2024-09-15T00:13:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2131e-05 2024-09-15T00:13:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908101+0000", 2024-09-15T00:13:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2596999999999997e-05 2024-09-15T00:13:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908110+0000", 2024-09-15T00:13:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8540000000000003e-06 2024-09-15T00:13:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.913753+0000", 2024-09-15T00:13:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056435260000000003 2024-09-15T00:13:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.049 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:16.050 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:16.050 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:16.051 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:16.051 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:16.052 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:16.056 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:16.055+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:16.157 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:16.155+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:16.200 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:16.200 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:16.208 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:16.258 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:16.256+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:16.353 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:16.354 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:16.359 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:16.357+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:16.362 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:16.459 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:16.457+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:16.467 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133834 3.1b e85/69 3:da9a78a3:::rbd_data.11395dfcf2b8.0000000000000000:head v 85'356, mlcod=85'354)", 2024-09-15T00:13:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.901938702000002, 2024-09-15T00:13:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026079054000000001, 2024-09-15T00:13:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563338+0000", 2024-09-15T00:13:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.418e-06 2024-09-15T00:13:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569385+0000", 2024-09-15T00:13:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006047335 2024-09-15T00:13:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569418+0000", 2024-09-15T00:13:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2957999999999999e-05 2024-09-15T00:13:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569432+0000", 2024-09-15T00:13:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3718e-05 2024-09-15T00:13:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569519+0000", 2024-09-15T00:13:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7022999999999997e-05 2024-09-15T00:13:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569540+0000", 2024-09-15T00:13:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1426000000000001e-05 2024-09-15T00:13:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589362+0000", 2024-09-15T00:13:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019821398 2024-09-15T00:13:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589415+0000", 2024-09-15T00:13:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3777999999999999e-05 2024-09-15T00:13:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133836 3.1 e85/67 3:849f9a98:::rbd_data.11395dfcf2b8.0000000000000002:head v 85'304, mlcod=84'301)", 2024-09-15T00:13:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.894243791999997, 2024-09-15T00:13:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023231600000000002, 2024-09-15T00:13:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571032+0000", 2024-09-15T00:13:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-15T00:13:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576635+0000", 2024-09-15T00:13:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056028700000000002 2024-09-15T00:13:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576650+0000", 2024-09-15T00:13:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.505e-05 2024-09-15T00:13:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576658+0000", 2024-09-15T00:13:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6879999999999999e-06 2024-09-15T00:13:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576717+0000", 2024-09-15T00:13:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175000000000002e-05 2024-09-15T00:13:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576742+0000", 2024-09-15T00:13:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4734000000000001e-05 2024-09-15T00:13:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594234+0000", 2024-09-15T00:13:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017492527000000001 2024-09-15T00:13:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594263+0000", 2024-09-15T00:13:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8629e-05 2024-09-15T00:13:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146609 3.b e98/69 3:d39b42ba:::rbd_data.11394501364.0000000000000001:head v 98'283, mlcod=93'281)", 2024-09-15T00:13:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.984525677000001, 2024-09-15T00:13:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020892945, 2024-09-15T00:13:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480751+0000", 2024-09-15T00:13:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6509999999999999e-06 2024-09-15T00:13:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482738+0000", 2024-09-15T00:13:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019867410000000002 2024-09-15T00:13:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482754+0000", 2024-09-15T00:13:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6171000000000002e-05 2024-09-15T00:13:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482769+0000", 2024-09-15T00:13:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4688000000000001e-05 2024-09-15T00:13:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482835+0000", 2024-09-15T00:13:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6526000000000006e-05 2024-09-15T00:13:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482858+0000", 2024-09-15T00:13:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2959000000000002e-05 2024-09-15T00:13:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501587+0000", 2024-09-15T00:13:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018728471 2024-09-15T00:13:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501642+0000", 2024-09-15T00:13:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5738000000000001e-05 2024-09-15T00:13:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146835 3.12 e98/68 3:4eee39d6:::rbd_data.11394501364.0000000000000000:head v 98'474, mlcod=93'472)", 2024-09-15T00:13:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.875898407000001, 2024-09-15T00:13:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020080434000000001, 2024-09-15T00:13:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589378+0000", 2024-09-15T00:13:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7770000000000001e-06 2024-09-15T00:13:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590678+0000", 2024-09-15T00:13:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012991750000000001 2024-09-15T00:13:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590699+0000", 2024-09-15T00:13:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1353e-05 2024-09-15T00:13:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590710+0000", 2024-09-15T00:13:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1245e-05 2024-09-15T00:13:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590800+0000", 2024-09-15T00:13:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9963000000000006e-05 2024-09-15T00:13:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590823+0000", 2024-09-15T00:13:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3289e-05 2024-09-15T00:13:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609368+0000", 2024-09-15T00:13:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018545078 2024-09-15T00:13:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609457+0000", 2024-09-15T00:13:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8554000000000006e-05 2024-09-15T00:13:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146837 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7201319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.865118943999999, 2024-09-15T00:13:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024751038, 2024-09-15T00:13:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146837 2024-09-15T00:13:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600157+0000", 2024-09-15T00:13:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:13:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601965+0000", 2024-09-15T00:13:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001807613 2024-09-15T00:13:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601985+0000", 2024-09-15T00:13:16.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0679e-05 2024-09-15T00:13:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601998+0000", 2024-09-15T00:13:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2955999999999999e-05 2024-09-15T00:13:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602070+0000", 2024-09-15T00:13:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1958000000000005e-05 2024-09-15T00:13:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602237+0000", 2024-09-15T00:13:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016621299999999999 2024-09-15T00:13:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602799+0000", 2024-09-15T00:13:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000562723 2024-09-15T00:13:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:16.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.620811+0000", 2024-09-15T00:13:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018011207000000001 2024-09-15T00:13:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624793+0000", 2024-09-15T00:13:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0039828770000000001 2024-09-15T00:13:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624855+0000", 2024-09-15T00:13:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.143e-05 2024-09-15T00:13:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624907+0000", 2024-09-15T00:13:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2262000000000002e-05 2024-09-15T00:13:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146936 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.711605625000001, 2024-09-15T00:13:16.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056847126999999997, 2024-09-15T00:13:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146936 2024-09-15T00:13:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753671+0000", 2024-09-15T00:13:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2839999999999999e-06 2024-09-15T00:13:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753677+0000", 2024-09-15T00:13:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0260000000000004e-06 2024-09-15T00:13:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753691+0000", 2024-09-15T00:13:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3891e-05 2024-09-15T00:13:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753698+0000", 2024-09-15T00:13:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8480000000000001e-06 2024-09-15T00:13:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753750+0000", 2024-09-15T00:13:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1338000000000002e-05 2024-09-15T00:13:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753814+0000", 2024-09-15T00:13:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4642999999999998e-05 2024-09-15T00:13:16.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.756359+0000", 2024-09-15T00:13:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025450619999999998 2024-09-15T00:13:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810435+0000", 2024-09-15T00:13:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.054075397999999997 2024-09-15T00:13:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810446+0000", 2024-09-15T00:13:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1005000000000001e-05 2024-09-15T00:13:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810482+0000", 2024-09-15T00:13:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6494e-05 2024-09-15T00:13:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810517+0000", 2024-09-15T00:13:16.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4137999999999997e-05 2024-09-15T00:13:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:146936 3.6 e98/97 ondisk, result = 0)", 2024-09-15T00:13:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.701822571000001, 2024-09-15T00:13:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046987205999999997, 2024-09-15T00:13:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.502 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763453+0000", 2024-09-15T00:13:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5199999999999996e-07 2024-09-15T00:13:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763457+0000", 2024-09-15T00:13:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8569999999999997e-06 2024-09-15T00:13:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763469+0000", 2024-09-15T00:13:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1977999999999999e-05 2024-09-15T00:13:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763476+0000", 2024-09-15T00:13:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:13:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810423+0000", 2024-09-15T00:13:16.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946993999999999 2024-09-15T00:13:16.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810431+0000", 2024-09-15T00:13:16.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0339999999999997e-06 2024-09-15T00:13:16.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810440+0000", 2024-09-15T00:13:16.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9989999999999997e-06 2024-09-15T00:13:16.507 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.507 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.507 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.507 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:151036 3.1f e103/68 3:f84a1a09:::rbd_data.113958cb0e55.000000000000000c:head v 103'452, mlcod=100'450)", 2024-09-15T00:13:16.507 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:16.507 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.128476342999999, 2024-09-15T00:13:16.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020618753, 2024-09-15T00:13:16.507 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.507 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.507 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.508 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:16.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:16.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:16.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:13:16.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340692+0000", 2024-09-15T00:13:16.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038928890000000001 2024-09-15T00:13:16.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340704+0000", 2024-09-15T00:13:16.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1497e-05 2024-09-15T00:13:16.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340709+0000", 2024-09-15T00:13:16.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1529999999999996e-06 2024-09-15T00:13:16.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340789+0000", 2024-09-15T00:13:16.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9907999999999995e-05 2024-09-15T00:13:16.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340810+0000", 2024-09-15T00:13:16.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1837000000000001e-05 2024-09-15T00:13:16.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357368+0000", 2024-09-15T00:13:16.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016557199000000002 2024-09-15T00:13:16.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357417+0000", 2024-09-15T00:13:16.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9783000000000001e-05 2024-09-15T00:13:16.512 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.512 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.512 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.512 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:151042 3.6 3:6419cde4:::rbd_data.113958cb0e55.0000000000000031:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:16.512 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:16.512 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.099398225000002, 2024-09-15T00:13:16.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025627375000000001, 2024-09-15T00:13:16.513 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.513 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.513 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.513 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:16.513 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:16.513 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:16.513 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 151042 2024-09-15T00:13:16.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.513 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:16.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:16.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:16.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7899999999999998e-07 2024-09-15T00:13:16.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365881+0000", 2024-09-15T00:13:16.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9770000000000002e-06 2024-09-15T00:13:16.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365890+0000", 2024-09-15T00:13:16.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5920000000000001e-06 2024-09-15T00:13:16.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365894+0000", 2024-09-15T00:13:16.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4569999999999998e-06 2024-09-15T00:13:16.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365927+0000", 2024-09-15T00:13:16.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2036000000000003e-05 2024-09-15T00:13:16.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365970+0000", 2024-09-15T00:13:16.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3197e-05 2024-09-15T00:13:16.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:16.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.367432+0000", 2024-09-15T00:13:16.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014621720000000001 2024-09-15T00:13:16.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:16.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391358+0000", 2024-09-15T00:13:16.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023925921999999999 2024-09-15T00:13:16.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:16.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391415+0000", 2024-09-15T00:13:16.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7253999999999998e-05 2024-09-15T00:13:16.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391456+0000", 2024-09-15T00:13:16.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0497999999999997e-05 2024-09-15T00:13:16.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391504+0000", 2024-09-15T00:13:16.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8591000000000003e-05 2024-09-15T00:13:16.519 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.519 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.519 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.519 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:151042 3.6 e103/97 ondisk, result = 0)", 2024-09-15T00:13:16.519 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:16.519 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.093655212000002, 2024-09-15T00:13:16.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019892730000000001, 2024-09-15T00:13:16.519 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.519 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:16.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8100000000000003e-07 2024-09-15T00:13:16.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371623+0000", 2024-09-15T00:13:16.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6290000000000002e-06 2024-09-15T00:13:16.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371632+0000", 2024-09-15T00:13:16.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6379999999999993e-06 2024-09-15T00:13:16.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371636+0000", 2024-09-15T00:13:16.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:13:16.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391403+0000", 2024-09-15T00:13:16.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019766982999999998 2024-09-15T00:13:16.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391411+0000", 2024-09-15T00:13:16.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7309999999999993e-06 2024-09-15T00:13:16.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391513+0000", 2024-09-15T00:13:16.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101648 2024-09-15T00:13:16.523 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.524 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.524 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.524 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152025 3.3 e103/68 3:c0279197:::rbd_data.113958cb0e55.0000000000000007:head v 103'464, mlcod=102'461)", 2024-09-15T00:13:16.524 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:16.524 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.949167616, 2024-09-15T00:13:16.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021264476000000001, 2024-09-15T00:13:16.524 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.525 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.525 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.525 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:16.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:16.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516109+0000", 2024-09-15T00:13:16.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.666e-06 2024-09-15T00:13:16.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.519997+0000", 2024-09-15T00:13:16.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003888449 2024-09-15T00:13:16.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520016+0000", 2024-09-15T00:13:16.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8059000000000001e-05 2024-09-15T00:13:16.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520026+0000", 2024-09-15T00:13:16.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9599999999999995e-06 2024-09-15T00:13:16.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520089+0000", 2024-09-15T00:13:16.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3074e-05 2024-09-15T00:13:16.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520108+0000", 2024-09-15T00:13:16.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8981999999999999e-05 2024-09-15T00:13:16.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537326+0000", 2024-09-15T00:13:16.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017218102999999998 2024-09-15T00:13:16.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537372+0000", 2024-09-15T00:13:16.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6183000000000002e-05 2024-09-15T00:13:16.532 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.532 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.532 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.532 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152284 3.f e103/68 3:f4272b5d:::rbd_data.113958cb0e55.0000000000000006:head v 103'370, mlcod=103'368)", 2024-09-15T00:13:16.532 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:16.532 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.900691084999998, 2024-09-15T00:13:16.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020145026999999999, 2024-09-15T00:13:16.532 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.532 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.533 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.533 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:16.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:16.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564585+0000", 2024-09-15T00:13:16.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7000000000000003e-07 2024-09-15T00:13:16.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568352+0000", 2024-09-15T00:13:16.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037675909999999998 2024-09-15T00:13:16.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568370+0000", 2024-09-15T00:13:16.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7158000000000001e-05 2024-09-15T00:13:16.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568378+0000", 2024-09-15T00:13:16.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6449999999999994e-06 2024-09-15T00:13:16.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568454+0000", 2024-09-15T00:13:16.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5285999999999997e-05 2024-09-15T00:13:16.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568472+0000", 2024-09-15T00:13:16.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8447000000000001e-05 2024-09-15T00:13:16.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584693+0000", 2024-09-15T00:13:16.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016221467999999999 2024-09-15T00:13:16.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584729+0000", 2024-09-15T00:13:16.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5462e-05 2024-09-15T00:13:16.538 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.538 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.538 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.538 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152500 3.17 e103/68 3:edff5adf:::rbd_data.113958cb0e55.000000000000000d:head v 103'461, mlcod=103'459)", 2024-09-15T00:13:16.538 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:16.538 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.823704500000002, 2024-09-15T00:13:16.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020217809, 2024-09-15T00:13:16.538 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.538 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.539 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.539 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:16.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:16.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641572+0000", 2024-09-15T00:13:16.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-15T00:13:16.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644961+0000", 2024-09-15T00:13:16.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033882610000000001 2024-09-15T00:13:16.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644979+0000", 2024-09-15T00:13:16.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8179e-05 2024-09-15T00:13:16.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644987+0000", 2024-09-15T00:13:16.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8860000000000004e-06 2024-09-15T00:13:16.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645044+0000", 2024-09-15T00:13:16.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7141000000000001e-05 2024-09-15T00:13:16.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645062+0000", 2024-09-15T00:13:16.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8258e-05 2024-09-15T00:13:16.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661749+0000", 2024-09-15T00:13:16.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016686553999999999 2024-09-15T00:13:16.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661788+0000", 2024-09-15T00:13:16.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9690000000000001e-05 2024-09-15T00:13:16.543 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.543 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.543 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.544 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153171 3.18 e103/69 3:1db186e8:::rbd_data.113958cb0e55.0000000000000002:head v 103'520, mlcod=103'518)", 2024-09-15T00:13:16.544 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:16.544 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.685219593999999, 2024-09-15T00:13:16.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022266838000000001, 2024-09-15T00:13:16.544 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.544 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.544 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.544 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:16.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:16.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780057+0000", 2024-09-15T00:13:16.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5880000000000001e-06 2024-09-15T00:13:16.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781273+0000", 2024-09-15T00:13:16.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001216087 2024-09-15T00:13:16.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781311+0000", 2024-09-15T00:13:16.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7656e-05 2024-09-15T00:13:16.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781325+0000", 2024-09-15T00:13:16.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4508999999999999e-05 2024-09-15T00:13:16.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781403+0000", 2024-09-15T00:13:16.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7927000000000001e-05 2024-09-15T00:13:16.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781422+0000", 2024-09-15T00:13:16.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8575000000000001e-05 2024-09-15T00:13:16.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802249+0000", 2024-09-15T00:13:16.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020827577 2024-09-15T00:13:16.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802322+0000", 2024-09-15T00:13:16.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2918999999999998e-05 2024-09-15T00:13:16.549 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.549 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.549 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.549 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153772 3.12 e103/68 3:4911a4f6:::rbd_data.113958cb0e55.0000000000000001:head v 103'490, mlcod=103'488)", 2024-09-15T00:13:16.549 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:16.549 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.582348967000001, 2024-09-15T00:13:16.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021181004, 2024-09-15T00:13:16.549 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.549 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.549 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.550 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:16.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:16.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882927+0000", 2024-09-15T00:13:16.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.426e-06 2024-09-15T00:13:16.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883966+0000", 2024-09-15T00:13:16.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010384610000000001 2024-09-15T00:13:16.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883985+0000", 2024-09-15T00:13:16.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9276e-05 2024-09-15T00:13:16.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883994+0000", 2024-09-15T00:13:16.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9490000000000002e-06 2024-09-15T00:13:16.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884054+0000", 2024-09-15T00:13:16.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9419999999999997e-05 2024-09-15T00:13:16.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884073+0000", 2024-09-15T00:13:16.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9137e-05 2024-09-15T00:13:16.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904038+0000", 2024-09-15T00:13:16.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019965092 2024-09-15T00:13:16.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904107+0000", 2024-09-15T00:13:16.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9243000000000001e-05 2024-09-15T00:13:16.554 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.554 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.554 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.554 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163112 3.3 e106/68 3:c668f3d5:::rbd_data.11395d771336.0000000000000001:head v 106'489, mlcod=105'487)", 2024-09-15T00:13:16.554 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:16.555 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.000451697999999, 2024-09-15T00:13:16.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019317183000000002, 2024-09-15T00:13:16.555 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.555 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.555 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.555 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:16.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:16.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464824+0000", 2024-09-15T00:13:16.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1100000000000005e-07 2024-09-15T00:13:16.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466874+0000", 2024-09-15T00:13:16.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002049497 2024-09-15T00:13:16.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466884+0000", 2024-09-15T00:13:16.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0424999999999999e-05 2024-09-15T00:13:16.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466890+0000", 2024-09-15T00:13:16.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-15T00:13:16.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466944+0000", 2024-09-15T00:13:16.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4592999999999997e-05 2024-09-15T00:13:16.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466958+0000", 2024-09-15T00:13:16.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3993e-05 2024-09-15T00:13:16.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484068+0000", 2024-09-15T00:13:16.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017109994 2024-09-15T00:13:16.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484140+0000", 2024-09-15T00:13:16.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2224000000000005e-05 2024-09-15T00:13:16.559 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.559 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.560 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.560 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163119 3.0 e106/94 3:06118613:::rbd_data.11395d771336.0000000000000004:head v 106'549, mlcod=105'547)", 2024-09-15T00:13:16.560 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:16.560 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.988836495000001, 2024-09-15T00:13:16.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017175867000000001, 2024-09-15T00:13:16.560 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.560 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.560 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.561 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:16.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:16.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:16.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6700000000000002e-07 2024-09-15T00:13:16.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477786+0000", 2024-09-15T00:13:16.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013468149999999999 2024-09-15T00:13:16.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477797+0000", 2024-09-15T00:13:16.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0635000000000001e-05 2024-09-15T00:13:16.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477805+0000", 2024-09-15T00:13:16.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2989999999999997e-06 2024-09-15T00:13:16.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477859+0000", 2024-09-15T00:13:16.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3699000000000002e-05 2024-09-15T00:13:16.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477878+0000", 2024-09-15T00:13:16.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9335e-05 2024-09-15T00:13:16.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493560+0000", 2024-09-15T00:13:16.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015681748999999998 2024-09-15T00:13:16.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493614+0000", 2024-09-15T00:13:16.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4367999999999999e-05 2024-09-15T00:13:16.565 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.565 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.565 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.565 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.5.0:5005 3.17 e110/68 3:eb0359e8:::rbd_data.113964deed84.0000000000000251:35 v 110'537, mlcod=109'534)", 2024-09-15T00:13:16.565 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:16.565 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.328986123, 2024-09-15T00:13:16.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015446896999999999, 2024-09-15T00:13:16.566 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.566 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.566 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.566 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:16.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:16.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136290+0000", 2024-09-15T00:13:16.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7400000000000003e-07 2024-09-15T00:13:16.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136294+0000", 2024-09-15T00:13:16.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4989999999999996e-06 2024-09-15T00:13:16.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136304+0000", 2024-09-15T00:13:16.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0319999999999999e-05 2024-09-15T00:13:16.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136310+0000", 2024-09-15T00:13:16.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.981e-06 2024-09-15T00:13:16.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136350+0000", 2024-09-15T00:13:16.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0088e-05 2024-09-15T00:13:16.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136364+0000", 2024-09-15T00:13:16.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3954999999999999e-05 2024-09-15T00:13:16.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151681+0000", 2024-09-15T00:13:16.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015316969999999999 2024-09-15T00:13:16.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151736+0000", 2024-09-15T00:13:16.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4410000000000003e-05 2024-09-15T00:13:16.570 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.570 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.570 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.570 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1758 3.10 e110/68 3:08149147:::rbd_data.113964deed84.0000000000000055:35 v 110'594, mlcod=109'591)", 2024-09-15T00:13:16.570 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:16.571 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.324687897, 2024-09-15T00:13:16.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016278652000000001, 2024-09-15T00:13:16.571 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.571 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.571 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.571 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:16.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:16.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:16.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:16.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140599+0000", 2024-09-15T00:13:16.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1109e-05 2024-09-15T00:13:16.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140605+0000", 2024-09-15T00:13:16.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-15T00:13:16.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140612+0000", 2024-09-15T00:13:16.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6089999999999997e-06 2024-09-15T00:13:16.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140656+0000", 2024-09-15T00:13:16.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3754999999999997e-05 2024-09-15T00:13:16.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140674+0000", 2024-09-15T00:13:16.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7640999999999999e-05 2024-09-15T00:13:16.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156823+0000", 2024-09-15T00:13:16.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016149047999999999 2024-09-15T00:13:16.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156866+0000", 2024-09-15T00:13:16.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3205999999999999e-05 2024-09-15T00:13:16.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.575 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.576 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.4.0:5722 3.e e110/95 3:74a58daf:::rbd_data.113964deed84.00000000000000c1:35 v 110'1287, mlcod=109'1284)", 2024-09-15T00:13:16.576 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:16.576 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315613541000001, 2024-09-15T00:13:16.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01830555, 2024-09-15T00:13:16.576 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:16.576 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:16.576 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:16.576 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:16.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:16.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:16.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:16.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:16.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:16.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:16.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149662+0000", 2024-09-15T00:13:16.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:13:16.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:16.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149666+0000", 2024-09-15T00:13:16.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2220000000000004e-06 2024-09-15T00:13:16.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:16.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149672+0000", 2024-09-15T00:13:16.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3410000000000002e-06 2024-09-15T00:13:16.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:16.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149676+0000", 2024-09-15T00:13:16.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.566e-06 2024-09-15T00:13:16.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:16.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149709+0000", 2024-09-15T00:13:16.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2403000000000001e-05 2024-09-15T00:13:16.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:16.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149719+0000", 2024-09-15T00:13:16.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0608000000000001e-05 2024-09-15T00:13:16.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:16.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167944+0000", 2024-09-15T00:13:16.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018225183999999998 2024-09-15T00:13:16.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:16.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:16.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:16.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167967+0000", 2024-09-15T00:13:16.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2705e-05 2024-09-15T00:13:16.580 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.581 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.581 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.581 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:16.581 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:16.581 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:16.583 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:16.583 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:16.580+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:16.683 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:16.681+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:16.726 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:16.726 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:16.726 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:16.726 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:16.734 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:16.818 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:16.816+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:16.878 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:16.878 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:16.878 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:16.878 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:16.878 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:16.885 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:16.882+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:16.887 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:16.986 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:16.983+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:17.031 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:13:17.036 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:17.037 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:17.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:17.037 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:17.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.037 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:17.037 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:17.037 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 110.85440890300001, 2024-09-15T00:13:17.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:13:17.037 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.037 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.037 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:17.038 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.038 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.038 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.038 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:13:17.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.038 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:17.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:17.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:13:17.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:13:17.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:13:17.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:13:17.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:13:17.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:13:17.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:13:17.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:13:17.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:13:17.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:13:17.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:13:17.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:13:17.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:17.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:13:17.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:13:17.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:17.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:13:17.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:13:17.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:17.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:13:17.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:13:17.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:17.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:13:17.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:13:17.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:13:17.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:13:17.044 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.044 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.044 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.044 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:13:17.045 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:17.045 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.124567724, 2024-09-15T00:13:17.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:13:17.045 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.045 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.045 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:17.045 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.045 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.045 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.045 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:13:17.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.045 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:17.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:17.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:13:17.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:13:17.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:13:17.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:13:17.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:13:17.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:13:17.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:13:17.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:13:17.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:13:17.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:13:17.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:13:17.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:13:17.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:17.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:13:17.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:13:17.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:17.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:13:17.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:13:17.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:17.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:13:17.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:13:17.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:17.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:13:17.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:13:17.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:13:17.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:13:17.051 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.051 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.051 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.051 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:13:17.051 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:17.051 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.11760936100001, 2024-09-15T00:13:17.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:13:17.052 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.052 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.052 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.052 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:17.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:17.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:13:17.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:13:17.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:13:17.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:13:17.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:13:17.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:13:17.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:13:17.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:13:17.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:13:17.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:13:17.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:13:17.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:13:17.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:13:17.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:13:17.056 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.056 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.056 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.056 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:17.056 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:17.056 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.370420523999996, 2024-09-15T00:13:17.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:13:17.056 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.056 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.057 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:17.057 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.057 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.057 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.057 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:13:17.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.057 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:17.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:17.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:13:17.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:13:17.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:13:17.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:13:17.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:13:17.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:13:17.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:13:17.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:13:17.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:13:17.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:13:17.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:13:17.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:13:17.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:13:17.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:13:17.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:13:17.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:17.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:13:17.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:13:17.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:17.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:13:17.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:13:17.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:17.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:13:17.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:13:17.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:13:17.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:13:17.062 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.063 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.063 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.063 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97493 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:17.063 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:17.063 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.180837804000006, 2024-09-15T00:13:17.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033259141999999998, 2024-09-15T00:13:17.063 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.063 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.063 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.063 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.064 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.064 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.064 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97493 2024-09-15T00:13:17.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.064 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:17.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:17.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:17.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-15T00:13:17.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853401+0000", 2024-09-15T00:13:17.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:13:17.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853407+0000", 2024-09-15T00:13:17.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-15T00:13:17.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853411+0000", 2024-09-15T00:13:17.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.481e-06 2024-09-15T00:13:17.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886544+0000", 2024-09-15T00:13:17.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033132861999999999 2024-09-15T00:13:17.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886587+0000", 2024-09-15T00:13:17.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2942000000000003e-05 2024-09-15T00:13:17.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886655+0000", 2024-09-15T00:13:17.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7711000000000003e-05 2024-09-15T00:13:17.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.068 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.068 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97494 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:17.068 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:17.068 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.180816680999996, 2024-09-15T00:13:17.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033392390000000001, 2024-09-15T00:13:17.068 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.068 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.068 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.069 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97494 2024-09-15T00:13:17.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.069 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:17.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:17.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:17.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-15T00:13:17.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853419+0000", 2024-09-15T00:13:17.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.917e-06 2024-09-15T00:13:17.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853425+0000", 2024-09-15T00:13:17.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:13:17.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853433+0000", 2024-09-15T00:13:17.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8120000000000008e-06 2024-09-15T00:13:17.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886680+0000", 2024-09-15T00:13:17.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033247207000000001 2024-09-15T00:13:17.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886726+0000", 2024-09-15T00:13:17.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5902000000000002e-05 2024-09-15T00:13:17.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886809+0000", 2024-09-15T00:13:17.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3475999999999998e-05 2024-09-15T00:13:17.073 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.073 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.073 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.073 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97495 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:17.073 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:17.073 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.180795728000007, 2024-09-15T00:13:17.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033640503000000002, 2024-09-15T00:13:17.073 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.074 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.074 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.074 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.074 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.074 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.074 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97495 2024-09-15T00:13:17.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.074 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:17.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:17.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:17.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:13:17.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853440+0000", 2024-09-15T00:13:17.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.804e-06 2024-09-15T00:13:17.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853446+0000", 2024-09-15T00:13:17.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5559999999999998e-06 2024-09-15T00:13:17.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853450+0000", 2024-09-15T00:13:17.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9519999999999996e-06 2024-09-15T00:13:17.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886904+0000", 2024-09-15T00:13:17.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033454207999999999 2024-09-15T00:13:17.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887007+0000", 2024-09-15T00:13:17.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102904 2024-09-15T00:13:17.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887078+0000", 2024-09-15T00:13:17.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1719999999999998e-05 2024-09-15T00:13:17.078 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.078 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.078 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.079 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97496 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:17.079 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:17.079 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.180778630000006, 2024-09-15T00:13:17.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033802431000000001, 2024-09-15T00:13:17.079 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.079 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.079 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.079 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.079 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.079 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.079 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97496 2024-09-15T00:13:17.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.080 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:17.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:17.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:17.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.53e-07 2024-09-15T00:13:17.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853462+0000", 2024-09-15T00:13:17.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-15T00:13:17.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853468+0000", 2024-09-15T00:13:17.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3609999999999997e-06 2024-09-15T00:13:17.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853473+0000", 2024-09-15T00:13:17.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-15T00:13:17.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887111+0000", 2024-09-15T00:13:17.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033638376999999997 2024-09-15T00:13:17.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887178+0000", 2024-09-15T00:13:17.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6877000000000004e-05 2024-09-15T00:13:17.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887257+0000", 2024-09-15T00:13:17.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9616999999999993e-05 2024-09-15T00:13:17.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.083 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.084 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97587 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:17.084 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:17.084 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.180755395999995, 2024-09-15T00:13:17.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033973642999999998, 2024-09-15T00:13:17.084 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.084 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.084 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.084 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.084 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.085 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.085 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97587 2024-09-15T00:13:17.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.085 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:17.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:17.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853479+0000", 2024-09-15T00:13:17.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:13:17.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853481+0000", 2024-09-15T00:13:17.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.136e-06 2024-09-15T00:13:17.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853487+0000", 2024-09-15T00:13:17.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2439999999999998e-06 2024-09-15T00:13:17.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853492+0000", 2024-09-15T00:13:17.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:13:17.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887284+0000", 2024-09-15T00:13:17.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033791827000000003 2024-09-15T00:13:17.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887361+0000", 2024-09-15T00:13:17.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6983000000000005e-05 2024-09-15T00:13:17.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887452+0000", 2024-09-15T00:13:17.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1236999999999994e-05 2024-09-15T00:13:17.089 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.089 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.089 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.089 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97588 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:17.089 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:17.089 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.180735239000001, 2024-09-15T00:13:17.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034134617999999999, 2024-09-15T00:13:17.089 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.089 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.090 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.090 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.090 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.090 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.090 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97588 2024-09-15T00:13:17.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.090 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:17.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:17.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853499+0000", 2024-09-15T00:13:17.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:13:17.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853501+0000", 2024-09-15T00:13:17.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.26e-06 2024-09-15T00:13:17.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853507+0000", 2024-09-15T00:13:17.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2899999999999999e-06 2024-09-15T00:13:17.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853512+0000", 2024-09-15T00:13:17.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8330000000000001e-06 2024-09-15T00:13:17.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887495+0000", 2024-09-15T00:13:17.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033982529999999997 2024-09-15T00:13:17.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887548+0000", 2024-09-15T00:13:17.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3445000000000001e-05 2024-09-15T00:13:17.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887633+0000", 2024-09-15T00:13:17.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4890999999999997e-05 2024-09-15T00:13:17.094 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.094 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.094 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.094 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126866 3.6 3.973c9fa6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.094 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:17.095 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.077413294000003, 2024-09-15T00:13:17.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153301000000003, 2024-09-15T00:13:17.095 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.095 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.095 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:17.095 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.095 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.095 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.095 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126866 2024-09-15T00:13:17.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.095 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:17.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:17.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956821+0000", 2024-09-15T00:13:17.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.91e-07 2024-09-15T00:13:17.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956822+0000", 2024-09-15T00:13:17.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.705e-06 2024-09-15T00:13:17.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956828+0000", 2024-09-15T00:13:17.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6760000000000004e-06 2024-09-15T00:13:17.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956833+0000", 2024-09-15T00:13:17.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5379999999999996e-06 2024-09-15T00:13:17.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989970+0000", 2024-09-15T00:13:17.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033137560000000003 2024-09-15T00:13:17.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989974+0000", 2024-09-15T00:13:17.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-15T00:13:17.099 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.099 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.099 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.099 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126951 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.099 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:13:17.099 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.077287611000003, 2024-09-15T00:13:17.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033152585999999998, 2024-09-15T00:13:17.099 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.099 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.099 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:17.100 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.100 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.100 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.100 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126951 2024-09-15T00:13:17.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.100 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:13:17.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:13:17.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956946+0000", 2024-09-15T00:13:17.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9999999999999999e-07 2024-09-15T00:13:17.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956953+0000", 2024-09-15T00:13:17.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9310000000000002e-06 2024-09-15T00:13:17.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956957+0000", 2024-09-15T00:13:17.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5379999999999998e-06 2024-09-15T00:13:17.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956959+0000", 2024-09-15T00:13:17.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6599999999999999e-06 2024-09-15T00:13:17.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990095+0000", 2024-09-15T00:13:17.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033135855999999998 2024-09-15T00:13:17.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990099+0000", 2024-09-15T00:13:17.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4010000000000001e-06 2024-09-15T00:13:17.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.103 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.104 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126952 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.104 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:17.104 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.077270622, 2024-09-15T00:13:17.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033156066999999997, 2024-09-15T00:13:17.104 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.104 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.104 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:17.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.105 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126952 2024-09-15T00:13:17.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.105 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:17.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:17.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:17.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:13:17.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956965+0000", 2024-09-15T00:13:17.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3379999999999999e-06 2024-09-15T00:13:17.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956968+0000", 2024-09-15T00:13:17.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0419999999999998e-06 2024-09-15T00:13:17.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956970+0000", 2024-09-15T00:13:17.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4899999999999999e-06 2024-09-15T00:13:17.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990113+0000", 2024-09-15T00:13:17.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033142642 2024-09-15T00:13:17.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990119+0000", 2024-09-15T00:13:17.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2979999999999997e-06 2024-09-15T00:13:17.108 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.108 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.108 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.108 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126954 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.109 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:17.109 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.077225126999998, 2024-09-15T00:13:17.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033158449, 2024-09-15T00:13:17.109 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.109 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.109 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:17.109 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.109 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.109 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.109 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126954 2024-09-15T00:13:17.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.109 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:17.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:17.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:17.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-15T00:13:17.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957010+0000", 2024-09-15T00:13:17.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3659999999999999e-06 2024-09-15T00:13:17.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957013+0000", 2024-09-15T00:13:17.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1140000000000002e-06 2024-09-15T00:13:17.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957016+0000", 2024-09-15T00:13:17.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1209999999999998e-06 2024-09-15T00:13:17.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990163+0000", 2024-09-15T00:13:17.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033146451 2024-09-15T00:13:17.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990167+0000", 2024-09-15T00:13:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0579999999999999e-06 2024-09-15T00:13:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126955 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.077213913000001, 2024-09-15T00:13:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033160617000000003, 2024-09-15T00:13:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.114 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:17.114 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.114 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.114 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.114 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126955 2024-09-15T00:13:17.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.114 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:17.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:17.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:17.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-15T00:13:17.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957021+0000", 2024-09-15T00:13:17.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-15T00:13:17.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957024+0000", 2024-09-15T00:13:17.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1599999999999998e-06 2024-09-15T00:13:17.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957027+0000", 2024-09-15T00:13:17.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000001e-06 2024-09-15T00:13:17.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990177+0000", 2024-09-15T00:13:17.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033149741000000003 2024-09-15T00:13:17.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990180+0000", 2024-09-15T00:13:17.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4910000000000001e-06 2024-09-15T00:13:17.117 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.117 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.117 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.118 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126956 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.118 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:17.119 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.077203791999999, 2024-09-15T00:13:17.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033163633999999997, 2024-09-15T00:13:17.120 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.120 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.120 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:17.120 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.120 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.120 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.120 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126956 2024-09-15T00:13:17.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.120 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:17.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:17.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:17.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:17.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957031+0000", 2024-09-15T00:13:17.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-06 2024-09-15T00:13:17.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957034+0000", 2024-09-15T00:13:17.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9960000000000002e-06 2024-09-15T00:13:17.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957037+0000", 2024-09-15T00:13:17.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6419999999999999e-06 2024-09-15T00:13:17.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990190+0000", 2024-09-15T00:13:17.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153152999999998 2024-09-15T00:13:17.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990193+0000", 2024-09-15T00:13:17.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6069999999999999e-06 2024-09-15T00:13:17.124 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.124 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.124 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.124 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126957 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.124 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:17.124 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.077194142000003, 2024-09-15T00:13:17.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033166442999999997, 2024-09-15T00:13:17.124 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.125 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.125 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:17.125 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.125 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.125 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.125 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126957 2024-09-15T00:13:17.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.125 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:17.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:17.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957040+0000", 2024-09-15T00:13:17.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:13:17.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957041+0000", 2024-09-15T00:13:17.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0440000000000001e-06 2024-09-15T00:13:17.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957044+0000", 2024-09-15T00:13:17.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0170000000000001e-06 2024-09-15T00:13:17.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957048+0000", 2024-09-15T00:13:17.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0319999999999997e-06 2024-09-15T00:13:17.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990202+0000", 2024-09-15T00:13:17.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033154640999999999 2024-09-15T00:13:17.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990206+0000", 2024-09-15T00:13:17.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.507e-06 2024-09-15T00:13:17.129 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.129 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.129 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.129 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126958 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.129 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:17.129 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.077182911999998, 2024-09-15T00:13:17.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033167613999999998, 2024-09-15T00:13:17.129 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.129 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.129 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:17.130 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.130 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.130 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.130 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126958 2024-09-15T00:13:17.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.130 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:17.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:17.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:17.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:13:17.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957053+0000", 2024-09-15T00:13:17.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.756e-06 2024-09-15T00:13:17.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957059+0000", 2024-09-15T00:13:17.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2770000000000002e-06 2024-09-15T00:13:17.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957063+0000", 2024-09-15T00:13:17.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2459999999999997e-06 2024-09-15T00:13:17.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990215+0000", 2024-09-15T00:13:17.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033151509000000003 2024-09-15T00:13:17.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990218+0000", 2024-09-15T00:13:17.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5099999999999999e-06 2024-09-15T00:13:17.133 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.133 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.134 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.134 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126959 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.134 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:17.134 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.077163908000003, 2024-09-15T00:13:17.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033161673000000003, 2024-09-15T00:13:17.134 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.134 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.134 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:17.134 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.134 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.135 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.135 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126959 2024-09-15T00:13:17.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.135 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:17.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:17.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:17.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:13:17.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957072+0000", 2024-09-15T00:13:17.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2239999999999998e-06 2024-09-15T00:13:17.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957079+0000", 2024-09-15T00:13:17.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.19e-06 2024-09-15T00:13:17.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957083+0000", 2024-09-15T00:13:17.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:13:17.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990228+0000", 2024-09-15T00:13:17.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033144818 2024-09-15T00:13:17.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990231+0000", 2024-09-15T00:13:17.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:13:17.138 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.138 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.139 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.139 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126960 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.139 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:17.139 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.077145100999999, 2024-09-15T00:13:17.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033155166, 2024-09-15T00:13:17.139 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.139 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.140 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:17.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.140 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126960 2024-09-15T00:13:17.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.140 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:17.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:17.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:17.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:13:17.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957091+0000", 2024-09-15T00:13:17.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:13:17.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957097+0000", 2024-09-15T00:13:17.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6050000000000003e-06 2024-09-15T00:13:17.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957102+0000", 2024-09-15T00:13:17.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5539999999999999e-06 2024-09-15T00:13:17.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990240+0000", 2024-09-15T00:13:17.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033138289000000001 2024-09-15T00:13:17.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990244+0000", 2024-09-15T00:13:17.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4740000000000001e-06 2024-09-15T00:13:17.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.144 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.144 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.144 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:17.146 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:17.146 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:17.143+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:17.246 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:17.243+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:17.293 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:17.293 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:17.293 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:17.293 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:17.302 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:17.387 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:17.384+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:17.447 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:17.447 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:17.447 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:17.448 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:17.444+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:17.455 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:17.548 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:17.546+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:17.601 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:17.602 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:17.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:17.602 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:17.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.602 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:13:17.602 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:17.602 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 89.355957484000001, 2024-09-15T00:13:17.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:13:17.602 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.602 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.602 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:17.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.603 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:13:17.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.603 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:17.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:17.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:13:17.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:13:17.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:13:17.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:13:17.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:13:17.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:13:17.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:13:17.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:17.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:13:17.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:13:17.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:13:17.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:13:17.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:17.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:13:17.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:13:17.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:17.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:13:17.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:13:17.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:17.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:13:17.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:13:17.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:17.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:13:17.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:13:17.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:13:17.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:13:17.607 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.607 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.607 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.608 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58878 3.8 3:12456482:::rbd_data.1139766160da.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:17.608 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:17.608 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 84.309651900000006, 2024-09-15T00:13:17.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061956199000000003, 2024-09-15T00:13:17.608 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.608 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.608 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:17.608 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.608 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.608 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.608 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58878 2024-09-15T00:13:17.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.608 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:17.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:17.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289151+0000", 2024-09-15T00:13:17.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:13:17.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289473+0000", 2024-09-15T00:13:17.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032183800000000002 2024-09-15T00:13:17.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289487+0000", 2024-09-15T00:13:17.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-05 2024-09-15T00:13:17.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289493+0000", 2024-09-15T00:13:17.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4489999999999996e-06 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289542+0000", 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8844999999999997e-05 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289642+0000", 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9607999999999999e-05 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.293908+0000", 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659350000000002 2024-09-15T00:13:17.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:17.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351022+0000", 2024-09-15T00:13:17.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057114535000000001 2024-09-15T00:13:17.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:17.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351033+0000", 2024-09-15T00:13:17.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0263e-05 2024-09-15T00:13:17.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:17.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351070+0000", 2024-09-15T00:13:17.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7666000000000002e-05 2024-09-15T00:13:17.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351106+0000", 2024-09-15T00:13:17.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999997e-05 2024-09-15T00:13:17.613 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.613 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.613 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.613 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58879 3.7 3:e169894d:::rbd_data.1139766160da.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:17.613 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:17.613 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 84.307300412000004, 2024-09-15T00:13:17.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043716294000000003, 2024-09-15T00:13:17.613 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.613 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.614 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:17.614 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.614 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.614 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.614 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58879 2024-09-15T00:13:17.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.614 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:17.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:17.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291502+0000", 2024-09-15T00:13:17.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-15T00:13:17.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291664+0000", 2024-09-15T00:13:17.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016216599999999999 2024-09-15T00:13:17.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291674+0000", 2024-09-15T00:13:17.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0276e-05 2024-09-15T00:13:17.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291680+0000", 2024-09-15T00:13:17.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2580000000000003e-06 2024-09-15T00:13:17.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291732+0000", 2024-09-15T00:13:17.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2114999999999998e-05 2024-09-15T00:13:17.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291866+0000", 2024-09-15T00:13:17.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013416900000000001 2024-09-15T00:13:17.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:13:17.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.296426+0000", 2024-09-15T00:13:17.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045595779999999999 2024-09-15T00:13:17.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:17.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335075+0000", 2024-09-15T00:13:17.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038649347000000001 2024-09-15T00:13:17.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:17.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335097+0000", 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2565000000000002e-05 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335161+0000", 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3928000000000002e-05 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335217+0000", 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6042000000000003e-05 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58880 3.8 3:13d91cfd:::rbd_data.1139766160da.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 84.303908304999993, 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056331463999999998, 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58880 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:17.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:17.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294896+0000", 2024-09-15T00:13:17.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.356e-06 2024-09-15T00:13:17.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295288+0000", 2024-09-15T00:13:17.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039223200000000001 2024-09-15T00:13:17.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295304+0000", 2024-09-15T00:13:17.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5937e-05 2024-09-15T00:13:17.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295312+0000", 2024-09-15T00:13:17.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1489999999999994e-06 2024-09-15T00:13:17.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333162+0000", 2024-09-15T00:13:17.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037849840000000003 2024-09-15T00:13:17.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333269+0000", 2024-09-15T00:13:17.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010749800000000001 2024-09-15T00:13:17.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:13:17.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.336198+0000", 2024-09-15T00:13:17.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029284739999999999 2024-09-15T00:13:17.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:17.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351137+0000", 2024-09-15T00:13:17.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014939103 2024-09-15T00:13:17.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:17.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351150+0000", 2024-09-15T00:13:17.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3542e-05 2024-09-15T00:13:17.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:17.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351191+0000", 2024-09-15T00:13:17.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0791000000000003e-05 2024-09-15T00:13:17.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351225+0000", 2024-09-15T00:13:17.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3541999999999998e-05 2024-09-15T00:13:17.624 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.624 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.624 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.624 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58878 3.8 e56/23 ondisk, result = 0)", 2024-09-15T00:13:17.624 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:17.625 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 84.297378522000002, 2024-09-15T00:13:17.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049689096000000002, 2024-09-15T00:13:17.625 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.625 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.625 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.625 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:17.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:17.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301425+0000", 2024-09-15T00:13:17.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.782e-06 2024-09-15T00:13:17.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301433+0000", 2024-09-15T00:13:17.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689999999999994e-06 2024-09-15T00:13:17.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301449+0000", 2024-09-15T00:13:17.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5767e-05 2024-09-15T00:13:17.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301458+0000", 2024-09-15T00:13:17.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:13:17.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351009+0000", 2024-09-15T00:13:17.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049551523 2024-09-15T00:13:17.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351017+0000", 2024-09-15T00:13:17.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8359999999999992e-06 2024-09-15T00:13:17.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351112+0000", 2024-09-15T00:13:17.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5173999999999994e-05 2024-09-15T00:13:17.628 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.628 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.628 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.628 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127004 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 6881280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.628 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:17.628 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.630473058, 2024-09-15T00:13:17.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735720000000003, 2024-09-15T00:13:17.629 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.629 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.629 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.629 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.629 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.629 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.629 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127004 2024-09-15T00:13:17.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.629 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:17.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:17.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:17.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:13:17.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968331+0000", 2024-09-15T00:13:17.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.849e-06 2024-09-15T00:13:17.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968334+0000", 2024-09-15T00:13:17.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7689999999999998e-06 2024-09-15T00:13:17.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968338+0000", 2024-09-15T00:13:17.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2490000000000001e-06 2024-09-15T00:13:17.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008987+0000", 2024-09-15T00:13:17.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040649117999999998 2024-09-15T00:13:17.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009027+0000", 2024-09-15T00:13:17.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0357e-05 2024-09-15T00:13:17.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009064+0000", 2024-09-15T00:13:17.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7036000000000003e-05 2024-09-15T00:13:17.632 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.632 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.633 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.633 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127005 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7004160~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.633 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:17.633 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.630460505999999, 2024-09-15T00:13:17.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040824358999999998, 2024-09-15T00:13:17.633 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.633 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.633 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.633 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.633 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.633 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.633 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127005 2024-09-15T00:13:17.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.634 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:17.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:17.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:17.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-15T00:13:17.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968343+0000", 2024-09-15T00:13:17.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3430000000000001e-06 2024-09-15T00:13:17.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968349+0000", 2024-09-15T00:13:17.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3080000000000004e-06 2024-09-15T00:13:17.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968355+0000", 2024-09-15T00:13:17.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9499999999999998e-06 2024-09-15T00:13:17.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009091+0000", 2024-09-15T00:13:17.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735645000000001 2024-09-15T00:13:17.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009130+0000", 2024-09-15T00:13:17.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8951e-05 2024-09-15T00:13:17.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009165+0000", 2024-09-15T00:13:17.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5926000000000001e-05 2024-09-15T00:13:17.637 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.637 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.637 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.637 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127006 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7127040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.637 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:17.638 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.630443448000001, 2024-09-15T00:13:17.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040912783000000001, 2024-09-15T00:13:17.638 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.638 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.638 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.638 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.638 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.638 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.638 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127006 2024-09-15T00:13:17.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.638 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:17.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:17.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:17.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:17.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968359+0000", 2024-09-15T00:13:17.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0419999999999999e-06 2024-09-15T00:13:17.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968363+0000", 2024-09-15T00:13:17.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5999999999999998e-06 2024-09-15T00:13:17.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968366+0000", 2024-09-15T00:13:17.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5179999999999999e-06 2024-09-15T00:13:17.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009196+0000", 2024-09-15T00:13:17.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040830313 2024-09-15T00:13:17.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009234+0000", 2024-09-15T00:13:17.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8562000000000001e-05 2024-09-15T00:13:17.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009271+0000", 2024-09-15T00:13:17.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6538000000000002e-05 2024-09-15T00:13:17.642 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.642 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.642 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.642 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127007 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7249920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.642 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:17.642 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.630433248999999, 2024-09-15T00:13:17.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041019218000000003, 2024-09-15T00:13:17.642 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.642 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.642 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.642 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.643 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.643 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.643 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127007 2024-09-15T00:13:17.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.643 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:17.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:17.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968369+0000", 2024-09-15T00:13:17.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-15T00:13:17.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968370+0000", 2024-09-15T00:13:17.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.012e-06 2024-09-15T00:13:17.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968374+0000", 2024-09-15T00:13:17.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3839999999999999e-06 2024-09-15T00:13:17.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968376+0000", 2024-09-15T00:13:17.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5339999999999998e-06 2024-09-15T00:13:17.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009292+0000", 2024-09-15T00:13:17.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040915092 2024-09-15T00:13:17.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009333+0000", 2024-09-15T00:13:17.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1105e-05 2024-09-15T00:13:17.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009388+0000", 2024-09-15T00:13:17.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4868999999999999e-05 2024-09-15T00:13:17.647 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.647 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.647 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.647 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127008 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7364608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.647 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:17.647 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.630422127999999, 2024-09-15T00:13:17.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041115290999999998, 2024-09-15T00:13:17.647 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.648 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.648 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.648 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.648 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.648 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.648 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127008 2024-09-15T00:13:17.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.648 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:17.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:17.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968380+0000", 2024-09-15T00:13:17.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3999999999999998e-07 2024-09-15T00:13:17.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968381+0000", 2024-09-15T00:13:17.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8700000000000004e-07 2024-09-15T00:13:17.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968385+0000", 2024-09-15T00:13:17.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9260000000000002e-06 2024-09-15T00:13:17.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968389+0000", 2024-09-15T00:13:17.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.685e-06 2024-09-15T00:13:17.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009419+0000", 2024-09-15T00:13:17.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041029852999999998 2024-09-15T00:13:17.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009458+0000", 2024-09-15T00:13:17.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8538000000000003e-05 2024-09-15T00:13:17.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009495+0000", 2024-09-15T00:13:17.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7061999999999999e-05 2024-09-15T00:13:17.651 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.651 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.651 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.652 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127009 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7487488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.652 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:17.652 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.630388011000001, 2024-09-15T00:13:17.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041233063, 2024-09-15T00:13:17.652 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.652 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.652 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.652 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.652 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.652 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.652 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127009 2024-09-15T00:13:17.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.653 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:17.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:17.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:17.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7199999999999999e-07 2024-09-15T00:13:17.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968417+0000", 2024-09-15T00:13:17.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3810000000000002e-06 2024-09-15T00:13:17.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968423+0000", 2024-09-15T00:13:17.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8820000000000003e-06 2024-09-15T00:13:17.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968433+0000", 2024-09-15T00:13:17.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0005e-05 2024-09-15T00:13:17.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009527+0000", 2024-09-15T00:13:17.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041093750999999998 2024-09-15T00:13:17.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009601+0000", 2024-09-15T00:13:17.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3979000000000004e-05 2024-09-15T00:13:17.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009647+0000", 2024-09-15T00:13:17.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5493e-05 2024-09-15T00:13:17.656 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.656 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.656 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.656 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127010 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7610368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.656 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:17.656 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.630359007000003, 2024-09-15T00:13:17.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041325924999999999, 2024-09-15T00:13:17.656 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.656 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.656 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.656 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.657 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.657 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.657 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127010 2024-09-15T00:13:17.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.657 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:17.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:17.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:17.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:13:17.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968445+0000", 2024-09-15T00:13:17.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:13:17.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968452+0000", 2024-09-15T00:13:17.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-15T00:13:17.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968456+0000", 2024-09-15T00:13:17.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5210000000000004e-06 2024-09-15T00:13:17.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009664+0000", 2024-09-15T00:13:17.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041207404000000003 2024-09-15T00:13:17.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009723+0000", 2024-09-15T00:13:17.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9036000000000002e-05 2024-09-15T00:13:17.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009768+0000", 2024-09-15T00:13:17.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5578000000000003e-05 2024-09-15T00:13:17.660 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.660 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.660 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.660 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127011 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7733248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.660 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:17.660 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.630340771999997, 2024-09-15T00:13:17.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041482441000000002, 2024-09-15T00:13:17.661 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.661 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.661 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.661 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.661 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.661 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.661 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127011 2024-09-15T00:13:17.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.661 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:17.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:17.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:17.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-15T00:13:17.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968466+0000", 2024-09-15T00:13:17.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:13:17.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968470+0000", 2024-09-15T00:13:17.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6469999999999997e-06 2024-09-15T00:13:17.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968473+0000", 2024-09-15T00:13:17.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.977e-06 2024-09-15T00:13:17.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009800+0000", 2024-09-15T00:13:17.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041326918999999997 2024-09-15T00:13:17.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009869+0000", 2024-09-15T00:13:17.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8935000000000003e-05 2024-09-15T00:13:17.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009943+0000", 2024-09-15T00:13:17.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4188000000000002e-05 2024-09-15T00:13:17.664 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.665 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.665 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.665 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127012 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7856128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.665 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:17.665 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.63032527, 2024-09-15T00:13:17.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041640756000000001, 2024-09-15T00:13:17.665 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.665 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.665 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.665 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.665 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.665 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.665 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127012 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968477+0000", 2024-09-15T00:13:17.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:17.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968482+0000", 2024-09-15T00:13:17.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6180000000000003e-06 2024-09-15T00:13:17.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968486+0000", 2024-09-15T00:13:17.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.501e-06 2024-09-15T00:13:17.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968489+0000", 2024-09-15T00:13:17.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1530000000000001e-06 2024-09-15T00:13:17.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009983+0000", 2024-09-15T00:13:17.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041493980999999999 2024-09-15T00:13:17.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010051+0000", 2024-09-15T00:13:17.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8039000000000004e-05 2024-09-15T00:13:17.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010117+0000", 2024-09-15T00:13:17.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6254000000000006e-05 2024-09-15T00:13:17.669 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.669 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.669 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.669 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127013 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7979008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.669 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:17.669 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.630309449999999, 2024-09-15T00:13:17.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041763863999999998, 2024-09-15T00:13:17.669 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.669 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.669 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.670 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.670 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.670 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.670 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127013 2024-09-15T00:13:17.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.670 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:17.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:17.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969767+0000", 2024-09-15T00:13:17.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001275006 2024-09-15T00:13:17.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969772+0000", 2024-09-15T00:13:17.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4769999999999997e-06 2024-09-15T00:13:17.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969781+0000", 2024-09-15T00:13:17.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1959999999999997e-06 2024-09-15T00:13:17.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969788+0000", 2024-09-15T00:13:17.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6359999999999994e-06 2024-09-15T00:13:17.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010162+0000", 2024-09-15T00:13:17.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040373291999999998 2024-09-15T00:13:17.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010212+0000", 2024-09-15T00:13:17.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0543e-05 2024-09-15T00:13:17.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010256+0000", 2024-09-15T00:13:17.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3714000000000002e-05 2024-09-15T00:13:17.673 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.673 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.673 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.673 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127015 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8224768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.673 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:17.674 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.628970514000002, 2024-09-15T00:13:17.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040710186000000002, 2024-09-15T00:13:17.674 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.674 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.674 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.674 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.674 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.674 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.674 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127015 2024-09-15T00:13:17.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.674 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:17.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:17.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969832+0000", 2024-09-15T00:13:17.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:13:17.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969834+0000", 2024-09-15T00:13:17.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2919999999999998e-06 2024-09-15T00:13:17.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969843+0000", 2024-09-15T00:13:17.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0480000000000004e-06 2024-09-15T00:13:17.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969848+0000", 2024-09-15T00:13:17.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-15T00:13:17.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010457+0000", 2024-09-15T00:13:17.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040609054999999998 2024-09-15T00:13:17.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010501+0000", 2024-09-15T00:13:17.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3396000000000003e-05 2024-09-15T00:13:17.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010541+0000", 2024-09-15T00:13:17.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0599000000000002e-05 2024-09-15T00:13:17.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.677 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.678 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127016 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:17.678 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:17.678 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.628919887999999, 2024-09-15T00:13:17.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040766216000000001, 2024-09-15T00:13:17.678 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.678 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.678 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.678 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127016 2024-09-15T00:13:17.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.679 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:17.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:17.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:17.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:13:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969886+0000", 2024-09-15T00:13:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8449999999999996e-06 2024-09-15T00:13:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969897+0000", 2024-09-15T00:13:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-15T00:13:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969904+0000", 2024-09-15T00:13:17.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1169999999999998e-06 2024-09-15T00:13:17.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010561+0000", 2024-09-15T00:13:17.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040657470000000001 2024-09-15T00:13:17.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010607+0000", 2024-09-15T00:13:17.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6108e-05 2024-09-15T00:13:17.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010648+0000", 2024-09-15T00:13:17.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0479000000000003e-05 2024-09-15T00:13:17.682 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.682 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.682 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.682 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127694 3.9 3:9390b205:::rbd_data.11396481e1ec.00000000000000df:head [set-alloc-hint object_size 1048576 write_size 1048576,write 819200~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T00:13:17.682 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:17.682 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.374200770999998, 2024-09-15T00:13:17.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90376419399999997, 2024-09-15T00:13:17.682 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.682 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.682 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:17.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.683 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.683 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.683 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127694 2024-09-15T00:13:17.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.683 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:17.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:17.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224602+0000", 2024-09-15T00:13:17.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5899999999999995e-07 2024-09-15T00:13:17.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224610+0000", 2024-09-15T00:13:17.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9130000000000003e-06 2024-09-15T00:13:17.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224620+0000", 2024-09-15T00:13:17.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0426e-05 2024-09-15T00:13:17.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224627+0000", 2024-09-15T00:13:17.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4390000000000003e-06 2024-09-15T00:13:17.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224675+0000", 2024-09-15T00:13:17.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7636000000000003e-05 2024-09-15T00:13:17.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:17.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224681+0000", 2024-09-15T00:13:17.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1460000000000001e-06 2024-09-15T00:13:17.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126388+0000", 2024-09-15T00:13:17.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90170724999999996 2024-09-15T00:13:17.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126526+0000", 2024-09-15T00:13:17.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013730599999999999 2024-09-15T00:13:17.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:17.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126684+0000", 2024-09-15T00:13:17.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000157935 2024-09-15T00:13:17.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:17.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.127932+0000", 2024-09-15T00:13:17.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012479240000000001 2024-09-15T00:13:17.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:17.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128269+0000", 2024-09-15T00:13:17.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033761099999999998 2024-09-15T00:13:17.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:17.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128324+0000", 2024-09-15T00:13:17.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5244999999999997e-05 2024-09-15T00:13:17.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128365+0000", 2024-09-15T00:13:17.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0503999999999997e-05 2024-09-15T00:13:17.688 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.688 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.688 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.688 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:134707 3.15 3:af1fbdb5:::rbd_data.113933a4f79a.000000000000001c:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T00:13:17.688 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:17.688 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.906233657000001, 2024-09-15T00:13:17.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057328626000000001, 2024-09-15T00:13:17.689 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.689 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.689 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:17.689 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:17.689 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:17.689 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:17.689 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 134707 2024-09-15T00:13:17.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.689 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:17.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:17.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692569+0000", 2024-09-15T00:13:17.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.339e-06 2024-09-15T00:13:17.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692574+0000", 2024-09-15T00:13:17.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-15T00:13:17.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692584+0000", 2024-09-15T00:13:17.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0363000000000001e-05 2024-09-15T00:13:17.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692590+0000", 2024-09-15T00:13:17.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.818e-06 2024-09-15T00:13:17.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692633+0000", 2024-09-15T00:13:17.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.265e-05 2024-09-15T00:13:17.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692718+0000", 2024-09-15T00:13:17.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5279999999999997e-05 2024-09-15T00:13:17.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:17.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.695940+0000", 2024-09-15T00:13:17.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003222052 2024-09-15T00:13:17.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:17.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749714+0000", 2024-09-15T00:13:17.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.053774126999999998 2024-09-15T00:13:17.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:17.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749788+0000", 2024-09-15T00:13:17.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3693000000000001e-05 2024-09-15T00:13:17.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:17.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749847+0000", 2024-09-15T00:13:17.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175999999999997e-05 2024-09-15T00:13:17.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749897+0000", 2024-09-15T00:13:17.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9311999999999998e-05 2024-09-15T00:13:17.694 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.694 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.694 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.694 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:134707 3.15 e86/55 ondisk, result = 0)", 2024-09-15T00:13:17.694 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:17.694 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.897672307999997, 2024-09-15T00:13:17.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048596066, 2024-09-15T00:13:17.694 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:17.694 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:17.694 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:17.694 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:17.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:17.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:17.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:17.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:17.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:17.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:17.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701131+0000", 2024-09-15T00:13:17.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3549999999999999e-06 2024-09-15T00:13:17.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:17.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701139+0000", 2024-09-15T00:13:17.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1650000000000006e-06 2024-09-15T00:13:17.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:17.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701156+0000", 2024-09-15T00:13:17.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6730000000000001e-05 2024-09-15T00:13:17.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:17.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701167+0000", 2024-09-15T00:13:17.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1504e-05 2024-09-15T00:13:17.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:17.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749629+0000", 2024-09-15T00:13:17.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048461952000000003 2024-09-15T00:13:17.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:17.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749640+0000", 2024-09-15T00:13:17.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0874e-05 2024-09-15T00:13:17.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:17.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:17.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:17.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749725+0000", 2024-09-15T00:13:17.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5486000000000001e-05 2024-09-15T00:13:17.698 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.698 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.698 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.698 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:17.698 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:17.698 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:17.699 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:17.700 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:17.697+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:17.800 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:17.798+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:17.805 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:17.805 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:17.805 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:17.805 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:17.813 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:17.901 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:17.899+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:17.916 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:17.917 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:17.917 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:17.917 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:17.917 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:17.925 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:18.002 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:18.000+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:18.015 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is set 2024-09-15T00:13:18.035 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:18.035 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:18.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:18.035 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:18.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.035 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:18.036 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:18.036 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 112.460795774, 2024-09-15T00:13:18.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:13:18.036 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.036 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.036 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:18.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.036 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:13:18.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.036 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:18.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:18.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:13:18.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:13:18.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:13:18.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:13:18.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:13:18.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:13:18.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:13:18.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:13:18.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:13:18.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:13:18.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:13:18.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:13:18.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:18.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:13:18.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:13:18.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:18.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:13:18.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:13:18.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:18.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:13:18.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:13:18.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:18.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:13:18.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:13:18.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:13:18.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:13:18.041 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.041 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.041 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.042 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:18.042 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:18.042 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 112.460761082, 2024-09-15T00:13:18.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:13:18.042 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.042 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.042 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:18.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.042 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:13:18.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.043 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:18.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:18.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:18.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:13:18.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:13:18.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:13:18.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:13:18.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:13:18.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:13:18.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:13:18.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:13:18.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:13:18.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:13:18.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:13:18.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:18.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:13:18.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:13:18.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:18.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:13:18.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:13:18.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:18.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:13:18.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:13:18.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:18.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:13:18.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:13:18.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:13:18.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:13:18.047 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.047 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.047 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.048 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.048 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:18.048 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.420087394000006, 2024-09-15T00:13:18.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:13:18.048 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.048 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.048 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.048 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.048 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.048 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.048 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:13:18.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.049 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:18.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:18.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:18.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:13:18.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:13:18.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:13:18.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:13:18.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:13:18.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:13:18.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:13:18.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:13:18.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:13:18.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:13:18.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:13:18.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:13:18.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:13:18.052 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.052 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.052 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.052 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.053 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:18.053 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.420070863000007, 2024-09-15T00:13:18.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:13:18.053 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.053 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.053 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.053 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.053 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.053 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.053 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:13:18.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.053 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:18.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:18.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:13:18.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:18.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:13:18.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:13:18.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:13:18.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:13:18.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:13:18.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:13:18.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:13:18.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:13:18.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:13:18.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:13:18.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:13:18.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:13:18.057 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.057 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.057 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.057 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.057 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:18.057 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.420054084, 2024-09-15T00:13:18.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:13:18.057 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.057 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.057 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.057 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.058 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.058 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.058 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:13:18.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.058 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:18.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:18.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:18.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:13:18.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:13:18.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:13:18.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:13:18.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:13:18.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:13:18.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:13:18.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:13:18.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:13:18.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:13:18.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:13:18.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:13:18.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:13:18.061 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.061 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.061 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.061 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.061 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:18.061 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.420036354000004, 2024-09-15T00:13:18.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:13:18.062 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.062 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.062 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.062 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.062 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.062 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.062 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:13:18.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.062 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:18.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:18.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:18.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:13:18.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:13:18.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:13:18.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:13:18.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:13:18.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:13:18.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:13:18.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:13:18.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:13:18.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:13:18.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:13:18.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:13:18.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:13:18.066 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.066 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.066 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.066 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.066 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:18.066 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.420019190000005, 2024-09-15T00:13:18.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:13:18.066 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.066 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.066 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.066 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.066 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.066 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.067 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:13:18.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.067 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:18.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:18.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:18.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:13:18.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:13:18.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:13:18.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:13:18.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:13:18.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:13:18.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:13:18.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:13:18.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:13:18.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:13:18.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:13:18.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:13:18.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:13:18.070 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.070 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.070 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.070 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.070 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:18.070 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.419997514000002, 2024-09-15T00:13:18.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:13:18.070 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:18.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:18.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:13:18.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:13:18.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:13:18.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:13:18.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:13:18.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:13:18.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:13:18.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:13:18.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:13:18.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:13:18.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:13:18.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:13:18.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:13:18.074 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.074 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.074 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.075 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.075 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:18.075 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.419951389999994, 2024-09-15T00:13:18.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:13:18.075 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.075 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.075 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.075 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.075 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.075 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.075 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:13:18.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.076 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:18.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:18.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:18.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:13:18.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:13:18.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:13:18.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:13:18.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:13:18.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:13:18.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:13:18.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:13:18.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:13:18.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:13:18.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:13:18.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:13:18.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:13:18.079 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.079 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.079 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.079 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.079 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:18.079 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.419921633000001, 2024-09-15T00:13:18.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:13:18.079 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.079 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.080 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.080 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.080 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.080 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.080 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:13:18.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.080 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:18.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:18.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:13:18.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:13:18.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:13:18.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:13:18.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:13:18.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:18.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:13:18.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:13:18.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:13:18.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:13:18.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:13:18.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:13:18.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:13:18.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:13:18.083 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.083 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.083 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.084 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.084 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:18.084 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.419895314000001, 2024-09-15T00:13:18.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:13:18.084 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.084 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.084 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.084 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.084 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.084 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.084 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:13:18.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.084 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:18.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:18.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:18.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:18.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:13:18.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:13:18.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:13:18.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:13:18.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:13:18.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:13:18.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:13:18.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:13:18.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:13:18.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:13:18.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:13:18.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:13:18.088 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.088 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.088 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.088 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.088 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:18.088 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.419859451999997, 2024-09-15T00:13:18.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:13:18.088 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.088 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.088 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.089 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.089 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.089 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.089 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:13:18.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.089 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:18.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:18.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:18.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:13:18.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:13:18.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:13:18.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:13:18.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:13:18.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:13:18.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:13:18.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:13:18.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:13:18.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:13:18.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:13:18.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:13:18.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:13:18.092 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.092 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.092 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.093 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.093 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:18.093 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.419837733999998, 2024-09-15T00:13:18.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:13:18.093 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.093 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.093 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.093 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.093 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.093 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.093 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:13:18.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.093 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:13:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:13:18.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:13:18.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:13:18.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:13:18.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:13:18.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:13:18.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:13:18.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:13:18.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:13:18.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:13:18.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:13:18.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:13:18.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:13:18.097 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.097 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.097 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.097 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.097 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:18.097 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.419641267000003, 2024-09-15T00:13:18.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:13:18.097 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.097 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.097 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.098 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.098 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.098 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:13:18.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.098 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:18.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:18.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:18.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:13:18.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:13:18.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:13:18.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:13:18.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:13:18.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:13:18.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:13:18.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:13:18.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:13:18.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:13:18.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:13:18.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:13:18.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:13:18.101 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.101 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.101 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.102 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.102 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:18.102 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.419615428, 2024-09-15T00:13:18.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:13:18.102 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.102 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.102 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.102 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.102 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.102 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.102 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:13:18.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.102 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:18.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:18.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:18.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:13:18.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:13:18.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:13:18.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:13:18.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:13:18.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:13:18.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:13:18.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:13:18.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:13:18.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:13:18.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:13:18.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:13:18.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:13:18.106 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.106 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.106 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.106 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.106 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:18.106 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.419553781999994, 2024-09-15T00:13:18.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:13:18.106 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.106 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.107 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.107 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.107 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.107 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.107 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:13:18.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.107 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:18.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:18.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:13:18.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:13:18.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:13:18.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:13:18.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:13:18.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:13:18.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:13:18.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:13:18.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:13:18.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:13:18.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:13:18.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:13:18.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:13:18.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:13:18.110 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.110 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.110 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.111 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.111 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:18.111 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.419514453999994, 2024-09-15T00:13:18.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:13:18.111 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.111 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.111 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.111 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.111 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.111 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.111 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:13:18.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.111 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:18.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:18.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:13:18.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:13:18.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:13:18.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:13:18.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:13:18.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:13:18.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:13:18.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:13:18.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:13:18.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:13:18.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:13:18.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:18.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:13:18.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:13:18.115 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.115 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.115 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.115 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.115 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:18.115 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.417789053999996, 2024-09-15T00:13:18.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:13:18.116 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.116 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.116 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.116 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.116 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.116 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.116 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:13:18.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.116 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:18.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:18.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:13:18.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:13:18.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:13:18.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:13:18.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:13:18.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:13:18.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:13:18.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:13:18.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:13:18.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:13:18.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:13:18.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:13:18.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:13:18.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:13:18.119 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.120 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.120 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.120 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:18.120 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:18.120 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 93.417766893000007, 2024-09-15T00:13:18.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:13:18.120 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.120 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.120 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:18.120 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.121 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.121 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.121 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:13:18.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.121 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:18.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:18.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:18.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:13:18.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:13:18.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:13:18.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:13:18.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:13:18.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:13:18.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:13:18.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:13:18.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:13:18.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:13:18.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:13:18.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:13:18.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:13:18.124 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.124 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.125 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.125 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:145034 3.6 3:618673df:::rbd_object_map.1139646a7d4e.000000000000002b:head [writefull 0~48] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T00:13:18.125 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:18.125 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.015513288999998, 2024-09-15T00:13:18.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0081079740000001, 2024-09-15T00:13:18.125 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:18.125 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:18.125 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:18.125 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:18.125 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:18.125 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:18.125 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 145034 2024-09-15T00:13:18.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.126 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:18.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:18.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:18.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:18.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:18.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:18.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:18.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017198+0000", 2024-09-15T00:13:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0100000000000004e-07 2024-09-15T00:13:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017202+0000", 2024-09-15T00:13:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:13:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017212+0000", 2024-09-15T00:13:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7189999999999996e-06 2024-09-15T00:13:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017219+0000", 2024-09-15T00:13:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7839999999999998e-06 2024-09-15T00:13:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017268+0000", 2024-09-15T00:13:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:13:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:13:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017277+0000", 2024-09-15T00:13:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1899999999999995e-06 2024-09-15T00:13:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019139+0000", 2024-09-15T00:13:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0018626129999999 2024-09-15T00:13:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019236+0000", 2024-09-15T00:13:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6513000000000002e-05 2024-09-15T00:13:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019318+0000", 2024-09-15T00:13:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1848000000000006e-05 2024-09-15T00:13:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:18.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.023151+0000", 2024-09-15T00:13:18.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038332169999999999 2024-09-15T00:13:18.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:18.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025217+0000", 2024-09-15T00:13:18.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020657610000000002 2024-09-15T00:13:18.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:18.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025267+0000", 2024-09-15T00:13:18.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0198000000000003e-05 2024-09-15T00:13:18.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:18.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:18.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:18.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025305+0000", 2024-09-15T00:13:18.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8065999999999998e-05 2024-09-15T00:13:18.131 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.131 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.131 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.131 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:18.131 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:18.131 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:18.131 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:13:18.133 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:18.134 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:18.130+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:18.234 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:18.232+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:18.239 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:18.239 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:18.239 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:18.239 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:18.248 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:18.258 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:18.258 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:18.275 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:18.275 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:18.290 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:18.291 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:18.306 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:18.307 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:18.324 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:18.324 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:18.335 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:18.332+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:18.340 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:18.340 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:18.357 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:18.357 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:18.374 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:18.375 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:18.395 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:18.395 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:18.404 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:18.448 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:18.446+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:18.507 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:18.507 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:18.515 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:18.537 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:18.535+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:18.638 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:18.635+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:18.659 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:18.659 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:18.668 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:18.739 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:18.737+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:18.771 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:18.771 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:18.780 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:18.840 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:18.838+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:18.884 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:18.884 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:18.892 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:18.977 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:18.975+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:19.041 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:19.041 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:19.042 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:19.039+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:19.052 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:19.143 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:19.140+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:19.202 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:19.202 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:19.211 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:19.255 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:19.253+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:19.314 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:19.314 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:19.323 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:19.367 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:19.364+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:19.425 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:19.426 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:19.434 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:19.447 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:19.445+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:19.548 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:19.547+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:19.578 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:19.579 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:19.588 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:19.650 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:19.648+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:19.691 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:19.692 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:19.700 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:19.751 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:19.749+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:19.803 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:19.803 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:19.812 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:19.852 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:19.850+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:19.915 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:19.915 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:19.923 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:19.952 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:19.950+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:20.028 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:20.028 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:20.034 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:13:20.039 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:20.081 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:20.078+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:20.153 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:20.151+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:20.196 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:20.196 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:20.205 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:20.255 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:20.253+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:20.332 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 0] out_osds: [7, 1, 2] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-15T00:13:20.332 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:13:20.332 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-15T00:13:20.333 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-15T00:13:20.351 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:20.351 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:20.355 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:20.354+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:20.361 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:20.385 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:13:20.457 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:20.454+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:20.526 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:20.526 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:20.530 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:20.530 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:20.537 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:20.549 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:20.549 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:20.566 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:20.567 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:20.567 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:20.564+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:20.583 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:20.583 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:20.600 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:20.600 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:20.617 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:20.617 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:20.633 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:20.634 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:20.650 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:20.651 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:20.654 INFO:tasks.ceph.mon.a.smithi171.stderr:2024-09-15T00:13:20.652+0000 7f0425d35640 -1 mon.a@0(leader).osd e132 definitely_dead 0 2024-09-15T00:13:20.658 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:20.658 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:20.658 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:20.659 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:20.659 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:20.656+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:20.669 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:20.759 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:20.757+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:20.814 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:20.814 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:20.814 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:20.822 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:20.860 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:20.858+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:20.961 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:20.960+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:20.966 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:20.966 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:20.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:20.967 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:20.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.967 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:13:20.967 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:20.967 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.721469866000007, 2024-09-15T00:13:20.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:13:20.967 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:20.967 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:20.967 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:20.967 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:20.967 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:20.967 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:20.968 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:13:20.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.968 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:20.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:20.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:20.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:20.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:20.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:20.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:20.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:20.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:13:20.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:13:20.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:20.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:13:20.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:13:20.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:20.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:13:20.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:13:20.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:20.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:13:20.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:20.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:20.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:13:20.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:13:20.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:20.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:13:20.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:13:20.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:20.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:13:20.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:13:20.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:20.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:13:20.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:13:20.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:20.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:13:20.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:13:20.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:20.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:13:20.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:13:20.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:20.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:13:20.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:13:20.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:20.972 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:20.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:20.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.973 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58878 3.8 3:12456482:::rbd_data.1139766160da.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:20.973 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:20.973 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.675164281999997, 2024-09-15T00:13:20.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061956199000000003, 2024-09-15T00:13:20.973 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:20.973 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:20.973 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:20.973 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:20.973 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:20.973 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:20.973 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58878 2024-09-15T00:13:20.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.973 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:20.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:20.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:20.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:20.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:20.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:20.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:20.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:20.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289151+0000", 2024-09-15T00:13:20.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:13:20.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:20.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289473+0000", 2024-09-15T00:13:20.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032183800000000002 2024-09-15T00:13:20.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:20.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289487+0000", 2024-09-15T00:13:20.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-05 2024-09-15T00:13:20.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:20.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289493+0000", 2024-09-15T00:13:20.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4489999999999996e-06 2024-09-15T00:13:20.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:20.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289542+0000", 2024-09-15T00:13:20.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8844999999999997e-05 2024-09-15T00:13:20.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:20.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289642+0000", 2024-09-15T00:13:20.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9607999999999999e-05 2024-09-15T00:13:20.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:13:20.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.293908+0000", 2024-09-15T00:13:20.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659350000000002 2024-09-15T00:13:20.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:20.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351022+0000", 2024-09-15T00:13:20.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057114535000000001 2024-09-15T00:13:20.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:20.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351033+0000", 2024-09-15T00:13:20.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0263e-05 2024-09-15T00:13:20.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:20.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351070+0000", 2024-09-15T00:13:20.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7666000000000002e-05 2024-09-15T00:13:20.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:20.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351106+0000", 2024-09-15T00:13:20.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999997e-05 2024-09-15T00:13:20.978 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:20.978 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:20.979 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:20.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.979 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58879 3.7 3:e169894d:::rbd_data.1139766160da.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:20.979 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:20.979 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.672812793999995, 2024-09-15T00:13:20.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043716294000000003, 2024-09-15T00:13:20.979 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:20.979 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:20.979 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:20.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:20.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:20.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:20.979 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58879 2024-09-15T00:13:20.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.980 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:20.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:20.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:20.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:20.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:20.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:20.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:20.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:20.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291502+0000", 2024-09-15T00:13:20.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-15T00:13:20.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:20.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291664+0000", 2024-09-15T00:13:20.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016216599999999999 2024-09-15T00:13:20.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:20.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291674+0000", 2024-09-15T00:13:20.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0276e-05 2024-09-15T00:13:20.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:20.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291680+0000", 2024-09-15T00:13:20.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2580000000000003e-06 2024-09-15T00:13:20.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:20.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291732+0000", 2024-09-15T00:13:20.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2114999999999998e-05 2024-09-15T00:13:20.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:20.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291866+0000", 2024-09-15T00:13:20.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013416900000000001 2024-09-15T00:13:20.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:13:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.296426+0000", 2024-09-15T00:13:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045595779999999999 2024-09-15T00:13:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335075+0000", 2024-09-15T00:13:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038649347000000001 2024-09-15T00:13:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335097+0000", 2024-09-15T00:13:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2565000000000002e-05 2024-09-15T00:13:20.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:20.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335161+0000", 2024-09-15T00:13:20.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3928000000000002e-05 2024-09-15T00:13:20.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:20.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335217+0000", 2024-09-15T00:13:20.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6042000000000003e-05 2024-09-15T00:13:20.984 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:20.984 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:20.984 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:20.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.985 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58880 3.8 3:13d91cfd:::rbd_data.1139766160da.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:20.985 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:20.985 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.669420686999999, 2024-09-15T00:13:20.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056331463999999998, 2024-09-15T00:13:20.985 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:20.985 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:20.985 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:20.985 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:20.985 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:20.985 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:20.985 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58880 2024-09-15T00:13:20.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.985 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:20.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:20.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:20.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:20.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:20.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:20.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:20.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:20.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294896+0000", 2024-09-15T00:13:20.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.356e-06 2024-09-15T00:13:20.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:20.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295288+0000", 2024-09-15T00:13:20.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039223200000000001 2024-09-15T00:13:20.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:20.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295304+0000", 2024-09-15T00:13:20.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5937e-05 2024-09-15T00:13:20.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:20.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295312+0000", 2024-09-15T00:13:20.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1489999999999994e-06 2024-09-15T00:13:20.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:20.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333162+0000", 2024-09-15T00:13:20.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037849840000000003 2024-09-15T00:13:20.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:20.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333269+0000", 2024-09-15T00:13:20.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010749800000000001 2024-09-15T00:13:20.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:13:20.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.336198+0000", 2024-09-15T00:13:20.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029284739999999999 2024-09-15T00:13:20.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:20.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351137+0000", 2024-09-15T00:13:20.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014939103 2024-09-15T00:13:20.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:20.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351150+0000", 2024-09-15T00:13:20.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3542e-05 2024-09-15T00:13:20.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:20.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351191+0000", 2024-09-15T00:13:20.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0791000000000003e-05 2024-09-15T00:13:20.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:20.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351225+0000", 2024-09-15T00:13:20.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3541999999999998e-05 2024-09-15T00:13:20.990 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:20.990 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:20.990 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:20.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.990 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58878 3.8 e56/23 ondisk, result = 0)", 2024-09-15T00:13:20.990 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:20.990 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.662890903999994, 2024-09-15T00:13:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049689096000000002, 2024-09-15T00:13:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301425+0000", 2024-09-15T00:13:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.782e-06 2024-09-15T00:13:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301433+0000", 2024-09-15T00:13:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689999999999994e-06 2024-09-15T00:13:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301449+0000", 2024-09-15T00:13:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5767e-05 2024-09-15T00:13:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301458+0000", 2024-09-15T00:13:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:13:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351009+0000", 2024-09-15T00:13:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049551523 2024-09-15T00:13:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351017+0000", 2024-09-15T00:13:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8359999999999992e-06 2024-09-15T00:13:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351112+0000", 2024-09-15T00:13:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5173999999999994e-05 2024-09-15T00:13:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127004 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 6881280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 58.995985439999998, 2024-09-15T00:13:20.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735720000000003, 2024-09-15T00:13:20.995 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:20.995 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:20.995 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:20.995 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:20.995 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:20.995 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:20.995 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127004 2024-09-15T00:13:20.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.995 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:20.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:20.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:20.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:20.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:20.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:20.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:20.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:20.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:20.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:13:20.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:20.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968331+0000", 2024-09-15T00:13:20.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.849e-06 2024-09-15T00:13:20.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:20.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968334+0000", 2024-09-15T00:13:20.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7689999999999998e-06 2024-09-15T00:13:20.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:20.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968338+0000", 2024-09-15T00:13:20.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2490000000000001e-06 2024-09-15T00:13:20.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:20.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008987+0000", 2024-09-15T00:13:20.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040649117999999998 2024-09-15T00:13:20.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:20.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009027+0000", 2024-09-15T00:13:20.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0357e-05 2024-09-15T00:13:20.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:20.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009064+0000", 2024-09-15T00:13:20.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7036000000000003e-05 2024-09-15T00:13:20.998 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:20.998 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:20.999 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:20.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:20.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:20.999 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127005 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7004160~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:20.999 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:20.999 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 58.995972887999997, 2024-09-15T00:13:20.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040824358999999998, 2024-09-15T00:13:20.999 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:20.999 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:20.999 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:20.999 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:20.999 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:20.999 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:20.999 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127005 2024-09-15T00:13:21.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.000 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:21.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:21.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:21.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:21.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:21.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:21.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:21.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-15T00:13:21.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:21.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968343+0000", 2024-09-15T00:13:21.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3430000000000001e-06 2024-09-15T00:13:21.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:21.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968349+0000", 2024-09-15T00:13:21.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3080000000000004e-06 2024-09-15T00:13:21.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968355+0000", 2024-09-15T00:13:21.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9499999999999998e-06 2024-09-15T00:13:21.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:21.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009091+0000", 2024-09-15T00:13:21.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735645000000001 2024-09-15T00:13:21.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:21.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009130+0000", 2024-09-15T00:13:21.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8951e-05 2024-09-15T00:13:21.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:21.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009165+0000", 2024-09-15T00:13:21.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5926000000000001e-05 2024-09-15T00:13:21.004 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.004 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:21.004 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.004 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127006 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7127040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:21.004 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:21.004 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 58.99595583, 2024-09-15T00:13:21.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040912783000000001, 2024-09-15T00:13:21.004 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:21.004 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:21.004 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:21.005 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:21.005 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:21.005 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.005 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127006 2024-09-15T00:13:21.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.005 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:21.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:21.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:21.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:21.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968359+0000", 2024-09-15T00:13:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0419999999999999e-06 2024-09-15T00:13:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968363+0000", 2024-09-15T00:13:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5999999999999998e-06 2024-09-15T00:13:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968366+0000", 2024-09-15T00:13:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5179999999999999e-06 2024-09-15T00:13:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009196+0000", 2024-09-15T00:13:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040830313 2024-09-15T00:13:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009234+0000", 2024-09-15T00:13:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8562000000000001e-05 2024-09-15T00:13:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009271+0000", 2024-09-15T00:13:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6538000000000002e-05 2024-09-15T00:13:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127007 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7249920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 58.995945630999998, 2024-09-15T00:13:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041019218000000003, 2024-09-15T00:13:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127007 2024-09-15T00:13:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968369+0000", 2024-09-15T00:13:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-15T00:13:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968370+0000", 2024-09-15T00:13:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.012e-06 2024-09-15T00:13:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968374+0000", 2024-09-15T00:13:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3839999999999999e-06 2024-09-15T00:13:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968376+0000", 2024-09-15T00:13:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5339999999999998e-06 2024-09-15T00:13:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009292+0000", 2024-09-15T00:13:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040915092 2024-09-15T00:13:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009333+0000", 2024-09-15T00:13:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1105e-05 2024-09-15T00:13:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009388+0000", 2024-09-15T00:13:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4868999999999999e-05 2024-09-15T00:13:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127008 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7364608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 58.995934509999998, 2024-09-15T00:13:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041115290999999998, 2024-09-15T00:13:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127008 2024-09-15T00:13:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968380+0000", 2024-09-15T00:13:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3999999999999998e-07 2024-09-15T00:13:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968381+0000", 2024-09-15T00:13:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8700000000000004e-07 2024-09-15T00:13:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968385+0000", 2024-09-15T00:13:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9260000000000002e-06 2024-09-15T00:13:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968389+0000", 2024-09-15T00:13:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.685e-06 2024-09-15T00:13:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009419+0000", 2024-09-15T00:13:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041029852999999998 2024-09-15T00:13:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009458+0000", 2024-09-15T00:13:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8538000000000003e-05 2024-09-15T00:13:21.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:21.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009495+0000", 2024-09-15T00:13:21.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7061999999999999e-05 2024-09-15T00:13:21.017 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.017 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:21.017 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.017 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127009 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7487488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:21.017 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:21.017 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 58.995900392999999, 2024-09-15T00:13:21.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041233063, 2024-09-15T00:13:21.018 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:21.018 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:21.018 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:21.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:21.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:21.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.018 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127009 2024-09-15T00:13:21.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.018 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:21.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:21.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:21.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:21.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:21.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:21.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:21.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7199999999999999e-07 2024-09-15T00:13:21.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:21.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968417+0000", 2024-09-15T00:13:21.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3810000000000002e-06 2024-09-15T00:13:21.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:21.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968423+0000", 2024-09-15T00:13:21.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8820000000000003e-06 2024-09-15T00:13:21.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968433+0000", 2024-09-15T00:13:21.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0005e-05 2024-09-15T00:13:21.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:21.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009527+0000", 2024-09-15T00:13:21.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041093750999999998 2024-09-15T00:13:21.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:21.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009601+0000", 2024-09-15T00:13:21.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3979000000000004e-05 2024-09-15T00:13:21.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:21.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009647+0000", 2024-09-15T00:13:21.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5493e-05 2024-09-15T00:13:21.021 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.021 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:21.021 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.022 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127010 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7610368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:21.022 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:21.022 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 58.995871389000001, 2024-09-15T00:13:21.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041325924999999999, 2024-09-15T00:13:21.022 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:21.022 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:21.022 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:21.022 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:21.022 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:21.022 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.022 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127010 2024-09-15T00:13:21.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.022 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:21.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:21.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:21.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:21.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:21.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:21.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:21.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:13:21.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:21.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968445+0000", 2024-09-15T00:13:21.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:13:21.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:21.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968452+0000", 2024-09-15T00:13:21.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-15T00:13:21.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968456+0000", 2024-09-15T00:13:21.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5210000000000004e-06 2024-09-15T00:13:21.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:21.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009664+0000", 2024-09-15T00:13:21.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041207404000000003 2024-09-15T00:13:21.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:21.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009723+0000", 2024-09-15T00:13:21.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9036000000000002e-05 2024-09-15T00:13:21.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:21.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009768+0000", 2024-09-15T00:13:21.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5578000000000003e-05 2024-09-15T00:13:21.026 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.026 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:21.026 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.026 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127011 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7733248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:21.026 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:21.026 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 58.995853154000002, 2024-09-15T00:13:21.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041482441000000002, 2024-09-15T00:13:21.026 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:21.026 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:21.026 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:21.026 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:21.027 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:21.027 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.027 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127011 2024-09-15T00:13:21.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.027 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:21.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:21.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:21.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:21.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:21.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:21.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:21.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-15T00:13:21.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:21.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968466+0000", 2024-09-15T00:13:21.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:13:21.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:21.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968470+0000", 2024-09-15T00:13:21.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6469999999999997e-06 2024-09-15T00:13:21.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968473+0000", 2024-09-15T00:13:21.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.977e-06 2024-09-15T00:13:21.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:21.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009800+0000", 2024-09-15T00:13:21.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041326918999999997 2024-09-15T00:13:21.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:21.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009869+0000", 2024-09-15T00:13:21.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8935000000000003e-05 2024-09-15T00:13:21.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:21.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009943+0000", 2024-09-15T00:13:21.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4188000000000002e-05 2024-09-15T00:13:21.030 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.030 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:21.030 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.030 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127012 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7856128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:21.030 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:21.030 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 58.995837651999999, 2024-09-15T00:13:21.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041640756000000001, 2024-09-15T00:13:21.031 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:21.031 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:21.031 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:21.031 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:21.031 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:21.031 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.031 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127012 2024-09-15T00:13:21.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.031 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:21.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:21.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:21.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:21.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:21.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:21.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968477+0000", 2024-09-15T00:13:21.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:21.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:21.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968482+0000", 2024-09-15T00:13:21.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6180000000000003e-06 2024-09-15T00:13:21.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:21.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968486+0000", 2024-09-15T00:13:21.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.501e-06 2024-09-15T00:13:21.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968489+0000", 2024-09-15T00:13:21.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1530000000000001e-06 2024-09-15T00:13:21.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:21.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009983+0000", 2024-09-15T00:13:21.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041493980999999999 2024-09-15T00:13:21.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:21.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010051+0000", 2024-09-15T00:13:21.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8039000000000004e-05 2024-09-15T00:13:21.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:21.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010117+0000", 2024-09-15T00:13:21.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6254000000000006e-05 2024-09-15T00:13:21.034 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.034 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:21.034 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.035 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127013 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7979008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:21.035 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:21.035 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 58.995821831999997, 2024-09-15T00:13:21.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041763863999999998, 2024-09-15T00:13:21.035 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:21.035 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:21.035 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:21.035 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:21.035 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:21.035 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.035 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127013 2024-09-15T00:13:21.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.035 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:21.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:21.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:21.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:21.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:21.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:21.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969767+0000", 2024-09-15T00:13:21.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001275006 2024-09-15T00:13:21.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:21.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969772+0000", 2024-09-15T00:13:21.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4769999999999997e-06 2024-09-15T00:13:21.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:21.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969781+0000", 2024-09-15T00:13:21.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1959999999999997e-06 2024-09-15T00:13:21.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969788+0000", 2024-09-15T00:13:21.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6359999999999994e-06 2024-09-15T00:13:21.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:21.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010162+0000", 2024-09-15T00:13:21.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040373291999999998 2024-09-15T00:13:21.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:21.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010212+0000", 2024-09-15T00:13:21.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0543e-05 2024-09-15T00:13:21.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:21.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010256+0000", 2024-09-15T00:13:21.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3714000000000002e-05 2024-09-15T00:13:21.039 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.039 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:21.039 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.039 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127015 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8224768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:21.039 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:21.039 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 58.994482896000001, 2024-09-15T00:13:21.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040710186000000002, 2024-09-15T00:13:21.039 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:21.039 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:21.040 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:21.040 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:21.040 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:21.040 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.040 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127015 2024-09-15T00:13:21.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.040 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:21.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:21.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:21.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:21.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:21.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:21.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969832+0000", 2024-09-15T00:13:21.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:13:21.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:21.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969834+0000", 2024-09-15T00:13:21.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2919999999999998e-06 2024-09-15T00:13:21.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:21.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969843+0000", 2024-09-15T00:13:21.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0480000000000004e-06 2024-09-15T00:13:21.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969848+0000", 2024-09-15T00:13:21.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-15T00:13:21.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:21.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010457+0000", 2024-09-15T00:13:21.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040609054999999998 2024-09-15T00:13:21.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:21.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010501+0000", 2024-09-15T00:13:21.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3396000000000003e-05 2024-09-15T00:13:21.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:21.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010541+0000", 2024-09-15T00:13:21.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0599000000000002e-05 2024-09-15T00:13:21.043 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.043 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:21.043 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.043 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127016 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:21.044 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:21.044 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 58.994432269999997, 2024-09-15T00:13:21.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040766216000000001, 2024-09-15T00:13:21.044 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:21.044 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:21.044 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:21.044 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:21.044 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:21.044 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.044 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127016 2024-09-15T00:13:21.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.044 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:21.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:21.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:21.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:21.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:21.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:21.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:21.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:13:21.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:21.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969886+0000", 2024-09-15T00:13:21.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8449999999999996e-06 2024-09-15T00:13:21.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:21.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969897+0000", 2024-09-15T00:13:21.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-15T00:13:21.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969904+0000", 2024-09-15T00:13:21.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1169999999999998e-06 2024-09-15T00:13:21.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:21.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010561+0000", 2024-09-15T00:13:21.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040657470000000001 2024-09-15T00:13:21.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:21.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010607+0000", 2024-09-15T00:13:21.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6108e-05 2024-09-15T00:13:21.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:21.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010648+0000", 2024-09-15T00:13:21.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0479000000000003e-05 2024-09-15T00:13:21.047 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.048 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:21.048 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.048 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127694 3.9 3:9390b205:::rbd_data.11396481e1ec.00000000000000df:head [set-alloc-hint object_size 1048576 write_size 1048576,write 819200~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T00:13:21.048 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:21.048 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 58.739713152999997, 2024-09-15T00:13:21.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90376419399999997, 2024-09-15T00:13:21.048 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:21.048 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:21.048 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:21.048 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:21.048 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:21.048 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.049 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127694 2024-09-15T00:13:21.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.049 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:21.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:21.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:21.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:21.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:21.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:21.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224602+0000", 2024-09-15T00:13:21.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5899999999999995e-07 2024-09-15T00:13:21.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:21.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224610+0000", 2024-09-15T00:13:21.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9130000000000003e-06 2024-09-15T00:13:21.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:21.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224620+0000", 2024-09-15T00:13:21.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0426e-05 2024-09-15T00:13:21.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224627+0000", 2024-09-15T00:13:21.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4390000000000003e-06 2024-09-15T00:13:21.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:21.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224675+0000", 2024-09-15T00:13:21.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7636000000000003e-05 2024-09-15T00:13:21.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:21.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224681+0000", 2024-09-15T00:13:21.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1460000000000001e-06 2024-09-15T00:13:21.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:21.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126388+0000", 2024-09-15T00:13:21.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90170724999999996 2024-09-15T00:13:21.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:21.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126526+0000", 2024-09-15T00:13:21.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013730599999999999 2024-09-15T00:13:21.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:21.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126684+0000", 2024-09-15T00:13:21.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000157935 2024-09-15T00:13:21.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:21.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.127932+0000", 2024-09-15T00:13:21.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012479240000000001 2024-09-15T00:13:21.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:21.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128269+0000", 2024-09-15T00:13:21.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033761099999999998 2024-09-15T00:13:21.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:21.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128324+0000", 2024-09-15T00:13:21.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5244999999999997e-05 2024-09-15T00:13:21.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:21.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128365+0000", 2024-09-15T00:13:21.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0503999999999997e-05 2024-09-15T00:13:21.054 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.054 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:21.054 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.055 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:134707 3.15 3:af1fbdb5:::rbd_data.113933a4f79a.000000000000001c:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T00:13:21.055 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:21.055 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.271746039, 2024-09-15T00:13:21.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057328626000000001, 2024-09-15T00:13:21.055 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:21.055 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:21.055 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:21.055 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:21.055 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:21.055 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.055 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 134707 2024-09-15T00:13:21.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.055 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:21.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:21.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:21.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:21.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:21.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:21.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692569+0000", 2024-09-15T00:13:21.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.339e-06 2024-09-15T00:13:21.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:21.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692574+0000", 2024-09-15T00:13:21.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-15T00:13:21.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:21.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692584+0000", 2024-09-15T00:13:21.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0363000000000001e-05 2024-09-15T00:13:21.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692590+0000", 2024-09-15T00:13:21.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.818e-06 2024-09-15T00:13:21.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:21.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692633+0000", 2024-09-15T00:13:21.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.265e-05 2024-09-15T00:13:21.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:21.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692718+0000", 2024-09-15T00:13:21.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5279999999999997e-05 2024-09-15T00:13:21.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:21.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.695940+0000", 2024-09-15T00:13:21.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003222052 2024-09-15T00:13:21.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:21.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749714+0000", 2024-09-15T00:13:21.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.053774126999999998 2024-09-15T00:13:21.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:21.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749788+0000", 2024-09-15T00:13:21.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3693000000000001e-05 2024-09-15T00:13:21.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:21.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749847+0000", 2024-09-15T00:13:21.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175999999999997e-05 2024-09-15T00:13:21.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:21.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749897+0000", 2024-09-15T00:13:21.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9311999999999998e-05 2024-09-15T00:13:21.060 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.060 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:21.060 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.060 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:134707 3.15 e86/55 ondisk, result = 0)", 2024-09-15T00:13:21.060 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:21.060 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.263184690000003, 2024-09-15T00:13:21.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048596066, 2024-09-15T00:13:21.061 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:21.061 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:21.061 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:21.061 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:21.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:21.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:21.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:21.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:21.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:21.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:21.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701131+0000", 2024-09-15T00:13:21.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3549999999999999e-06 2024-09-15T00:13:21.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:21.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701139+0000", 2024-09-15T00:13:21.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1650000000000006e-06 2024-09-15T00:13:21.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:21.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701156+0000", 2024-09-15T00:13:21.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6730000000000001e-05 2024-09-15T00:13:21.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701167+0000", 2024-09-15T00:13:21.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1504e-05 2024-09-15T00:13:21.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:21.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749629+0000", 2024-09-15T00:13:21.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048461952000000003 2024-09-15T00:13:21.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:21.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749640+0000", 2024-09-15T00:13:21.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0874e-05 2024-09-15T00:13:21.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:21.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:21.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:21.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749725+0000", 2024-09-15T00:13:21.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5486000000000001e-05 2024-09-15T00:13:21.064 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.064 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:21.064 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.064 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:21.064 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:21.064 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:21.066 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:21.066 INFO:teuthology.orchestra.run.smithi171.stderr:marked in osd.1. 2024-09-15T00:13:21.067 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is set 2024-09-15T00:13:21.067 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-15T00:13:21.068 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:21.064+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:21.167 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:21.165+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:21.174 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:21.174 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:21.174 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:21.174 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:21.183 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:21.268 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:21.266+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:21.285 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:21.285 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:21.285 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:21.285 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:21.285 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:21.294 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:21.378 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:21.376+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:21.438 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:21.438 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:21.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:21.438 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:21.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.438 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:21.438 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:21.438 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 115.863320196, 2024-09-15T00:13:21.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:13:21.439 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.439 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.439 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:21.439 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.439 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.439 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.439 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:13:21.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.439 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:21.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:21.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:13:21.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:13:21.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:13:21.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:13:21.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:13:21.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:13:21.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:13:21.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:13:21.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:13:21.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:13:21.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:13:21.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:13:21.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:21.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:13:21.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:13:21.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:21.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:13:21.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:13:21.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:21.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:13:21.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:13:21.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:21.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:13:21.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:13:21.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:13:21.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:13:21.444 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.444 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.444 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.444 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:21.445 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:21.445 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 115.863285504, 2024-09-15T00:13:21.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:13:21.445 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.445 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.445 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:21.445 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.445 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.445 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.445 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:13:21.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.445 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:21.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:21.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:21.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:13:21.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:13:21.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:13:21.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:13:21.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:13:21.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:13:21.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:13:21.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:13:21.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:13:21.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:13:21.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:13:21.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:21.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:13:21.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:13:21.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:21.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:13:21.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:13:21.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:21.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:13:21.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:13:21.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:21.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:13:21.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:13:21.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:13:21.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:13:21.450 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.450 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.450 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.450 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.450 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:21.450 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.822611816000006, 2024-09-15T00:13:21.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:13:21.451 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.451 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.451 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.451 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.451 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.451 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.451 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:13:21.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.451 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:21.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:21.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:21.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:13:21.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:13:21.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:13:21.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:13:21.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:13:21.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:13:21.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:13:21.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:13:21.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:13:21.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:13:21.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:13:21.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:13:21.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:13:21.454 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.455 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.455 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.455 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.455 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:21.455 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.822595285000006, 2024-09-15T00:13:21.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:13:21.455 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.455 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.455 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.455 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.455 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.456 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.456 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:13:21.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.456 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:21.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:21.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:13:21.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:21.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:13:21.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:13:21.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:13:21.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:13:21.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:13:21.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:13:21.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:13:21.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:13:21.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:13:21.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:13:21.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:13:21.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:13:21.459 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.459 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.459 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.459 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.459 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:21.460 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.822578505999999, 2024-09-15T00:13:21.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:13:21.460 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.460 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.460 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.460 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.460 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.460 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.460 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:13:21.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.460 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:21.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:21.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:21.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:13:21.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:13:21.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:13:21.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:13:21.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:13:21.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:13:21.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:13:21.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:13:21.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:13:21.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:13:21.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:13:21.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:13:21.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:13:21.463 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.463 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.464 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.464 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.464 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:21.464 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.822560776000003, 2024-09-15T00:13:21.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:13:21.464 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.464 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.464 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.464 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.464 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.464 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.465 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:13:21.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.465 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:21.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:21.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:21.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:13:21.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:13:21.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:13:21.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:13:21.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:13:21.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:13:21.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:13:21.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:13:21.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:13:21.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:13:21.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:13:21.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:13:21.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:13:21.468 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.468 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.468 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.468 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.468 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:21.468 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.822543612000004, 2024-09-15T00:13:21.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:13:21.469 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.469 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.469 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.469 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.469 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.469 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.469 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:13:21.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.469 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:21.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:21.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:21.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:13:21.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:13:21.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:13:21.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:13:21.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:13:21.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:13:21.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:13:21.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:13:21.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:13:21.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:13:21.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:13:21.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:13:21.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:13:21.472 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.472 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.472 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.473 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.473 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:21.473 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.822521936000001, 2024-09-15T00:13:21.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:13:21.473 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.473 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.473 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.473 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.473 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.473 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.473 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:13:21.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.474 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:21.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:21.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:21.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:13:21.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:13:21.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:13:21.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:13:21.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:13:21.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:13:21.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:13:21.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:13:21.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:13:21.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:13:21.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:13:21.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:13:21.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:13:21.477 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.477 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.477 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.477 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.477 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:21.477 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.822475811999993, 2024-09-15T00:13:21.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:13:21.477 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.477 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.478 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.478 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.478 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.478 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.478 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:13:21.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.478 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:21.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:21.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:21.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:13:21.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:13:21.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:13:21.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:13:21.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:13:21.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:13:21.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:13:21.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:13:21.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:13:21.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:13:21.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:13:21.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:13:21.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:13:21.481 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.481 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.481 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.482 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.482 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:21.482 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.822446055, 2024-09-15T00:13:21.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:13:21.482 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.482 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.482 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.482 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.482 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.482 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.482 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:13:21.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.483 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:21.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:21.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:13:21.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:13:21.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:13:21.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:13:21.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:13:21.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:21.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:13:21.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:13:21.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:13:21.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:13:21.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:13:21.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:13:21.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:13:21.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:13:21.486 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.486 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.486 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.486 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.486 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:21.486 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.822419736000001, 2024-09-15T00:13:21.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:13:21.486 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.486 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.486 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.487 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.487 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.487 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.487 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:13:21.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.487 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:21.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:21.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:21.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:21.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:13:21.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:13:21.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:13:21.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:13:21.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:13:21.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:13:21.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:13:21.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:13:21.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:13:21.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:13:21.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:13:21.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:13:21.490 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.490 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.490 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.490 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.491 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:21.491 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.822383873999996, 2024-09-15T00:13:21.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:13:21.491 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.491 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.491 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.491 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.491 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.491 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.491 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:13:21.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.491 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:21.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:21.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:21.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:13:21.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:13:21.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:13:21.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:13:21.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:13:21.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:13:21.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:13:21.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:13:21.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:13:21.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:13:21.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:13:21.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:13:21.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:13:21.495 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.495 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.495 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.495 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.495 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:21.495 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.822362155999997, 2024-09-15T00:13:21.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:13:21.495 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.495 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.496 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.496 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.496 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.496 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.496 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:13:21.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.496 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:21.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:21.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:13:21.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:13:21.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:13:21.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:13:21.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:13:21.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:13:21.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:13:21.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:13:21.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:13:21.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:13:21.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:13:21.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:13:21.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:13:21.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:13:21.499 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.499 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.499 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.500 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.500 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:21.500 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.822165689000002, 2024-09-15T00:13:21.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:13:21.500 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.500 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.500 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.500 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:13:21.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.500 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:21.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:21.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:21.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:13:21.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:13:21.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:13:21.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:13:21.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:13:21.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:13:21.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:13:21.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:13:21.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:13:21.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:13:21.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:13:21.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:13:21.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:13:21.504 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.504 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.504 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.504 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.504 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:21.504 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.822139849999999, 2024-09-15T00:13:21.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:13:21.505 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.505 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.505 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.505 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.505 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.505 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.505 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:13:21.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.505 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:21.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:21.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:21.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:13:21.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:13:21.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:13:21.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:13:21.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:13:21.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:13:21.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:13:21.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:13:21.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:13:21.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:13:21.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:13:21.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:13:21.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:13:21.509 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.509 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.509 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.509 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.509 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:21.509 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.822078203999993, 2024-09-15T00:13:21.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:13:21.509 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.509 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.510 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.510 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.510 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.510 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.510 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:13:21.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.510 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:21.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:21.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:13:21.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:13:21.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:13:21.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:13:21.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:13:21.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:13:21.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:13:21.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:13:21.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:13:21.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:13:21.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:13:21.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:13:21.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:13:21.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:13:21.513 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.513 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.514 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.514 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.514 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:21.514 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.822038875999993, 2024-09-15T00:13:21.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:13:21.514 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.514 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.514 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.514 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.514 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.514 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.514 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:13:21.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.515 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:21.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:21.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:13:21.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:13:21.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:13:21.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:13:21.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:13:21.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:13:21.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:13:21.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:13:21.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:13:21.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:13:21.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:13:21.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:21.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:13:21.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:13:21.518 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.518 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.518 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.518 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.518 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:21.518 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.820313475999995, 2024-09-15T00:13:21.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:13:21.519 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.519 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.519 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.519 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.519 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.519 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.519 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:13:21.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.519 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:21.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:21.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:13:21.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:13:21.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:13:21.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:13:21.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:13:21.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:13:21.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:13:21.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:13:21.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:13:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:13:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:13:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:13:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:13:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:13:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 96.820291315000006, 2024-09-15T00:13:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:13:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:13:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:13:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:13:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:13:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:13:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:13:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:13:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:13:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:13:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:13:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:13:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:13:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:13:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:13:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:145034 3.6 3:618673df:::rbd_object_map.1139646a7d4e.000000000000002b:head [writefull 0~48] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T00:13:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.418037710999997, 2024-09-15T00:13:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0081079740000001, 2024-09-15T00:13:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 145034 2024-09-15T00:13:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017198+0000", 2024-09-15T00:13:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0100000000000004e-07 2024-09-15T00:13:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017202+0000", 2024-09-15T00:13:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:13:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017212+0000", 2024-09-15T00:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7189999999999996e-06 2024-09-15T00:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017219+0000", 2024-09-15T00:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7839999999999998e-06 2024-09-15T00:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017268+0000", 2024-09-15T00:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:13:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017277+0000", 2024-09-15T00:13:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1899999999999995e-06 2024-09-15T00:13:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019139+0000", 2024-09-15T00:13:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0018626129999999 2024-09-15T00:13:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019236+0000", 2024-09-15T00:13:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6513000000000002e-05 2024-09-15T00:13:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019318+0000", 2024-09-15T00:13:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1848000000000006e-05 2024-09-15T00:13:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.023151+0000", 2024-09-15T00:13:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038332169999999999 2024-09-15T00:13:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025217+0000", 2024-09-15T00:13:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020657610000000002 2024-09-15T00:13:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025267+0000", 2024-09-15T00:13:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0198000000000003e-05 2024-09-15T00:13:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025305+0000", 2024-09-15T00:13:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8065999999999998e-05 2024-09-15T00:13:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.533 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:21.535 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:21.535 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:21.533+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:21.636 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:21.634+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:21.679 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:21.679 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:21.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.679 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:174123 3.16 3.98a54356 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-15T00:13:21.679 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:13:21.048130+0000", 2024-09-15T00:13:21.679 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.62945848800000004, 2024-09-15T00:13:21.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.62948052499999996, 2024-09-15T00:13:21.679 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.679 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.679 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "waiting for peered", 2024-09-15T00:13:21.679 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.680 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.680 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.680 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 174123 2024-09-15T00:13:21.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.680 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.048130+0000", 2024-09-15T00:13:21.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.048130+0000", 2024-09-15T00:13:21.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.048132+0000", 2024-09-15T00:13:21.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1519999999999999e-06 2024-09-15T00:13:21.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.048139+0000", 2024-09-15T00:13:21.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0480000000000001e-06 2024-09-15T00:13:21.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.048154+0000", 2024-09-15T00:13:21.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4250000000000001e-05 2024-09-15T00:13:21.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.048164+0000", 2024-09-15T00:13:21.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0456e-05 2024-09-15T00:13:21.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.049381+0000", 2024-09-15T00:13:21.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0012168070000000001 2024-09-15T00:13:21.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:13:21.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.049391+0000", 2024-09-15T00:13:21.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0044000000000001e-05 2024-09-15T00:13:21.683 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.683 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.683 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.683 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:174124 3.16 3.98a54356 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-15T00:13:21.683 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:13:21.049243+0000", 2024-09-15T00:13:21.683 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.62834579599999996, 2024-09-15T00:13:21.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.62840852000000003, 2024-09-15T00:13:21.683 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.683 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.684 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "waiting for peered", 2024-09-15T00:13:21.684 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.684 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.684 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.684 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 174124 2024-09-15T00:13:21.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.684 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.684 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.684 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.049243+0000", 2024-09-15T00:13:21.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.684 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.685 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.049243+0000", 2024-09-15T00:13:21.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.685 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.049244+0000", 2024-09-15T00:13:21.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.015e-06 2024-09-15T00:13:21.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.685 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.049249+0000", 2024-09-15T00:13:21.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8420000000000004e-06 2024-09-15T00:13:21.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.049258+0000", 2024-09-15T00:13:21.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8589999999999994e-06 2024-09-15T00:13:21.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.049265+0000", 2024-09-15T00:13:21.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2130000000000002e-06 2024-09-15T00:13:21.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.049424+0000", 2024-09-15T00:13:21.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000159223 2024-09-15T00:13:21.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:13:21.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:21.049434+0000", 2024-09-15T00:13:21.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8679999999999994e-06 2024-09-15T00:13:21.687 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.687 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.687 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.687 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.687 INFO:teuthology.orchestra.run.smithi184.stdout: ], 2024-09-15T00:13:21.687 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 2 2024-09-15T00:13:21.687 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:21.688 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:21.736 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:21.734+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:21.791 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:21.791 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:21.791 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:21.791 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:21.791 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:21.800 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:21.837 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:21.835+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:21.938 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:21.936+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:21.944 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:21.944 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:21.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:21.945 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:21.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.945 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.945 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:21.945 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 97.332664328000007, 2024-09-15T00:13:21.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:13:21.945 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.945 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.945 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.945 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.945 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.945 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.946 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:13:21.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.946 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:21.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:21.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:21.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:13:21.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:13:21.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:13:21.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:13:21.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:13:21.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:13:21.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:13:21.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:13:21.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:13:21.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:13:21.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:13:21.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:13:21.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:13:21.949 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.949 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.949 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.949 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.949 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:21.949 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 97.332643098999995, 2024-09-15T00:13:21.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:13:21.950 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.950 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.950 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.950 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.950 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.950 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.950 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:13:21.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.950 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:21.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:21.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:13:21.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:13:21.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:13:21.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:21.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:13:21.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:13:21.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:13:21.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:13:21.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:13:21.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:13:21.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:13:21.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:13:21.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:13:21.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:13:21.954 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.954 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.954 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.954 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.954 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:21.954 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 97.332617557999995, 2024-09-15T00:13:21.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:13:21.954 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.955 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.955 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.955 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.955 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.955 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.955 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:13:21.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.955 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:21.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:21.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:21.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:21.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:13:21.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:13:21.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:13:21.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:13:21.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:13:21.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:13:21.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:13:21.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:13:21.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:13:21.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:13:21.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:13:21.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:13:21.958 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.958 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.958 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.959 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.959 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:21.959 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 97.332595083000001, 2024-09-15T00:13:21.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:13:21.959 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.959 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.959 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.959 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.959 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.959 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.959 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:13:21.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.959 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:21.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:21.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:13:21.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:13:21.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:13:21.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:13:21.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:13:21.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:13:21.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:13:21.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:13:21.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:13:21.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:13:21.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:13:21.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:13:21.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:13:21.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:13:21.963 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.963 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.963 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.963 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.963 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:21.963 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 97.332574573000002, 2024-09-15T00:13:21.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:13:21.963 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.963 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.963 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.963 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.964 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.964 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.964 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:13:21.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.964 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:21.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:21.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:21.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:13:21.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:13:21.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:13:21.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:13:21.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:13:21.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:13:21.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:13:21.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:13:21.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:13:21.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:13:21.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:13:21.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:13:21.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:13:21.967 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.967 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.967 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.967 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.967 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:21.967 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 97.332554258000002, 2024-09-15T00:13:21.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:13:21.968 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.968 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.968 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.968 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.968 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.968 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.968 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:13:21.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.968 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:21.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:21.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:13:21.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:13:21.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:13:21.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:13:21.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:13:21.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:13:21.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:13:21.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:13:21.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:13:21.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:13:21.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:13:21.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:13:21.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:13:21.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:13:21.971 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.971 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.972 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.972 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.972 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:21.972 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 97.332529481999998, 2024-09-15T00:13:21.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:13:21.972 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.972 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.972 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.972 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.972 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.972 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.973 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:13:21.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.973 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:21.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:21.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:21.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:13:21.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:13:21.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:13:21.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:13:21.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:13:21.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:13:21.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:13:21.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:13:21.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:13:21.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:13:21.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:13:21.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:13:21.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:13:21.976 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.976 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.976 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.976 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:21.976 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:21.976 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 97.153188095000004, 2024-09-15T00:13:21.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:13:21.977 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.977 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.977 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:21.977 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.977 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.977 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.977 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:13:21.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.977 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:21.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:21.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:13:21.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:13:21.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:13:21.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:13:21.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:13:21.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:13:21.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:13:21.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:13:21.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:13:21.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:13:21.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:13:21.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:13:21.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:21.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:13:21.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:13:21.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:21.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:13:21.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:13:21.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:21.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:13:21.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:13:21.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:21.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:13:21.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:13:21.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:13:21.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:13:21.982 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.982 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.982 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.982 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:13:21.982 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:21.982 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 97.136234642999995, 2024-09-15T00:13:21.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:13:21.982 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.982 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.982 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.983 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:21.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:21.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:13:21.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:13:21.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:13:21.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:13:21.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:13:21.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:13:21.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:13:21.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:13:21.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:13:21.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:13:21.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:13:21.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:13:21.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:13:21.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:13:21.986 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.986 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.986 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.986 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:13:21.986 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:21.986 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.841803736000003, 2024-09-15T00:13:21.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:13:21.987 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.987 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.987 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:21.987 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.987 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.987 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.987 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:13:21.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.987 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:21.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:21.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:13:21.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:13:21.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:13:21.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:13:21.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:13:21.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:13:21.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:13:21.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:13:21.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:13:21.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:13:21.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:13:21.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:13:21.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:21.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:13:21.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:13:21.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:21.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:13:21.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:13:21.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:21.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:13:21.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:13:21.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:21.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:13:21.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:13:21.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:13:21.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:13:21.992 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.992 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.992 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.993 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:13:21.993 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:21.993 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 83.829347921999997, 2024-09-15T00:13:21.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:13:21.993 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.993 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.993 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:21.993 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:21.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:13:21.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:13:21.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:13:21.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:13:21.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:13:21.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:13:21.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:21.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:13:21.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:13:21.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:21.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:13:21.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:13:21.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.996 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:21.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:21.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.996 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:146935 3.16 3:6c79e092:::rbd_data.11394501364.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:21.997 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:21.997 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.188568404000002, 2024-09-15T00:13:21.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.058987488999999997, 2024-09-15T00:13:21.997 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:21.997 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:21.997 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:21.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:21.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:21.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:21.997 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 146935 2024-09-15T00:13:21.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.997 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:21.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753644+0000", 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6199999999999997e-07 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:21.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753648+0000", 2024-09-15T00:13:21.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-15T00:13:21.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:21.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753659+0000", 2024-09-15T00:13:21.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0703000000000001e-05 2024-09-15T00:13:21.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:21.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753664+0000", 2024-09-15T00:13:21.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.04e-06 2024-09-15T00:13:21.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:21.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:21.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:21.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753709+0000", 2024-09-15T00:13:22.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4728000000000003e-05 2024-09-15T00:13:22.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753756+0000", 2024-09-15T00:13:22.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7808e-05 2024-09-15T00:13:22.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:22.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.757051+0000", 2024-09-15T00:13:22.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294249 2024-09-15T00:13:22.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812553+0000", 2024-09-15T00:13:22.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055501807 2024-09-15T00:13:22.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812564+0000", 2024-09-15T00:13:22.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1467999999999999e-05 2024-09-15T00:13:22.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812598+0000", 2024-09-15T00:13:22.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3553000000000002e-05 2024-09-15T00:13:22.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812630+0000", 2024-09-15T00:13:22.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2895000000000003e-05 2024-09-15T00:13:22.002 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.002 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.002 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.002 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:146935 3.16 e98/95 ondisk, result = 0)", 2024-09-15T00:13:22.002 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:22.002 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.177765708000003, 2024-09-15T00:13:22.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048111830000000001, 2024-09-15T00:13:22.002 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.002 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.002 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.003 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:22.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:22.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764447+0000", 2024-09-15T00:13:22.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0500000000000002e-07 2024-09-15T00:13:22.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764450+0000", 2024-09-15T00:13:22.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4400000000000001e-06 2024-09-15T00:13:22.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764462+0000", 2024-09-15T00:13:22.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1603000000000001e-05 2024-09-15T00:13:22.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764469+0000", 2024-09-15T00:13:22.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5089999999999996e-06 2024-09-15T00:13:22.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812541+0000", 2024-09-15T00:13:22.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048071585 2024-09-15T00:13:22.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812548+0000", 2024-09-15T00:13:22.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9979999999999997e-06 2024-09-15T00:13:22.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812557+0000", 2024-09-15T00:13:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7899999999999994e-06 2024-09-15T00:13:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:152505 3.17 3:ea3f268c:::rbd_data.113958cb0e55.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.275659746000002, 2024-09-15T00:13:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.060817959999999997, 2024-09-15T00:13:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.007 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.007 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 152505 2024-09-15T00:13:22.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.007 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:22.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:22.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666553+0000", 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666562+0000", 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0909999999999999e-06 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666572+0000", 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0214000000000001e-05 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666577+0000", 2024-09-15T00:13:22.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3800000000000002e-06 2024-09-15T00:13:22.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666617+0000", 2024-09-15T00:13:22.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9304000000000002e-05 2024-09-15T00:13:22.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666672+0000", 2024-09-15T00:13:22.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5206e-05 2024-09-15T00:13:22.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:22.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.670821+0000", 2024-09-15T00:13:22.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041493169999999996 2024-09-15T00:13:22.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727290+0000", 2024-09-15T00:13:22.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.056468580999999997 2024-09-15T00:13:22.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727302+0000", 2024-09-15T00:13:22.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2605e-05 2024-09-15T00:13:22.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727337+0000", 2024-09-15T00:13:22.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4783999999999998e-05 2024-09-15T00:13:22.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727370+0000", 2024-09-15T00:13:22.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2366000000000001e-05 2024-09-15T00:13:22.011 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.011 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.011 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.012 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:152505 3.17 e103/68 ondisk, result = 0)", 2024-09-15T00:13:22.012 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:22.012 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.263581309999999, 2024-09-15T00:13:22.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048665938999999998, 2024-09-15T00:13:22.012 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.012 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.012 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.012 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:22.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:22.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678631+0000", 2024-09-15T00:13:22.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5099999999999998e-07 2024-09-15T00:13:22.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678634+0000", 2024-09-15T00:13:22.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.478e-06 2024-09-15T00:13:22.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678647+0000", 2024-09-15T00:13:22.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2085e-05 2024-09-15T00:13:22.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678653+0000", 2024-09-15T00:13:22.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2099999999999998e-06 2024-09-15T00:13:22.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727277+0000", 2024-09-15T00:13:22.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048624405000000002 2024-09-15T00:13:22.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727285+0000", 2024-09-15T00:13:22.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2060000000000007e-06 2024-09-15T00:13:22.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727296+0000", 2024-09-15T00:13:22.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0604e-05 2024-09-15T00:13:22.016 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.016 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.016 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.017 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153959 3.c 3:326b8454:::rbd_data.113958cb0e55.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:22.017 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:22.017 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.852353363999999, 2024-09-15T00:13:22.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.068102041000000002, 2024-09-15T00:13:22.017 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.018 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.018 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:22.018 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.018 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.018 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.018 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153959 2024-09-15T00:13:22.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.019 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:22.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:22.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089859+0000", 2024-09-15T00:13:22.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1800000000000004e-07 2024-09-15T00:13:22.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089864+0000", 2024-09-15T00:13:22.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2079999999999999e-06 2024-09-15T00:13:22.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089880+0000", 2024-09-15T00:13:22.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5695000000000001e-05 2024-09-15T00:13:22.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089889+0000", 2024-09-15T00:13:22.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.935e-06 2024-09-15T00:13:22.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089942+0000", 2024-09-15T00:13:22.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3219e-05 2024-09-15T00:13:22.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.090066+0000", 2024-09-15T00:13:22.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012385099999999999 2024-09-15T00:13:22.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:22.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.095262+0000", 2024-09-15T00:13:22.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051963950000000004 2024-09-15T00:13:22.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157865+0000", 2024-09-15T00:13:22.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.062602865999999993 2024-09-15T00:13:22.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157891+0000", 2024-09-15T00:13:22.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5778e-05 2024-09-15T00:13:22.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157924+0000", 2024-09-15T00:13:22.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3133000000000003e-05 2024-09-15T00:13:22.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157960+0000", 2024-09-15T00:13:22.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6043e-05 2024-09-15T00:13:22.023 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.023 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.023 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.024 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:153959 3.c e103/23 ondisk, result = 0)", 2024-09-15T00:13:22.024 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:22.024 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.839697705999999, 2024-09-15T00:13:22.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055370211000000003, 2024-09-15T00:13:22.024 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.024 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.024 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.024 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:22.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102515+0000", 2024-09-15T00:13:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.206e-06 2024-09-15T00:13:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102520+0000", 2024-09-15T00:13:22.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1209999999999998e-06 2024-09-15T00:13:22.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102537+0000", 2024-09-15T00:13:22.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:22.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102546+0000", 2024-09-15T00:13:22.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2329999999999995e-06 2024-09-15T00:13:22.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157847+0000", 2024-09-15T00:13:22.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055300798999999998 2024-09-15T00:13:22.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157857+0000", 2024-09-15T00:13:22.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.046e-05 2024-09-15T00:13:22.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157884+0000", 2024-09-15T00:13:22.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.671e-05 2024-09-15T00:13:22.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.027 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.028 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163346 3.1d 3:baa43c12:::rbd_data.11395d771336.0000000000000065:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:13:22.028 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:22.028 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.171083133, 2024-09-15T00:13:22.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047558544000000001, 2024-09-15T00:13:22.028 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.028 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.028 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:22.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.028 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163346 2024-09-15T00:13:22.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.029 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:22.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:22.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771130+0000", 2024-09-15T00:13:22.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5939999999999999e-06 2024-09-15T00:13:22.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771141+0000", 2024-09-15T00:13:22.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1001e-05 2024-09-15T00:13:22.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771154+0000", 2024-09-15T00:13:22.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2921e-05 2024-09-15T00:13:22.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771160+0000", 2024-09-15T00:13:22.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3849999999999999e-06 2024-09-15T00:13:22.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771207+0000", 2024-09-15T00:13:22.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6888999999999998e-05 2024-09-15T00:13:22.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771306+0000", 2024-09-15T00:13:22.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9259000000000006e-05 2024-09-15T00:13:22.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:22.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.774692+0000", 2024-09-15T00:13:22.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033854850000000001 2024-09-15T00:13:22.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818600+0000", 2024-09-15T00:13:22.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043908006999999999 2024-09-15T00:13:22.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818616+0000", 2024-09-15T00:13:22.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6688e-05 2024-09-15T00:13:22.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818651+0000", 2024-09-15T00:13:22.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4885000000000002e-05 2024-09-15T00:13:22.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818687+0000", 2024-09-15T00:13:22.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5429999999999998e-05 2024-09-15T00:13:22.033 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.033 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.033 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.034 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163578 3.1c 3:3efd71ab:::rbd_data.11395d771336.0000000000000060:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-15T00:13:22.034 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:22.034 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.771660384, 2024-09-15T00:13:22.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.085154081000000006, 2024-09-15T00:13:22.034 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.034 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.034 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:22.034 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.034 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.034 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.034 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163578 2024-09-15T00:13:22.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.034 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:22.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:22.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170552+0000", 2024-09-15T00:13:22.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8599999999999997e-07 2024-09-15T00:13:22.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170557+0000", 2024-09-15T00:13:22.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-15T00:13:22.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170567+0000", 2024-09-15T00:13:22.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0628999999999999e-05 2024-09-15T00:13:22.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170573+0000", 2024-09-15T00:13:22.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1649999999999997e-06 2024-09-15T00:13:22.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170619+0000", 2024-09-15T00:13:22.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6405e-05 2024-09-15T00:13:22.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170666+0000", 2024-09-15T00:13:22.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7259000000000003e-05 2024-09-15T00:13:22.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:22.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.175060+0000", 2024-09-15T00:13:22.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043940170000000001 2024-09-15T00:13:22.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255559+0000", 2024-09-15T00:13:22.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080499065999999994 2024-09-15T00:13:22.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255616+0000", 2024-09-15T00:13:22.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6422000000000003e-05 2024-09-15T00:13:22.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255655+0000", 2024-09-15T00:13:22.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9632999999999998e-05 2024-09-15T00:13:22.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255705+0000", 2024-09-15T00:13:22.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.973e-05 2024-09-15T00:13:22.039 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.039 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.039 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.039 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:163578 3.1c e107/23 ondisk, result = 0)", 2024-09-15T00:13:22.039 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:22.040 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.756438292999999, 2024-09-15T00:13:22.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069939984999999996, 2024-09-15T00:13:22.040 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.040 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.040 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.040 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:22.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:22.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185774+0000", 2024-09-15T00:13:22.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.093e-06 2024-09-15T00:13:22.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185779+0000", 2024-09-15T00:13:22.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9470000000000002e-06 2024-09-15T00:13:22.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185794+0000", 2024-09-15T00:13:22.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4450000000000001e-05 2024-09-15T00:13:22.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185801+0000", 2024-09-15T00:13:22.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9189999999999999e-06 2024-09-15T00:13:22.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255604+0000", 2024-09-15T00:13:22.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069802201999999994 2024-09-15T00:13:22.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255611+0000", 2024-09-15T00:13:22.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7740000000000004e-06 2024-09-15T00:13:22.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255713+0000", 2024-09-15T00:13:22.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0001016 2024-09-15T00:13:22.043 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.043 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.043 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.043 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.043 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.043 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:22.045 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:22.045 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:22.042+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:22.146 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:22.144+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:22.150 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:22.150 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:22.150 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:22.151 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:22.160 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:22.247 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:22.245+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:22.304 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:22.305 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:22.305 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:22.305 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:22.305 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:22.313 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:22.348 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:22.345+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:22.417 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:22.417 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:22.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:22.417 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 97.747497855000006, 2024-09-15T00:13:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:13:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:13:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:13:22.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:13:22.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:13:22.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:13:22.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:13:22.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:13:22.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:13:22.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:13:22.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:13:22.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:13:22.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:13:22.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:13:22.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:22.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:13:22.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:13:22.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:13:22.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:13:22.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:13:22.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:13:22.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:13:22.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:13:22.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:13:22.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:13:22.423 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.423 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.423 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.424 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125618 3.10 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:22.424 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:22.424 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 60.576358773000003, 2024-09-15T00:13:22.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030860942999999998, 2024-09-15T00:13:22.424 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.424 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.424 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:22.424 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.424 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.424 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.424 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125618 2024-09-15T00:13:22.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.424 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:22.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:22.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838499+0000", 2024-09-15T00:13:22.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6189999999999998e-06 2024-09-15T00:13:22.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839599+0000", 2024-09-15T00:13:22.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001099764 2024-09-15T00:13:22.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839620+0000", 2024-09-15T00:13:22.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0764e-05 2024-09-15T00:13:22.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839631+0000", 2024-09-15T00:13:22.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1335000000000001e-05 2024-09-15T00:13:22.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839725+0000", 2024-09-15T00:13:22.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4217999999999998e-05 2024-09-15T00:13:22.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839910+0000", 2024-09-15T00:13:22.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000184298 2024-09-15T00:13:22.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:22.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843171+0000", 2024-09-15T00:13:22.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032613540000000002 2024-09-15T00:13:22.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869187+0000", 2024-09-15T00:13:22.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026015467 2024-09-15T00:13:22.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869240+0000", 2024-09-15T00:13:22.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3733000000000002e-05 2024-09-15T00:13:22.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869309+0000", 2024-09-15T00:13:22.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8381999999999997e-05 2024-09-15T00:13:22.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869358+0000", 2024-09-15T00:13:22.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-15T00:13:22.429 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.429 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.429 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.430 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125619 3.a 3:507b29bf:::rbd_data.11396481e1ec.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:22.430 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:22.430 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 60.575196644000002, 2024-09-15T00:13:22.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045768870000000003, 2024-09-15T00:13:22.430 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.430 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.430 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:22.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.430 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125619 2024-09-15T00:13:22.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.430 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:22.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:22.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839660+0000", 2024-09-15T00:13:22.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.22e-06 2024-09-15T00:13:22.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840511+0000", 2024-09-15T00:13:22.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000851284 2024-09-15T00:13:22.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840522+0000", 2024-09-15T00:13:22.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0569e-05 2024-09-15T00:13:22.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840527+0000", 2024-09-15T00:13:22.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.412e-06 2024-09-15T00:13:22.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840573+0000", 2024-09-15T00:13:22.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5247000000000002e-05 2024-09-15T00:13:22.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840682+0000", 2024-09-15T00:13:22.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109368 2024-09-15T00:13:22.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:22.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843976+0000", 2024-09-15T00:13:22.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294445 2024-09-15T00:13:22.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870833+0000", 2024-09-15T00:13:22.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026856944000000001 2024-09-15T00:13:22.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885344+0000", 2024-09-15T00:13:22.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014510744000000001 2024-09-15T00:13:22.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885382+0000", 2024-09-15T00:13:22.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7821000000000003e-05 2024-09-15T00:13:22.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885428+0000", 2024-09-15T00:13:22.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5816000000000003e-05 2024-09-15T00:13:22.435 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.435 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.435 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.435 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125621 3.a 3:53594db9:::rbd_data.11396481e1ec.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:22.435 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:22.436 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 60.561673265000003, 2024-09-15T00:13:22.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032481797, 2024-09-15T00:13:22.436 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.436 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.436 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:22.436 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.436 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.436 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.436 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125621 2024-09-15T00:13:22.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.436 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:22.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:22.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853185+0000", 2024-09-15T00:13:22.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.43e-06 2024-09-15T00:13:22.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854125+0000", 2024-09-15T00:13:22.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000940539 2024-09-15T00:13:22.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854141+0000", 2024-09-15T00:13:22.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5568000000000001e-05 2024-09-15T00:13:22.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854148+0000", 2024-09-15T00:13:22.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6529999999999994e-06 2024-09-15T00:13:22.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870872+0000", 2024-09-15T00:13:22.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016723135 2024-09-15T00:13:22.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870975+0000", 2024-09-15T00:13:22.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010314099999999999 2024-09-15T00:13:22.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:22.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.873082+0000", 2024-09-15T00:13:22.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021069859999999999 2024-09-15T00:13:22.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885439+0000", 2024-09-15T00:13:22.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012357343999999999 2024-09-15T00:13:22.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885569+0000", 2024-09-15T00:13:22.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012995199999999999 2024-09-15T00:13:22.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885624+0000", 2024-09-15T00:13:22.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5281999999999997e-05 2024-09-15T00:13:22.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885664+0000", 2024-09-15T00:13:22.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9767000000000001e-05 2024-09-15T00:13:22.441 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.441 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.441 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.441 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:147075 3.0 3:06ab39f2:::rbd_data.11394501364.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:22.441 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:22.441 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.465774883999998, 2024-09-15T00:13:22.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046402906000000001, 2024-09-15T00:13:22.442 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.442 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.442 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:22.442 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.442 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.442 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.442 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 147075 2024-09-15T00:13:22.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.442 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:22.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:22.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:22.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4399999999999999e-07 2024-09-15T00:13:22.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949086+0000", 2024-09-15T00:13:22.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.138e-06 2024-09-15T00:13:22.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949096+0000", 2024-09-15T00:13:22.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0438999999999999e-05 2024-09-15T00:13:22.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949101+0000", 2024-09-15T00:13:22.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8999999999999997e-06 2024-09-15T00:13:22.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949145+0000", 2024-09-15T00:13:22.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4141000000000003e-05 2024-09-15T00:13:22.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949195+0000", 2024-09-15T00:13:22.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9657999999999998e-05 2024-09-15T00:13:22.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:22.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.950923+0000", 2024-09-15T00:13:22.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001728753 2024-09-15T00:13:22.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995390+0000", 2024-09-15T00:13:22.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044466335000000003 2024-09-15T00:13:22.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995401+0000", 2024-09-15T00:13:22.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1671000000000001e-05 2024-09-15T00:13:22.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995432+0000", 2024-09-15T00:13:22.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0704999999999998e-05 2024-09-15T00:13:22.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995484+0000", 2024-09-15T00:13:22.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1422000000000003e-05 2024-09-15T00:13:22.448 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.448 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.448 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.448 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:147075 3.0 e98/94 ondisk, result = 0)", 2024-09-15T00:13:22.448 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:22.448 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.457975163, 2024-09-15T00:13:22.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038514459000000001, 2024-09-15T00:13:22.448 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.448 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.449 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.449 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:22.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:22.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956881+0000", 2024-09-15T00:13:22.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0599999999999999e-07 2024-09-15T00:13:22.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956885+0000", 2024-09-15T00:13:22.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3500000000000001e-06 2024-09-15T00:13:22.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956897+0000", 2024-09-15T00:13:22.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2303999999999999e-05 2024-09-15T00:13:22.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956902+0000", 2024-09-15T00:13:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5169999999999999e-06 2024-09-15T00:13:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995377+0000", 2024-09-15T00:13:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038474544999999999 2024-09-15T00:13:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995385+0000", 2024-09-15T00:13:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6240000000000002e-06 2024-09-15T00:13:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995395+0000", 2024-09-15T00:13:22.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0213e-05 2024-09-15T00:13:22.452 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.452 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.452 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.452 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153172 3.14 3:2a7843e3:::rbd_data.113958cb0e55.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 983040~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:22.452 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:22.452 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.637086099000001, 2024-09-15T00:13:22.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.033232668, 2024-09-15T00:13:22.452 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.452 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.452 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:22.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.453 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153172 2024-09-15T00:13:22.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.453 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:22.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777770+0000", 2024-09-15T00:13:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.017e-06 2024-09-15T00:13:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788009+0000", 2024-09-15T00:13:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010238146 2024-09-15T00:13:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788024+0000", 2024-09-15T00:13:22.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5582000000000001e-05 2024-09-15T00:13:22.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788048+0000", 2024-09-15T00:13:22.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4173e-05 2024-09-15T00:13:22.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788097+0000", 2024-09-15T00:13:22.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8498999999999998e-05 2024-09-15T00:13:22.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788201+0000", 2024-09-15T00:13:22.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010398799999999999 2024-09-15T00:13:22.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:22.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788393+0000", 2024-09-15T00:13:22.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000192337 2024-09-15T00:13:22.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810637+0000", 2024-09-15T00:13:22.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.022244202000000001 2024-09-15T00:13:22.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810903+0000", 2024-09-15T00:13:22.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00026574600000000003 2024-09-15T00:13:22.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810953+0000", 2024-09-15T00:13:22.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9509000000000003e-05 2024-09-15T00:13:22.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.811002+0000", 2024-09-15T00:13:22.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9468999999999997e-05 2024-09-15T00:13:22.458 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.458 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.458 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.458 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170014 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:22.458 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:22.458 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.567790992999999, 2024-09-15T00:13:22.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030291116999999999, 2024-09-15T00:13:22.458 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.458 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.458 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.458 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.458 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.458 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.459 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170014 2024-09-15T00:13:22.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.459 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:22.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:22.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:22.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:22.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847073+0000", 2024-09-15T00:13:22.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9589999999999995e-06 2024-09-15T00:13:22.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847080+0000", 2024-09-15T00:13:22.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2930000000000003e-06 2024-09-15T00:13:22.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847086+0000", 2024-09-15T00:13:22.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.631e-06 2024-09-15T00:13:22.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877213+0000", 2024-09-15T00:13:22.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030127017999999998 2024-09-15T00:13:22.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877281+0000", 2024-09-15T00:13:22.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7916999999999994e-05 2024-09-15T00:13:22.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877356+0000", 2024-09-15T00:13:22.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4796999999999996e-05 2024-09-15T00:13:22.462 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.462 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.462 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.462 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170017 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1310720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:22.462 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:22.462 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.567762342000002, 2024-09-15T00:13:22.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030437623, 2024-09-15T00:13:22.463 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.463 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.463 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.463 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.463 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.463 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.463 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170017 2024-09-15T00:13:22.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.463 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:22.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:22.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847094+0000", 2024-09-15T00:13:22.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:13:22.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847096+0000", 2024-09-15T00:13:22.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.136e-06 2024-09-15T00:13:22.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847104+0000", 2024-09-15T00:13:22.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7190000000000001e-06 2024-09-15T00:13:22.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847109+0000", 2024-09-15T00:13:22.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3279999999999996e-06 2024-09-15T00:13:22.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877384+0000", 2024-09-15T00:13:22.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030275348000000001 2024-09-15T00:13:22.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877451+0000", 2024-09-15T00:13:22.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6509999999999998e-05 2024-09-15T00:13:22.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877531+0000", 2024-09-15T00:13:22.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9993000000000005e-05 2024-09-15T00:13:22.467 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.467 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.467 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.467 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170018 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1351680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:22.467 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:22.467 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.567739997, 2024-09-15T00:13:22.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030591884999999999, 2024-09-15T00:13:22.467 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.467 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.467 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.467 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.467 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.467 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.468 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170018 2024-09-15T00:13:22.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.468 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:22.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:22.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:22.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:13:22.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847118+0000", 2024-09-15T00:13:22.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.03e-06 2024-09-15T00:13:22.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847124+0000", 2024-09-15T00:13:22.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:22.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847129+0000", 2024-09-15T00:13:22.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8600000000000001e-06 2024-09-15T00:13:22.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877560+0000", 2024-09-15T00:13:22.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030431145999999999 2024-09-15T00:13:22.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877629+0000", 2024-09-15T00:13:22.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8793000000000004e-05 2024-09-15T00:13:22.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877707+0000", 2024-09-15T00:13:22.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8139999999999994e-05 2024-09-15T00:13:22.471 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.471 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.471 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.471 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170019 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1392640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:22.471 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:22.471 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.567719249, 2024-09-15T00:13:22.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030769023, 2024-09-15T00:13:22.472 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.472 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.472 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.472 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.472 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.472 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.472 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170019 2024-09-15T00:13:22.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.472 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:22.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:22.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847137+0000", 2024-09-15T00:13:22.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:13:22.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847139+0000", 2024-09-15T00:13:22.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.142e-06 2024-09-15T00:13:22.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847146+0000", 2024-09-15T00:13:22.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8120000000000002e-06 2024-09-15T00:13:22.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847157+0000", 2024-09-15T00:13:22.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1093e-05 2024-09-15T00:13:22.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877735+0000", 2024-09-15T00:13:22.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578195999999998 2024-09-15T00:13:22.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877810+0000", 2024-09-15T00:13:22.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5563999999999997e-05 2024-09-15T00:13:22.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877905+0000", 2024-09-15T00:13:22.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4809000000000007e-05 2024-09-15T00:13:22.475 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.475 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.476 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.476 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170020 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:22.476 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:22.476 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.567692009000002, 2024-09-15T00:13:22.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030917150000000001, 2024-09-15T00:13:22.476 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.476 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.476 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.476 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.476 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.476 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.477 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170020 2024-09-15T00:13:22.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.477 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:22.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:22.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:22.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:13:22.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847166+0000", 2024-09-15T00:13:22.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2790000000000001e-06 2024-09-15T00:13:22.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847173+0000", 2024-09-15T00:13:22.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3969999999999999e-06 2024-09-15T00:13:22.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847177+0000", 2024-09-15T00:13:22.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8620000000000002e-06 2024-09-15T00:13:22.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877934+0000", 2024-09-15T00:13:22.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030756143999999999 2024-09-15T00:13:22.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878002+0000", 2024-09-15T00:13:22.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7951e-05 2024-09-15T00:13:22.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878081+0000", 2024-09-15T00:13:22.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9087999999999999e-05 2024-09-15T00:13:22.480 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.480 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.480 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.480 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170094 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:22.480 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:22.481 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.567434760000001, 2024-09-15T00:13:22.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030836142, 2024-09-15T00:13:22.481 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.481 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.481 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.481 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.481 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.481 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.481 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170094 2024-09-15T00:13:22.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.481 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:22.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:22.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:22.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3499999999999996e-07 2024-09-15T00:13:22.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847424+0000", 2024-09-15T00:13:22.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2850000000000001e-06 2024-09-15T00:13:22.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847430+0000", 2024-09-15T00:13:22.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3260000000000001e-06 2024-09-15T00:13:22.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847435+0000", 2024-09-15T00:13:22.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.356e-06 2024-09-15T00:13:22.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878110+0000", 2024-09-15T00:13:22.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030675021 2024-09-15T00:13:22.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878177+0000", 2024-09-15T00:13:22.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6506000000000003e-05 2024-09-15T00:13:22.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878257+0000", 2024-09-15T00:13:22.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0012999999999994e-05 2024-09-15T00:13:22.485 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.485 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.485 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.485 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170096 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 458752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:22.485 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:22.485 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.567392804000001, 2024-09-15T00:13:22.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030977030999999999, 2024-09-15T00:13:22.485 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.485 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.485 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.485 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.485 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.486 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.486 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170096 2024-09-15T00:13:22.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.486 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:22.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:22.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847465+0000", 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0559999999999999e-06 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847472+0000", 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3130000000000004e-06 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847477+0000", 2024-09-15T00:13:22.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9930000000000003e-06 2024-09-15T00:13:22.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878288+0000", 2024-09-15T00:13:22.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030811609 2024-09-15T00:13:22.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878358+0000", 2024-09-15T00:13:22.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0215000000000005e-05 2024-09-15T00:13:22.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878440+0000", 2024-09-15T00:13:22.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1366999999999995e-05 2024-09-15T00:13:22.489 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.489 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.489 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.489 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170097 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:22.489 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:22.489 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.567372719000002, 2024-09-15T00:13:22.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031134757999999998, 2024-09-15T00:13:22.489 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.489 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.490 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.490 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.490 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.490 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.490 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170097 2024-09-15T00:13:22.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.490 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:22.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:22.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:22.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:13:22.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847485+0000", 2024-09-15T00:13:22.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9190000000000002e-06 2024-09-15T00:13:22.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847492+0000", 2024-09-15T00:13:22.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7109999999999998e-06 2024-09-15T00:13:22.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847497+0000", 2024-09-15T00:13:22.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0259999999999998e-06 2024-09-15T00:13:22.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878469+0000", 2024-09-15T00:13:22.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030972115000000001 2024-09-15T00:13:22.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878538+0000", 2024-09-15T00:13:22.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8669000000000003e-05 2024-09-15T00:13:22.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878618+0000", 2024-09-15T00:13:22.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9874000000000001e-05 2024-09-15T00:13:22.493 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.493 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.493 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.494 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170099 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 532480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:22.494 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:22.494 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.567321156999999, 2024-09-15T00:13:22.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031262183999999998, 2024-09-15T00:13:22.494 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.494 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.494 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.494 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.494 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.494 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.494 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170099 2024-09-15T00:13:22.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.494 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:22.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:22.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847535+0000", 2024-09-15T00:13:22.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:13:22.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847537+0000", 2024-09-15T00:13:22.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:22.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847544+0000", 2024-09-15T00:13:22.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9630000000000001e-06 2024-09-15T00:13:22.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847549+0000", 2024-09-15T00:13:22.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8910000000000003e-06 2024-09-15T00:13:22.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878646+0000", 2024-09-15T00:13:22.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031097400000000001 2024-09-15T00:13:22.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878717+0000", 2024-09-15T00:13:22.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0993000000000003e-05 2024-09-15T00:13:22.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878797+0000", 2024-09-15T00:13:22.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9425000000000006e-05 2024-09-15T00:13:22.498 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.498 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.498 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.498 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170101 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:22.498 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:22.498 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.567279778, 2024-09-15T00:13:22.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031674568, 2024-09-15T00:13:22.498 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.498 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.498 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.499 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.499 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.499 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.499 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170101 2024-09-15T00:13:22.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.499 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:22.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:22.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:22.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:13:22.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847578+0000", 2024-09-15T00:13:22.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0389999999999999e-06 2024-09-15T00:13:22.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847585+0000", 2024-09-15T00:13:22.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.336e-06 2024-09-15T00:13:22.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847596+0000", 2024-09-15T00:13:22.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015999999999999e-05 2024-09-15T00:13:22.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878825+0000", 2024-09-15T00:13:22.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031229848000000001 2024-09-15T00:13:22.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878899+0000", 2024-09-15T00:13:22.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:22.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879250+0000", 2024-09-15T00:13:22.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035110000000000002 2024-09-15T00:13:22.502 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.502 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.502 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.503 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170102 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 614400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:22.503 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:22.503 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.567252481000001, 2024-09-15T00:13:22.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031577371999999999, 2024-09-15T00:13:22.503 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.503 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.503 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.503 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170102 2024-09-15T00:13:22.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.503 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:22.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:22.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:22.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:13:22.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847610+0000", 2024-09-15T00:13:22.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6329999999999999e-06 2024-09-15T00:13:22.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847618+0000", 2024-09-15T00:13:22.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4599999999999997e-06 2024-09-15T00:13:22.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847623+0000", 2024-09-15T00:13:22.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2909999999999998e-06 2024-09-15T00:13:22.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879016+0000", 2024-09-15T00:13:22.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031393227000000003 2024-09-15T00:13:22.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879091+0000", 2024-09-15T00:13:22.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5384999999999997e-05 2024-09-15T00:13:22.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879180+0000", 2024-09-15T00:13:22.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8941000000000001e-05 2024-09-15T00:13:22.507 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.507 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.507 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.507 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170132 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:22.507 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:22.507 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.567205100999999, 2024-09-15T00:13:22.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031722579000000001, 2024-09-15T00:13:22.507 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.507 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.507 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.508 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.508 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.508 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170132 2024-09-15T00:13:22.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.508 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:22.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:22.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847651+0000", 2024-09-15T00:13:22.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1900000000000003e-07 2024-09-15T00:13:22.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847653+0000", 2024-09-15T00:13:22.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0329999999999998e-06 2024-09-15T00:13:22.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847659+0000", 2024-09-15T00:13:22.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4169999999999997e-06 2024-09-15T00:13:22.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847670+0000", 2024-09-15T00:13:22.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1031e-05 2024-09-15T00:13:22.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879205+0000", 2024-09-15T00:13:22.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031534982000000003 2024-09-15T00:13:22.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879286+0000", 2024-09-15T00:13:22.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0675999999999998e-05 2024-09-15T00:13:22.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879373+0000", 2024-09-15T00:13:22.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6921000000000003e-05 2024-09-15T00:13:22.511 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.511 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.511 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.511 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170133 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:22.511 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:22.511 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 24.567177948000001, 2024-09-15T00:13:22.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031894817999999998, 2024-09-15T00:13:22.512 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.512 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.512 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.512 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.512 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.512 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.512 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170133 2024-09-15T00:13:22.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.512 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:22.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853540+0000", 2024-09-15T00:13:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058619409999999999 2024-09-15T00:13:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853545+0000", 2024-09-15T00:13:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3430000000000003e-06 2024-09-15T00:13:22.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853554+0000", 2024-09-15T00:13:22.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4069999999999999e-06 2024-09-15T00:13:22.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853561+0000", 2024-09-15T00:13:22.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5529999999999998e-06 2024-09-15T00:13:22.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879399+0000", 2024-09-15T00:13:22.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025837941 2024-09-15T00:13:22.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879477+0000", 2024-09-15T00:13:22.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8367000000000004e-05 2024-09-15T00:13:22.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879572+0000", 2024-09-15T00:13:22.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5266000000000006e-05 2024-09-15T00:13:22.515 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.515 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.515 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.515 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.515 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.516 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:22.517 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:22.517 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:22.515+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:22.618 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:22.616+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:22.661 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:22.661 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:22.661 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:22.662 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:22.666 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:13:22.670 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:22.718 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:22.716+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:22.773 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:22.773 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:22.773 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:22.773 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:22.773 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:22.782 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:22.799 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:22.799 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:22.815 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:22.815 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:22.819 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:22.816+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:22.831 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:22.832 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:22.847 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:22.847 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:22.865 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:22.865 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:22.881 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:22.882 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:22.889 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:22.889 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:22.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:22.889 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:22.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.890 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:22.890 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:22.890 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.808060403, 2024-09-15T00:13:22.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:13:22.890 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.890 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.890 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:22.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.890 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:13:22.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.891 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:22.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:22.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:13:22.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:13:22.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:13:22.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:13:22.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:13:22.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:13:22.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:13:22.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:13:22.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:13:22.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:13:22.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:13:22.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:13:22.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:22.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:13:22.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:13:22.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:13:22.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:13:22.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:13:22.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:13:22.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:13:22.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:13:22.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:13:22.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:13:22.895 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.896 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.896 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.896 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:13:22.896 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:22.896 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.795319861, 2024-09-15T00:13:22.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:13:22.896 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.896 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.896 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.896 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:22.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:22.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:13:22.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:13:22.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:13:22.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:13:22.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:13:22.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:13:22.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:13:22.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:13:22.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:13:22.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:13:22.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:13:22.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:13:22.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:13:22.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:13:22.900 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.900 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.900 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.900 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:22.900 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:22.900 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.662964102, 2024-09-15T00:13:22.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:13:22.901 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.901 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.901 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:22.901 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.901 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.901 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.901 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:13:22.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.901 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:22.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:22.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:13:22.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:13:22.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:13:22.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:13:22.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:13:22.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:13:22.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:13:22.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:13:22.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:13:22.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:13:22.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:13:22.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:13:22.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:22.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:13:22.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:13:22.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:13:22.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:13:22.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:13:22.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:13:22.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:13:22.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:13:22.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:13:22.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:13:22.906 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.906 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.906 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.906 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:13:22.906 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:22.906 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.649502457, 2024-09-15T00:13:22.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:13:22.906 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.907 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.907 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.907 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:22.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:22.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:13:22.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:13:22.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:13:22.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:13:22.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:13:22.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:13:22.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:13:22.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:13:22.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:13:22.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:13:22.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:13:22.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:13:22.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:13:22.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:13:22.910 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.910 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.910 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.910 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:13:22.910 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:22.910 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 107.464500974, 2024-09-15T00:13:22.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:13:22.911 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.911 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.911 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:22.911 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.911 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.911 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.911 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:13:22.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.911 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:22.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:22.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:13:22.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:13:22.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:13:22.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:13:22.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:13:22.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:13:22.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:13:22.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:13:22.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:13:22.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:13:22.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:13:22.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:13:22.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:22.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:13:22.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:13:22.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:13:22.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:13:22.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:13:22.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:13:22.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:13:22.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:13:22.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:13:22.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:13:22.916 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.916 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.916 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.916 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:13:22.916 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:22.916 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 107.45162470699999, 2024-09-15T00:13:22.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:13:22.916 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.916 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.916 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.917 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:22.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:22.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:13:22.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:13:22.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:13:22.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:13:22.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:13:22.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:13:22.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:13:22.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:13:22.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:13:22.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:13:22.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:13:22.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:13:22.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:13:22.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:13:22.920 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.920 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.920 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.921 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:13:22.921 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:22.921 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 101.993109563, 2024-09-15T00:13:22.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:13:22.921 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.921 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.921 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:22.921 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.921 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.921 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.921 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:13:22.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.921 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:22.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:22.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:13:22.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:13:22.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:13:22.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:13:22.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:13:22.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:13:22.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:13:22.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:13:22.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:13:22.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:13:22.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:13:22.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:13:22.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:22.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:13:22.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:13:22.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:13:22.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:13:22.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:13:22.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:13:22.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:13:22.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:13:22.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:13:22.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:13:22.926 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.927 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.927 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.927 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:22.927 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:22.927 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 98.275599002999996, 2024-09-15T00:13:22.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:13:22.927 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.927 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.927 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.928 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.928 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:13:22.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.928 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:22.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:22.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:13:22.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:13:22.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:13:22.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:13:22.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:13:22.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:13:22.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:13:22.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:13:22.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:13:22.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:13:22.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:13:22.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:13:22.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:13:22.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:13:22.931 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.931 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.931 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.932 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:22.932 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:22.932 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 98.275580394000002, 2024-09-15T00:13:22.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:13:22.932 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.932 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.932 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.932 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.932 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.932 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.932 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:13:22.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.932 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:22.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:22.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:22.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:13:22.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:13:22.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:13:22.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:13:22.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:13:22.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:13:22.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:13:22.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:13:22.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:13:22.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:13:22.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:13:22.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:13:22.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:13:22.936 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.936 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.936 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.936 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:22.936 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:22.936 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 98.275555069999996, 2024-09-15T00:13:22.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:13:22.937 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.937 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.937 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.937 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.937 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.937 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.937 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:13:22.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.937 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:22.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:22.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:22.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:13:22.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:13:22.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:13:22.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:13:22.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:13:22.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:13:22.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:13:22.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:13:22.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:13:22.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:13:22.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:13:22.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:13:22.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:13:22.941 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.941 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.941 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.941 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:22.941 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:22.941 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 98.275536746, 2024-09-15T00:13:22.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:13:22.941 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.941 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.942 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.942 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.942 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.942 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.942 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:13:22.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.942 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:22.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:22.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:13:22.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:13:22.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:13:22.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:13:22.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:13:22.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:22.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:13:22.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:13:22.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:13:22.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:13:22.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:13:22.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:13:22.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:13:22.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:13:22.946 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.946 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.946 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.946 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:22.946 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:22.946 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 98.275354172999997, 2024-09-15T00:13:22.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:13:22.946 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.946 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.946 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.947 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.947 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.947 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.947 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:13:22.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.947 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:22.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:22.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:22.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:13:22.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:13:22.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:13:22.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:13:22.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:13:22.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:13:22.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:13:22.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:13:22.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:13:22.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:13:22.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:13:22.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:13:22.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:13:22.950 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.950 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.950 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.951 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:22.951 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:22.951 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 98.275325331999994, 2024-09-15T00:13:22.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:13:22.951 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.951 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.951 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.951 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.951 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.951 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.951 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:13:22.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.951 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:22.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:22.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:22.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:13:22.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:13:22.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:13:22.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:13:22.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:13:22.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:13:22.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:13:22.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:13:22.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:13:22.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:13:22.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:13:22.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:13:22.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:13:22.955 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.955 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.955 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.956 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:22.956 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:22.956 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 98.275303120999993, 2024-09-15T00:13:22.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:13:22.956 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.956 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.956 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.956 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.956 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.956 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.956 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:13:22.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.956 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:22.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:22.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:22.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:13:22.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:13:22.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:13:22.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:13:22.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:13:22.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:13:22.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:13:22.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:13:22.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:13:22.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:13:22.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:13:22.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:13:22.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:13:22.960 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.960 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.960 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.960 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:22.960 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:22.961 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 98.275281961999994, 2024-09-15T00:13:22.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:13:22.961 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.961 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.961 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.961 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.961 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.961 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.961 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:13:22.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.961 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:22.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:22.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:22.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:13:22.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:13:22.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:13:22.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:13:22.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:13:22.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:13:22.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:13:22.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:13:22.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:13:22.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:13:22.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:13:22.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:13:22.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:13:22.965 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.965 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.965 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.966 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:22.966 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:22.966 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 98.275252014000003, 2024-09-15T00:13:22.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:13:22.967 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.967 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.967 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.967 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.967 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.967 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.968 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:13:22.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.968 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:22.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:22.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:22.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:13:22.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:13:22.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:13:22.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:13:22.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:13:22.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:13:22.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:13:22.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:13:22.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:13:22.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:13:22.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:13:22.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:13:22.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:13:22.976 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.976 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.976 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.977 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:22.977 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:22.977 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 98.275230050000005, 2024-09-15T00:13:22.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:13:22.977 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.977 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.977 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.977 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.977 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.977 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.977 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:13:22.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.977 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:22.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:22.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:22.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:13:22.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:13:22.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:13:22.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:13:22.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:13:22.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:13:22.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:13:22.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:13:22.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:13:22.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:13:22.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:13:22.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:13:22.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:13:22.981 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.981 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.981 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.981 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:22.981 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:22.981 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 98.275212623000002, 2024-09-15T00:13:22.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:13:22.981 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.981 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.981 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.981 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.982 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.982 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.982 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:13:22.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.982 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:22.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:22.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:13:22.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:13:22.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:13:22.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:13:22.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:13:22.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:22.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:13:22.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:13:22.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:13:22.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:13:22.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:13:22.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:13:22.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:13:22.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:13:22.985 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.985 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.985 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.985 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:22.985 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:22.986 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 98.275190824000006, 2024-09-15T00:13:22.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:13:22.986 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.986 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.986 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:22.986 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.986 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.986 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.986 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:13:22.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.986 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:22.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:22.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:13:22.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:13:22.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:13:22.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:13:22.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:13:22.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:13:22.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:13:22.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:13:22.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:13:22.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:13:22.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:13:22.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:13:22.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:13:22.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:13:22.989 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.990 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.990 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.990 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:22.990 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:22.990 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 98.225962464999995, 2024-09-15T00:13:22.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:13:22.990 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:22.990 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:22.990 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:22.991 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:22.991 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:22.991 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:22.991 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:13:22.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.991 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:22.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:22.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:22.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:22.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:22.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:22.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:22.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:13:22.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:13:22.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:22.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:13:22.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:13:22.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:22.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:13:22.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:13:22.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:22.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:13:22.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:13:22.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:22.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:13:22.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:13:22.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:22.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:13:22.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:13:22.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:22.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:13:22.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:13:22.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:22.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:13:22.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:13:22.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:22.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:13:22.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:13:22.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:22.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:13:22.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:13:22.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:22.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:22.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:22.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:13:22.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:13:22.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.996 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:22.996 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:22.996 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:22.997 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:22.998 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:22.999 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:22.999 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:22.999 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:22.996+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:23.000 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:23.069 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:13:23.098 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:23.096+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:23.151 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:23.151 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:23.161 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:23.199 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:23.197+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:23.294 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:23.294 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:23.299 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:23.297+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:23.303 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:23.400 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:23.399+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:23.448 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:23.448 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:23.457 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:23.501 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:23.498+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:23.561 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:23.561 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:23.569 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:23.611 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:23.609+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:23.703 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:23.701+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:23.713 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:23.714 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:23.722 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:23.804 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:23.802+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:23.842 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:23.842 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:23.852 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:23.905 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:23.902+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:23.977 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:23.978 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:23.987 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:24.027 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:24.025+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:24.060 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is unset 2024-09-15T00:13:24.107 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:24.106+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:24.148 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:24.148 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:24.160 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:24.217 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:24.216+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:24.318 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:24.316+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:24.328 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:24.328 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:24.339 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:24.422 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:24.421+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:24.490 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:24.490 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:24.499 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:24.540 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:24.539+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:24.619 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:24.618+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:24.649 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:24.650 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:24.659 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:24.742 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:24.741+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:24.810 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:24.811 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:24.821 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:24.821 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:24.819+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:24.922 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:24.920+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:24.928 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:24.928 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:24.937 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:25.002 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:13:25.022 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:25.021+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:25.089 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:25.090 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:25.099 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:25.140 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:25.138+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:25.149 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:25.150 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:25.168 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:25.168 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:25.185 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:25.185 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:25.201 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:25.201 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:25.218 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:25.218 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:25.225 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:25.223+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:25.236 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:25.236 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:25.245 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:25.245 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:25.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:25.245 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:25.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.245 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:25.245 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:25.245 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 119.670462336, 2024-09-15T00:13:25.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:13:25.246 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.246 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.246 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:25.246 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.246 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.246 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.246 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:13:25.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.246 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:25.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:25.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:13:25.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:13:25.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:13:25.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:13:25.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:13:25.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:13:25.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:13:25.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:13:25.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:13:25.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:13:25.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:13:25.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:13:25.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:25.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:13:25.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:13:25.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:25.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:13:25.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:13:25.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:25.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:13:25.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:13:25.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:25.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:13:25.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:13:25.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:13:25.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:13:25.252 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.252 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.252 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.252 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:25.252 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:25.252 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 119.670427644, 2024-09-15T00:13:25.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:13:25.252 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.252 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.252 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:25.252 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.252 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.252 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.253 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:13:25.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.253 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:25.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:25.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:25.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:13:25.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:13:25.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:13:25.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:13:25.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:13:25.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:13:25.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:13:25.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:13:25.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:13:25.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:13:25.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:13:25.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:25.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:13:25.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:13:25.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:25.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:13:25.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:13:25.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:25.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:13:25.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:13:25.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:25.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:13:25.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:13:25.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:13:25.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:13:25.258 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.258 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.258 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.258 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.258 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:25.258 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.629753956, 2024-09-15T00:13:25.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:13:25.258 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.258 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.258 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.258 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.259 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.259 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.259 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:13:25.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.259 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:25.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:25.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:25.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:13:25.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:13:25.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:13:25.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:13:25.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:13:25.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:13:25.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:13:25.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:13:25.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:13:25.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:13:25.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:13:25.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:13:25.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:13:25.262 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.262 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.262 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.629737425, 2024-09-15T00:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:25.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:25.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:13:25.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:25.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:13:25.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:13:25.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:13:25.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:13:25.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:13:25.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:13:25.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:13:25.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:13:25.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:13:25.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:13:25.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:13:25.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:13:25.267 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.267 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.267 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.267 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.267 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:25.267 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.629720646, 2024-09-15T00:13:25.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:13:25.268 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.268 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.268 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.268 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.268 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.268 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.268 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:13:25.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.269 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:25.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:25.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:25.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:13:25.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:13:25.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:13:25.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:13:25.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:13:25.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:13:25.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:13:25.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:13:25.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:13:25.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:13:25.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:13:25.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.272 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.272 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:13:25.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:13:25.272 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.272 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.272 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.272 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.272 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:25.272 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.629702916, 2024-09-15T00:13:25.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:13:25.272 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.272 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.273 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.273 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.273 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.273 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.273 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:13:25.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.273 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:25.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:25.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:25.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:13:25.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:13:25.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:13:25.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:13:25.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:13:25.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:13:25.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:13:25.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:13:25.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:13:25.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.276 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:13:25.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:13:25.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:13:25.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:13:25.277 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.277 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.278 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.278 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.278 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:25.278 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.629685752, 2024-09-15T00:13:25.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:13:25.279 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.279 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.279 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.279 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.279 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.279 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.279 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:13:25.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.280 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:25.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:25.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:25.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:13:25.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:13:25.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:13:25.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:13:25.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:13:25.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:13:25.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:13:25.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:13:25.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:13:25.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:13:25.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:13:25.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:13:25.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:13:25.285 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.285 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.285 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.286 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.286 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:25.286 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.629664076, 2024-09-15T00:13:25.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:13:25.286 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.286 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.286 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.286 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.286 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.286 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.286 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:13:25.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.286 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:25.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:25.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:25.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:13:25.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:13:25.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:13:25.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:13:25.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:13:25.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:13:25.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:13:25.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:13:25.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:13:25.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:13:25.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:13:25.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:13:25.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:13:25.290 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.290 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.290 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.290 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.290 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:25.290 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.629617952, 2024-09-15T00:13:25.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:13:25.290 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.290 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.290 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:25.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:13:25.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:13:25.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:13:25.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:13:25.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:13:25.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:13:25.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:13:25.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:13:25.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:13:25.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:13:25.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:13:25.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:13:25.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:13:25.294 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.294 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.294 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.294 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.294 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:25.294 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.629588195, 2024-09-15T00:13:25.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:13:25.294 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.294 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:25.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:13:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:13:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:13:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:13:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:13:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:25.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:13:25.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:13:25.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:13:25.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:13:25.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:13:25.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:13:25.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:13:25.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:13:25.298 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.298 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.629561876, 2024-09-15T00:13:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:13:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:13:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:13:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:13:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:13:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:13:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:13:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:13:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:13:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:13:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:13:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:13:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:13:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:13:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.62952601400001, 2024-09-15T00:13:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:13:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:13:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:13:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:13:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:13:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:13:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:13:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:13:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:13:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:13:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:13:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:13:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:13:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:13:25.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:13:25.308 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.308 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.308 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.62950429599999, 2024-09-15T00:13:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:13:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:13:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:13:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:13:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:13:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:13:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:13:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:13:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:13:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:13:25.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:13:25.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:13:25.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:13:25.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:13:25.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:13:25.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:13:25.313 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.313 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.313 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.313 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.313 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:25.313 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.629307829, 2024-09-15T00:13:25.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:13:25.314 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.314 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.314 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.314 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.314 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.314 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.314 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:13:25.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.314 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:25.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:25.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:25.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:13:25.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:13:25.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:13:25.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:13:25.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:13:25.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:13:25.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:13:25.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:13:25.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:13:25.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:13:25.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:13:25.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:13:25.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:13:25.317 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.317 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.318 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.318 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.318 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:25.318 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.62928199, 2024-09-15T00:13:25.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:13:25.318 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.318 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.318 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.318 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.318 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.318 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.318 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:13:25.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.319 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:25.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:25.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:25.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:13:25.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:13:25.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:13:25.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:13:25.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:13:25.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:13:25.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:13:25.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:13:25.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:13:25.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:13:25.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:13:25.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:13:25.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:13:25.322 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.322 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.322 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.322 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.322 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:25.322 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.629220344, 2024-09-15T00:13:25.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:13:25.322 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.322 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.323 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.323 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.323 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.323 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.323 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:13:25.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.323 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:25.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:25.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:13:25.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:13:25.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:13:25.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:13:25.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:13:25.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:13:25.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:13:25.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:13:25.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:13:25.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:13:25.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:13:25.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:13:25.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:13:25.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:13:25.326 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.326 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.327 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.327 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.327 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:25.327 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.629181016, 2024-09-15T00:13:25.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:13:25.327 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.327 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.327 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.327 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.327 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.327 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.327 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:13:25.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.328 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:25.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:25.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:13:25.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:13:25.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:13:25.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:13:25.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:13:25.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:13:25.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:13:25.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:13:25.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:13:25.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:13:25.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:13:25.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:25.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:13:25.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:13:25.332 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.332 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.332 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.332 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.332 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:25.332 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.62745561600001, 2024-09-15T00:13:25.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:13:25.332 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.332 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.332 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.333 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.333 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.333 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.333 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:13:25.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.333 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:25.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:25.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:13:25.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:13:25.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:13:25.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:13:25.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:13:25.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:13:25.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:13:25.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:13:25.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:13:25.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:13:25.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:13:25.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:13:25.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:13:25.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:13:25.336 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.336 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.337 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.337 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.337 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:25.337 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.627433455, 2024-09-15T00:13:25.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:13:25.337 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.337 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.337 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.337 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.337 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.337 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.338 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:13:25.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.338 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:25.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:25.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:25.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:13:25.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:13:25.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:13:25.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:13:25.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:13:25.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:13:25.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:13:25.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:13:25.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:13:25.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:13:25.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:13:25.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:13:25.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:13:25.341 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.341 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.341 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.341 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:145034 3.6 3:618673df:::rbd_object_map.1139646a7d4e.000000000000002b:head [writefull 0~48] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T00:13:25.341 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:25.341 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.225179851, 2024-09-15T00:13:25.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0081079740000001, 2024-09-15T00:13:25.342 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.342 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.342 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:25.342 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.342 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.342 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.342 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 145034 2024-09-15T00:13:25.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.342 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:25.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:25.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017198+0000", 2024-09-15T00:13:25.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0100000000000004e-07 2024-09-15T00:13:25.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017202+0000", 2024-09-15T00:13:25.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:13:25.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017212+0000", 2024-09-15T00:13:25.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7189999999999996e-06 2024-09-15T00:13:25.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017219+0000", 2024-09-15T00:13:25.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7839999999999998e-06 2024-09-15T00:13:25.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017268+0000", 2024-09-15T00:13:25.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:13:25.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:13:25.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017277+0000", 2024-09-15T00:13:25.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1899999999999995e-06 2024-09-15T00:13:25.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019139+0000", 2024-09-15T00:13:25.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0018626129999999 2024-09-15T00:13:25.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019236+0000", 2024-09-15T00:13:25.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6513000000000002e-05 2024-09-15T00:13:25.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:25.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019318+0000", 2024-09-15T00:13:25.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1848000000000006e-05 2024-09-15T00:13:25.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:25.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.023151+0000", 2024-09-15T00:13:25.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038332169999999999 2024-09-15T00:13:25.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:25.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025217+0000", 2024-09-15T00:13:25.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020657610000000002 2024-09-15T00:13:25.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:25.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025267+0000", 2024-09-15T00:13:25.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0198000000000003e-05 2024-09-15T00:13:25.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025305+0000", 2024-09-15T00:13:25.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8065999999999998e-05 2024-09-15T00:13:25.349 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.349 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.349 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.349 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.349 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.349 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:25.350 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:25.350 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:25.351 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:25.352 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:25.352 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:25.349+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:25.353 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:25.451 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:25.449+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:25.496 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:25.496 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:25.496 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:25.496 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:25.505 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:25.551 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:25.550+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:25.648 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:25.649 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:25.649 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:25.649 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:25.649 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:25.653 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:25.651+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:25.658 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:25.753 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:25.752+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:25.761 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:25.761 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:25.761 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:25.761 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:25.761 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.761 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.762 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:25.762 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 101.14926706200001, 2024-09-15T00:13:25.762 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:13:25.762 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.762 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.762 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.762 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.762 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.762 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.762 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:13:25.762 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.762 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.763 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.763 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.763 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:25.763 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.763 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.763 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.763 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.763 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:25.763 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.763 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.763 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.763 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.763 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:25.763 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:13:25.763 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.764 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.764 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.764 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:13:25.764 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:13:25.764 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.764 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.764 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.764 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:13:25.764 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:13:25.764 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.764 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.764 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.764 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:13:25.764 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:13:25.764 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.765 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.765 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.765 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:13:25.765 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:13:25.765 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.765 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.765 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.765 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:13:25.765 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:13:25.765 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.765 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.765 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.765 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:13:25.765 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:13:25.765 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.766 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.766 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.766 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.766 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.766 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.766 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:25.766 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 101.14924583299999, 2024-09-15T00:13:25.766 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:13:25.766 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.766 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.766 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.766 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.766 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.766 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.766 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:13:25.767 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:13:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.769 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.769 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.769 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:13:25.769 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:13:25.769 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.769 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.769 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.769 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:13:25.769 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:13:25.769 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.770 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.770 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.770 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:13:25.770 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:13:25.770 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.770 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.770 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.770 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.770 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.770 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.770 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:25.770 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 101.149220292, 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.771 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.772 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.772 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:25.772 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.772 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.772 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.772 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:25.772 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:25.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.772 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.772 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.772 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:13:25.772 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:13:25.773 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.773 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.773 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.773 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:13:25.773 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:13:25.773 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.773 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.773 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.773 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:13:25.773 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:13:25.773 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.773 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.773 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.773 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:13:25.773 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.774 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:25.775 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 101.149197817, 2024-09-15T00:13:25.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:13:25.775 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.775 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.775 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.775 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.775 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.775 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.775 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:13:25.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.775 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.775 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.775 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.775 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:25.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:25.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:13:25.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:13:25.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:13:25.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:13:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:13:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:13:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:13:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:13:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:13:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:13:25.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:13:25.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:13:25.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:13:25.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:13:25.779 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.779 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.779 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.779 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.779 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.779 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.779 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:25.779 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 101.149177307, 2024-09-15T00:13:25.779 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:13:25.779 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.779 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.779 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.779 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.779 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.779 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.780 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:13:25.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.780 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.780 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.780 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:25.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.780 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.780 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:25.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.780 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:13:25.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:13:25.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.782 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.782 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:13:25.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:13:25.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.782 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.782 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:13:25.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:13:25.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.782 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.782 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:13:25.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:13:25.783 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.783 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.783 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.783 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.783 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:25.783 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 101.149156992, 2024-09-15T00:13:25.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:13:25.783 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.783 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.783 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.784 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.784 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.784 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.784 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:13:25.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.784 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:25.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:25.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.786 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.786 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:13:25.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:13:25.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.786 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.786 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:13:25.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:13:25.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.786 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.786 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:13:25.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:13:25.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 101.149132216, 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.787 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.788 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.788 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.788 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:13:25.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.788 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:25.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:25.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:13:25.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:13:25.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:13:25.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:13:25.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:13:25.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:13:25.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.969790829, 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.792 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:25.792 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.792 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.792 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.792 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:13:25.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.792 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:25.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:13:25.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:13:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.95283737699999, 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:13:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:13:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:13:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:13:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:13:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:13:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:13:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:13:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:13:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.658406470000003, 2024-09-15T00:13:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:13:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:13:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:13:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:13:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:13:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:13:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:13:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:13:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:13:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:13:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.645950655999997, 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:13:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:13:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:13:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:13:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:13:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:13:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:146935 3.16 3:6c79e092:::rbd_data.11394501364.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.005171138000001, 2024-09-15T00:13:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.058987488999999997, 2024-09-15T00:13:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 146935 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753644+0000", 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6199999999999997e-07 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753648+0000", 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753659+0000", 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0703000000000001e-05 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753664+0000", 2024-09-15T00:13:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.04e-06 2024-09-15T00:13:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753709+0000", 2024-09-15T00:13:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4728000000000003e-05 2024-09-15T00:13:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753756+0000", 2024-09-15T00:13:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7808e-05 2024-09-15T00:13:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.757051+0000", 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294249 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812553+0000", 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055501807 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812564+0000", 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1467999999999999e-05 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812598+0000", 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3553000000000002e-05 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812630+0000", 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2895000000000003e-05 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:146935 3.16 e98/95 ondisk, result = 0)", 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 40.994368442000003, 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048111830000000001, 2024-09-15T00:13:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764447+0000", 2024-09-15T00:13:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0500000000000002e-07 2024-09-15T00:13:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764450+0000", 2024-09-15T00:13:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4400000000000001e-06 2024-09-15T00:13:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764462+0000", 2024-09-15T00:13:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1603000000000001e-05 2024-09-15T00:13:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764469+0000", 2024-09-15T00:13:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5089999999999996e-06 2024-09-15T00:13:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812541+0000", 2024-09-15T00:13:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048071585 2024-09-15T00:13:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812548+0000", 2024-09-15T00:13:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9979999999999997e-06 2024-09-15T00:13:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812557+0000", 2024-09-15T00:13:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7899999999999994e-06 2024-09-15T00:13:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:152505 3.17 3:ea3f268c:::rbd_data.113958cb0e55.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.092262480000002, 2024-09-15T00:13:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.060817959999999997, 2024-09-15T00:13:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 152505 2024-09-15T00:13:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666553+0000", 2024-09-15T00:13:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-15T00:13:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666562+0000", 2024-09-15T00:13:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0909999999999999e-06 2024-09-15T00:13:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666572+0000", 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0214000000000001e-05 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666577+0000", 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3800000000000002e-06 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666617+0000", 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9304000000000002e-05 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666672+0000", 2024-09-15T00:13:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5206e-05 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.670821+0000", 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041493169999999996 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727290+0000", 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.056468580999999997 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727302+0000", 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2605e-05 2024-09-15T00:13:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727337+0000", 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4783999999999998e-05 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727370+0000", 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2366000000000001e-05 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:152505 3.17 e103/68 ondisk, result = 0)", 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.080184043999999, 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048665938999999998, 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678631+0000", 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5099999999999998e-07 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678634+0000", 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.478e-06 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678647+0000", 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2085e-05 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678653+0000", 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2099999999999998e-06 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727277+0000", 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048624405000000002 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727285+0000", 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2060000000000007e-06 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727296+0000", 2024-09-15T00:13:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0604e-05 2024-09-15T00:13:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153959 3.c 3:326b8454:::rbd_data.113958cb0e55.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.668956098000002, 2024-09-15T00:13:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.068102041000000002, 2024-09-15T00:13:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153959 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089859+0000", 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1800000000000004e-07 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089864+0000", 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2079999999999999e-06 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089880+0000", 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5695000000000001e-05 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089889+0000", 2024-09-15T00:13:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.935e-06 2024-09-15T00:13:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089942+0000", 2024-09-15T00:13:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3219e-05 2024-09-15T00:13:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.090066+0000", 2024-09-15T00:13:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012385099999999999 2024-09-15T00:13:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.095262+0000", 2024-09-15T00:13:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051963950000000004 2024-09-15T00:13:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157865+0000", 2024-09-15T00:13:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.062602865999999993 2024-09-15T00:13:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157891+0000", 2024-09-15T00:13:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5778e-05 2024-09-15T00:13:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157924+0000", 2024-09-15T00:13:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3133000000000003e-05 2024-09-15T00:13:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157960+0000", 2024-09-15T00:13:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6043e-05 2024-09-15T00:13:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:153959 3.c e103/23 ondisk, result = 0)", 2024-09-15T00:13:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.656300440000003, 2024-09-15T00:13:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055370211000000003, 2024-09-15T00:13:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102515+0000", 2024-09-15T00:13:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.206e-06 2024-09-15T00:13:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102520+0000", 2024-09-15T00:13:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1209999999999998e-06 2024-09-15T00:13:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102537+0000", 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102546+0000", 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2329999999999995e-06 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157847+0000", 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055300798999999998 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157857+0000", 2024-09-15T00:13:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.046e-05 2024-09-15T00:13:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157884+0000", 2024-09-15T00:13:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.671e-05 2024-09-15T00:13:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163346 3.1d 3:baa43c12:::rbd_data.11395d771336.0000000000000065:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:13:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.987685867, 2024-09-15T00:13:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047558544000000001, 2024-09-15T00:13:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163346 2024-09-15T00:13:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771130+0000", 2024-09-15T00:13:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5939999999999999e-06 2024-09-15T00:13:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771141+0000", 2024-09-15T00:13:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1001e-05 2024-09-15T00:13:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771154+0000", 2024-09-15T00:13:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2921e-05 2024-09-15T00:13:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771160+0000", 2024-09-15T00:13:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3849999999999999e-06 2024-09-15T00:13:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771207+0000", 2024-09-15T00:13:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6888999999999998e-05 2024-09-15T00:13:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771306+0000", 2024-09-15T00:13:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9259000000000006e-05 2024-09-15T00:13:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.774692+0000", 2024-09-15T00:13:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033854850000000001 2024-09-15T00:13:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818600+0000", 2024-09-15T00:13:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043908006999999999 2024-09-15T00:13:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818616+0000", 2024-09-15T00:13:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6688e-05 2024-09-15T00:13:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818651+0000", 2024-09-15T00:13:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4885000000000002e-05 2024-09-15T00:13:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818687+0000", 2024-09-15T00:13:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5429999999999998e-05 2024-09-15T00:13:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163578 3.1c 3:3efd71ab:::rbd_data.11395d771336.0000000000000060:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-15T00:13:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.588263118, 2024-09-15T00:13:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.085154081000000006, 2024-09-15T00:13:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163578 2024-09-15T00:13:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170552+0000", 2024-09-15T00:13:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8599999999999997e-07 2024-09-15T00:13:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170557+0000", 2024-09-15T00:13:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-15T00:13:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170567+0000", 2024-09-15T00:13:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0628999999999999e-05 2024-09-15T00:13:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170573+0000", 2024-09-15T00:13:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1649999999999997e-06 2024-09-15T00:13:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170619+0000", 2024-09-15T00:13:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6405e-05 2024-09-15T00:13:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170666+0000", 2024-09-15T00:13:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7259000000000003e-05 2024-09-15T00:13:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.175060+0000", 2024-09-15T00:13:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043940170000000001 2024-09-15T00:13:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255559+0000", 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080499065999999994 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255616+0000", 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6422000000000003e-05 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255655+0000", 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9632999999999998e-05 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255705+0000", 2024-09-15T00:13:25.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.973e-05 2024-09-15T00:13:25.851 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.851 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.851 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.851 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:163578 3.1c e107/23 ondisk, result = 0)", 2024-09-15T00:13:25.851 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:25.851 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.573041026999999, 2024-09-15T00:13:25.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069939984999999996, 2024-09-15T00:13:25.851 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:25.851 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:25.851 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:25.851 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:25.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:25.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:25.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:25.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:25.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:25.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:25.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185774+0000", 2024-09-15T00:13:25.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.093e-06 2024-09-15T00:13:25.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:25.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185779+0000", 2024-09-15T00:13:25.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9470000000000002e-06 2024-09-15T00:13:25.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:25.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185794+0000", 2024-09-15T00:13:25.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4450000000000001e-05 2024-09-15T00:13:25.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:25.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185801+0000", 2024-09-15T00:13:25.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9189999999999999e-06 2024-09-15T00:13:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255604+0000", 2024-09-15T00:13:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069802201999999994 2024-09-15T00:13:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255611+0000", 2024-09-15T00:13:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7740000000000004e-06 2024-09-15T00:13:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:25.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255713+0000", 2024-09-15T00:13:25.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0001016 2024-09-15T00:13:25.855 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.855 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.855 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.855 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:25.855 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:25.855 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:25.857 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:25.857 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:25.855+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:25.957 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:25.956+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:26.000 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:26.000 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:26.000 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:26.000 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:26.009 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:26.058 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:26.057+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:26.069 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 0, 1] out_osds: [7, 2] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-15T00:13:26.069 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:13:26.069 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T00:13:26.069 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T00:13:26.079 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:13:26.115 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:26.115 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:26.115 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:26.115 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:26.116 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:26.123 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:26.167 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:26.165+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:26.216 INFO:teuthology.orchestra.run.smithi171.stderr:no valid command found; 10 closest matches: 2024-09-15T00:13:26.216 INFO:teuthology.orchestra.run.smithi171.stderr:0 2024-09-15T00:13:26.216 INFO:teuthology.orchestra.run.smithi171.stderr:1 2024-09-15T00:13:26.216 INFO:teuthology.orchestra.run.smithi171.stderr:2 2024-09-15T00:13:26.216 INFO:teuthology.orchestra.run.smithi171.stderr:abort 2024-09-15T00:13:26.216 INFO:teuthology.orchestra.run.smithi171.stderr:assert 2024-09-15T00:13:26.216 INFO:teuthology.orchestra.run.smithi171.stderr:bench [] [] [] [] 2024-09-15T00:13:26.216 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs debug_inject_read_zeros 2024-09-15T00:13:26.216 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs files list 2024-09-15T00:13:26.216 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs stats 2024-09-15T00:13:26.216 INFO:teuthology.orchestra.run.smithi171.stderr:bluestore allocator dump block 2024-09-15T00:13:26.216 INFO:teuthology.orchestra.run.smithi171.stderr:Error EINVAL: invalid command 2024-09-15T00:13:26.218 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T00:13:26.219 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T00:13:26.219 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-15T00:13:26.227 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:26.227 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:26.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:26.228 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:26.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.228 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:26.228 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:26.228 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 101.557710431, 2024-09-15T00:13:26.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:13:26.228 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.228 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.228 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:13:26.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:13:26.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:13:26.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:13:26.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:13:26.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:13:26.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:13:26.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:13:26.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:13:26.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:13:26.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:13:26.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:13:26.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:26.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:13:26.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:13:26.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:26.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:13:26.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:13:26.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:26.233 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:13:26.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:13:26.233 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.233 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:26.233 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:13:26.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:13:26.233 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.233 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.233 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:13:26.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:13:26.233 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.233 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.234 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.234 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125618 3.10 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:26.234 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:26.234 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.386571348999993, 2024-09-15T00:13:26.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030860942999999998, 2024-09-15T00:13:26.234 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.234 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.234 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:26.234 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.234 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.234 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.234 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125618 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838499+0000", 2024-09-15T00:13:26.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6189999999999998e-06 2024-09-15T00:13:26.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839599+0000", 2024-09-15T00:13:26.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001099764 2024-09-15T00:13:26.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839620+0000", 2024-09-15T00:13:26.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0764e-05 2024-09-15T00:13:26.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839631+0000", 2024-09-15T00:13:26.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1335000000000001e-05 2024-09-15T00:13:26.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839725+0000", 2024-09-15T00:13:26.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4217999999999998e-05 2024-09-15T00:13:26.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839910+0000", 2024-09-15T00:13:26.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000184298 2024-09-15T00:13:26.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:26.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843171+0000", 2024-09-15T00:13:26.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032613540000000002 2024-09-15T00:13:26.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.238 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:26.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869187+0000", 2024-09-15T00:13:26.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026015467 2024-09-15T00:13:26.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.238 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:26.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869240+0000", 2024-09-15T00:13:26.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3733000000000002e-05 2024-09-15T00:13:26.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:26.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869309+0000", 2024-09-15T00:13:26.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8381999999999997e-05 2024-09-15T00:13:26.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869358+0000", 2024-09-15T00:13:26.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-15T00:13:26.239 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.239 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.239 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.240 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125619 3.a 3:507b29bf:::rbd_data.11396481e1ec.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:26.240 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:26.240 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.38540922, 2024-09-15T00:13:26.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045768870000000003, 2024-09-15T00:13:26.240 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.240 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.240 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:26.240 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.240 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.240 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.240 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125619 2024-09-15T00:13:26.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.240 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.241 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:26.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.241 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.241 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:26.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.241 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.241 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839660+0000", 2024-09-15T00:13:26.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.22e-06 2024-09-15T00:13:26.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.242 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.242 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840511+0000", 2024-09-15T00:13:26.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000851284 2024-09-15T00:13:26.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.242 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.242 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840522+0000", 2024-09-15T00:13:26.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0569e-05 2024-09-15T00:13:26.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.242 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.242 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840527+0000", 2024-09-15T00:13:26.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.412e-06 2024-09-15T00:13:26.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840573+0000", 2024-09-15T00:13:26.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5247000000000002e-05 2024-09-15T00:13:26.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840682+0000", 2024-09-15T00:13:26.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109368 2024-09-15T00:13:26.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:26.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843976+0000", 2024-09-15T00:13:26.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294445 2024-09-15T00:13:26.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:26.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870833+0000", 2024-09-15T00:13:26.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026856944000000001 2024-09-15T00:13:26.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:26.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885344+0000", 2024-09-15T00:13:26.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014510744000000001 2024-09-15T00:13:26.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:26.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885382+0000", 2024-09-15T00:13:26.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7821000000000003e-05 2024-09-15T00:13:26.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885428+0000", 2024-09-15T00:13:26.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5816000000000003e-05 2024-09-15T00:13:26.245 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.245 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.245 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.245 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125621 3.a 3:53594db9:::rbd_data.11396481e1ec.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:26.245 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:26.245 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.371885840999994, 2024-09-15T00:13:26.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032481797, 2024-09-15T00:13:26.246 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.246 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.246 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:26.246 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.246 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.246 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.246 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125621 2024-09-15T00:13:26.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.246 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:26.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:26.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853185+0000", 2024-09-15T00:13:26.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.43e-06 2024-09-15T00:13:26.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854125+0000", 2024-09-15T00:13:26.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000940539 2024-09-15T00:13:26.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854141+0000", 2024-09-15T00:13:26.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5568000000000001e-05 2024-09-15T00:13:26.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854148+0000", 2024-09-15T00:13:26.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6529999999999994e-06 2024-09-15T00:13:26.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870872+0000", 2024-09-15T00:13:26.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016723135 2024-09-15T00:13:26.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870975+0000", 2024-09-15T00:13:26.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010314099999999999 2024-09-15T00:13:26.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:26.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.873082+0000", 2024-09-15T00:13:26.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021069859999999999 2024-09-15T00:13:26.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:26.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885439+0000", 2024-09-15T00:13:26.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012357343999999999 2024-09-15T00:13:26.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:26.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885569+0000", 2024-09-15T00:13:26.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012995199999999999 2024-09-15T00:13:26.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:26.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885624+0000", 2024-09-15T00:13:26.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5281999999999997e-05 2024-09-15T00:13:26.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885664+0000", 2024-09-15T00:13:26.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9767000000000001e-05 2024-09-15T00:13:26.251 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.251 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.251 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.251 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:147075 3.0 3:06ab39f2:::rbd_data.11394501364.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:26.251 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:26.251 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.275987460000003, 2024-09-15T00:13:26.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046402906000000001, 2024-09-15T00:13:26.251 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.251 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.251 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:26.252 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.252 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.252 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.252 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 147075 2024-09-15T00:13:26.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.252 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:26.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:26.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:26.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4399999999999999e-07 2024-09-15T00:13:26.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949086+0000", 2024-09-15T00:13:26.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.138e-06 2024-09-15T00:13:26.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949096+0000", 2024-09-15T00:13:26.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0438999999999999e-05 2024-09-15T00:13:26.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949101+0000", 2024-09-15T00:13:26.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8999999999999997e-06 2024-09-15T00:13:26.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949145+0000", 2024-09-15T00:13:26.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4141000000000003e-05 2024-09-15T00:13:26.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949195+0000", 2024-09-15T00:13:26.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9657999999999998e-05 2024-09-15T00:13:26.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:26.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.950923+0000", 2024-09-15T00:13:26.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001728753 2024-09-15T00:13:26.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:26.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995390+0000", 2024-09-15T00:13:26.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044466335000000003 2024-09-15T00:13:26.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:26.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995401+0000", 2024-09-15T00:13:26.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1671000000000001e-05 2024-09-15T00:13:26.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:26.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995432+0000", 2024-09-15T00:13:26.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0704999999999998e-05 2024-09-15T00:13:26.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995484+0000", 2024-09-15T00:13:26.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1422000000000003e-05 2024-09-15T00:13:26.256 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.256 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.257 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.257 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:147075 3.0 e98/94 ondisk, result = 0)", 2024-09-15T00:13:26.257 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:26.257 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.268187738999998, 2024-09-15T00:13:26.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038514459000000001, 2024-09-15T00:13:26.257 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.257 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.257 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.257 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:26.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:26.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956881+0000", 2024-09-15T00:13:26.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0599999999999999e-07 2024-09-15T00:13:26.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956885+0000", 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3500000000000001e-06 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956897+0000", 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2303999999999999e-05 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956902+0000", 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5169999999999999e-06 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995377+0000", 2024-09-15T00:13:26.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038474544999999999 2024-09-15T00:13:26.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995385+0000", 2024-09-15T00:13:26.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6240000000000002e-06 2024-09-15T00:13:26.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995395+0000", 2024-09-15T00:13:26.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0213e-05 2024-09-15T00:13:26.260 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.260 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.260 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.261 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153172 3.14 3:2a7843e3:::rbd_data.113958cb0e55.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 983040~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:26.261 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:26.261 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.447298674999999, 2024-09-15T00:13:26.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.033232668, 2024-09-15T00:13:26.261 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.261 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.261 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:26.261 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.261 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.261 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.261 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153172 2024-09-15T00:13:26.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.261 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:26.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:26.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777770+0000", 2024-09-15T00:13:26.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.017e-06 2024-09-15T00:13:26.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788009+0000", 2024-09-15T00:13:26.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010238146 2024-09-15T00:13:26.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788024+0000", 2024-09-15T00:13:26.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5582000000000001e-05 2024-09-15T00:13:26.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788048+0000", 2024-09-15T00:13:26.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4173e-05 2024-09-15T00:13:26.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788097+0000", 2024-09-15T00:13:26.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8498999999999998e-05 2024-09-15T00:13:26.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788201+0000", 2024-09-15T00:13:26.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010398799999999999 2024-09-15T00:13:26.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:26.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788393+0000", 2024-09-15T00:13:26.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000192337 2024-09-15T00:13:26.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:26.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810637+0000", 2024-09-15T00:13:26.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.022244202000000001 2024-09-15T00:13:26.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:26.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810903+0000", 2024-09-15T00:13:26.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00026574600000000003 2024-09-15T00:13:26.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:26.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810953+0000", 2024-09-15T00:13:26.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9509000000000003e-05 2024-09-15T00:13:26.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.811002+0000", 2024-09-15T00:13:26.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9468999999999997e-05 2024-09-15T00:13:26.266 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.266 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.266 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.266 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170014 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:26.266 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:26.266 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.378003569000001, 2024-09-15T00:13:26.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030291116999999999, 2024-09-15T00:13:26.267 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.267 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.267 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.267 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.267 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.267 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.267 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170014 2024-09-15T00:13:26.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.267 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:26.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:26.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:26.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:26.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847073+0000", 2024-09-15T00:13:26.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9589999999999995e-06 2024-09-15T00:13:26.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847080+0000", 2024-09-15T00:13:26.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2930000000000003e-06 2024-09-15T00:13:26.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847086+0000", 2024-09-15T00:13:26.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.631e-06 2024-09-15T00:13:26.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877213+0000", 2024-09-15T00:13:26.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030127017999999998 2024-09-15T00:13:26.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877281+0000", 2024-09-15T00:13:26.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7916999999999994e-05 2024-09-15T00:13:26.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877356+0000", 2024-09-15T00:13:26.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4796999999999996e-05 2024-09-15T00:13:26.271 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.271 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.271 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.271 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170017 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1310720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:26.271 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:26.272 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.377974918, 2024-09-15T00:13:26.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030437623, 2024-09-15T00:13:26.272 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.272 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.272 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.272 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.272 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.272 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.272 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170017 2024-09-15T00:13:26.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.273 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:26.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:26.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847094+0000", 2024-09-15T00:13:26.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:13:26.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847096+0000", 2024-09-15T00:13:26.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.136e-06 2024-09-15T00:13:26.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847104+0000", 2024-09-15T00:13:26.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7190000000000001e-06 2024-09-15T00:13:26.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847109+0000", 2024-09-15T00:13:26.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3279999999999996e-06 2024-09-15T00:13:26.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877384+0000", 2024-09-15T00:13:26.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030275348000000001 2024-09-15T00:13:26.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877451+0000", 2024-09-15T00:13:26.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6509999999999998e-05 2024-09-15T00:13:26.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.276 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877531+0000", 2024-09-15T00:13:26.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9993000000000005e-05 2024-09-15T00:13:26.276 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.276 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.276 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.276 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170018 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1351680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:26.276 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:26.276 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.377952573000002, 2024-09-15T00:13:26.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030591884999999999, 2024-09-15T00:13:26.276 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170018 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847118+0000", 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.03e-06 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847124+0000", 2024-09-15T00:13:26.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:26.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847129+0000", 2024-09-15T00:13:26.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8600000000000001e-06 2024-09-15T00:13:26.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877560+0000", 2024-09-15T00:13:26.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030431145999999999 2024-09-15T00:13:26.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877629+0000", 2024-09-15T00:13:26.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8793000000000004e-05 2024-09-15T00:13:26.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877707+0000", 2024-09-15T00:13:26.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8139999999999994e-05 2024-09-15T00:13:26.280 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.280 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.280 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.280 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170019 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1392640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:26.280 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:26.280 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.377931825000001, 2024-09-15T00:13:26.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030769023, 2024-09-15T00:13:26.281 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.281 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.281 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.281 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.281 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.281 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.281 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170019 2024-09-15T00:13:26.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.281 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:26.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:26.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847137+0000", 2024-09-15T00:13:26.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:13:26.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847139+0000", 2024-09-15T00:13:26.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.142e-06 2024-09-15T00:13:26.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847146+0000", 2024-09-15T00:13:26.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8120000000000002e-06 2024-09-15T00:13:26.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847157+0000", 2024-09-15T00:13:26.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1093e-05 2024-09-15T00:13:26.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877735+0000", 2024-09-15T00:13:26.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578195999999998 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877810+0000", 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5563999999999997e-05 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877905+0000", 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4809000000000007e-05 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.284 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170020 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:26.285 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:26.285 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.377904585, 2024-09-15T00:13:26.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030917150000000001, 2024-09-15T00:13:26.285 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.285 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.285 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.285 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.285 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.285 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.285 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170020 2024-09-15T00:13:26.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:13:26.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847166+0000", 2024-09-15T00:13:26.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2790000000000001e-06 2024-09-15T00:13:26.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847173+0000", 2024-09-15T00:13:26.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3969999999999999e-06 2024-09-15T00:13:26.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847177+0000", 2024-09-15T00:13:26.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8620000000000002e-06 2024-09-15T00:13:26.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877934+0000", 2024-09-15T00:13:26.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030756143999999999 2024-09-15T00:13:26.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878002+0000", 2024-09-15T00:13:26.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7951e-05 2024-09-15T00:13:26.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878081+0000", 2024-09-15T00:13:26.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9087999999999999e-05 2024-09-15T00:13:26.289 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.289 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.289 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.289 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170094 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:26.289 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:26.289 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.377647335999999, 2024-09-15T00:13:26.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030836142, 2024-09-15T00:13:26.289 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.289 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.289 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.290 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170094 2024-09-15T00:13:26.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.290 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.290 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.290 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:26.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.290 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.290 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:26.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:26.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3499999999999996e-07 2024-09-15T00:13:26.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847424+0000", 2024-09-15T00:13:26.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2850000000000001e-06 2024-09-15T00:13:26.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847430+0000", 2024-09-15T00:13:26.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3260000000000001e-06 2024-09-15T00:13:26.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847435+0000", 2024-09-15T00:13:26.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.356e-06 2024-09-15T00:13:26.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878110+0000", 2024-09-15T00:13:26.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030675021 2024-09-15T00:13:26.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878177+0000", 2024-09-15T00:13:26.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6506000000000003e-05 2024-09-15T00:13:26.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878257+0000", 2024-09-15T00:13:26.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0012999999999994e-05 2024-09-15T00:13:26.293 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.293 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.293 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.293 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170096 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 458752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:26.293 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:26.294 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.377605379999999, 2024-09-15T00:13:26.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030977030999999999, 2024-09-15T00:13:26.294 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.294 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.294 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.294 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.294 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.294 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.294 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170096 2024-09-15T00:13:26.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.294 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:26.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:26.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:26.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:26.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847465+0000", 2024-09-15T00:13:26.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0559999999999999e-06 2024-09-15T00:13:26.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847472+0000", 2024-09-15T00:13:26.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3130000000000004e-06 2024-09-15T00:13:26.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847477+0000", 2024-09-15T00:13:26.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9930000000000003e-06 2024-09-15T00:13:26.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878288+0000", 2024-09-15T00:13:26.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030811609 2024-09-15T00:13:26.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878358+0000", 2024-09-15T00:13:26.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0215000000000005e-05 2024-09-15T00:13:26.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878440+0000", 2024-09-15T00:13:26.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1366999999999995e-05 2024-09-15T00:13:26.297 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.298 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.298 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.298 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170097 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:26.298 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:26.298 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.377585294999999, 2024-09-15T00:13:26.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031134757999999998, 2024-09-15T00:13:26.298 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.298 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.298 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.298 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.298 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.298 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.299 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170097 2024-09-15T00:13:26.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.299 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:26.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:26.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:26.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:13:26.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847485+0000", 2024-09-15T00:13:26.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9190000000000002e-06 2024-09-15T00:13:26.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847492+0000", 2024-09-15T00:13:26.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7109999999999998e-06 2024-09-15T00:13:26.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847497+0000", 2024-09-15T00:13:26.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0259999999999998e-06 2024-09-15T00:13:26.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878469+0000", 2024-09-15T00:13:26.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030972115000000001 2024-09-15T00:13:26.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878538+0000", 2024-09-15T00:13:26.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8669000000000003e-05 2024-09-15T00:13:26.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878618+0000", 2024-09-15T00:13:26.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9874000000000001e-05 2024-09-15T00:13:26.302 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.302 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.302 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.302 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170099 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 532480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:26.302 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:26.302 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.377533733, 2024-09-15T00:13:26.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031262183999999998, 2024-09-15T00:13:26.303 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.303 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.303 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.303 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.303 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.303 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.303 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170099 2024-09-15T00:13:26.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.303 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:26.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:26.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847535+0000", 2024-09-15T00:13:26.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:13:26.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847537+0000", 2024-09-15T00:13:26.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:26.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847544+0000", 2024-09-15T00:13:26.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9630000000000001e-06 2024-09-15T00:13:26.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847549+0000", 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8910000000000003e-06 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878646+0000", 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031097400000000001 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878717+0000", 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0993000000000003e-05 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878797+0000", 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9425000000000006e-05 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170101 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.377492354000001, 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031674568, 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.307 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.308 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170101 2024-09-15T00:13:26.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.308 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:26.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:26.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:26.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:13:26.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847578+0000", 2024-09-15T00:13:26.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0389999999999999e-06 2024-09-15T00:13:26.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847585+0000", 2024-09-15T00:13:26.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.336e-06 2024-09-15T00:13:26.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847596+0000", 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015999999999999e-05 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878825+0000", 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031229848000000001 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878899+0000", 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879250+0000", 2024-09-15T00:13:26.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035110000000000002 2024-09-15T00:13:26.311 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.311 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.311 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.311 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170102 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 614400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:26.311 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:26.311 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.377465056999998, 2024-09-15T00:13:26.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031577371999999999, 2024-09-15T00:13:26.311 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.311 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.312 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.312 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.312 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.312 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.312 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170102 2024-09-15T00:13:26.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.312 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:26.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:26.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:26.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:13:26.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847610+0000", 2024-09-15T00:13:26.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6329999999999999e-06 2024-09-15T00:13:26.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847618+0000", 2024-09-15T00:13:26.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4599999999999997e-06 2024-09-15T00:13:26.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847623+0000", 2024-09-15T00:13:26.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2909999999999998e-06 2024-09-15T00:13:26.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879016+0000", 2024-09-15T00:13:26.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031393227000000003 2024-09-15T00:13:26.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879091+0000", 2024-09-15T00:13:26.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5384999999999997e-05 2024-09-15T00:13:26.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879180+0000", 2024-09-15T00:13:26.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8941000000000001e-05 2024-09-15T00:13:26.315 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.315 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.315 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.316 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170132 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:26.316 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:26.316 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.377417677, 2024-09-15T00:13:26.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031722579000000001, 2024-09-15T00:13:26.316 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.316 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.316 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.316 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.316 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.316 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.317 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170132 2024-09-15T00:13:26.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.317 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:26.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:26.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847651+0000", 2024-09-15T00:13:26.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1900000000000003e-07 2024-09-15T00:13:26.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847653+0000", 2024-09-15T00:13:26.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0329999999999998e-06 2024-09-15T00:13:26.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847659+0000", 2024-09-15T00:13:26.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4169999999999997e-06 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847670+0000", 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1031e-05 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879205+0000", 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031534982000000003 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879286+0000", 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0675999999999998e-05 2024-09-15T00:13:26.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879373+0000", 2024-09-15T00:13:26.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6921000000000003e-05 2024-09-15T00:13:26.320 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.320 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.320 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.320 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170133 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:26.320 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:26.320 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.377390523999999, 2024-09-15T00:13:26.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031894817999999998, 2024-09-15T00:13:26.321 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.321 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.321 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.321 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.324 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.324 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.324 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170133 2024-09-15T00:13:26.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.325 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:26.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:26.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853540+0000", 2024-09-15T00:13:26.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058619409999999999 2024-09-15T00:13:26.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853545+0000", 2024-09-15T00:13:26.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3430000000000003e-06 2024-09-15T00:13:26.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853554+0000", 2024-09-15T00:13:26.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4069999999999999e-06 2024-09-15T00:13:26.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853561+0000", 2024-09-15T00:13:26.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5529999999999998e-06 2024-09-15T00:13:26.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879399+0000", 2024-09-15T00:13:26.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025837941 2024-09-15T00:13:26.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879477+0000", 2024-09-15T00:13:26.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8367000000000004e-05 2024-09-15T00:13:26.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879572+0000", 2024-09-15T00:13:26.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5266000000000006e-05 2024-09-15T00:13:26.332 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.332 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.332 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.332 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.332 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.332 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:26.334 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:26.332+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:26.335 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:26.373 INFO:teuthology.orchestra.run.smithi171.stderr:no valid command found; 10 closest matches: 2024-09-15T00:13:26.374 INFO:teuthology.orchestra.run.smithi171.stderr:0 2024-09-15T00:13:26.374 INFO:teuthology.orchestra.run.smithi171.stderr:1 2024-09-15T00:13:26.374 INFO:teuthology.orchestra.run.smithi171.stderr:2 2024-09-15T00:13:26.374 INFO:teuthology.orchestra.run.smithi171.stderr:abort 2024-09-15T00:13:26.374 INFO:teuthology.orchestra.run.smithi171.stderr:assert 2024-09-15T00:13:26.374 INFO:teuthology.orchestra.run.smithi171.stderr:bench [] [] [] [] 2024-09-15T00:13:26.374 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs debug_inject_read_zeros 2024-09-15T00:13:26.374 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs files list 2024-09-15T00:13:26.374 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs stats 2024-09-15T00:13:26.374 INFO:teuthology.orchestra.run.smithi171.stderr:bluestore allocator dump block 2024-09-15T00:13:26.374 INFO:teuthology.orchestra.run.smithi171.stderr:Error EINVAL: invalid command 2024-09-15T00:13:26.376 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T00:13:26.377 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T00:13:26.377 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-15T00:13:26.435 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:26.433+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:26.480 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:26.480 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:26.480 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:26.480 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:26.489 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:26.566 INFO:teuthology.orchestra.run.smithi171.stderr:no valid command found; 10 closest matches: 2024-09-15T00:13:26.566 INFO:teuthology.orchestra.run.smithi171.stderr:0 2024-09-15T00:13:26.567 INFO:teuthology.orchestra.run.smithi171.stderr:1 2024-09-15T00:13:26.567 INFO:teuthology.orchestra.run.smithi171.stderr:2 2024-09-15T00:13:26.567 INFO:teuthology.orchestra.run.smithi171.stderr:abort 2024-09-15T00:13:26.567 INFO:teuthology.orchestra.run.smithi171.stderr:assert 2024-09-15T00:13:26.567 INFO:teuthology.orchestra.run.smithi171.stderr:bench [] [] [] [] 2024-09-15T00:13:26.567 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs debug_inject_read_zeros 2024-09-15T00:13:26.567 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs files list 2024-09-15T00:13:26.567 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs stats 2024-09-15T00:13:26.567 INFO:teuthology.orchestra.run.smithi171.stderr:bluestore allocator dump block 2024-09-15T00:13:26.567 INFO:teuthology.orchestra.run.smithi171.stderr:Error EINVAL: invalid command 2024-09-15T00:13:26.569 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T00:13:26.570 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T00:13:26.570 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-15T00:13:26.573 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:26.571+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:26.631 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:26.631 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:26.631 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:26.637 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:26.636+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:26.640 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:26.717 INFO:teuthology.orchestra.run.smithi171.stderr:no valid command found; 10 closest matches: 2024-09-15T00:13:26.717 INFO:teuthology.orchestra.run.smithi171.stderr:0 2024-09-15T00:13:26.717 INFO:teuthology.orchestra.run.smithi171.stderr:1 2024-09-15T00:13:26.717 INFO:teuthology.orchestra.run.smithi171.stderr:2 2024-09-15T00:13:26.717 INFO:teuthology.orchestra.run.smithi171.stderr:abort 2024-09-15T00:13:26.718 INFO:teuthology.orchestra.run.smithi171.stderr:assert 2024-09-15T00:13:26.718 INFO:teuthology.orchestra.run.smithi171.stderr:bench [] [] [] [] 2024-09-15T00:13:26.718 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs debug_inject_read_zeros 2024-09-15T00:13:26.718 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs files list 2024-09-15T00:13:26.718 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs stats 2024-09-15T00:13:26.718 INFO:teuthology.orchestra.run.smithi171.stderr:bluestore allocator dump block 2024-09-15T00:13:26.718 INFO:teuthology.orchestra.run.smithi171.stderr:Error EINVAL: invalid command 2024-09-15T00:13:26.720 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T00:13:26.720 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T00:13:26.720 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T00:13:26.738 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:26.736+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:26.785 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:26.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.704411347, 2024-09-15T00:13:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:13:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.786 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:13:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:13:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:13:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:13:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:13:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:13:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:13:26.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:13:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:13:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:13:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:13:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:13:26.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:13:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:13:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:13:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:13:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:13:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:13:26.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:13:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:13:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:13:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:13:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:13:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:13:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.691670805, 2024-09-15T00:13:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:13:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:13:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:13:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:13:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:13:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:13:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:13:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:13:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:13:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:13:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:13:26.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:13:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:13:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:13:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:13:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.795 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.55931504599999, 2024-09-15T00:13:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:13:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:13:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:13:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:13:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:13:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:13:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:13:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:13:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:13:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:13:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:13:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:13:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:13:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:13:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:13:26.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:13:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:13:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:13:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:13:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:13:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:26.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:13:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:13:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:13:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:13:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:13:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.545853401, 2024-09-15T00:13:26.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:13:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:13:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:13:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:13:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:13:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:13:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:13:26.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:13:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:13:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:13:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:13:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:13:26.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:13:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:13:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:13:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:13:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.36085191799999, 2024-09-15T00:13:26.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:13:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:13:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:13:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:13:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:13:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:13:26.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:13:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:13:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:13:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:13:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:13:26.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:13:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:13:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:13:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:13:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:13:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:26.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:13:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:13:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:13:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:13:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:13:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:13:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.347975651, 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:13:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:13:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:13:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:13:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:13:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:13:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:13:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:13:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.814 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.889460507, 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:13:26.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:13:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:13:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:13:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:13:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:13:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:13:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:13:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:13:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:13:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:13:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:13:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:13:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:13:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:13:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:13:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:13:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:13:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:13:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:13:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:13:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:13:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.820 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.171949947, 2024-09-15T00:13:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:13:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:13:26.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:13:26.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:13:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:13:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:13:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:13:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:13:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:13:26.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:13:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:13:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:13:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:13:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:13:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:13:26.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.17193133799999, 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.825 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:13:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:13:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:13:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:13:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:13:26.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:13:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:13:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:13:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:13:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:13:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:13:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:13:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:13:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:13:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.171906014, 2024-09-15T00:13:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:13:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:13:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:13:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:13:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:13:26.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:13:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:13:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:13:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:13:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:13:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:13:26.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:13:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:13:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:13:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:13:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.833 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.17188769000001, 2024-09-15T00:13:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:13:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:13:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:13:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:13:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:13:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:13:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:13:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:13:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:13:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:13:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:13:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:13:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:13:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:13:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:13:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.837 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.171705117, 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.838 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:13:26.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:13:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:13:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:13:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:13:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:13:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:13:26.841 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.171676276, 2024-09-15T00:13:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:13:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.842 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:26.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:13:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:13:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:13:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:13:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:13:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:13:26.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:13:26.845 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.171654065, 2024-09-15T00:13:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:13:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.846 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:13:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:13:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:13:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:13:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:13:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:13:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:13:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:13:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:13:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:13:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:13:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:13:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.171632906, 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:13:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:13:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:13:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:13:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:13:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:13:26.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:13:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.17160295799999, 2024-09-15T00:13:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:13:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:13:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:13:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:13:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:13:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:13:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:13:26.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:13:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:13:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:13:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:13:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:13:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:13:26.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:13:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:13:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:26.858 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.171580994, 2024-09-15T00:13:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:13:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:13:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:26.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:13:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:13:26.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.862 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.17156356700001, 2024-09-15T00:13:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:13:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:13:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:13:26.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:13:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:13:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:13:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:13:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:13:26.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:13:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:13:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:13:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:13:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:13:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.171541768, 2024-09-15T00:13:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:13:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:13:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:13:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:13:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:13:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:13:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:13:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:13:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:13:26.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:13:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:13:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:13:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:13:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:13:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:13:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:13:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 102.122313409, 2024-09-15T00:13:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:13:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:26.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:13:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:13:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:13:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:13:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:13:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:13:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:13:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:13:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:13:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:13:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:13:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:13:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:13:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:13:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:13:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:13:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:13:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:13:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:13:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:13:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:13:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:13:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:13:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:26.877 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:26.877 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:26.878 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:26.879 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:26.876+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:26.911 INFO:teuthology.orchestra.run.smithi171.stderr:no valid command found; 10 closest matches: 2024-09-15T00:13:26.911 INFO:teuthology.orchestra.run.smithi171.stderr:0 2024-09-15T00:13:26.911 INFO:teuthology.orchestra.run.smithi171.stderr:1 2024-09-15T00:13:26.911 INFO:teuthology.orchestra.run.smithi171.stderr:2 2024-09-15T00:13:26.911 INFO:teuthology.orchestra.run.smithi171.stderr:abort 2024-09-15T00:13:26.911 INFO:teuthology.orchestra.run.smithi171.stderr:assert 2024-09-15T00:13:26.911 INFO:teuthology.orchestra.run.smithi171.stderr:bench [] [] [] [] 2024-09-15T00:13:26.911 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs debug_inject_read_zeros 2024-09-15T00:13:26.911 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs files list 2024-09-15T00:13:26.911 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs stats 2024-09-15T00:13:26.911 INFO:teuthology.orchestra.run.smithi171.stderr:bluestore allocator dump block 2024-09-15T00:13:26.912 INFO:teuthology.orchestra.run.smithi171.stderr:Error EINVAL: invalid command 2024-09-15T00:13:26.914 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T00:13:26.914 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T00:13:26.914 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T00:13:26.979 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:26.977+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:26.987 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:26.987 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:26.987 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:26.987 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:26.996 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:27.025 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is unset 2024-09-15T00:13:27.085 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:27.084+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:27.121 INFO:teuthology.orchestra.run.smithi171.stderr:no valid command found; 10 closest matches: 2024-09-15T00:13:27.121 INFO:teuthology.orchestra.run.smithi171.stderr:0 2024-09-15T00:13:27.121 INFO:teuthology.orchestra.run.smithi171.stderr:1 2024-09-15T00:13:27.121 INFO:teuthology.orchestra.run.smithi171.stderr:2 2024-09-15T00:13:27.121 INFO:teuthology.orchestra.run.smithi171.stderr:abort 2024-09-15T00:13:27.121 INFO:teuthology.orchestra.run.smithi171.stderr:assert 2024-09-15T00:13:27.121 INFO:teuthology.orchestra.run.smithi171.stderr:bench [] [] [] [] 2024-09-15T00:13:27.121 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs debug_inject_read_zeros 2024-09-15T00:13:27.121 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs files list 2024-09-15T00:13:27.122 INFO:teuthology.orchestra.run.smithi171.stderr:bluefs stats 2024-09-15T00:13:27.122 INFO:teuthology.orchestra.run.smithi171.stderr:bluestore allocator dump block 2024-09-15T00:13:27.122 INFO:teuthology.orchestra.run.smithi171.stderr:Error EINVAL: invalid command 2024-09-15T00:13:27.124 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T00:13:27.124 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T00:13:27.151 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:27.152 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:27.160 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:27.181 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:27.179+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:27.282 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:27.280+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:27.306 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:27.306 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:27.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:27.306 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:27.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.306 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9, version: 37'47, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9@37'47, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:27.306 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:27.306 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.44971791, 2024-09-15T00:13:27.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054905559999999997, 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853793+0000", 2024-09-15T00:13:27.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9599999999999999e-07 2024-09-15T00:13:27.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854798+0000", 2024-09-15T00:13:27.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010050770000000001 2024-09-15T00:13:27.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854814+0000", 2024-09-15T00:13:27.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6478999999999999e-05 2024-09-15T00:13:27.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854825+0000", 2024-09-15T00:13:27.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0492999999999999e-05 2024-09-15T00:13:27.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854889+0000", 2024-09-15T00:13:27.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4134999999999995e-05 2024-09-15T00:13:27.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854898+0000", 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6260000000000001e-06 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.859283+0000", 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043852500000000003 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15, version: 59'205, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15@59'205, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.323497454999998, 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059047939999999997, 2024-09-15T00:13:27.310 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.311 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.311 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.311 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:27.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:27.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:27.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.44e-07 2024-09-15T00:13:27.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980977+0000", 2024-09-15T00:13:27.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00096402300000000005 2024-09-15T00:13:27.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980995+0000", 2024-09-15T00:13:27.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7214000000000002e-05 2024-09-15T00:13:27.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981007+0000", 2024-09-15T00:13:27.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2028000000000001e-05 2024-09-15T00:13:27.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981069+0000", 2024-09-15T00:13:27.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1977000000000006e-05 2024-09-15T00:13:27.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981077+0000", 2024-09-15T00:13:27.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2120000000000003e-06 2024-09-15T00:13:27.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.985918+0000", 2024-09-15T00:13:27.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048406960000000002 2024-09-15T00:13:27.314 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.314 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.314 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.314 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head, version: 62'221, data_included: [0~262144,393216~786432], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head@62'221, size: 1572864, copy_subset: [(0, 1572864)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:27.315 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:27.315 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.301229190000001, 2024-09-15T00:13:27.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054170770000000002, 2024-09-15T00:13:27.315 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.315 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.315 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.315 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:27.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:27.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002282+0000", 2024-09-15T00:13:27.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4499999999999997e-07 2024-09-15T00:13:27.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003196+0000", 2024-09-15T00:13:27.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091476399999999999 2024-09-15T00:13:27.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003213+0000", 2024-09-15T00:13:27.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6554000000000001e-05 2024-09-15T00:13:27.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003230+0000", 2024-09-15T00:13:27.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6617999999999999e-05 2024-09-15T00:13:27.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003292+0000", 2024-09-15T00:13:27.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2193999999999994e-05 2024-09-15T00:13:27.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003300+0000", 2024-09-15T00:13:27.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1429999999999998e-06 2024-09-15T00:13:27.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.007698+0000", 2024-09-15T00:13:27.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043981589999999996 2024-09-15T00:13:27.318 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.318 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.318 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.319 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:27.319 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:27.319 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.25049911, 2024-09-15T00:13:27.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067633270000000004, 2024-09-15T00:13:27.319 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.319 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.319 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.319 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:27.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053012+0000", 2024-09-15T00:13:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1659999999999999e-06 2024-09-15T00:13:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054086+0000", 2024-09-15T00:13:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010732789999999999 2024-09-15T00:13:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054100+0000", 2024-09-15T00:13:27.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4123e-05 2024-09-15T00:13:27.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054114+0000", 2024-09-15T00:13:27.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4442e-05 2024-09-15T00:13:27.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054192+0000", 2024-09-15T00:13:27.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8288000000000007e-05 2024-09-15T00:13:27.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054200+0000", 2024-09-15T00:13:27.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-15T00:13:27.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.059774+0000", 2024-09-15T00:13:27.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055741230000000003 2024-09-15T00:13:27.322 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.322 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.322 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.323 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:27.323 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:27.323 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.242181781999999, 2024-09-15T00:13:27.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049777950000000001, 2024-09-15T00:13:27.323 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.323 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.323 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.323 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:27.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:27.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061330+0000", 2024-09-15T00:13:27.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.11e-06 2024-09-15T00:13:27.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062350+0000", 2024-09-15T00:13:27.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010207969999999999 2024-09-15T00:13:27.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062365+0000", 2024-09-15T00:13:27.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4157e-05 2024-09-15T00:13:27.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062380+0000", 2024-09-15T00:13:27.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5129e-05 2024-09-15T00:13:27.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062467+0000", 2024-09-15T00:13:27.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7247e-05 2024-09-15T00:13:27.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062475+0000", 2024-09-15T00:13:27.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4300000000000006e-06 2024-09-15T00:13:27.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.066306+0000", 2024-09-15T00:13:27.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038309250000000002 2024-09-15T00:13:27.326 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.326 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.327 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.327 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:27.327 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:27.327 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.236298823999999, 2024-09-15T00:13:27.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052454889999999999, 2024-09-15T00:13:27.327 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.327 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.327 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.327 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:27.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:27.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067212+0000", 2024-09-15T00:13:27.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-07 2024-09-15T00:13:27.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068119+0000", 2024-09-15T00:13:27.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090671199999999995 2024-09-15T00:13:27.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068135+0000", 2024-09-15T00:13:27.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5739e-05 2024-09-15T00:13:27.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068150+0000", 2024-09-15T00:13:27.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5693e-05 2024-09-15T00:13:27.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068236+0000", 2024-09-15T00:13:27.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6100000000000006e-05 2024-09-15T00:13:27.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068245+0000", 2024-09-15T00:13:27.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1960000000000008e-06 2024-09-15T00:13:27.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.072457+0000", 2024-09-15T00:13:27.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042123719999999998 2024-09-15T00:13:27.331 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.331 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.331 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.331 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:27.331 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:27.331 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.230320474999999, 2024-09-15T00:13:27.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049532020000000003, 2024-09-15T00:13:27.332 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.332 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.332 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.332 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:27.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:27.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:27.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:13:27.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074164+0000", 2024-09-15T00:13:27.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097354400000000002 2024-09-15T00:13:27.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074178+0000", 2024-09-15T00:13:27.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4134999999999999e-05 2024-09-15T00:13:27.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074192+0000", 2024-09-15T00:13:27.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.451e-05 2024-09-15T00:13:27.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074272+0000", 2024-09-15T00:13:27.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9989999999999998e-05 2024-09-15T00:13:27.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074280+0000", 2024-09-15T00:13:27.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9990000000000008e-06 2024-09-15T00:13:27.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078143+0000", 2024-09-15T00:13:27.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00386258 2024-09-15T00:13:27.335 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.335 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.335 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.335 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [4194304~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:27.335 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:27.335 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.224609662999999, 2024-09-15T00:13:27.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049408730000000001, 2024-09-15T00:13:27.335 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.336 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.336 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.336 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:27.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:27.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:27.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2399999999999999e-07 2024-09-15T00:13:27.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079875+0000", 2024-09-15T00:13:27.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097349100000000005 2024-09-15T00:13:27.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079889+0000", 2024-09-15T00:13:27.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4148000000000001e-05 2024-09-15T00:13:27.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079904+0000", 2024-09-15T00:13:27.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5159e-05 2024-09-15T00:13:27.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079964+0000", 2024-09-15T00:13:27.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0204999999999997e-05 2024-09-15T00:13:27.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079971+0000", 2024-09-15T00:13:27.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2490000000000004e-06 2024-09-15T00:13:27.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.083841+0000", 2024-09-15T00:13:27.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038701970000000001 2024-09-15T00:13:27.339 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.339 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.339 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.339 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [5242880~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 6291456, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:27.339 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:27.339 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.218904683000002, 2024-09-15T00:13:27.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052525740000000003, 2024-09-15T00:13:27.339 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.340 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.340 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.340 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:27.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:27.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:27.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:13:27.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085579+0000", 2024-09-15T00:13:27.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097307600000000002 2024-09-15T00:13:27.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085592+0000", 2024-09-15T00:13:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3277e-05 2024-09-15T00:13:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085608+0000", 2024-09-15T00:13:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5773e-05 2024-09-15T00:13:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085687+0000", 2024-09-15T00:13:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8956999999999996e-05 2024-09-15T00:13:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085694+0000", 2024-09-15T00:13:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.956e-06 2024-09-15T00:13:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.089858+0000", 2024-09-15T00:13:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041641159999999998 2024-09-15T00:13:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:172552 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-15T00:13:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:27.344 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.114447375000001, 2024-09-15T00:13:27.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.997031471, 2024-09-15T00:13:27.344 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.344 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.344 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.344 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:27.344 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:27.344 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:27.344 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 172552 2024-09-15T00:13:27.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.344 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:27.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:27.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189064+0000", 2024-09-15T00:13:27.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.198e-06 2024-09-15T00:13:27.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189069+0000", 2024-09-15T00:13:27.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9069999999999998e-06 2024-09-15T00:13:27.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189080+0000", 2024-09-15T00:13:27.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:13:27.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189086+0000", 2024-09-15T00:13:27.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5429999999999999e-06 2024-09-15T00:13:27.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189128+0000", 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2255000000000001e-05 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189135+0000", 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6200000000000001e-06 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185787+0000", 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99665157199999999 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185926+0000", 2024-09-15T00:13:27.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013970399999999999 2024-09-15T00:13:27.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.186094+0000", 2024-09-15T00:13:27.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016813099999999999 2024-09-15T00:13:27.348 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.348 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.348 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.348 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9, version: 37'60, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9@37'60, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:27.348 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:27.348 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.055587016, 2024-09-15T00:13:27.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049454060000000003, 2024-09-15T00:13:27.349 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.349 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.349 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.349 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:27.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:27.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247924+0000", 2024-09-15T00:13:27.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:13:27.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248394+0000", 2024-09-15T00:13:27.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047063399999999999 2024-09-15T00:13:27.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248411+0000", 2024-09-15T00:13:27.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6384000000000001e-05 2024-09-15T00:13:27.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248441+0000", 2024-09-15T00:13:27.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0559999999999999e-05 2024-09-15T00:13:27.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248499+0000", 2024-09-15T00:13:27.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7274000000000001e-05 2024-09-15T00:13:27.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248507+0000", 2024-09-15T00:13:27.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2390000000000002e-06 2024-09-15T00:13:27.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.252869+0000", 2024-09-15T00:13:27.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043617200000000004 2024-09-15T00:13:27.352 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.352 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.352 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.352 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head, version: 59'393, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head@59'393, size: 5701632, copy_subset: [(0, 5701632)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:27.353 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:27.353 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.926138749, 2024-09-15T00:13:27.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004807469, 2024-09-15T00:13:27.353 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.353 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.353 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.353 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:27.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:27.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:27.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4799999999999998e-07 2024-09-15T00:13:27.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377830+0000", 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000457954 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377847+0000", 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6741000000000001e-05 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377858+0000", 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0951999999999999e-05 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377929+0000", 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0697000000000004e-05 2024-09-15T00:13:27.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377941+0000", 2024-09-15T00:13:27.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2529e-05 2024-09-15T00:13:27.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.382179+0000", 2024-09-15T00:13:27.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042379480000000001 2024-09-15T00:13:27.356 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.356 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.357 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head, version: 88'473, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head@88'473, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:27.357 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:27.357 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.844019316000001, 2024-09-15T00:13:27.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005823272, 2024-09-15T00:13:27.357 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.357 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.357 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.357 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:27.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:27.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459492+0000", 2024-09-15T00:13:27.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9800000000000003e-07 2024-09-15T00:13:27.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459940+0000", 2024-09-15T00:13:27.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044854699999999999 2024-09-15T00:13:27.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459957+0000", 2024-09-15T00:13:27.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6586e-05 2024-09-15T00:13:27.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459968+0000", 2024-09-15T00:13:27.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.145e-05 2024-09-15T00:13:27.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460043+0000", 2024-09-15T00:13:27.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4451999999999998e-05 2024-09-15T00:13:27.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460057+0000", 2024-09-15T00:13:27.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.417e-05 2024-09-15T00:13:27.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.465314+0000", 2024-09-15T00:13:27.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052574689999999999 2024-09-15T00:13:27.361 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.361 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.362 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:27.362 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:27.362 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.608843135000001, 2024-09-15T00:13:27.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051811399999999999, 2024-09-15T00:13:27.362 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.362 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.362 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.362 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:27.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:27.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694668+0000", 2024-09-15T00:13:27.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4600000000000005e-07 2024-09-15T00:13:27.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695136+0000", 2024-09-15T00:13:27.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00046864399999999999 2024-09-15T00:13:27.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695153+0000", 2024-09-15T00:13:27.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6218e-05 2024-09-15T00:13:27.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695189+0000", 2024-09-15T00:13:27.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6393000000000002e-05 2024-09-15T00:13:27.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695263+0000", 2024-09-15T00:13:27.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4175e-05 2024-09-15T00:13:27.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695271+0000", 2024-09-15T00:13:27.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2800000000000003e-06 2024-09-15T00:13:27.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.699848+0000", 2024-09-15T00:13:27.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045768839999999998 2024-09-15T00:13:27.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.366 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.366 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:27.366 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:27.366 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.600744927000001, 2024-09-15T00:13:27.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048716510000000003, 2024-09-15T00:13:27.366 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.366 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.367 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.367 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:27.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:27.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702783+0000", 2024-09-15T00:13:27.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7268000000000001e-05 2024-09-15T00:13:27.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703221+0000", 2024-09-15T00:13:27.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00043866899999999998 2024-09-15T00:13:27.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703237+0000", 2024-09-15T00:13:27.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5553000000000002e-05 2024-09-15T00:13:27.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703247+0000", 2024-09-15T00:13:27.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0392e-05 2024-09-15T00:13:27.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703358+0000", 2024-09-15T00:13:27.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110676 2024-09-15T00:13:27.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703369+0000", 2024-09-15T00:13:27.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0716e-05 2024-09-15T00:13:27.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.707637+0000", 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042683770000000003 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18, version: 62'274, data_included: [0~786432], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18@62'274, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.554663845, 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005453014, 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.370 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.371 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:27.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:27.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748847+0000", 2024-09-15T00:13:27.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8299999999999997e-07 2024-09-15T00:13:27.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749010+0000", 2024-09-15T00:13:27.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000163144 2024-09-15T00:13:27.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749027+0000", 2024-09-15T00:13:27.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6518999999999999e-05 2024-09-15T00:13:27.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749037+0000", 2024-09-15T00:13:27.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0438000000000001e-05 2024-09-15T00:13:27.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749112+0000", 2024-09-15T00:13:27.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4946999999999999e-05 2024-09-15T00:13:27.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749123+0000", 2024-09-15T00:13:27.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1014e-05 2024-09-15T00:13:27.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.754299+0000", 2024-09-15T00:13:27.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051763690000000001 2024-09-15T00:13:27.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.374 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.374 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20, version: 80'334, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20@80'334, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 20=[]:{20: [20]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:27.374 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:27.374 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.500026801000001, 2024-09-15T00:13:27.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048933570000000001, 2024-09-15T00:13:27.375 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.375 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.375 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.375 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:27.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:27.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803484+0000", 2024-09-15T00:13:27.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:13:27.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803938+0000", 2024-09-15T00:13:27.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045364599999999999 2024-09-15T00:13:27.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803954+0000", 2024-09-15T00:13:27.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6456999999999998e-05 2024-09-15T00:13:27.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803965+0000", 2024-09-15T00:13:27.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.076e-05 2024-09-15T00:13:27.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804023+0000", 2024-09-15T00:13:27.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7787000000000001e-05 2024-09-15T00:13:27.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804031+0000", 2024-09-15T00:13:27.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2160000000000006e-06 2024-09-15T00:13:27.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.808377+0000", 2024-09-15T00:13:27.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043458969999999996 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [0~131072,196608~786432,1048576~131072], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.463240205000002, 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047737029999999998, 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:27.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:27.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840271+0000", 2024-09-15T00:13:27.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:13:27.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840721+0000", 2024-09-15T00:13:27.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045029800000000002 2024-09-15T00:13:27.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840738+0000", 2024-09-15T00:13:27.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6671999999999999e-05 2024-09-15T00:13:27.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840749+0000", 2024-09-15T00:13:27.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1251e-05 2024-09-15T00:13:27.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840817+0000", 2024-09-15T00:13:27.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8046000000000005e-05 2024-09-15T00:13:27.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840825+0000", 2024-09-15T00:13:27.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2519999999999999e-06 2024-09-15T00:13:27.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.845044+0000", 2024-09-15T00:13:27.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042185900000000004 2024-09-15T00:13:27.382 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.383 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.383 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.383 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [1179648~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2228224, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:27.383 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:27.383 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.455216157999999, 2024-09-15T00:13:27.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053056819999999999, 2024-09-15T00:13:27.383 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.383 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.383 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.383 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:27.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:27.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848295+0000", 2024-09-15T00:13:27.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4000000000000001e-07 2024-09-15T00:13:27.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848728+0000", 2024-09-15T00:13:27.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000433709 2024-09-15T00:13:27.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848745+0000", 2024-09-15T00:13:27.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6167999999999998e-05 2024-09-15T00:13:27.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848788+0000", 2024-09-15T00:13:27.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3269000000000002e-05 2024-09-15T00:13:27.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848857+0000", 2024-09-15T00:13:27.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9453000000000001e-05 2024-09-15T00:13:27.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848866+0000", 2024-09-15T00:13:27.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2779999999999993e-06 2024-09-15T00:13:27.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.853600+0000", 2024-09-15T00:13:27.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047341650000000002 2024-09-15T00:13:27.387 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.387 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.387 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.387 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head, version: 93'455, data_included: [0~98304,114688~16384,147456~589824,753664~65536,835584~16384,868352~262144], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head@93'455, size: 2342912, copy_subset: [(0, 2342912)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1130496, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:27.387 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:27.387 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.396082132, 2024-09-15T00:13:27.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063250900000000002, 2024-09-15T00:13:27.387 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:27.388 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:27.388 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:27.388 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:27.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:27.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:27.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:27.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:27.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:27.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:27.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907429+0000", 2024-09-15T00:13:27.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.85e-07 2024-09-15T00:13:27.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:27.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908008+0000", 2024-09-15T00:13:27.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00057949599999999998 2024-09-15T00:13:27.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:27.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908026+0000", 2024-09-15T00:13:27.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7700999999999999e-05 2024-09-15T00:13:27.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908038+0000", 2024-09-15T00:13:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2131e-05 2024-09-15T00:13:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908101+0000", 2024-09-15T00:13:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2596999999999997e-05 2024-09-15T00:13:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908110+0000", 2024-09-15T00:13:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8540000000000003e-06 2024-09-15T00:13:27.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:27.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:27.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:27.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.913753+0000", 2024-09-15T00:13:27.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056435260000000003 2024-09-15T00:13:27.391 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.391 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.391 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.391 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:27.391 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:27.391 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:27.391 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:13:27.393 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:27.391+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:27.394 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:27.494 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:27.492+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:27.541 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:27.541 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:27.541 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:27.541 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:27.550 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:27.556 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:27.557 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:27.574 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:27.574 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:27.591 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:27.591 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:27.595 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:27.594+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:27.607 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:27.608 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:27.624 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:27.624 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:27.641 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:27.641 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:27.659 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:27.659 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:27.663 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:27.663 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:27.672 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:27.676 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:27.677 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:27.728 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:27.727+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:27.780 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:27.780 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:27.789 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:27.829 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:27.828+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:27.898 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:27.897+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:27.933 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:27.933 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:27.942 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:27.999 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:27.998+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:28.098 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:28.098 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:28.099 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:28.098+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:28.107 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:28.200 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:28.199+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:28.254 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:28.254 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:28.263 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:28.306 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:28.305+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:28.367 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:28.367 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:28.377 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:28.402 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:28.401+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:28.504 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:28.502+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:28.521 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:28.521 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:28.530 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:28.605 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:28.604+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:28.673 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:28.674 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:28.683 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:28.727 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:28.725+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:28.793 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:28.793 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:28.802 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:28.806 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:28.804+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:28.907 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:28.906+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:28.944 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:28.945 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:28.953 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:29.009 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:29.007+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:29.045 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:13:29.056 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:29.056 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:29.065 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:29.148 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:29.147+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:29.211 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:29.209+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:29.212 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:29.212 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:29.221 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:29.312 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:29.310+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:29.365 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:29.365 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:29.374 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:29.413 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:29.411+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:29.478 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:29.478 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:29.487 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:29.531 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:29.529+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:29.589 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:29.589 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:29.600 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:29.615 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:29.613+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:29.689 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:13:29.716 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:29.714+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:29.744 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:29.745 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:29.753 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:29.810 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:29.810 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:29.816 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:29.815+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:29.827 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:29.827 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:29.843 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:29.843 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:29.859 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:29.859 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:29.876 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:29.877 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:29.892 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:29.893 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:29.898 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:29.898 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:29.907 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:29.908 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:29.909 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:29.917 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:29.916+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:29.925 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:29.925 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:30.010 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:30.011 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:30.011 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:30.011 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:30.018 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:30.016+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:30.018 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is set 2024-09-15T00:13:30.021 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:30.119 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:30.118+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:30.174 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:30.174 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:30.174 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:30.174 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:30.174 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:30.183 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:30.221 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:30.219+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:30.288 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:30.288 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:30.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:30.288 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:30.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.288 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:30.288 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:30.288 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 124.20655253300001, 2024-09-15T00:13:30.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:13:30.289 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.289 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.289 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:30.289 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.289 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.289 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.289 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:13:30.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.289 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:30.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.290 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.290 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:30.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.290 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.290 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:13:30.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:13:30.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.290 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.290 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:13:30.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:13:30.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:13:30.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:13:30.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:13:30.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:13:30.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:13:30.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:13:30.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:13:30.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:13:30.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:30.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:13:30.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:13:30.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:30.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:13:30.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:13:30.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:30.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:13:30.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:13:30.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:30.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:13:30.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:13:30.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:13:30.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:13:30.294 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.294 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.294 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.294 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:13:30.294 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:30.294 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 124.193811991, 2024-09-15T00:13:30.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:13:30.294 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.294 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.294 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:30.294 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:30.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:30.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:13:30.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:13:30.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:13:30.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:13:30.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:13:30.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:13:30.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:13:30.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:13:30.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:13:30.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:13:30.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:13:30.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:13:30.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:13:30.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:13:30.298 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.298 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.298 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.298 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:30.298 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:30.298 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 124.061456232, 2024-09-15T00:13:30.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:13:30.298 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.298 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.299 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:30.299 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.299 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.299 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.299 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:13:30.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.299 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:30.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:30.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:13:30.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:13:30.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:13:30.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:13:30.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:13:30.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:13:30.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:13:30.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:13:30.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:13:30.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:13:30.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:13:30.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:13:30.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:30.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:13:30.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:13:30.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:30.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:13:30.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:13:30.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:30.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:13:30.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:13:30.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:30.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:13:30.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:13:30.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:13:30.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:13:30.304 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.304 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.304 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.304 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:13:30.304 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:30.304 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 124.04799458700001, 2024-09-15T00:13:30.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:13:30.304 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.304 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.305 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:30.305 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:30.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:30.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:13:30.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:13:30.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:13:30.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:13:30.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:13:30.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:13:30.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:13:30.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:13:30.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:13:30.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:13:30.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:13:30.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:13:30.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:13:30.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:13:30.308 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.308 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.308 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.308 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:13:30.308 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:30.309 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.862993104, 2024-09-15T00:13:30.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:13:30.309 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.309 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.309 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:30.309 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.309 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.309 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.309 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:13:30.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.309 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:30.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:30.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:13:30.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:13:30.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:13:30.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:13:30.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:13:30.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:13:30.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:13:30.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:13:30.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:13:30.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:13:30.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:13:30.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:13:30.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:30.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:13:30.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:13:30.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:30.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:13:30.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:13:30.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:30.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:13:30.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:13:30.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:30.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:13:30.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:13:30.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:13:30.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:13:30.314 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.314 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.314 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.314 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:13:30.314 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:30.315 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.850116837, 2024-09-15T00:13:30.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:13:30.315 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.315 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.315 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:30.315 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:30.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:30.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:13:30.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:13:30.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:13:30.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:13:30.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:13:30.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:13:30.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:13:30.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:13:30.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:13:30.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:13:30.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:13:30.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:13:30.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:13:30.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:13:30.318 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.318 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.319 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.319 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:13:30.319 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:30.319 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 109.391601693, 2024-09-15T00:13:30.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:13:30.319 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.319 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.319 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:30.319 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.319 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.319 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.319 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:13:30.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.320 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:30.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:30.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:13:30.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:13:30.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:13:30.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:13:30.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:13:30.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:13:30.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:13:30.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:13:30.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:13:30.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:13:30.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:13:30.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:13:30.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:30.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:13:30.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:13:30.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:30.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:13:30.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:13:30.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:30.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:13:30.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:13:30.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:30.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:13:30.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:13:30.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:13:30.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:13:30.324 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.324 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.325 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.325 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:30.325 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:30.325 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.674091133, 2024-09-15T00:13:30.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:13:30.325 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.325 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.325 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:30.325 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.325 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.325 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.326 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:13:30.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.326 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:30.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:30.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:13:30.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:13:30.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:13:30.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:13:30.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:13:30.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:13:30.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:13:30.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:13:30.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:13:30.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:13:30.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:13:30.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:13:30.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:13:30.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:13:30.329 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.329 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.329 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.329 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:30.329 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:30.329 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.674072524, 2024-09-15T00:13:30.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:13:30.329 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.330 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.330 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:30.330 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.330 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.330 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.330 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:13:30.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.330 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:30.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:30.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:30.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:13:30.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:13:30.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:13:30.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:13:30.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:13:30.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:13:30.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:13:30.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:13:30.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:13:30.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:13:30.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:13:30.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:13:30.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:13:30.334 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.334 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.334 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.334 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:30.334 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:30.334 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.6740472, 2024-09-15T00:13:30.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:13:30.334 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.334 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.334 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:30.334 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.334 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.334 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.335 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:13:30.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.335 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:30.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:30.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:30.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:13:30.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:13:30.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:13:30.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:13:30.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:13:30.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:13:30.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:13:30.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:13:30.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:13:30.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:13:30.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:13:30.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:13:30.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:13:30.338 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.338 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.338 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.338 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:30.338 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:30.338 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.67402887599999, 2024-09-15T00:13:30.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:13:30.338 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.338 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.339 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:30.339 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.339 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.339 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.339 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:13:30.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.339 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:30.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:30.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:13:30.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:13:30.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:13:30.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:13:30.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:13:30.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:30.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:13:30.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:13:30.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:13:30.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:13:30.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:13:30.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:13:30.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:13:30.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:13:30.342 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.342 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.342 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.342 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:30.343 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:30.343 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.673846303, 2024-09-15T00:13:30.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:13:30.343 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.343 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.343 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:30.343 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.343 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.343 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.343 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:13:30.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.343 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:30.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:30.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:30.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:13:30.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:13:30.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:13:30.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:13:30.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:13:30.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:13:30.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:13:30.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:13:30.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:13:30.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:13:30.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:13:30.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:13:30.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:13:30.347 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.347 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.347 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.347 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:30.347 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:30.347 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.673817462, 2024-09-15T00:13:30.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:13:30.347 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.347 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.347 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:30.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.348 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.348 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:13:30.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.348 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:30.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:30.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:30.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:13:30.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:13:30.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:13:30.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:13:30.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:13:30.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:13:30.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:13:30.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:13:30.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:13:30.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:13:30.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:13:30.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:13:30.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:13:30.351 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.351 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.351 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.351 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:30.351 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:30.351 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.673795251, 2024-09-15T00:13:30.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:13:30.351 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.352 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.352 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:30.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.352 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:13:30.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.352 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:30.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:30.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:30.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:13:30.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:13:30.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:13:30.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:13:30.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:13:30.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:13:30.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:13:30.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:13:30.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:13:30.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:13:30.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:13:30.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:13:30.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:13:30.355 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.355 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.355 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.356 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:30.356 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:30.356 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.673774092, 2024-09-15T00:13:30.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:13:30.356 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.356 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.356 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:30.356 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.356 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.356 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.356 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:13:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:13:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:13:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:13:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:13:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:13:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:13:30.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:13:30.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:13:30.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:13:30.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:13:30.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:13:30.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:13:30.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:13:30.360 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.360 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.360 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.360 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:30.360 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:30.360 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.673744144, 2024-09-15T00:13:30.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:13:30.360 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.360 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.360 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:30.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.361 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:13:30.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.361 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:30.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:30.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:30.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:13:30.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:13:30.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:13:30.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:13:30.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:13:30.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:13:30.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:13:30.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:13:30.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:13:30.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:13:30.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:13:30.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:13:30.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:13:30.364 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.364 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.364 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.365 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:30.365 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:30.365 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.67372218, 2024-09-15T00:13:30.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:13:30.365 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.365 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.365 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:30.365 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.365 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.365 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.365 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:13:30.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.366 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:30.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:30.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:30.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:13:30.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:13:30.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:13:30.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:13:30.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:13:30.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:13:30.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:13:30.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:13:30.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:13:30.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:13:30.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:13:30.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:13:30.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:13:30.369 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.369 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.369 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.369 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:30.369 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:30.369 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.673704753, 2024-09-15T00:13:30.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:13:30.369 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.369 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.369 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:30.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.370 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:13:30.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.370 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:30.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:30.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:13:30.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:13:30.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:13:30.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:13:30.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:13:30.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:30.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:13:30.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:13:30.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:13:30.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:13:30.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:13:30.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:13:30.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:13:30.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:13:30.373 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.373 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.374 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.374 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:30.374 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:30.374 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.673682954, 2024-09-15T00:13:30.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:13:30.374 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.374 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.374 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:30.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.374 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:13:30.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.375 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:30.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:30.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:13:30.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:13:30.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:13:30.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:13:30.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:13:30.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:13:30.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:13:30.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:13:30.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:13:30.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:13:30.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:13:30.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:13:30.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:13:30.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:13:30.378 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.378 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.378 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.378 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:30.378 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:30.379 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.624454595, 2024-09-15T00:13:30.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:13:30.379 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:30.379 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:30.379 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:30.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:30.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:30.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.379 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:13:30.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.379 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:30.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:30.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:30.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:30.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:30.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:30.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:30.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:13:30.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:13:30.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:30.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:13:30.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:13:30.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:30.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:13:30.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:13:30.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:13:30.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:13:30.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:30.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:13:30.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:13:30.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:30.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:13:30.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:13:30.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:30.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:13:30.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:13:30.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:30.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:13:30.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:13:30.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:30.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:13:30.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:13:30.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:30.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:13:30.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:13:30.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:30.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:30.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:30.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:13:30.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:13:30.384 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.384 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.384 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.384 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:30.384 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:30.384 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:30.386 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:30.386 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:30.384+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:30.486 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:30.484+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:30.490 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:30.491 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:30.491 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:30.491 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:30.499 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:30.587 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:30.586+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:30.603 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:30.603 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:30.604 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:30.604 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:30.612 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:30.688 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:30.687+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:30.757 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:30.758 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:30.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:30.758 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:30.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.758 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9, version: 37'47, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9@37'47, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:30.758 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:30.758 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.901477527000001, 2024-09-15T00:13:30.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054905559999999997, 2024-09-15T00:13:30.758 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.758 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.758 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.758 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:30.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:30.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853793+0000", 2024-09-15T00:13:30.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9599999999999999e-07 2024-09-15T00:13:30.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854798+0000", 2024-09-15T00:13:30.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010050770000000001 2024-09-15T00:13:30.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854814+0000", 2024-09-15T00:13:30.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6478999999999999e-05 2024-09-15T00:13:30.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854825+0000", 2024-09-15T00:13:30.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0492999999999999e-05 2024-09-15T00:13:30.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854889+0000", 2024-09-15T00:13:30.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4134999999999995e-05 2024-09-15T00:13:30.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854898+0000", 2024-09-15T00:13:30.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6260000000000001e-06 2024-09-15T00:13:30.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.859283+0000", 2024-09-15T00:13:30.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043852500000000003 2024-09-15T00:13:30.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.762 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.762 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15, version: 59'205, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15@59'205, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:30.762 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:30.762 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.775257071999999, 2024-09-15T00:13:30.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059047939999999997, 2024-09-15T00:13:30.762 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.762 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.762 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.44e-07 2024-09-15T00:13:30.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980977+0000", 2024-09-15T00:13:30.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00096402300000000005 2024-09-15T00:13:30.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980995+0000", 2024-09-15T00:13:30.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7214000000000002e-05 2024-09-15T00:13:30.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981007+0000", 2024-09-15T00:13:30.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2028000000000001e-05 2024-09-15T00:13:30.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981069+0000", 2024-09-15T00:13:30.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1977000000000006e-05 2024-09-15T00:13:30.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981077+0000", 2024-09-15T00:13:30.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2120000000000003e-06 2024-09-15T00:13:30.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.985918+0000", 2024-09-15T00:13:30.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048406960000000002 2024-09-15T00:13:30.766 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.766 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.766 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.766 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head, version: 62'221, data_included: [0~262144,393216~786432], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head@62'221, size: 1572864, copy_subset: [(0, 1572864)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:30.766 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:30.766 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.752988807000001, 2024-09-15T00:13:30.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054170770000000002, 2024-09-15T00:13:30.766 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.767 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.767 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.767 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:30.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:30.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002282+0000", 2024-09-15T00:13:30.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4499999999999997e-07 2024-09-15T00:13:30.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003196+0000", 2024-09-15T00:13:30.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091476399999999999 2024-09-15T00:13:30.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003213+0000", 2024-09-15T00:13:30.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6554000000000001e-05 2024-09-15T00:13:30.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003230+0000", 2024-09-15T00:13:30.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6617999999999999e-05 2024-09-15T00:13:30.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003292+0000", 2024-09-15T00:13:30.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2193999999999994e-05 2024-09-15T00:13:30.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003300+0000", 2024-09-15T00:13:30.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1429999999999998e-06 2024-09-15T00:13:30.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.007698+0000", 2024-09-15T00:13:30.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043981589999999996 2024-09-15T00:13:30.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.770 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.770 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:30.770 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:30.771 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.702258727, 2024-09-15T00:13:30.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067633270000000004, 2024-09-15T00:13:30.771 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.771 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.771 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.771 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:30.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:30.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053012+0000", 2024-09-15T00:13:30.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1659999999999999e-06 2024-09-15T00:13:30.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054086+0000", 2024-09-15T00:13:30.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010732789999999999 2024-09-15T00:13:30.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054100+0000", 2024-09-15T00:13:30.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4123e-05 2024-09-15T00:13:30.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054114+0000", 2024-09-15T00:13:30.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4442e-05 2024-09-15T00:13:30.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054192+0000", 2024-09-15T00:13:30.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8288000000000007e-05 2024-09-15T00:13:30.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054200+0000", 2024-09-15T00:13:30.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-15T00:13:30.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.059774+0000", 2024-09-15T00:13:30.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055741230000000003 2024-09-15T00:13:30.774 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.774 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.775 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.775 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:30.775 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:30.775 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.693941399, 2024-09-15T00:13:30.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049777950000000001, 2024-09-15T00:13:30.775 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.775 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.775 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.775 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:30.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:30.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061330+0000", 2024-09-15T00:13:30.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.11e-06 2024-09-15T00:13:30.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062350+0000", 2024-09-15T00:13:30.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010207969999999999 2024-09-15T00:13:30.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062365+0000", 2024-09-15T00:13:30.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4157e-05 2024-09-15T00:13:30.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062380+0000", 2024-09-15T00:13:30.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5129e-05 2024-09-15T00:13:30.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062467+0000", 2024-09-15T00:13:30.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7247e-05 2024-09-15T00:13:30.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062475+0000", 2024-09-15T00:13:30.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4300000000000006e-06 2024-09-15T00:13:30.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.066306+0000", 2024-09-15T00:13:30.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038309250000000002 2024-09-15T00:13:30.779 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.779 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.779 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.779 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:30.779 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:30.779 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.688058440999999, 2024-09-15T00:13:30.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052454889999999999, 2024-09-15T00:13:30.779 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.779 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.779 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.779 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:30.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:30.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067212+0000", 2024-09-15T00:13:30.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-07 2024-09-15T00:13:30.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068119+0000", 2024-09-15T00:13:30.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090671199999999995 2024-09-15T00:13:30.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068135+0000", 2024-09-15T00:13:30.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5739e-05 2024-09-15T00:13:30.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068150+0000", 2024-09-15T00:13:30.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5693e-05 2024-09-15T00:13:30.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068236+0000", 2024-09-15T00:13:30.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6100000000000006e-05 2024-09-15T00:13:30.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068245+0000", 2024-09-15T00:13:30.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1960000000000008e-06 2024-09-15T00:13:30.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.072457+0000", 2024-09-15T00:13:30.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042123719999999998 2024-09-15T00:13:30.783 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.783 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.783 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.784 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:30.784 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:30.784 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.682080092, 2024-09-15T00:13:30.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049532020000000003, 2024-09-15T00:13:30.784 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.784 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.784 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.784 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:30.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:30.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:30.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:13:30.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074164+0000", 2024-09-15T00:13:30.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097354400000000002 2024-09-15T00:13:30.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074178+0000", 2024-09-15T00:13:30.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4134999999999999e-05 2024-09-15T00:13:30.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074192+0000", 2024-09-15T00:13:30.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.451e-05 2024-09-15T00:13:30.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074272+0000", 2024-09-15T00:13:30.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9989999999999998e-05 2024-09-15T00:13:30.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074280+0000", 2024-09-15T00:13:30.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9990000000000008e-06 2024-09-15T00:13:30.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078143+0000", 2024-09-15T00:13:30.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00386258 2024-09-15T00:13:30.787 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.787 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.787 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.788 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [4194304~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:30.788 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:30.788 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.676369279999999, 2024-09-15T00:13:30.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049408730000000001, 2024-09-15T00:13:30.788 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.788 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.788 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.788 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:30.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:30.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:30.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2399999999999999e-07 2024-09-15T00:13:30.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079875+0000", 2024-09-15T00:13:30.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097349100000000005 2024-09-15T00:13:30.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079889+0000", 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4148000000000001e-05 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079904+0000", 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5159e-05 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079964+0000", 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0204999999999997e-05 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079971+0000", 2024-09-15T00:13:30.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2490000000000004e-06 2024-09-15T00:13:30.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.083841+0000", 2024-09-15T00:13:30.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038701970000000001 2024-09-15T00:13:30.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.791 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.791 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [5242880~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 6291456, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:30.792 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:30.792 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.670664299999999, 2024-09-15T00:13:30.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052525740000000003, 2024-09-15T00:13:30.792 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.792 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.792 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.792 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:30.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:30.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:30.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:13:30.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085579+0000", 2024-09-15T00:13:30.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097307600000000002 2024-09-15T00:13:30.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085592+0000", 2024-09-15T00:13:30.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3277e-05 2024-09-15T00:13:30.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085608+0000", 2024-09-15T00:13:30.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5773e-05 2024-09-15T00:13:30.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085687+0000", 2024-09-15T00:13:30.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8956999999999996e-05 2024-09-15T00:13:30.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085694+0000", 2024-09-15T00:13:30.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.956e-06 2024-09-15T00:13:30.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.089858+0000", 2024-09-15T00:13:30.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041641159999999998 2024-09-15T00:13:30.795 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.795 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.795 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.796 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:172552 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-15T00:13:30.796 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:30.796 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.566206992000001, 2024-09-15T00:13:30.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.997031471, 2024-09-15T00:13:30.796 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.796 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.796 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.796 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:30.796 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:30.796 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:30.796 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 172552 2024-09-15T00:13:30.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.797 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:30.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:30.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189064+0000", 2024-09-15T00:13:30.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.198e-06 2024-09-15T00:13:30.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189069+0000", 2024-09-15T00:13:30.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9069999999999998e-06 2024-09-15T00:13:30.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189080+0000", 2024-09-15T00:13:30.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:13:30.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189086+0000", 2024-09-15T00:13:30.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5429999999999999e-06 2024-09-15T00:13:30.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189128+0000", 2024-09-15T00:13:30.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2255000000000001e-05 2024-09-15T00:13:30.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:30.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189135+0000", 2024-09-15T00:13:30.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6200000000000001e-06 2024-09-15T00:13:30.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185787+0000", 2024-09-15T00:13:30.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99665157199999999 2024-09-15T00:13:30.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185926+0000", 2024-09-15T00:13:30.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013970399999999999 2024-09-15T00:13:30.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.186094+0000", 2024-09-15T00:13:30.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016813099999999999 2024-09-15T00:13:30.801 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.801 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.801 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.801 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9, version: 37'60, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9@37'60, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:30.801 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:30.801 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.507346633000001, 2024-09-15T00:13:30.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049454060000000003, 2024-09-15T00:13:30.801 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.801 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.801 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.801 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:30.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:30.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247924+0000", 2024-09-15T00:13:30.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:13:30.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248394+0000", 2024-09-15T00:13:30.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047063399999999999 2024-09-15T00:13:30.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248411+0000", 2024-09-15T00:13:30.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6384000000000001e-05 2024-09-15T00:13:30.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248441+0000", 2024-09-15T00:13:30.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0559999999999999e-05 2024-09-15T00:13:30.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248499+0000", 2024-09-15T00:13:30.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7274000000000001e-05 2024-09-15T00:13:30.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248507+0000", 2024-09-15T00:13:30.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2390000000000002e-06 2024-09-15T00:13:30.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.252869+0000", 2024-09-15T00:13:30.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043617200000000004 2024-09-15T00:13:30.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.805 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.805 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head, version: 59'393, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head@59'393, size: 5701632, copy_subset: [(0, 5701632)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:30.805 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:30.805 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.377898366, 2024-09-15T00:13:30.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004807469, 2024-09-15T00:13:30.805 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.805 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.806 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.806 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:30.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:30.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:30.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4799999999999998e-07 2024-09-15T00:13:30.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377830+0000", 2024-09-15T00:13:30.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000457954 2024-09-15T00:13:30.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377847+0000", 2024-09-15T00:13:30.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6741000000000001e-05 2024-09-15T00:13:30.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377858+0000", 2024-09-15T00:13:30.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0951999999999999e-05 2024-09-15T00:13:30.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377929+0000", 2024-09-15T00:13:30.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0697000000000004e-05 2024-09-15T00:13:30.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377941+0000", 2024-09-15T00:13:30.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2529e-05 2024-09-15T00:13:30.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.382179+0000", 2024-09-15T00:13:30.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042379480000000001 2024-09-15T00:13:30.809 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.809 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.809 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.809 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head, version: 88'473, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head@88'473, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:30.809 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:30.809 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.295778933000001, 2024-09-15T00:13:30.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005823272, 2024-09-15T00:13:30.810 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.810 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.810 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.810 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:30.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:30.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459492+0000", 2024-09-15T00:13:30.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9800000000000003e-07 2024-09-15T00:13:30.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459940+0000", 2024-09-15T00:13:30.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044854699999999999 2024-09-15T00:13:30.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459957+0000", 2024-09-15T00:13:30.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6586e-05 2024-09-15T00:13:30.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459968+0000", 2024-09-15T00:13:30.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.145e-05 2024-09-15T00:13:30.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460043+0000", 2024-09-15T00:13:30.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4451999999999998e-05 2024-09-15T00:13:30.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460057+0000", 2024-09-15T00:13:30.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.417e-05 2024-09-15T00:13:30.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.465314+0000", 2024-09-15T00:13:30.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052574689999999999 2024-09-15T00:13:30.813 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.813 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.813 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.813 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:30.813 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:30.813 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.060602752000001, 2024-09-15T00:13:30.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051811399999999999, 2024-09-15T00:13:30.814 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.814 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.814 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.814 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:30.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:30.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694668+0000", 2024-09-15T00:13:30.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4600000000000005e-07 2024-09-15T00:13:30.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695136+0000", 2024-09-15T00:13:30.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00046864399999999999 2024-09-15T00:13:30.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695153+0000", 2024-09-15T00:13:30.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6218e-05 2024-09-15T00:13:30.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695189+0000", 2024-09-15T00:13:30.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6393000000000002e-05 2024-09-15T00:13:30.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695263+0000", 2024-09-15T00:13:30.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4175e-05 2024-09-15T00:13:30.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695271+0000", 2024-09-15T00:13:30.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2800000000000003e-06 2024-09-15T00:13:30.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.699848+0000", 2024-09-15T00:13:30.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045768839999999998 2024-09-15T00:13:30.817 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.817 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.817 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.817 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:30.817 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:30.817 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.052504544000001, 2024-09-15T00:13:30.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048716510000000003, 2024-09-15T00:13:30.818 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.818 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.818 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.818 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:30.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:30.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702783+0000", 2024-09-15T00:13:30.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7268000000000001e-05 2024-09-15T00:13:30.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703221+0000", 2024-09-15T00:13:30.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00043866899999999998 2024-09-15T00:13:30.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703237+0000", 2024-09-15T00:13:30.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5553000000000002e-05 2024-09-15T00:13:30.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703247+0000", 2024-09-15T00:13:30.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0392e-05 2024-09-15T00:13:30.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703358+0000", 2024-09-15T00:13:30.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110676 2024-09-15T00:13:30.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703369+0000", 2024-09-15T00:13:30.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0716e-05 2024-09-15T00:13:30.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.707637+0000", 2024-09-15T00:13:30.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042683770000000003 2024-09-15T00:13:30.821 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.821 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.821 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.821 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18, version: 62'274, data_included: [0~786432], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18@62'274, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:30.821 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:30.821 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.006423462000001, 2024-09-15T00:13:30.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005453014, 2024-09-15T00:13:30.822 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.822 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.822 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.822 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:30.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:30.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748847+0000", 2024-09-15T00:13:30.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8299999999999997e-07 2024-09-15T00:13:30.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749010+0000", 2024-09-15T00:13:30.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000163144 2024-09-15T00:13:30.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749027+0000", 2024-09-15T00:13:30.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6518999999999999e-05 2024-09-15T00:13:30.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749037+0000", 2024-09-15T00:13:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0438000000000001e-05 2024-09-15T00:13:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749112+0000", 2024-09-15T00:13:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4946999999999999e-05 2024-09-15T00:13:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749123+0000", 2024-09-15T00:13:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1014e-05 2024-09-15T00:13:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.754299+0000", 2024-09-15T00:13:30.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051763690000000001 2024-09-15T00:13:30.825 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.825 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.825 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.825 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20, version: 80'334, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20@80'334, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 20=[]:{20: [20]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:30.825 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:30.825 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.951786418000001, 2024-09-15T00:13:30.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048933570000000001, 2024-09-15T00:13:30.825 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.826 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.826 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.826 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:30.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:30.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803484+0000", 2024-09-15T00:13:30.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:13:30.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803938+0000", 2024-09-15T00:13:30.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045364599999999999 2024-09-15T00:13:30.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803954+0000", 2024-09-15T00:13:30.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6456999999999998e-05 2024-09-15T00:13:30.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803965+0000", 2024-09-15T00:13:30.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.076e-05 2024-09-15T00:13:30.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804023+0000", 2024-09-15T00:13:30.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7787000000000001e-05 2024-09-15T00:13:30.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804031+0000", 2024-09-15T00:13:30.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2160000000000006e-06 2024-09-15T00:13:30.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.808377+0000", 2024-09-15T00:13:30.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043458969999999996 2024-09-15T00:13:30.829 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.829 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.829 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.829 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [0~131072,196608~786432,1048576~131072], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:30.829 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:30.829 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.914999821999999, 2024-09-15T00:13:30.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047737029999999998, 2024-09-15T00:13:30.829 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.829 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.830 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.830 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:30.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:30.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840271+0000", 2024-09-15T00:13:30.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:13:30.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840721+0000", 2024-09-15T00:13:30.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045029800000000002 2024-09-15T00:13:30.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840738+0000", 2024-09-15T00:13:30.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6671999999999999e-05 2024-09-15T00:13:30.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840749+0000", 2024-09-15T00:13:30.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1251e-05 2024-09-15T00:13:30.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840817+0000", 2024-09-15T00:13:30.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8046000000000005e-05 2024-09-15T00:13:30.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840825+0000", 2024-09-15T00:13:30.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2519999999999999e-06 2024-09-15T00:13:30.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.845044+0000", 2024-09-15T00:13:30.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042185900000000004 2024-09-15T00:13:30.833 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.833 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.833 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.833 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [1179648~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2228224, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:30.833 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:30.833 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.906975774999999, 2024-09-15T00:13:30.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053056819999999999, 2024-09-15T00:13:30.833 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.836 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.836 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.836 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:30.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:30.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848295+0000", 2024-09-15T00:13:30.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4000000000000001e-07 2024-09-15T00:13:30.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848728+0000", 2024-09-15T00:13:30.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000433709 2024-09-15T00:13:30.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848745+0000", 2024-09-15T00:13:30.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6167999999999998e-05 2024-09-15T00:13:30.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848788+0000", 2024-09-15T00:13:30.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3269000000000002e-05 2024-09-15T00:13:30.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848857+0000", 2024-09-15T00:13:30.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9453000000000001e-05 2024-09-15T00:13:30.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848866+0000", 2024-09-15T00:13:30.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2779999999999993e-06 2024-09-15T00:13:30.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.853600+0000", 2024-09-15T00:13:30.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047341650000000002 2024-09-15T00:13:30.839 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.839 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.839 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.840 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head, version: 93'455, data_included: [0~98304,114688~16384,147456~589824,753664~65536,835584~16384,868352~262144], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head@93'455, size: 2342912, copy_subset: [(0, 2342912)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1130496, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:30.840 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:30.840 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.847841749000001, 2024-09-15T00:13:30.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063250900000000002, 2024-09-15T00:13:30.840 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:30.840 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:30.840 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:30.840 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:30.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:30.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:30.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:30.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:30.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:30.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:30.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907429+0000", 2024-09-15T00:13:30.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.85e-07 2024-09-15T00:13:30.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:30.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908008+0000", 2024-09-15T00:13:30.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00057949599999999998 2024-09-15T00:13:30.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:30.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908026+0000", 2024-09-15T00:13:30.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7700999999999999e-05 2024-09-15T00:13:30.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:30.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908038+0000", 2024-09-15T00:13:30.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2131e-05 2024-09-15T00:13:30.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:30.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908101+0000", 2024-09-15T00:13:30.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2596999999999997e-05 2024-09-15T00:13:30.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:30.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908110+0000", 2024-09-15T00:13:30.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8540000000000003e-06 2024-09-15T00:13:30.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:30.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:30.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:30.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.913753+0000", 2024-09-15T00:13:30.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056435260000000003 2024-09-15T00:13:30.843 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.843 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.843 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:30.844 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:30.844 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:30.845 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:30.846 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:30.843+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:30.945 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:30.944+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:30.950 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:30.950 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:30.959 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:31.046 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:31.044+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:31.110 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:31.111 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:31.111 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:31.111 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:31.111 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:31.120 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:31.162 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:31.160+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:31.248 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:31.247+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:31.271 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:31.272 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:31.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:31.272 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:31.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.272 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133834 3.1b e85/69 3:da9a78a3:::rbd_data.11395dfcf2b8.0000000000000000:head v 85'356, mlcod=85'354)", 2024-09-15T00:13:31.272 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:31.272 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.705493281999999, 2024-09-15T00:13:31.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026079054000000001, 2024-09-15T00:13:31.272 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.272 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.272 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.272 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:31.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:31.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563338+0000", 2024-09-15T00:13:31.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.418e-06 2024-09-15T00:13:31.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569385+0000", 2024-09-15T00:13:31.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006047335 2024-09-15T00:13:31.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569418+0000", 2024-09-15T00:13:31.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2957999999999999e-05 2024-09-15T00:13:31.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569432+0000", 2024-09-15T00:13:31.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3718e-05 2024-09-15T00:13:31.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569519+0000", 2024-09-15T00:13:31.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7022999999999997e-05 2024-09-15T00:13:31.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569540+0000", 2024-09-15T00:13:31.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1426000000000001e-05 2024-09-15T00:13:31.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589362+0000", 2024-09-15T00:13:31.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019821398 2024-09-15T00:13:31.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589415+0000", 2024-09-15T00:13:31.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3777999999999999e-05 2024-09-15T00:13:31.276 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.277 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.277 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.277 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133836 3.1 e85/67 3:849f9a98:::rbd_data.11395dfcf2b8.0000000000000002:head v 85'304, mlcod=84'301)", 2024-09-15T00:13:31.277 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:31.277 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.697798372000001, 2024-09-15T00:13:31.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023231600000000002, 2024-09-15T00:13:31.277 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.277 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.277 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.277 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:31.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:31.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571032+0000", 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576635+0000", 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056028700000000002 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576650+0000", 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.505e-05 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576658+0000", 2024-09-15T00:13:31.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6879999999999999e-06 2024-09-15T00:13:31.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576717+0000", 2024-09-15T00:13:31.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175000000000002e-05 2024-09-15T00:13:31.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576742+0000", 2024-09-15T00:13:31.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4734000000000001e-05 2024-09-15T00:13:31.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594234+0000", 2024-09-15T00:13:31.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017492527000000001 2024-09-15T00:13:31.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594263+0000", 2024-09-15T00:13:31.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8629e-05 2024-09-15T00:13:31.281 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.281 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.281 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.281 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146609 3.b e98/69 3:d39b42ba:::rbd_data.11394501364.0000000000000001:head v 98'283, mlcod=93'281)", 2024-09-15T00:13:31.281 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:31.282 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.788080256999997, 2024-09-15T00:13:31.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020892945, 2024-09-15T00:13:31.282 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.282 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.282 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.282 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:31.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:31.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480751+0000", 2024-09-15T00:13:31.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6509999999999999e-06 2024-09-15T00:13:31.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482738+0000", 2024-09-15T00:13:31.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019867410000000002 2024-09-15T00:13:31.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482754+0000", 2024-09-15T00:13:31.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6171000000000002e-05 2024-09-15T00:13:31.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482769+0000", 2024-09-15T00:13:31.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4688000000000001e-05 2024-09-15T00:13:31.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482835+0000", 2024-09-15T00:13:31.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6526000000000006e-05 2024-09-15T00:13:31.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482858+0000", 2024-09-15T00:13:31.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2959000000000002e-05 2024-09-15T00:13:31.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501587+0000", 2024-09-15T00:13:31.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018728471 2024-09-15T00:13:31.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501642+0000", 2024-09-15T00:13:31.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5738000000000001e-05 2024-09-15T00:13:31.286 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.286 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.286 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.286 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146835 3.12 e98/68 3:4eee39d6:::rbd_data.11394501364.0000000000000000:head v 98'474, mlcod=93'472)", 2024-09-15T00:13:31.286 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:31.286 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.679452986999998, 2024-09-15T00:13:31.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020080434000000001, 2024-09-15T00:13:31.287 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.287 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.287 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.287 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:31.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:31.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589378+0000", 2024-09-15T00:13:31.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7770000000000001e-06 2024-09-15T00:13:31.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590678+0000", 2024-09-15T00:13:31.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012991750000000001 2024-09-15T00:13:31.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590699+0000", 2024-09-15T00:13:31.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1353e-05 2024-09-15T00:13:31.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590710+0000", 2024-09-15T00:13:31.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1245e-05 2024-09-15T00:13:31.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590800+0000", 2024-09-15T00:13:31.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9963000000000006e-05 2024-09-15T00:13:31.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590823+0000", 2024-09-15T00:13:31.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3289e-05 2024-09-15T00:13:31.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609368+0000", 2024-09-15T00:13:31.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018545078 2024-09-15T00:13:31.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609457+0000", 2024-09-15T00:13:31.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8554000000000006e-05 2024-09-15T00:13:31.291 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.291 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.291 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.292 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146837 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7201319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:31.292 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:31.292 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.668673523999999, 2024-09-15T00:13:31.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024751038, 2024-09-15T00:13:31.292 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.292 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.292 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.292 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.292 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.292 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.292 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146837 2024-09-15T00:13:31.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.292 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:31.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:31.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600157+0000", 2024-09-15T00:13:31.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:13:31.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601965+0000", 2024-09-15T00:13:31.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001807613 2024-09-15T00:13:31.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601985+0000", 2024-09-15T00:13:31.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0679e-05 2024-09-15T00:13:31.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601998+0000", 2024-09-15T00:13:31.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2955999999999999e-05 2024-09-15T00:13:31.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602070+0000", 2024-09-15T00:13:31.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1958000000000005e-05 2024-09-15T00:13:31.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602237+0000", 2024-09-15T00:13:31.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016621299999999999 2024-09-15T00:13:31.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:31.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602799+0000", 2024-09-15T00:13:31.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000562723 2024-09-15T00:13:31.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:31.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.620811+0000", 2024-09-15T00:13:31.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018011207000000001 2024-09-15T00:13:31.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:31.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624793+0000", 2024-09-15T00:13:31.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0039828770000000001 2024-09-15T00:13:31.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624855+0000", 2024-09-15T00:13:31.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.143e-05 2024-09-15T00:13:31.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624907+0000", 2024-09-15T00:13:31.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2262000000000002e-05 2024-09-15T00:13:31.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.297 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.297 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146936 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:31.298 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:31.298 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.515160205000001, 2024-09-15T00:13:31.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056847126999999997, 2024-09-15T00:13:31.298 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.298 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.298 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.298 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146936 2024-09-15T00:13:31.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.298 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:31.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:31.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753671+0000", 2024-09-15T00:13:31.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2839999999999999e-06 2024-09-15T00:13:31.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753677+0000", 2024-09-15T00:13:31.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0260000000000004e-06 2024-09-15T00:13:31.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753691+0000", 2024-09-15T00:13:31.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3891e-05 2024-09-15T00:13:31.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753698+0000", 2024-09-15T00:13:31.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8480000000000001e-06 2024-09-15T00:13:31.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753750+0000", 2024-09-15T00:13:31.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1338000000000002e-05 2024-09-15T00:13:31.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753814+0000", 2024-09-15T00:13:31.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4642999999999998e-05 2024-09-15T00:13:31.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.756359+0000", 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025450619999999998 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810435+0000", 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.054075397999999997 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810446+0000", 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1005000000000001e-05 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810482+0000", 2024-09-15T00:13:31.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6494e-05 2024-09-15T00:13:31.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810517+0000", 2024-09-15T00:13:31.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4137999999999997e-05 2024-09-15T00:13:31.303 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.303 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.303 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.303 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:146936 3.6 e98/97 ondisk, result = 0)", 2024-09-15T00:13:31.303 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:31.304 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.505377150999998, 2024-09-15T00:13:31.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046987205999999997, 2024-09-15T00:13:31.304 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.304 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.304 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:31.304 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:31.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:31.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763453+0000", 2024-09-15T00:13:31.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5199999999999996e-07 2024-09-15T00:13:31.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763457+0000", 2024-09-15T00:13:31.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8569999999999997e-06 2024-09-15T00:13:31.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763469+0000", 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1977999999999999e-05 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763476+0000", 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810423+0000", 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946993999999999 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810431+0000", 2024-09-15T00:13:31.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0339999999999997e-06 2024-09-15T00:13:31.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810440+0000", 2024-09-15T00:13:31.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9989999999999997e-06 2024-09-15T00:13:31.307 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.307 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.307 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.307 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:151036 3.1f e103/68 3:f84a1a09:::rbd_data.113958cb0e55.000000000000000c:head v 103'452, mlcod=100'450)", 2024-09-15T00:13:31.307 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:31.308 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.932030922999999, 2024-09-15T00:13:31.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020618753, 2024-09-15T00:13:31.308 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.308 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.308 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.308 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:31.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:31.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:31.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:13:31.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340692+0000", 2024-09-15T00:13:31.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038928890000000001 2024-09-15T00:13:31.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340704+0000", 2024-09-15T00:13:31.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1497e-05 2024-09-15T00:13:31.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340709+0000", 2024-09-15T00:13:31.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1529999999999996e-06 2024-09-15T00:13:31.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340789+0000", 2024-09-15T00:13:31.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9907999999999995e-05 2024-09-15T00:13:31.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340810+0000", 2024-09-15T00:13:31.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1837000000000001e-05 2024-09-15T00:13:31.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357368+0000", 2024-09-15T00:13:31.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016557199000000002 2024-09-15T00:13:31.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357417+0000", 2024-09-15T00:13:31.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9783000000000001e-05 2024-09-15T00:13:31.312 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.312 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.312 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.312 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:151042 3.6 3:6419cde4:::rbd_data.113958cb0e55.0000000000000031:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:31.312 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:31.312 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.902952804999998, 2024-09-15T00:13:31.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025627375000000001, 2024-09-15T00:13:31.313 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.313 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.313 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.313 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.313 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.313 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.313 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 151042 2024-09-15T00:13:31.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.313 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7899999999999998e-07 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365881+0000", 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9770000000000002e-06 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365890+0000", 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5920000000000001e-06 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365894+0000", 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4569999999999998e-06 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365927+0000", 2024-09-15T00:13:31.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2036000000000003e-05 2024-09-15T00:13:31.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365970+0000", 2024-09-15T00:13:31.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3197e-05 2024-09-15T00:13:31.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:31.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.367432+0000", 2024-09-15T00:13:31.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014621720000000001 2024-09-15T00:13:31.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:31.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391358+0000", 2024-09-15T00:13:31.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023925921999999999 2024-09-15T00:13:31.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:31.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391415+0000", 2024-09-15T00:13:31.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7253999999999998e-05 2024-09-15T00:13:31.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391456+0000", 2024-09-15T00:13:31.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0497999999999997e-05 2024-09-15T00:13:31.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391504+0000", 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8591000000000003e-05 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:151042 3.6 e103/97 ondisk, result = 0)", 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.897209791999998, 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019892730000000001, 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:31.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:31.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:31.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8100000000000003e-07 2024-09-15T00:13:31.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371623+0000", 2024-09-15T00:13:31.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6290000000000002e-06 2024-09-15T00:13:31.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371632+0000", 2024-09-15T00:13:31.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6379999999999993e-06 2024-09-15T00:13:31.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371636+0000", 2024-09-15T00:13:31.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:13:31.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391403+0000", 2024-09-15T00:13:31.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019766982999999998 2024-09-15T00:13:31.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391411+0000", 2024-09-15T00:13:31.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7309999999999993e-06 2024-09-15T00:13:31.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391513+0000", 2024-09-15T00:13:31.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101648 2024-09-15T00:13:31.322 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.322 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.322 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.322 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152025 3.3 e103/68 3:c0279197:::rbd_data.113958cb0e55.0000000000000007:head v 103'464, mlcod=102'461)", 2024-09-15T00:13:31.322 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:31.322 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.752722196000001, 2024-09-15T00:13:31.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021264476000000001, 2024-09-15T00:13:31.322 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.322 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.323 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.323 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:31.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:31.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516109+0000", 2024-09-15T00:13:31.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.666e-06 2024-09-15T00:13:31.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.519997+0000", 2024-09-15T00:13:31.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003888449 2024-09-15T00:13:31.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520016+0000", 2024-09-15T00:13:31.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8059000000000001e-05 2024-09-15T00:13:31.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520026+0000", 2024-09-15T00:13:31.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9599999999999995e-06 2024-09-15T00:13:31.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520089+0000", 2024-09-15T00:13:31.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3074e-05 2024-09-15T00:13:31.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520108+0000", 2024-09-15T00:13:31.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8981999999999999e-05 2024-09-15T00:13:31.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537326+0000", 2024-09-15T00:13:31.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017218102999999998 2024-09-15T00:13:31.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537372+0000", 2024-09-15T00:13:31.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6183000000000002e-05 2024-09-15T00:13:31.326 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.326 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.327 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.327 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152284 3.f e103/68 3:f4272b5d:::rbd_data.113958cb0e55.0000000000000006:head v 103'370, mlcod=103'368)", 2024-09-15T00:13:31.327 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:31.327 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.704245665000002, 2024-09-15T00:13:31.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020145026999999999, 2024-09-15T00:13:31.327 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.327 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.327 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.327 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:31.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:31.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564585+0000", 2024-09-15T00:13:31.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7000000000000003e-07 2024-09-15T00:13:31.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568352+0000", 2024-09-15T00:13:31.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037675909999999998 2024-09-15T00:13:31.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568370+0000", 2024-09-15T00:13:31.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7158000000000001e-05 2024-09-15T00:13:31.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568378+0000", 2024-09-15T00:13:31.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6449999999999994e-06 2024-09-15T00:13:31.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568454+0000", 2024-09-15T00:13:31.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5285999999999997e-05 2024-09-15T00:13:31.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568472+0000", 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8447000000000001e-05 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584693+0000", 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016221467999999999 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584729+0000", 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5462e-05 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.332 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152500 3.17 e103/68 3:edff5adf:::rbd_data.113958cb0e55.000000000000000d:head v 103'461, mlcod=103'459)", 2024-09-15T00:13:31.332 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:31.332 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.627259080000002, 2024-09-15T00:13:31.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020217809, 2024-09-15T00:13:31.332 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.332 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.332 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.332 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:31.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:31.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641572+0000", 2024-09-15T00:13:31.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-15T00:13:31.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644961+0000", 2024-09-15T00:13:31.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033882610000000001 2024-09-15T00:13:31.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644979+0000", 2024-09-15T00:13:31.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8179e-05 2024-09-15T00:13:31.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644987+0000", 2024-09-15T00:13:31.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8860000000000004e-06 2024-09-15T00:13:31.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645044+0000", 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7141000000000001e-05 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645062+0000", 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8258e-05 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661749+0000", 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016686553999999999 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661788+0000", 2024-09-15T00:13:31.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9690000000000001e-05 2024-09-15T00:13:31.336 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.336 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.336 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.336 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153171 3.18 e103/69 3:1db186e8:::rbd_data.113958cb0e55.0000000000000002:head v 103'520, mlcod=103'518)", 2024-09-15T00:13:31.336 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:31.336 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.488774174, 2024-09-15T00:13:31.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022266838000000001, 2024-09-15T00:13:31.336 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.336 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.336 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.337 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:31.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:31.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780057+0000", 2024-09-15T00:13:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5880000000000001e-06 2024-09-15T00:13:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781273+0000", 2024-09-15T00:13:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001216087 2024-09-15T00:13:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781311+0000", 2024-09-15T00:13:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7656e-05 2024-09-15T00:13:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781325+0000", 2024-09-15T00:13:31.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4508999999999999e-05 2024-09-15T00:13:31.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781403+0000", 2024-09-15T00:13:31.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7927000000000001e-05 2024-09-15T00:13:31.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781422+0000", 2024-09-15T00:13:31.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8575000000000001e-05 2024-09-15T00:13:31.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802249+0000", 2024-09-15T00:13:31.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020827577 2024-09-15T00:13:31.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802322+0000", 2024-09-15T00:13:31.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2918999999999998e-05 2024-09-15T00:13:31.340 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.340 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.340 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.341 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153772 3.12 e103/68 3:4911a4f6:::rbd_data.113958cb0e55.0000000000000001:head v 103'490, mlcod=103'488)", 2024-09-15T00:13:31.341 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:31.341 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.385903546999998, 2024-09-15T00:13:31.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021181004, 2024-09-15T00:13:31.341 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.341 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.341 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.341 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882927+0000", 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.426e-06 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883966+0000", 2024-09-15T00:13:31.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010384610000000001 2024-09-15T00:13:31.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883985+0000", 2024-09-15T00:13:31.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9276e-05 2024-09-15T00:13:31.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883994+0000", 2024-09-15T00:13:31.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9490000000000002e-06 2024-09-15T00:13:31.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884054+0000", 2024-09-15T00:13:31.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9419999999999997e-05 2024-09-15T00:13:31.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884073+0000", 2024-09-15T00:13:31.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9137e-05 2024-09-15T00:13:31.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904038+0000", 2024-09-15T00:13:31.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019965092 2024-09-15T00:13:31.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904107+0000", 2024-09-15T00:13:31.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9243000000000001e-05 2024-09-15T00:13:31.345 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.345 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.345 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.345 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163112 3.3 e106/68 3:c668f3d5:::rbd_data.11395d771336.0000000000000001:head v 106'489, mlcod=105'487)", 2024-09-15T00:13:31.345 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:31.345 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.804006278000003, 2024-09-15T00:13:31.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019317183000000002, 2024-09-15T00:13:31.345 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.345 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.345 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.345 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464824+0000", 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1100000000000005e-07 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466874+0000", 2024-09-15T00:13:31.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002049497 2024-09-15T00:13:31.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466884+0000", 2024-09-15T00:13:31.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0424999999999999e-05 2024-09-15T00:13:31.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466890+0000", 2024-09-15T00:13:31.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-15T00:13:31.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466944+0000", 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4592999999999997e-05 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466958+0000", 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3993e-05 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484068+0000", 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017109994 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484140+0000", 2024-09-15T00:13:31.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2224000000000005e-05 2024-09-15T00:13:31.349 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.349 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.349 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.349 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163119 3.0 e106/94 3:06118613:::rbd_data.11395d771336.0000000000000004:head v 106'549, mlcod=105'547)", 2024-09-15T00:13:31.349 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:31.349 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.792391074999998, 2024-09-15T00:13:31.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017175867000000001, 2024-09-15T00:13:31.349 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.349 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.350 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.350 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:31.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:31.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:31.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6700000000000002e-07 2024-09-15T00:13:31.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477786+0000", 2024-09-15T00:13:31.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013468149999999999 2024-09-15T00:13:31.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477797+0000", 2024-09-15T00:13:31.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0635000000000001e-05 2024-09-15T00:13:31.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477805+0000", 2024-09-15T00:13:31.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2989999999999997e-06 2024-09-15T00:13:31.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477859+0000", 2024-09-15T00:13:31.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3699000000000002e-05 2024-09-15T00:13:31.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477878+0000", 2024-09-15T00:13:31.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9335e-05 2024-09-15T00:13:31.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493560+0000", 2024-09-15T00:13:31.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015681748999999998 2024-09-15T00:13:31.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493614+0000", 2024-09-15T00:13:31.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4367999999999999e-05 2024-09-15T00:13:31.353 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.353 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.353 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.5.0:5005 3.17 e110/68 3:eb0359e8:::rbd_data.113964deed84.0000000000000251:35 v 110'537, mlcod=109'534)", 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.132540702999997, 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015446896999999999, 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136290+0000", 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7400000000000003e-07 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136294+0000", 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4989999999999996e-06 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136304+0000", 2024-09-15T00:13:31.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0319999999999999e-05 2024-09-15T00:13:31.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136310+0000", 2024-09-15T00:13:31.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.981e-06 2024-09-15T00:13:31.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136350+0000", 2024-09-15T00:13:31.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0088e-05 2024-09-15T00:13:31.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136364+0000", 2024-09-15T00:13:31.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3954999999999999e-05 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151681+0000", 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015316969999999999 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151736+0000", 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4410000000000003e-05 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1758 3.10 e110/68 3:08149147:::rbd_data.113964deed84.0000000000000055:35 v 110'594, mlcod=109'591)", 2024-09-15T00:13:31.358 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:31.358 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.128242477000001, 2024-09-15T00:13:31.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016278652000000001, 2024-09-15T00:13:31.358 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.358 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.358 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.358 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:31.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140599+0000", 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1109e-05 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140605+0000", 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-15T00:13:31.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140612+0000", 2024-09-15T00:13:31.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6089999999999997e-06 2024-09-15T00:13:31.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140656+0000", 2024-09-15T00:13:31.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3754999999999997e-05 2024-09-15T00:13:31.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140674+0000", 2024-09-15T00:13:31.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7640999999999999e-05 2024-09-15T00:13:31.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156823+0000", 2024-09-15T00:13:31.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016149047999999999 2024-09-15T00:13:31.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156866+0000", 2024-09-15T00:13:31.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3205999999999999e-05 2024-09-15T00:13:31.361 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.361 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.361 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.362 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.4.0:5722 3.e e110/95 3:74a58daf:::rbd_data.113964deed84.00000000000000c1:35 v 110'1287, mlcod=109'1284)", 2024-09-15T00:13:31.362 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:31.362 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.119168121000001, 2024-09-15T00:13:31.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01830555, 2024-09-15T00:13:31.362 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.362 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.362 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.362 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:31.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:31.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149662+0000", 2024-09-15T00:13:31.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:13:31.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149666+0000", 2024-09-15T00:13:31.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2220000000000004e-06 2024-09-15T00:13:31.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149672+0000", 2024-09-15T00:13:31.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3410000000000002e-06 2024-09-15T00:13:31.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149676+0000", 2024-09-15T00:13:31.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.566e-06 2024-09-15T00:13:31.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149709+0000", 2024-09-15T00:13:31.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2403000000000001e-05 2024-09-15T00:13:31.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149719+0000", 2024-09-15T00:13:31.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0608000000000001e-05 2024-09-15T00:13:31.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167944+0000", 2024-09-15T00:13:31.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018225183999999998 2024-09-15T00:13:31.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167967+0000", 2024-09-15T00:13:31.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2705e-05 2024-09-15T00:13:31.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.366 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.366 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.366 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:31.367 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:31.368 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:31.366+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:31.469 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:31.467+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:31.471 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:31.471 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:31.471 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:31.480 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:31.569 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:31.568+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:31.624 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:31.624 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:31.624 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:31.625 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:31.625 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:31.633 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:31.678 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:31.677+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:31.738 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 125.55613630400001, 2024-09-15T00:13:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:13:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:13:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:13:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:13:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:13:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:13:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:13:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:13:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:13:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:13:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:13:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:13:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:13:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:13:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:13:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:13:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:13:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:13:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:13:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:13:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:13:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:13:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:13:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:13:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:13:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.826295125, 2024-09-15T00:13:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:13:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:13:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:13:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:13:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:13:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:13:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:13:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:13:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:13:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:13:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:13:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:13:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:13:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:13:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:13:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:13:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:13:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:13:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:13:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:13:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:13:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:13:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:13:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:13:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:13:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.81933676200001, 2024-09-15T00:13:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:13:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:13:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:13:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:13:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:13:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:13:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:13:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:13:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:13:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:13:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:13:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:13:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:13:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:13:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:13:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 107.072147925, 2024-09-15T00:13:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:13:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:13:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:13:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:13:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:13:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:13:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:13:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:13:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:13:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:13:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:13:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:13:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:13:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:13:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:13:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:13:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:13:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:13:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:13:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:13:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:13:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:13:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:13:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:13:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:13:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97493 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.882565205000006, 2024-09-15T00:13:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033259141999999998, 2024-09-15T00:13:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97493 2024-09-15T00:13:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-15T00:13:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853401+0000", 2024-09-15T00:13:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:13:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853407+0000", 2024-09-15T00:13:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-15T00:13:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853411+0000", 2024-09-15T00:13:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.481e-06 2024-09-15T00:13:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886544+0000", 2024-09-15T00:13:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033132861999999999 2024-09-15T00:13:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886587+0000", 2024-09-15T00:13:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2942000000000003e-05 2024-09-15T00:13:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886655+0000", 2024-09-15T00:13:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7711000000000003e-05 2024-09-15T00:13:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97494 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.882544081999995, 2024-09-15T00:13:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033392390000000001, 2024-09-15T00:13:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97494 2024-09-15T00:13:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-15T00:13:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853419+0000", 2024-09-15T00:13:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.917e-06 2024-09-15T00:13:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853425+0000", 2024-09-15T00:13:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:13:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853433+0000", 2024-09-15T00:13:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8120000000000008e-06 2024-09-15T00:13:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886680+0000", 2024-09-15T00:13:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033247207000000001 2024-09-15T00:13:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886726+0000", 2024-09-15T00:13:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5902000000000002e-05 2024-09-15T00:13:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886809+0000", 2024-09-15T00:13:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3475999999999998e-05 2024-09-15T00:13:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97495 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.882523129000006, 2024-09-15T00:13:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033640503000000002, 2024-09-15T00:13:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97495 2024-09-15T00:13:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:13:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853440+0000", 2024-09-15T00:13:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.804e-06 2024-09-15T00:13:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853446+0000", 2024-09-15T00:13:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5559999999999998e-06 2024-09-15T00:13:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853450+0000", 2024-09-15T00:13:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9519999999999996e-06 2024-09-15T00:13:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886904+0000", 2024-09-15T00:13:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033454207999999999 2024-09-15T00:13:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887007+0000", 2024-09-15T00:13:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102904 2024-09-15T00:13:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887078+0000", 2024-09-15T00:13:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1719999999999998e-05 2024-09-15T00:13:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97496 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.882506031000005, 2024-09-15T00:13:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033802431000000001, 2024-09-15T00:13:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97496 2024-09-15T00:13:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.53e-07 2024-09-15T00:13:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853462+0000", 2024-09-15T00:13:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-15T00:13:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853468+0000", 2024-09-15T00:13:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3609999999999997e-06 2024-09-15T00:13:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853473+0000", 2024-09-15T00:13:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-15T00:13:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887111+0000", 2024-09-15T00:13:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033638376999999997 2024-09-15T00:13:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887178+0000", 2024-09-15T00:13:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6877000000000004e-05 2024-09-15T00:13:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887257+0000", 2024-09-15T00:13:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9616999999999993e-05 2024-09-15T00:13:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97587 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.882482796999994, 2024-09-15T00:13:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033973642999999998, 2024-09-15T00:13:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97587 2024-09-15T00:13:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853479+0000", 2024-09-15T00:13:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:13:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853481+0000", 2024-09-15T00:13:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.136e-06 2024-09-15T00:13:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853487+0000", 2024-09-15T00:13:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2439999999999998e-06 2024-09-15T00:13:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853492+0000", 2024-09-15T00:13:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:13:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887284+0000", 2024-09-15T00:13:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033791827000000003 2024-09-15T00:13:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887361+0000", 2024-09-15T00:13:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6983000000000005e-05 2024-09-15T00:13:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887452+0000", 2024-09-15T00:13:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1236999999999994e-05 2024-09-15T00:13:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97588 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.88246264, 2024-09-15T00:13:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034134617999999999, 2024-09-15T00:13:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97588 2024-09-15T00:13:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853499+0000", 2024-09-15T00:13:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:13:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853501+0000", 2024-09-15T00:13:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.26e-06 2024-09-15T00:13:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853507+0000", 2024-09-15T00:13:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2899999999999999e-06 2024-09-15T00:13:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853512+0000", 2024-09-15T00:13:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8330000000000001e-06 2024-09-15T00:13:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887495+0000", 2024-09-15T00:13:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033982529999999997 2024-09-15T00:13:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887548+0000", 2024-09-15T00:13:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3445000000000001e-05 2024-09-15T00:13:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887633+0000", 2024-09-15T00:13:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4890999999999997e-05 2024-09-15T00:13:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126866 3.6 3.973c9fa6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.779140694999995, 2024-09-15T00:13:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153301000000003, 2024-09-15T00:13:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126866 2024-09-15T00:13:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956821+0000", 2024-09-15T00:13:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.91e-07 2024-09-15T00:13:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956822+0000", 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.705e-06 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956828+0000", 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6760000000000004e-06 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956833+0000", 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5379999999999996e-06 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989970+0000", 2024-09-15T00:13:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033137560000000003 2024-09-15T00:13:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989974+0000", 2024-09-15T00:13:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-15T00:13:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126952 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.778998023, 2024-09-15T00:13:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033156066999999997, 2024-09-15T00:13:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126952 2024-09-15T00:13:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:13:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956965+0000", 2024-09-15T00:13:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3379999999999999e-06 2024-09-15T00:13:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956968+0000", 2024-09-15T00:13:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0419999999999998e-06 2024-09-15T00:13:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956970+0000", 2024-09-15T00:13:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4899999999999999e-06 2024-09-15T00:13:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990113+0000", 2024-09-15T00:13:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033142642 2024-09-15T00:13:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990119+0000", 2024-09-15T00:13:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2979999999999997e-06 2024-09-15T00:13:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126954 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.778952528000005, 2024-09-15T00:13:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033158449, 2024-09-15T00:13:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126954 2024-09-15T00:13:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-15T00:13:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957010+0000", 2024-09-15T00:13:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3659999999999999e-06 2024-09-15T00:13:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957013+0000", 2024-09-15T00:13:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1140000000000002e-06 2024-09-15T00:13:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957016+0000", 2024-09-15T00:13:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1209999999999998e-06 2024-09-15T00:13:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990163+0000", 2024-09-15T00:13:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033146451 2024-09-15T00:13:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990167+0000", 2024-09-15T00:13:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0579999999999999e-06 2024-09-15T00:13:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126955 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.778941313999994, 2024-09-15T00:13:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033160617000000003, 2024-09-15T00:13:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126955 2024-09-15T00:13:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-15T00:13:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957021+0000", 2024-09-15T00:13:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-15T00:13:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957024+0000", 2024-09-15T00:13:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1599999999999998e-06 2024-09-15T00:13:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957027+0000", 2024-09-15T00:13:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000001e-06 2024-09-15T00:13:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990177+0000", 2024-09-15T00:13:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033149741000000003 2024-09-15T00:13:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990180+0000", 2024-09-15T00:13:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4910000000000001e-06 2024-09-15T00:13:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126956 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.778931193000005, 2024-09-15T00:13:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033163633999999997, 2024-09-15T00:13:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126956 2024-09-15T00:13:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957031+0000", 2024-09-15T00:13:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-06 2024-09-15T00:13:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957034+0000", 2024-09-15T00:13:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9960000000000002e-06 2024-09-15T00:13:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957037+0000", 2024-09-15T00:13:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6419999999999999e-06 2024-09-15T00:13:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990190+0000", 2024-09-15T00:13:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153152999999998 2024-09-15T00:13:31.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990193+0000", 2024-09-15T00:13:31.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6069999999999999e-06 2024-09-15T00:13:31.808 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.808 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.808 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.808 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126957 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:31.808 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:31.808 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.778921542999996, 2024-09-15T00:13:31.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033166442999999997, 2024-09-15T00:13:31.808 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.809 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.809 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:31.809 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.809 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.809 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.809 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126957 2024-09-15T00:13:31.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.809 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:31.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:31.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957040+0000", 2024-09-15T00:13:31.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:13:31.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957041+0000", 2024-09-15T00:13:31.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0440000000000001e-06 2024-09-15T00:13:31.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957044+0000", 2024-09-15T00:13:31.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0170000000000001e-06 2024-09-15T00:13:31.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957048+0000", 2024-09-15T00:13:31.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0319999999999997e-06 2024-09-15T00:13:31.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990202+0000", 2024-09-15T00:13:31.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033154640999999999 2024-09-15T00:13:31.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990206+0000", 2024-09-15T00:13:31.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.507e-06 2024-09-15T00:13:31.812 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.812 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.812 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.812 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126958 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:31.812 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:31.812 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.778910312999997, 2024-09-15T00:13:31.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033167613999999998, 2024-09-15T00:13:31.812 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.812 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.813 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:31.813 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.813 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.813 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.813 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126958 2024-09-15T00:13:31.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.813 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:31.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:31.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:31.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:13:31.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957053+0000", 2024-09-15T00:13:31.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.756e-06 2024-09-15T00:13:31.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957059+0000", 2024-09-15T00:13:31.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2770000000000002e-06 2024-09-15T00:13:31.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957063+0000", 2024-09-15T00:13:31.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2459999999999997e-06 2024-09-15T00:13:31.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990215+0000", 2024-09-15T00:13:31.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033151509000000003 2024-09-15T00:13:31.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990218+0000", 2024-09-15T00:13:31.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5099999999999999e-06 2024-09-15T00:13:31.816 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.816 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.816 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.817 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126959 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:31.817 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:31.817 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.778891309000002, 2024-09-15T00:13:31.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033161673000000003, 2024-09-15T00:13:31.817 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.817 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.817 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:31.817 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.817 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.817 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.817 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126959 2024-09-15T00:13:31.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.817 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:31.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:31.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:31.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:13:31.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957072+0000", 2024-09-15T00:13:31.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2239999999999998e-06 2024-09-15T00:13:31.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957079+0000", 2024-09-15T00:13:31.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.19e-06 2024-09-15T00:13:31.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957083+0000", 2024-09-15T00:13:31.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:13:31.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990228+0000", 2024-09-15T00:13:31.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033144818 2024-09-15T00:13:31.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990231+0000", 2024-09-15T00:13:31.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:13:31.820 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.820 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.820 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.820 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126960 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:31.820 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:31.821 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.778872501999999, 2024-09-15T00:13:31.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033155166, 2024-09-15T00:13:31.821 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.821 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.821 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:31.821 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.821 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.821 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.821 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126960 2024-09-15T00:13:31.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.821 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:31.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:31.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:31.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:13:31.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957091+0000", 2024-09-15T00:13:31.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:13:31.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957097+0000", 2024-09-15T00:13:31.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6050000000000003e-06 2024-09-15T00:13:31.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957102+0000", 2024-09-15T00:13:31.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5539999999999999e-06 2024-09-15T00:13:31.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990240+0000", 2024-09-15T00:13:31.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033138289000000001 2024-09-15T00:13:31.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990244+0000", 2024-09-15T00:13:31.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4740000000000001e-06 2024-09-15T00:13:31.824 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.824 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.824 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.824 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:174575 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138)", 2024-09-15T00:13:31.824 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:26.073788+0000", 2024-09-15T00:13:31.825 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.6621731870000005, 2024-09-15T00:13:31.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.95024539600000002, 2024-09-15T00:13:31.825 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:31.825 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:31.825 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:31.825 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:31.825 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:31.825 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:31.825 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 174575 2024-09-15T00:13:31.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.825 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:31.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:31.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073788+0000", 2024-09-15T00:13:31.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:31.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073788+0000", 2024-09-15T00:13:31.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:31.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:31.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073789+0000", 2024-09-15T00:13:31.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3999999999999999e-06 2024-09-15T00:13:31.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:31.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073796+0000", 2024-09-15T00:13:31.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5529999999999998e-06 2024-09-15T00:13:31.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:31.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073807+0000", 2024-09-15T00:13:31.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0825000000000001e-05 2024-09-15T00:13:31.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:31.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073812+0000", 2024-09-15T00:13:31.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6690000000000003e-06 2024-09-15T00:13:31.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073870+0000", 2024-09-15T00:13:31.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7241999999999998e-05 2024-09-15T00:13:31.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:31.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073878+0000", 2024-09-15T00:13:31.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0760000000000003e-06 2024-09-15T00:13:31.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:31.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:27.023756+0000", 2024-09-15T00:13:31.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94987846899999995 2024-09-15T00:13:31.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:31.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:27.023891+0000", 2024-09-15T00:13:31.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000134785 2024-09-15T00:13:31.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:31.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:31.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:31.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:27.024033+0000", 2024-09-15T00:13:31.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014237700000000001 2024-09-15T00:13:31.829 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.829 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.829 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.829 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:31.830 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:31.830 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:31.831 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:31.831 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:31.829+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:31.932 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:31.931+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:31.937 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:13:31.978 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:31.978 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:31.978 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:31.978 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:31.987 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:32.033 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:32.032+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:32.037 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:13:32.064 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:32.065 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:32.082 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:32.082 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:32.099 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:32.099 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:32.117 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:32.117 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:32.125 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 0, 1] out_osds: [7, 2] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-15T00:13:32.125 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T00:13:32.141 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:32.142 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:32.142 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:32.140+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:32.151 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:32.151 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:32.166 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:32.168 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:32.169 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:32.187 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:32.187 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:32.206 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:32.206 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:32.234 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:32.233+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:32.306 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:32.306 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:32.319 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:32.335 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:32.334+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:32.424 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:32.424 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:32.434 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:32.435 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:32.434+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:32.536 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:32.535+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:32.537 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:32.537 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:32.546 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:32.637 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:32.636+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:32.689 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:32.689 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:32.697 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:32.738 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:32.736+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:32.800 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:32.801 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:32.810 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:32.855 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:32.853+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:32.914 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:32.914 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:32.923 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:32.965 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:32.963+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:33.041 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:33.039+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:33.075 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:33.136 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is set 2024-09-15T00:13:33.142 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:33.140+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:33.145 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T00:13:33.164 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-15T00:13:33.187 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:33.187 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:33.198 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:33.243 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:33.241+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:33.307 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:33.307 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:33.316 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:33.357 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:33.355+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:33.445 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:33.443+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:33.459 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:33.459 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:33.467 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:33.546 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:33.545+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:33.571 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:33.571 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:33.581 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:33.665 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:33.663+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:33.724 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:33.724 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:33.733 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:33.748 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:33.747+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:33.849 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:33.847+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:33.875 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:33.876 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:33.884 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:33.950 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:33.948+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:33.991 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:33.991 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:34.000 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:34.051 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:34.049+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:34.104 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:34.104 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:34.112 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:34.141 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.4 to 1 (10000) 2024-09-15T00:13:34.158 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-15T00:13:34.158 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:34.156+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:34.220 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:13:34.259 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:34.260 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:34.260 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:34.258+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:34.272 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:34.357 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:34.356+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:34.375 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:34.375 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:34.393 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:34.393 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:34.409 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:34.410 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:34.423 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:34.423 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:34.423 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:34.423 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:34.427 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:34.427 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:34.434 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:34.446 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:34.446 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:34.462 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:34.463 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:34.463 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:34.461+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:34.480 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:34.480 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:34.497 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:34.497 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:34.555 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:34.555 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:34.555 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:34.555 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:34.555 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:34.556 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:34.554+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:34.564 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:34.656 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:34.655+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:34.669 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:34.669 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:34.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:34.670 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:34.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.670 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133834 3.1b e85/69 3:da9a78a3:::rbd_data.11395dfcf2b8.0000000000000000:head v 85'356, mlcod=85'354)", 2024-09-15T00:13:34.670 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:34.670 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.103566422, 2024-09-15T00:13:34.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026079054000000001, 2024-09-15T00:13:34.670 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.670 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.670 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.670 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:34.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:34.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563338+0000", 2024-09-15T00:13:34.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.418e-06 2024-09-15T00:13:34.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569385+0000", 2024-09-15T00:13:34.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006047335 2024-09-15T00:13:34.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569418+0000", 2024-09-15T00:13:34.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2957999999999999e-05 2024-09-15T00:13:34.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569432+0000", 2024-09-15T00:13:34.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3718e-05 2024-09-15T00:13:34.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569519+0000", 2024-09-15T00:13:34.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7022999999999997e-05 2024-09-15T00:13:34.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569540+0000", 2024-09-15T00:13:34.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1426000000000001e-05 2024-09-15T00:13:34.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589362+0000", 2024-09-15T00:13:34.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019821398 2024-09-15T00:13:34.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589415+0000", 2024-09-15T00:13:34.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3777999999999999e-05 2024-09-15T00:13:34.674 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.674 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.674 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.674 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133836 3.1 e85/67 3:849f9a98:::rbd_data.11395dfcf2b8.0000000000000002:head v 85'304, mlcod=84'301)", 2024-09-15T00:13:34.674 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:34.674 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.095871512000002, 2024-09-15T00:13:34.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023231600000000002, 2024-09-15T00:13:34.674 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571032+0000", 2024-09-15T00:13:34.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-15T00:13:34.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576635+0000", 2024-09-15T00:13:34.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056028700000000002 2024-09-15T00:13:34.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576650+0000", 2024-09-15T00:13:34.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.505e-05 2024-09-15T00:13:34.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576658+0000", 2024-09-15T00:13:34.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6879999999999999e-06 2024-09-15T00:13:34.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576717+0000", 2024-09-15T00:13:34.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175000000000002e-05 2024-09-15T00:13:34.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576742+0000", 2024-09-15T00:13:34.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4734000000000001e-05 2024-09-15T00:13:34.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594234+0000", 2024-09-15T00:13:34.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017492527000000001 2024-09-15T00:13:34.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594263+0000", 2024-09-15T00:13:34.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8629e-05 2024-09-15T00:13:34.679 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.679 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.679 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.679 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146609 3.b e98/69 3:d39b42ba:::rbd_data.11394501364.0000000000000001:head v 98'283, mlcod=93'281)", 2024-09-15T00:13:34.680 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:34.680 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.186153396999998, 2024-09-15T00:13:34.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020892945, 2024-09-15T00:13:34.680 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.680 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.680 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.680 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:34.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:34.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480751+0000", 2024-09-15T00:13:34.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6509999999999999e-06 2024-09-15T00:13:34.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482738+0000", 2024-09-15T00:13:34.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019867410000000002 2024-09-15T00:13:34.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482754+0000", 2024-09-15T00:13:34.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6171000000000002e-05 2024-09-15T00:13:34.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482769+0000", 2024-09-15T00:13:34.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4688000000000001e-05 2024-09-15T00:13:34.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482835+0000", 2024-09-15T00:13:34.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6526000000000006e-05 2024-09-15T00:13:34.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482858+0000", 2024-09-15T00:13:34.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2959000000000002e-05 2024-09-15T00:13:34.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501587+0000", 2024-09-15T00:13:34.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018728471 2024-09-15T00:13:34.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501642+0000", 2024-09-15T00:13:34.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5738000000000001e-05 2024-09-15T00:13:34.684 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.685 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.685 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.685 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146835 3.12 e98/68 3:4eee39d6:::rbd_data.11394501364.0000000000000000:head v 98'474, mlcod=93'472)", 2024-09-15T00:13:34.685 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:34.685 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.077526126999999, 2024-09-15T00:13:34.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020080434000000001, 2024-09-15T00:13:34.685 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.685 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.685 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.685 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:34.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:34.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589378+0000", 2024-09-15T00:13:34.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7770000000000001e-06 2024-09-15T00:13:34.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590678+0000", 2024-09-15T00:13:34.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012991750000000001 2024-09-15T00:13:34.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590699+0000", 2024-09-15T00:13:34.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1353e-05 2024-09-15T00:13:34.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590710+0000", 2024-09-15T00:13:34.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1245e-05 2024-09-15T00:13:34.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590800+0000", 2024-09-15T00:13:34.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9963000000000006e-05 2024-09-15T00:13:34.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590823+0000", 2024-09-15T00:13:34.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3289e-05 2024-09-15T00:13:34.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609368+0000", 2024-09-15T00:13:34.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018545078 2024-09-15T00:13:34.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609457+0000", 2024-09-15T00:13:34.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8554000000000006e-05 2024-09-15T00:13:34.692 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.692 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.692 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.693 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146837 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7201319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:34.693 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:34.693 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.066746664, 2024-09-15T00:13:34.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024751038, 2024-09-15T00:13:34.693 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.693 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.693 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.693 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:34.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:34.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:34.694 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146837 2024-09-15T00:13:34.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.694 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:34.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:34.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600157+0000", 2024-09-15T00:13:34.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:13:34.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601965+0000", 2024-09-15T00:13:34.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001807613 2024-09-15T00:13:34.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601985+0000", 2024-09-15T00:13:34.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0679e-05 2024-09-15T00:13:34.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601998+0000", 2024-09-15T00:13:34.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2955999999999999e-05 2024-09-15T00:13:34.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602070+0000", 2024-09-15T00:13:34.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1958000000000005e-05 2024-09-15T00:13:34.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602237+0000", 2024-09-15T00:13:34.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016621299999999999 2024-09-15T00:13:34.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:34.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602799+0000", 2024-09-15T00:13:34.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000562723 2024-09-15T00:13:34.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:34.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.620811+0000", 2024-09-15T00:13:34.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018011207000000001 2024-09-15T00:13:34.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:34.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624793+0000", 2024-09-15T00:13:34.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0039828770000000001 2024-09-15T00:13:34.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624855+0000", 2024-09-15T00:13:34.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.143e-05 2024-09-15T00:13:34.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624907+0000", 2024-09-15T00:13:34.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2262000000000002e-05 2024-09-15T00:13:34.699 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.699 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.699 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.699 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146936 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:34.699 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:34.699 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.913233345000002, 2024-09-15T00:13:34.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056847126999999997, 2024-09-15T00:13:34.700 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.700 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.700 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.700 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:34.700 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:34.700 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:34.700 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146936 2024-09-15T00:13:34.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.700 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:34.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:34.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753671+0000", 2024-09-15T00:13:34.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2839999999999999e-06 2024-09-15T00:13:34.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753677+0000", 2024-09-15T00:13:34.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0260000000000004e-06 2024-09-15T00:13:34.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753691+0000", 2024-09-15T00:13:34.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3891e-05 2024-09-15T00:13:34.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753698+0000", 2024-09-15T00:13:34.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8480000000000001e-06 2024-09-15T00:13:34.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753750+0000", 2024-09-15T00:13:34.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1338000000000002e-05 2024-09-15T00:13:34.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753814+0000", 2024-09-15T00:13:34.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4642999999999998e-05 2024-09-15T00:13:34.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:34.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.756359+0000", 2024-09-15T00:13:34.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025450619999999998 2024-09-15T00:13:34.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:34.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810435+0000", 2024-09-15T00:13:34.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.054075397999999997 2024-09-15T00:13:34.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:34.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810446+0000", 2024-09-15T00:13:34.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1005000000000001e-05 2024-09-15T00:13:34.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810482+0000", 2024-09-15T00:13:34.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6494e-05 2024-09-15T00:13:34.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810517+0000", 2024-09-15T00:13:34.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4137999999999997e-05 2024-09-15T00:13:34.705 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.705 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.706 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.706 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:146936 3.6 e98/97 ondisk, result = 0)", 2024-09-15T00:13:34.706 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:34.706 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.903450290999999, 2024-09-15T00:13:34.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046987205999999997, 2024-09-15T00:13:34.706 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.706 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.706 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:34.706 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:34.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:34.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763453+0000", 2024-09-15T00:13:34.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5199999999999996e-07 2024-09-15T00:13:34.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763457+0000", 2024-09-15T00:13:34.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8569999999999997e-06 2024-09-15T00:13:34.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763469+0000", 2024-09-15T00:13:34.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1977999999999999e-05 2024-09-15T00:13:34.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763476+0000", 2024-09-15T00:13:34.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:13:34.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810423+0000", 2024-09-15T00:13:34.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946993999999999 2024-09-15T00:13:34.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810431+0000", 2024-09-15T00:13:34.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0339999999999997e-06 2024-09-15T00:13:34.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810440+0000", 2024-09-15T00:13:34.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9989999999999997e-06 2024-09-15T00:13:34.710 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.710 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.710 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.710 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:151036 3.1f e103/68 3:f84a1a09:::rbd_data.113958cb0e55.000000000000000c:head v 103'452, mlcod=100'450)", 2024-09-15T00:13:34.710 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:34.710 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.330104063, 2024-09-15T00:13:34.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020618753, 2024-09-15T00:13:34.711 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.711 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.711 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.711 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:34.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:34.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:34.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:13:34.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340692+0000", 2024-09-15T00:13:34.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038928890000000001 2024-09-15T00:13:34.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340704+0000", 2024-09-15T00:13:34.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1497e-05 2024-09-15T00:13:34.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340709+0000", 2024-09-15T00:13:34.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1529999999999996e-06 2024-09-15T00:13:34.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340789+0000", 2024-09-15T00:13:34.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9907999999999995e-05 2024-09-15T00:13:34.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340810+0000", 2024-09-15T00:13:34.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1837000000000001e-05 2024-09-15T00:13:34.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357368+0000", 2024-09-15T00:13:34.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016557199000000002 2024-09-15T00:13:34.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357417+0000", 2024-09-15T00:13:34.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9783000000000001e-05 2024-09-15T00:13:34.715 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.715 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.715 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.715 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:151042 3.6 3:6419cde4:::rbd_data.113958cb0e55.0000000000000031:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:34.715 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:34.715 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.301025944999999, 2024-09-15T00:13:34.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025627375000000001, 2024-09-15T00:13:34.715 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.715 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.715 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.716 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:34.716 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:34.716 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:34.716 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 151042 2024-09-15T00:13:34.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.716 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:34.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:34.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:34.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7899999999999998e-07 2024-09-15T00:13:34.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365881+0000", 2024-09-15T00:13:34.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9770000000000002e-06 2024-09-15T00:13:34.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365890+0000", 2024-09-15T00:13:34.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5920000000000001e-06 2024-09-15T00:13:34.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365894+0000", 2024-09-15T00:13:34.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4569999999999998e-06 2024-09-15T00:13:34.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365927+0000", 2024-09-15T00:13:34.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2036000000000003e-05 2024-09-15T00:13:34.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365970+0000", 2024-09-15T00:13:34.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3197e-05 2024-09-15T00:13:34.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:34.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.367432+0000", 2024-09-15T00:13:34.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014621720000000001 2024-09-15T00:13:34.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:34.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391358+0000", 2024-09-15T00:13:34.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023925921999999999 2024-09-15T00:13:34.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:34.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391415+0000", 2024-09-15T00:13:34.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7253999999999998e-05 2024-09-15T00:13:34.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391456+0000", 2024-09-15T00:13:34.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0497999999999997e-05 2024-09-15T00:13:34.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391504+0000", 2024-09-15T00:13:34.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8591000000000003e-05 2024-09-15T00:13:34.721 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.721 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.721 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.721 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:151042 3.6 e103/97 ondisk, result = 0)", 2024-09-15T00:13:34.721 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:34.721 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.295282931999999, 2024-09-15T00:13:34.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019892730000000001, 2024-09-15T00:13:34.721 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.721 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.722 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:34.722 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:34.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:34.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:34.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8100000000000003e-07 2024-09-15T00:13:34.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371623+0000", 2024-09-15T00:13:34.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6290000000000002e-06 2024-09-15T00:13:34.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371632+0000", 2024-09-15T00:13:34.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6379999999999993e-06 2024-09-15T00:13:34.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371636+0000", 2024-09-15T00:13:34.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:13:34.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391403+0000", 2024-09-15T00:13:34.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019766982999999998 2024-09-15T00:13:34.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391411+0000", 2024-09-15T00:13:34.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7309999999999993e-06 2024-09-15T00:13:34.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391513+0000", 2024-09-15T00:13:34.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101648 2024-09-15T00:13:34.725 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.725 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.725 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.725 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152025 3.3 e103/68 3:c0279197:::rbd_data.113958cb0e55.0000000000000007:head v 103'464, mlcod=102'461)", 2024-09-15T00:13:34.725 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:34.725 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.150795336000002, 2024-09-15T00:13:34.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021264476000000001, 2024-09-15T00:13:34.726 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.726 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.726 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.726 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:34.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:34.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516109+0000", 2024-09-15T00:13:34.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.666e-06 2024-09-15T00:13:34.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.519997+0000", 2024-09-15T00:13:34.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003888449 2024-09-15T00:13:34.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520016+0000", 2024-09-15T00:13:34.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8059000000000001e-05 2024-09-15T00:13:34.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520026+0000", 2024-09-15T00:13:34.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9599999999999995e-06 2024-09-15T00:13:34.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520089+0000", 2024-09-15T00:13:34.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3074e-05 2024-09-15T00:13:34.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520108+0000", 2024-09-15T00:13:34.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8981999999999999e-05 2024-09-15T00:13:34.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537326+0000", 2024-09-15T00:13:34.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017218102999999998 2024-09-15T00:13:34.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537372+0000", 2024-09-15T00:13:34.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6183000000000002e-05 2024-09-15T00:13:34.729 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.729 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.729 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.730 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152284 3.f e103/68 3:f4272b5d:::rbd_data.113958cb0e55.0000000000000006:head v 103'370, mlcod=103'368)", 2024-09-15T00:13:34.730 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:34.730 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.102318805000003, 2024-09-15T00:13:34.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020145026999999999, 2024-09-15T00:13:34.730 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.730 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.730 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.730 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:34.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:34.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564585+0000", 2024-09-15T00:13:34.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7000000000000003e-07 2024-09-15T00:13:34.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568352+0000", 2024-09-15T00:13:34.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037675909999999998 2024-09-15T00:13:34.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568370+0000", 2024-09-15T00:13:34.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7158000000000001e-05 2024-09-15T00:13:34.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568378+0000", 2024-09-15T00:13:34.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6449999999999994e-06 2024-09-15T00:13:34.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568454+0000", 2024-09-15T00:13:34.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5285999999999997e-05 2024-09-15T00:13:34.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568472+0000", 2024-09-15T00:13:34.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8447000000000001e-05 2024-09-15T00:13:34.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584693+0000", 2024-09-15T00:13:34.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016221467999999999 2024-09-15T00:13:34.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584729+0000", 2024-09-15T00:13:34.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5462e-05 2024-09-15T00:13:34.734 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.734 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.734 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.734 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152500 3.17 e103/68 3:edff5adf:::rbd_data.113958cb0e55.000000000000000d:head v 103'461, mlcod=103'459)", 2024-09-15T00:13:34.734 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:34.734 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.025332220000003, 2024-09-15T00:13:34.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020217809, 2024-09-15T00:13:34.734 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.734 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.734 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.735 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:34.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:34.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641572+0000", 2024-09-15T00:13:34.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-15T00:13:34.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644961+0000", 2024-09-15T00:13:34.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033882610000000001 2024-09-15T00:13:34.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644979+0000", 2024-09-15T00:13:34.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8179e-05 2024-09-15T00:13:34.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644987+0000", 2024-09-15T00:13:34.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8860000000000004e-06 2024-09-15T00:13:34.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645044+0000", 2024-09-15T00:13:34.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7141000000000001e-05 2024-09-15T00:13:34.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645062+0000", 2024-09-15T00:13:34.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8258e-05 2024-09-15T00:13:34.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661749+0000", 2024-09-15T00:13:34.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016686553999999999 2024-09-15T00:13:34.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661788+0000", 2024-09-15T00:13:34.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9690000000000001e-05 2024-09-15T00:13:34.738 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.738 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.738 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.738 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153171 3.18 e103/69 3:1db186e8:::rbd_data.113958cb0e55.0000000000000002:head v 103'520, mlcod=103'518)", 2024-09-15T00:13:34.738 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:34.738 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 44.886847314000001, 2024-09-15T00:13:34.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022266838000000001, 2024-09-15T00:13:34.739 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.739 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.739 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.739 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:34.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:34.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780057+0000", 2024-09-15T00:13:34.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5880000000000001e-06 2024-09-15T00:13:34.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781273+0000", 2024-09-15T00:13:34.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001216087 2024-09-15T00:13:34.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781311+0000", 2024-09-15T00:13:34.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7656e-05 2024-09-15T00:13:34.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781325+0000", 2024-09-15T00:13:34.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4508999999999999e-05 2024-09-15T00:13:34.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781403+0000", 2024-09-15T00:13:34.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7927000000000001e-05 2024-09-15T00:13:34.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781422+0000", 2024-09-15T00:13:34.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8575000000000001e-05 2024-09-15T00:13:34.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802249+0000", 2024-09-15T00:13:34.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020827577 2024-09-15T00:13:34.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802322+0000", 2024-09-15T00:13:34.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2918999999999998e-05 2024-09-15T00:13:34.742 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.742 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153772 3.12 e103/68 3:4911a4f6:::rbd_data.113958cb0e55.0000000000000001:head v 103'490, mlcod=103'488)", 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 44.783976686999999, 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021181004, 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:34.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882927+0000", 2024-09-15T00:13:34.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.426e-06 2024-09-15T00:13:34.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883966+0000", 2024-09-15T00:13:34.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010384610000000001 2024-09-15T00:13:34.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883985+0000", 2024-09-15T00:13:34.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9276e-05 2024-09-15T00:13:34.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883994+0000", 2024-09-15T00:13:34.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9490000000000002e-06 2024-09-15T00:13:34.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884054+0000", 2024-09-15T00:13:34.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9419999999999997e-05 2024-09-15T00:13:34.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884073+0000", 2024-09-15T00:13:34.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9137e-05 2024-09-15T00:13:34.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904038+0000", 2024-09-15T00:13:34.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019965092 2024-09-15T00:13:34.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904107+0000", 2024-09-15T00:13:34.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9243000000000001e-05 2024-09-15T00:13:34.747 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.747 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.747 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.747 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163112 3.3 e106/68 3:c668f3d5:::rbd_data.11395d771336.0000000000000001:head v 106'489, mlcod=105'487)", 2024-09-15T00:13:34.747 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:34.747 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.202079417999997, 2024-09-15T00:13:34.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019317183000000002, 2024-09-15T00:13:34.747 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.748 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.748 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.748 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:34.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:34.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464824+0000", 2024-09-15T00:13:34.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1100000000000005e-07 2024-09-15T00:13:34.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466874+0000", 2024-09-15T00:13:34.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002049497 2024-09-15T00:13:34.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466884+0000", 2024-09-15T00:13:34.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0424999999999999e-05 2024-09-15T00:13:34.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466890+0000", 2024-09-15T00:13:34.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-15T00:13:34.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466944+0000", 2024-09-15T00:13:34.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4592999999999997e-05 2024-09-15T00:13:34.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466958+0000", 2024-09-15T00:13:34.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3993e-05 2024-09-15T00:13:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484068+0000", 2024-09-15T00:13:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017109994 2024-09-15T00:13:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484140+0000", 2024-09-15T00:13:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2224000000000005e-05 2024-09-15T00:13:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163119 3.0 e106/94 3:06118613:::rbd_data.11395d771336.0000000000000004:head v 106'549, mlcod=105'547)", 2024-09-15T00:13:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.190464214999999, 2024-09-15T00:13:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017175867000000001, 2024-09-15T00:13:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6700000000000002e-07 2024-09-15T00:13:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477786+0000", 2024-09-15T00:13:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013468149999999999 2024-09-15T00:13:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477797+0000", 2024-09-15T00:13:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0635000000000001e-05 2024-09-15T00:13:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477805+0000", 2024-09-15T00:13:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2989999999999997e-06 2024-09-15T00:13:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477859+0000", 2024-09-15T00:13:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3699000000000002e-05 2024-09-15T00:13:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477878+0000", 2024-09-15T00:13:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9335e-05 2024-09-15T00:13:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493560+0000", 2024-09-15T00:13:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015681748999999998 2024-09-15T00:13:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493614+0000", 2024-09-15T00:13:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4367999999999999e-05 2024-09-15T00:13:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.5.0:5005 3.17 e110/68 3:eb0359e8:::rbd_data.113964deed84.0000000000000251:35 v 110'537, mlcod=109'534)", 2024-09-15T00:13:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.530613842999998, 2024-09-15T00:13:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015446896999999999, 2024-09-15T00:13:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136290+0000", 2024-09-15T00:13:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7400000000000003e-07 2024-09-15T00:13:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136294+0000", 2024-09-15T00:13:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4989999999999996e-06 2024-09-15T00:13:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136304+0000", 2024-09-15T00:13:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0319999999999999e-05 2024-09-15T00:13:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136310+0000", 2024-09-15T00:13:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.981e-06 2024-09-15T00:13:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136350+0000", 2024-09-15T00:13:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0088e-05 2024-09-15T00:13:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136364+0000", 2024-09-15T00:13:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3954999999999999e-05 2024-09-15T00:13:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151681+0000", 2024-09-15T00:13:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015316969999999999 2024-09-15T00:13:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151736+0000", 2024-09-15T00:13:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4410000000000003e-05 2024-09-15T00:13:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1758 3.10 e110/68 3:08149147:::rbd_data.113964deed84.0000000000000055:35 v 110'594, mlcod=109'591)", 2024-09-15T00:13:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.526315617000002, 2024-09-15T00:13:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016278652000000001, 2024-09-15T00:13:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140599+0000", 2024-09-15T00:13:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1109e-05 2024-09-15T00:13:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140605+0000", 2024-09-15T00:13:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-15T00:13:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140612+0000", 2024-09-15T00:13:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6089999999999997e-06 2024-09-15T00:13:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140656+0000", 2024-09-15T00:13:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3754999999999997e-05 2024-09-15T00:13:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140674+0000", 2024-09-15T00:13:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7640999999999999e-05 2024-09-15T00:13:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156823+0000", 2024-09-15T00:13:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016149047999999999 2024-09-15T00:13:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156866+0000", 2024-09-15T00:13:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3205999999999999e-05 2024-09-15T00:13:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.4.0:5722 3.e e110/95 3:74a58daf:::rbd_data.113964deed84.00000000000000c1:35 v 110'1287, mlcod=109'1284)", 2024-09-15T00:13:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.517241261000002, 2024-09-15T00:13:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01830555, 2024-09-15T00:13:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149662+0000", 2024-09-15T00:13:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:13:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149666+0000", 2024-09-15T00:13:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2220000000000004e-06 2024-09-15T00:13:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149672+0000", 2024-09-15T00:13:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3410000000000002e-06 2024-09-15T00:13:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149676+0000", 2024-09-15T00:13:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.566e-06 2024-09-15T00:13:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149709+0000", 2024-09-15T00:13:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2403000000000001e-05 2024-09-15T00:13:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149719+0000", 2024-09-15T00:13:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0608000000000001e-05 2024-09-15T00:13:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167944+0000", 2024-09-15T00:13:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018225183999999998 2024-09-15T00:13:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167967+0000", 2024-09-15T00:13:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2705e-05 2024-09-15T00:13:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:34.770 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:34.771 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:34.771 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:34.769+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:34.872 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:34.870+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:34.876 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:34.876 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:34.877 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:34.877 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:34.886 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:34.973 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:34.971+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:35.015 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.5 to 1 (10000) 2024-09-15T00:13:35.036 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-15T00:13:35.039 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:35.039 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:35.039 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:35.039 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:35.039 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:35.052 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:35.093 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:35.092+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:35.157 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:13:35.175 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:35.173+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:35.216 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:35.216 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:35.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:35.216 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:35.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.217 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:35.217 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:35.217 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 129.034038872, 2024-09-15T00:13:35.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:13:35.217 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.217 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.217 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:35.217 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.217 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.217 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.217 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:13:35.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.218 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:35.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:35.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:13:35.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:13:35.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:13:35.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:13:35.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:13:35.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:13:35.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:13:35.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:13:35.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:13:35.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:13:35.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:13:35.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:13:35.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:35.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:13:35.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:13:35.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:35.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:13:35.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:13:35.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:35.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:13:35.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:13:35.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:35.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:13:35.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:13:35.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:13:35.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:13:35.222 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.222 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.222 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.223 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:13:35.223 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:35.223 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 119.30419769300001, 2024-09-15T00:13:35.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:13:35.223 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.223 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.223 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:35.223 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.223 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.223 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.223 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:13:35.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.223 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:35.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:35.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:13:35.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:13:35.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:13:35.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:13:35.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:13:35.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:13:35.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:13:35.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:13:35.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:13:35.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:13:35.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:13:35.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:13:35.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:35.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:13:35.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:13:35.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:35.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:13:35.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:13:35.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:35.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:13:35.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:13:35.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:35.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:13:35.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:13:35.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:13:35.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:13:35.228 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.228 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.228 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.228 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:13:35.229 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:35.229 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 119.29723933, 2024-09-15T00:13:35.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:13:35.229 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.229 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.229 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.229 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:35.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:35.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:13:35.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:13:35.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:13:35.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:13:35.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:13:35.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:13:35.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:13:35.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:13:35.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:13:35.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:13:35.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:13:35.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:13:35.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:13:35.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:13:35.232 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.232 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.232 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.233 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:35.233 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:35.233 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 110.550050493, 2024-09-15T00:13:35.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:13:35.233 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.233 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.233 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:35.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.233 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:13:35.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.233 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:35.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:35.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:13:35.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:13:35.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:13:35.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:13:35.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:13:35.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:13:35.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:13:35.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:13:35.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:13:35.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:13:35.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:13:35.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:13:35.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:13:35.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:13:35.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:13:35.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:35.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:13:35.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:13:35.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:35.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:13:35.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:13:35.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:35.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:13:35.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:13:35.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:13:35.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:13:35.238 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.238 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.238 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.238 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97493 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:35.239 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:35.239 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.360467772999996, 2024-09-15T00:13:35.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033259141999999998, 2024-09-15T00:13:35.239 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.239 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.239 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.239 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.239 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.239 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.239 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97493 2024-09-15T00:13:35.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.239 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:35.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:35.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:35.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-15T00:13:35.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853401+0000", 2024-09-15T00:13:35.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:13:35.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853407+0000", 2024-09-15T00:13:35.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-15T00:13:35.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853411+0000", 2024-09-15T00:13:35.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.481e-06 2024-09-15T00:13:35.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886544+0000", 2024-09-15T00:13:35.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033132861999999999 2024-09-15T00:13:35.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886587+0000", 2024-09-15T00:13:35.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2942000000000003e-05 2024-09-15T00:13:35.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886655+0000", 2024-09-15T00:13:35.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7711000000000003e-05 2024-09-15T00:13:35.243 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.243 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.243 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.243 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97494 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:35.243 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:35.243 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.36044665, 2024-09-15T00:13:35.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033392390000000001, 2024-09-15T00:13:35.243 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.243 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.243 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.243 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.243 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.243 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.244 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97494 2024-09-15T00:13:35.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.244 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:35.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:35.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:35.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-15T00:13:35.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853419+0000", 2024-09-15T00:13:35.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.917e-06 2024-09-15T00:13:35.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853425+0000", 2024-09-15T00:13:35.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:13:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853433+0000", 2024-09-15T00:13:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8120000000000008e-06 2024-09-15T00:13:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886680+0000", 2024-09-15T00:13:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033247207000000001 2024-09-15T00:13:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886726+0000", 2024-09-15T00:13:35.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5902000000000002e-05 2024-09-15T00:13:35.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886809+0000", 2024-09-15T00:13:35.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3475999999999998e-05 2024-09-15T00:13:35.247 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.247 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.247 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.247 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97495 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:35.247 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:35.247 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.360425696999997, 2024-09-15T00:13:35.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033640503000000002, 2024-09-15T00:13:35.248 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.248 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.248 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.248 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.248 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.248 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.248 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97495 2024-09-15T00:13:35.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.248 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:35.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:35.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:35.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:13:35.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853440+0000", 2024-09-15T00:13:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.804e-06 2024-09-15T00:13:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853446+0000", 2024-09-15T00:13:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5559999999999998e-06 2024-09-15T00:13:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853450+0000", 2024-09-15T00:13:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9519999999999996e-06 2024-09-15T00:13:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886904+0000", 2024-09-15T00:13:35.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033454207999999999 2024-09-15T00:13:35.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887007+0000", 2024-09-15T00:13:35.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102904 2024-09-15T00:13:35.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887078+0000", 2024-09-15T00:13:35.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1719999999999998e-05 2024-09-15T00:13:35.251 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.251 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.252 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.252 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97496 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:35.252 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:35.252 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.360408598999996, 2024-09-15T00:13:35.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033802431000000001, 2024-09-15T00:13:35.252 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.252 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.252 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.252 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.252 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.252 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.252 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97496 2024-09-15T00:13:35.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.253 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:35.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:35.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:35.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.53e-07 2024-09-15T00:13:35.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853462+0000", 2024-09-15T00:13:35.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-15T00:13:35.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853468+0000", 2024-09-15T00:13:35.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3609999999999997e-06 2024-09-15T00:13:35.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853473+0000", 2024-09-15T00:13:35.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-15T00:13:35.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887111+0000", 2024-09-15T00:13:35.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033638376999999997 2024-09-15T00:13:35.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887178+0000", 2024-09-15T00:13:35.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6877000000000004e-05 2024-09-15T00:13:35.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887257+0000", 2024-09-15T00:13:35.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9616999999999993e-05 2024-09-15T00:13:35.256 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.256 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.256 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.256 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97587 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:35.256 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:35.256 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.360385364999999, 2024-09-15T00:13:35.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033973642999999998, 2024-09-15T00:13:35.256 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.256 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.256 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.257 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.257 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.257 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.257 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97587 2024-09-15T00:13:35.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.257 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:35.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:35.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853479+0000", 2024-09-15T00:13:35.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:13:35.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853481+0000", 2024-09-15T00:13:35.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.136e-06 2024-09-15T00:13:35.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853487+0000", 2024-09-15T00:13:35.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2439999999999998e-06 2024-09-15T00:13:35.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853492+0000", 2024-09-15T00:13:35.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:13:35.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887284+0000", 2024-09-15T00:13:35.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033791827000000003 2024-09-15T00:13:35.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887361+0000", 2024-09-15T00:13:35.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6983000000000005e-05 2024-09-15T00:13:35.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887452+0000", 2024-09-15T00:13:35.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1236999999999994e-05 2024-09-15T00:13:35.260 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.260 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.260 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.261 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97588 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:35.261 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:35.261 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.360365208000005, 2024-09-15T00:13:35.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034134617999999999, 2024-09-15T00:13:35.261 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.261 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.261 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.261 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.261 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.261 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.261 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97588 2024-09-15T00:13:35.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.261 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853499+0000", 2024-09-15T00:13:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:13:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853501+0000", 2024-09-15T00:13:35.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.26e-06 2024-09-15T00:13:35.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853507+0000", 2024-09-15T00:13:35.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2899999999999999e-06 2024-09-15T00:13:35.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853512+0000", 2024-09-15T00:13:35.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8330000000000001e-06 2024-09-15T00:13:35.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887495+0000", 2024-09-15T00:13:35.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033982529999999997 2024-09-15T00:13:35.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887548+0000", 2024-09-15T00:13:35.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3445000000000001e-05 2024-09-15T00:13:35.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887633+0000", 2024-09-15T00:13:35.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4890999999999997e-05 2024-09-15T00:13:35.265 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.265 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.265 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.265 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126866 3.6 3.973c9fa6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.265 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:35.265 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.257043263, 2024-09-15T00:13:35.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153301000000003, 2024-09-15T00:13:35.265 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.265 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.265 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:35.266 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.266 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.266 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.266 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126866 2024-09-15T00:13:35.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.266 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:35.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:35.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956821+0000", 2024-09-15T00:13:35.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.91e-07 2024-09-15T00:13:35.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956822+0000", 2024-09-15T00:13:35.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.705e-06 2024-09-15T00:13:35.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956828+0000", 2024-09-15T00:13:35.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6760000000000004e-06 2024-09-15T00:13:35.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956833+0000", 2024-09-15T00:13:35.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5379999999999996e-06 2024-09-15T00:13:35.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989970+0000", 2024-09-15T00:13:35.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033137560000000003 2024-09-15T00:13:35.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989974+0000", 2024-09-15T00:13:35.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-15T00:13:35.269 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.269 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.269 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.269 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126952 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.269 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:35.269 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.256900591000004, 2024-09-15T00:13:35.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033156066999999997, 2024-09-15T00:13:35.269 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.269 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.270 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:35.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.270 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126952 2024-09-15T00:13:35.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.270 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:35.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:35.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:35.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:13:35.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956965+0000", 2024-09-15T00:13:35.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3379999999999999e-06 2024-09-15T00:13:35.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956968+0000", 2024-09-15T00:13:35.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0419999999999998e-06 2024-09-15T00:13:35.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956970+0000", 2024-09-15T00:13:35.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4899999999999999e-06 2024-09-15T00:13:35.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990113+0000", 2024-09-15T00:13:35.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033142642 2024-09-15T00:13:35.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990119+0000", 2024-09-15T00:13:35.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2979999999999997e-06 2024-09-15T00:13:35.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.273 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.273 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126954 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.273 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:35.273 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.256855095999995, 2024-09-15T00:13:35.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033158449, 2024-09-15T00:13:35.273 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.273 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.274 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:35.274 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.274 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.274 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.274 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126954 2024-09-15T00:13:35.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.274 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:35.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:35.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:35.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-15T00:13:35.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957010+0000", 2024-09-15T00:13:35.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3659999999999999e-06 2024-09-15T00:13:35.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957013+0000", 2024-09-15T00:13:35.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1140000000000002e-06 2024-09-15T00:13:35.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957016+0000", 2024-09-15T00:13:35.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1209999999999998e-06 2024-09-15T00:13:35.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990163+0000", 2024-09-15T00:13:35.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033146451 2024-09-15T00:13:35.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990167+0000", 2024-09-15T00:13:35.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0579999999999999e-06 2024-09-15T00:13:35.277 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.277 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.277 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.277 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126955 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.277 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.256843881999998, 2024-09-15T00:13:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033160617000000003, 2024-09-15T00:13:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126955 2024-09-15T00:13:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-15T00:13:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957021+0000", 2024-09-15T00:13:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-15T00:13:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957024+0000", 2024-09-15T00:13:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1599999999999998e-06 2024-09-15T00:13:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957027+0000", 2024-09-15T00:13:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000001e-06 2024-09-15T00:13:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990177+0000", 2024-09-15T00:13:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033149741000000003 2024-09-15T00:13:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990180+0000", 2024-09-15T00:13:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4910000000000001e-06 2024-09-15T00:13:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126956 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.256833760999996, 2024-09-15T00:13:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033163633999999997, 2024-09-15T00:13:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126956 2024-09-15T00:13:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957031+0000", 2024-09-15T00:13:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-06 2024-09-15T00:13:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957034+0000", 2024-09-15T00:13:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9960000000000002e-06 2024-09-15T00:13:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957037+0000", 2024-09-15T00:13:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6419999999999999e-06 2024-09-15T00:13:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990190+0000", 2024-09-15T00:13:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153152999999998 2024-09-15T00:13:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990193+0000", 2024-09-15T00:13:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6069999999999999e-06 2024-09-15T00:13:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126957 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.256824111, 2024-09-15T00:13:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033166442999999997, 2024-09-15T00:13:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126957 2024-09-15T00:13:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957040+0000", 2024-09-15T00:13:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:13:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957041+0000", 2024-09-15T00:13:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0440000000000001e-06 2024-09-15T00:13:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957044+0000", 2024-09-15T00:13:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0170000000000001e-06 2024-09-15T00:13:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957048+0000", 2024-09-15T00:13:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0319999999999997e-06 2024-09-15T00:13:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990202+0000", 2024-09-15T00:13:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033154640999999999 2024-09-15T00:13:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990206+0000", 2024-09-15T00:13:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.507e-06 2024-09-15T00:13:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126958 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.256812881000002, 2024-09-15T00:13:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033167613999999998, 2024-09-15T00:13:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126958 2024-09-15T00:13:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:13:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957053+0000", 2024-09-15T00:13:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.756e-06 2024-09-15T00:13:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957059+0000", 2024-09-15T00:13:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2770000000000002e-06 2024-09-15T00:13:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957063+0000", 2024-09-15T00:13:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2459999999999997e-06 2024-09-15T00:13:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990215+0000", 2024-09-15T00:13:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033151509000000003 2024-09-15T00:13:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990218+0000", 2024-09-15T00:13:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5099999999999999e-06 2024-09-15T00:13:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126959 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.256793877000007, 2024-09-15T00:13:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033161673000000003, 2024-09-15T00:13:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126959 2024-09-15T00:13:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:13:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957072+0000", 2024-09-15T00:13:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2239999999999998e-06 2024-09-15T00:13:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957079+0000", 2024-09-15T00:13:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.19e-06 2024-09-15T00:13:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957083+0000", 2024-09-15T00:13:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:13:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990228+0000", 2024-09-15T00:13:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033144818 2024-09-15T00:13:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990231+0000", 2024-09-15T00:13:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:13:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126960 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.256775070000003, 2024-09-15T00:13:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033155166, 2024-09-15T00:13:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126960 2024-09-15T00:13:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:13:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957091+0000", 2024-09-15T00:13:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:13:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957097+0000", 2024-09-15T00:13:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6050000000000003e-06 2024-09-15T00:13:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957102+0000", 2024-09-15T00:13:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5539999999999999e-06 2024-09-15T00:13:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990240+0000", 2024-09-15T00:13:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033138289000000001 2024-09-15T00:13:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990244+0000", 2024-09-15T00:13:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4740000000000001e-06 2024-09-15T00:13:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:174575 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138)", 2024-09-15T00:13:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:26.073788+0000", 2024-09-15T00:13:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.1400757549999998, 2024-09-15T00:13:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.95024539600000002, 2024-09-15T00:13:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 174575 2024-09-15T00:13:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073788+0000", 2024-09-15T00:13:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073788+0000", 2024-09-15T00:13:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073789+0000", 2024-09-15T00:13:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3999999999999999e-06 2024-09-15T00:13:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073796+0000", 2024-09-15T00:13:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5529999999999998e-06 2024-09-15T00:13:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073807+0000", 2024-09-15T00:13:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0825000000000001e-05 2024-09-15T00:13:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073812+0000", 2024-09-15T00:13:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6690000000000003e-06 2024-09-15T00:13:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073870+0000", 2024-09-15T00:13:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7241999999999998e-05 2024-09-15T00:13:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073878+0000", 2024-09-15T00:13:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0760000000000003e-06 2024-09-15T00:13:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:27.023756+0000", 2024-09-15T00:13:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94987846899999995 2024-09-15T00:13:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:27.023891+0000", 2024-09-15T00:13:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000134785 2024-09-15T00:13:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:27.024033+0000", 2024-09-15T00:13:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014237700000000001 2024-09-15T00:13:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.308 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:35.310 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:35.310 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:35.308+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:35.410 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:35.409+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:35.457 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:35.457 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:35.457 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:35.457 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:35.465 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:35.550 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:35.548+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:35.610 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:35.610 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:35.610 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:35.612 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:35.610+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:35.618 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:35.713 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:35.711+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:35.764 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:13:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 107.518697745, 2024-09-15T00:13:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:13:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:13:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:13:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:13:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:13:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:13:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:13:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:13:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:13:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:13:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:13:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:13:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:13:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:13:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:13:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:13:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:13:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:13:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:13:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:13:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:13:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:13:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:13:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58878 3.8 3:12456482:::rbd_data.1139766160da.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 102.472392161, 2024-09-15T00:13:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061956199000000003, 2024-09-15T00:13:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58878 2024-09-15T00:13:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289151+0000", 2024-09-15T00:13:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:13:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289473+0000", 2024-09-15T00:13:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032183800000000002 2024-09-15T00:13:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289487+0000", 2024-09-15T00:13:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-05 2024-09-15T00:13:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289493+0000", 2024-09-15T00:13:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4489999999999996e-06 2024-09-15T00:13:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289542+0000", 2024-09-15T00:13:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8844999999999997e-05 2024-09-15T00:13:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289642+0000", 2024-09-15T00:13:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9607999999999999e-05 2024-09-15T00:13:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:13:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.293908+0000", 2024-09-15T00:13:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659350000000002 2024-09-15T00:13:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351022+0000", 2024-09-15T00:13:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057114535000000001 2024-09-15T00:13:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351033+0000", 2024-09-15T00:13:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0263e-05 2024-09-15T00:13:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351070+0000", 2024-09-15T00:13:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7666000000000002e-05 2024-09-15T00:13:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351106+0000", 2024-09-15T00:13:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999997e-05 2024-09-15T00:13:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58879 3.7 3:e169894d:::rbd_data.1139766160da.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 102.470040673, 2024-09-15T00:13:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043716294000000003, 2024-09-15T00:13:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58879 2024-09-15T00:13:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291502+0000", 2024-09-15T00:13:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-15T00:13:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291664+0000", 2024-09-15T00:13:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016216599999999999 2024-09-15T00:13:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291674+0000", 2024-09-15T00:13:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0276e-05 2024-09-15T00:13:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291680+0000", 2024-09-15T00:13:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2580000000000003e-06 2024-09-15T00:13:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291732+0000", 2024-09-15T00:13:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2114999999999998e-05 2024-09-15T00:13:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291866+0000", 2024-09-15T00:13:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013416900000000001 2024-09-15T00:13:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:13:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.296426+0000", 2024-09-15T00:13:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045595779999999999 2024-09-15T00:13:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335075+0000", 2024-09-15T00:13:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038649347000000001 2024-09-15T00:13:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335097+0000", 2024-09-15T00:13:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2565000000000002e-05 2024-09-15T00:13:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335161+0000", 2024-09-15T00:13:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3928000000000002e-05 2024-09-15T00:13:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335217+0000", 2024-09-15T00:13:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6042000000000003e-05 2024-09-15T00:13:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58880 3.8 3:13d91cfd:::rbd_data.1139766160da.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 102.466648566, 2024-09-15T00:13:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056331463999999998, 2024-09-15T00:13:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58880 2024-09-15T00:13:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294896+0000", 2024-09-15T00:13:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.356e-06 2024-09-15T00:13:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295288+0000", 2024-09-15T00:13:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039223200000000001 2024-09-15T00:13:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295304+0000", 2024-09-15T00:13:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5937e-05 2024-09-15T00:13:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295312+0000", 2024-09-15T00:13:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1489999999999994e-06 2024-09-15T00:13:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333162+0000", 2024-09-15T00:13:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037849840000000003 2024-09-15T00:13:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333269+0000", 2024-09-15T00:13:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010749800000000001 2024-09-15T00:13:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:13:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.336198+0000", 2024-09-15T00:13:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029284739999999999 2024-09-15T00:13:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351137+0000", 2024-09-15T00:13:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014939103 2024-09-15T00:13:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351150+0000", 2024-09-15T00:13:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3542e-05 2024-09-15T00:13:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351191+0000", 2024-09-15T00:13:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0791000000000003e-05 2024-09-15T00:13:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351225+0000", 2024-09-15T00:13:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3541999999999998e-05 2024-09-15T00:13:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58878 3.8 e56/23 ondisk, result = 0)", 2024-09-15T00:13:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 102.460118783, 2024-09-15T00:13:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049689096000000002, 2024-09-15T00:13:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301425+0000", 2024-09-15T00:13:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.782e-06 2024-09-15T00:13:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301433+0000", 2024-09-15T00:13:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689999999999994e-06 2024-09-15T00:13:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301449+0000", 2024-09-15T00:13:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5767e-05 2024-09-15T00:13:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301458+0000", 2024-09-15T00:13:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:13:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351009+0000", 2024-09-15T00:13:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049551523 2024-09-15T00:13:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351017+0000", 2024-09-15T00:13:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8359999999999992e-06 2024-09-15T00:13:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351112+0000", 2024-09-15T00:13:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5173999999999994e-05 2024-09-15T00:13:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127004 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 6881280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.793213319000003, 2024-09-15T00:13:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735720000000003, 2024-09-15T00:13:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127004 2024-09-15T00:13:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:13:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968331+0000", 2024-09-15T00:13:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.849e-06 2024-09-15T00:13:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968334+0000", 2024-09-15T00:13:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7689999999999998e-06 2024-09-15T00:13:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968338+0000", 2024-09-15T00:13:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2490000000000001e-06 2024-09-15T00:13:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008987+0000", 2024-09-15T00:13:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040649117999999998 2024-09-15T00:13:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009027+0000", 2024-09-15T00:13:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0357e-05 2024-09-15T00:13:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009064+0000", 2024-09-15T00:13:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7036000000000003e-05 2024-09-15T00:13:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127005 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7004160~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.793200767000002, 2024-09-15T00:13:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040824358999999998, 2024-09-15T00:13:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127005 2024-09-15T00:13:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-15T00:13:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968343+0000", 2024-09-15T00:13:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3430000000000001e-06 2024-09-15T00:13:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968349+0000", 2024-09-15T00:13:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3080000000000004e-06 2024-09-15T00:13:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968355+0000", 2024-09-15T00:13:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9499999999999998e-06 2024-09-15T00:13:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009091+0000", 2024-09-15T00:13:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735645000000001 2024-09-15T00:13:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009130+0000", 2024-09-15T00:13:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8951e-05 2024-09-15T00:13:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009165+0000", 2024-09-15T00:13:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5926000000000001e-05 2024-09-15T00:13:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127006 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7127040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.793183709000004, 2024-09-15T00:13:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040912783000000001, 2024-09-15T00:13:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127006 2024-09-15T00:13:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968359+0000", 2024-09-15T00:13:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0419999999999999e-06 2024-09-15T00:13:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968363+0000", 2024-09-15T00:13:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5999999999999998e-06 2024-09-15T00:13:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968366+0000", 2024-09-15T00:13:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5179999999999999e-06 2024-09-15T00:13:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009196+0000", 2024-09-15T00:13:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040830313 2024-09-15T00:13:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009234+0000", 2024-09-15T00:13:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8562000000000001e-05 2024-09-15T00:13:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009271+0000", 2024-09-15T00:13:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6538000000000002e-05 2024-09-15T00:13:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127007 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7249920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.793173510000003, 2024-09-15T00:13:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041019218000000003, 2024-09-15T00:13:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127007 2024-09-15T00:13:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968369+0000", 2024-09-15T00:13:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-15T00:13:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968370+0000", 2024-09-15T00:13:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.012e-06 2024-09-15T00:13:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968374+0000", 2024-09-15T00:13:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3839999999999999e-06 2024-09-15T00:13:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968376+0000", 2024-09-15T00:13:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5339999999999998e-06 2024-09-15T00:13:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009292+0000", 2024-09-15T00:13:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040915092 2024-09-15T00:13:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009333+0000", 2024-09-15T00:13:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1105e-05 2024-09-15T00:13:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009388+0000", 2024-09-15T00:13:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4868999999999999e-05 2024-09-15T00:13:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127008 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7364608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.793162389000003, 2024-09-15T00:13:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041115290999999998, 2024-09-15T00:13:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127008 2024-09-15T00:13:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968380+0000", 2024-09-15T00:13:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3999999999999998e-07 2024-09-15T00:13:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968381+0000", 2024-09-15T00:13:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8700000000000004e-07 2024-09-15T00:13:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968385+0000", 2024-09-15T00:13:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9260000000000002e-06 2024-09-15T00:13:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968389+0000", 2024-09-15T00:13:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.685e-06 2024-09-15T00:13:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009419+0000", 2024-09-15T00:13:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041029852999999998 2024-09-15T00:13:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009458+0000", 2024-09-15T00:13:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8538000000000003e-05 2024-09-15T00:13:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009495+0000", 2024-09-15T00:13:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7061999999999999e-05 2024-09-15T00:13:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127009 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7487488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.793128272000004, 2024-09-15T00:13:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041233063, 2024-09-15T00:13:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127009 2024-09-15T00:13:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7199999999999999e-07 2024-09-15T00:13:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968417+0000", 2024-09-15T00:13:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3810000000000002e-06 2024-09-15T00:13:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968423+0000", 2024-09-15T00:13:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8820000000000003e-06 2024-09-15T00:13:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968433+0000", 2024-09-15T00:13:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0005e-05 2024-09-15T00:13:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009527+0000", 2024-09-15T00:13:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041093750999999998 2024-09-15T00:13:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009601+0000", 2024-09-15T00:13:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3979000000000004e-05 2024-09-15T00:13:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009647+0000", 2024-09-15T00:13:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5493e-05 2024-09-15T00:13:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127010 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7610368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.793099268000006, 2024-09-15T00:13:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041325924999999999, 2024-09-15T00:13:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127010 2024-09-15T00:13:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:13:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968445+0000", 2024-09-15T00:13:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:13:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968452+0000", 2024-09-15T00:13:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-15T00:13:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968456+0000", 2024-09-15T00:13:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5210000000000004e-06 2024-09-15T00:13:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009664+0000", 2024-09-15T00:13:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041207404000000003 2024-09-15T00:13:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009723+0000", 2024-09-15T00:13:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9036000000000002e-05 2024-09-15T00:13:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009768+0000", 2024-09-15T00:13:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5578000000000003e-05 2024-09-15T00:13:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127011 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7733248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.793081032999993, 2024-09-15T00:13:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041482441000000002, 2024-09-15T00:13:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127011 2024-09-15T00:13:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-15T00:13:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968466+0000", 2024-09-15T00:13:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:13:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968470+0000", 2024-09-15T00:13:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6469999999999997e-06 2024-09-15T00:13:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968473+0000", 2024-09-15T00:13:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.977e-06 2024-09-15T00:13:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009800+0000", 2024-09-15T00:13:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041326918999999997 2024-09-15T00:13:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009869+0000", 2024-09-15T00:13:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8935000000000003e-05 2024-09-15T00:13:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009943+0000", 2024-09-15T00:13:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4188000000000002e-05 2024-09-15T00:13:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127012 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7856128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.793065530999996, 2024-09-15T00:13:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041640756000000001, 2024-09-15T00:13:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127012 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968477+0000", 2024-09-15T00:13:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968482+0000", 2024-09-15T00:13:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6180000000000003e-06 2024-09-15T00:13:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968486+0000", 2024-09-15T00:13:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.501e-06 2024-09-15T00:13:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968489+0000", 2024-09-15T00:13:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1530000000000001e-06 2024-09-15T00:13:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009983+0000", 2024-09-15T00:13:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041493980999999999 2024-09-15T00:13:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010051+0000", 2024-09-15T00:13:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8039000000000004e-05 2024-09-15T00:13:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010117+0000", 2024-09-15T00:13:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6254000000000006e-05 2024-09-15T00:13:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127013 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7979008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.793049710999995, 2024-09-15T00:13:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041763863999999998, 2024-09-15T00:13:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127013 2024-09-15T00:13:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969767+0000", 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001275006 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969772+0000", 2024-09-15T00:13:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4769999999999997e-06 2024-09-15T00:13:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969781+0000", 2024-09-15T00:13:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1959999999999997e-06 2024-09-15T00:13:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969788+0000", 2024-09-15T00:13:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6359999999999994e-06 2024-09-15T00:13:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010162+0000", 2024-09-15T00:13:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040373291999999998 2024-09-15T00:13:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010212+0000", 2024-09-15T00:13:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0543e-05 2024-09-15T00:13:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010256+0000", 2024-09-15T00:13:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3714000000000002e-05 2024-09-15T00:13:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127015 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8224768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.791710774999999, 2024-09-15T00:13:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040710186000000002, 2024-09-15T00:13:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127015 2024-09-15T00:13:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969832+0000", 2024-09-15T00:13:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:13:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969834+0000", 2024-09-15T00:13:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2919999999999998e-06 2024-09-15T00:13:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969843+0000", 2024-09-15T00:13:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0480000000000004e-06 2024-09-15T00:13:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969848+0000", 2024-09-15T00:13:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-15T00:13:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010457+0000", 2024-09-15T00:13:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040609054999999998 2024-09-15T00:13:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010501+0000", 2024-09-15T00:13:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3396000000000003e-05 2024-09-15T00:13:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010541+0000", 2024-09-15T00:13:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0599000000000002e-05 2024-09-15T00:13:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127016 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.791660148999995, 2024-09-15T00:13:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040766216000000001, 2024-09-15T00:13:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127016 2024-09-15T00:13:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:13:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969886+0000", 2024-09-15T00:13:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8449999999999996e-06 2024-09-15T00:13:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969897+0000", 2024-09-15T00:13:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-15T00:13:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969904+0000", 2024-09-15T00:13:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1169999999999998e-06 2024-09-15T00:13:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010561+0000", 2024-09-15T00:13:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040657470000000001 2024-09-15T00:13:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010607+0000", 2024-09-15T00:13:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6108e-05 2024-09-15T00:13:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010648+0000", 2024-09-15T00:13:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0479000000000003e-05 2024-09-15T00:13:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127694 3.9 3:9390b205:::rbd_data.11396481e1ec.00000000000000df:head [set-alloc-hint object_size 1048576 write_size 1048576,write 819200~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T00:13:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.536941032000001, 2024-09-15T00:13:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90376419399999997, 2024-09-15T00:13:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127694 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224602+0000", 2024-09-15T00:13:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5899999999999995e-07 2024-09-15T00:13:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224610+0000", 2024-09-15T00:13:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9130000000000003e-06 2024-09-15T00:13:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224620+0000", 2024-09-15T00:13:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0426e-05 2024-09-15T00:13:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224627+0000", 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4390000000000003e-06 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224675+0000", 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7636000000000003e-05 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224681+0000", 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1460000000000001e-06 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126388+0000", 2024-09-15T00:13:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90170724999999996 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126526+0000", 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013730599999999999 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126684+0000", 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000157935 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.127932+0000", 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012479240000000001 2024-09-15T00:13:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128269+0000", 2024-09-15T00:13:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033761099999999998 2024-09-15T00:13:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128324+0000", 2024-09-15T00:13:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5244999999999997e-05 2024-09-15T00:13:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128365+0000", 2024-09-15T00:13:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0503999999999997e-05 2024-09-15T00:13:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:134707 3.15 3:af1fbdb5:::rbd_data.113933a4f79a.000000000000001c:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T00:13:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.068973917999998, 2024-09-15T00:13:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057328626000000001, 2024-09-15T00:13:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 134707 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692569+0000", 2024-09-15T00:13:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.339e-06 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692574+0000", 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692584+0000", 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0363000000000001e-05 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692590+0000", 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.818e-06 2024-09-15T00:13:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692633+0000", 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.265e-05 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692718+0000", 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5279999999999997e-05 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.695940+0000", 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003222052 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749714+0000", 2024-09-15T00:13:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.053774126999999998 2024-09-15T00:13:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749788+0000", 2024-09-15T00:13:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3693000000000001e-05 2024-09-15T00:13:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749847+0000", 2024-09-15T00:13:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175999999999997e-05 2024-09-15T00:13:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749897+0000", 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9311999999999998e-05 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:134707 3.15 e86/55 ondisk, result = 0)", 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.060412568999993, 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048596066, 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701131+0000", 2024-09-15T00:13:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3549999999999999e-06 2024-09-15T00:13:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701139+0000", 2024-09-15T00:13:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1650000000000006e-06 2024-09-15T00:13:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701156+0000", 2024-09-15T00:13:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6730000000000001e-05 2024-09-15T00:13:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701167+0000", 2024-09-15T00:13:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1504e-05 2024-09-15T00:13:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749629+0000", 2024-09-15T00:13:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048461952000000003 2024-09-15T00:13:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749640+0000", 2024-09-15T00:13:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0874e-05 2024-09-15T00:13:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749725+0000", 2024-09-15T00:13:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5486000000000001e-05 2024-09-15T00:13:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:35.862 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:35.863 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:35.864 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:35.862+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:35.964 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:35.962+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:35.969 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:35.969 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:35.969 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:35.969 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:35.978 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:36.066 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:36.064+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:36.121 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:36.122 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:36.122 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:36.130 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:36.159 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.6 to 1 (10000) 2024-09-15T00:13:36.160 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is unset 2024-09-15T00:13:36.166 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:36.165+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:36.184 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T00:13:36.268 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:36.267+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:36.281 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:36.281 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:36.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:36.281 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:36.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.281 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:36.282 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:36.282 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 130.70671352700001, 2024-09-15T00:13:36.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:13:36.282 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.282 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.282 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:36.282 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.282 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.282 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.282 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:13:36.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.282 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:36.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:36.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:13:36.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:13:36.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:13:36.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:13:36.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:13:36.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:13:36.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:13:36.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:13:36.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:13:36.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:13:36.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:13:36.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:13:36.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:36.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:13:36.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:13:36.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:36.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:13:36.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:13:36.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:36.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:13:36.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:13:36.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:36.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:13:36.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:13:36.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:13:36.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:13:36.288 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.288 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.288 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.288 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:36.288 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:36.289 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 130.70667883499999, 2024-09-15T00:13:36.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:13:36.289 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.289 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.289 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:36.289 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.289 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.289 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.289 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:13:36.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.289 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.290 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.290 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:36.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.290 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.290 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:36.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.290 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:36.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:13:36.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:13:36.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:13:36.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:13:36.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:13:36.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:13:36.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:13:36.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:13:36.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:13:36.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:13:36.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:13:36.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:36.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:13:36.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:13:36.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:36.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:13:36.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:13:36.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:36.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:13:36.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:13:36.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:36.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:13:36.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:13:36.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:13:36.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:13:36.295 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.295 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.295 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.295 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.296 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:36.296 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.66600514700001, 2024-09-15T00:13:36.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:13:36.296 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.296 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.296 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.296 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.296 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.296 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.296 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:13:36.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.296 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:36.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:36.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:36.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:13:36.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:13:36.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:13:36.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:13:36.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:13:36.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:13:36.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:13:36.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:13:36.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:13:36.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:13:36.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:13:36.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:13:36.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:13:36.300 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.300 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.300 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.301 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.301 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:36.301 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.66598861600001, 2024-09-15T00:13:36.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:13:36.301 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.301 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.301 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.301 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.301 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.301 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.301 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:13:36.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.302 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:36.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:36.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:13:36.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:36.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:13:36.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:13:36.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:13:36.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:13:36.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:13:36.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:13:36.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:13:36.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:13:36.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:13:36.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:13:36.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:13:36.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:13:36.305 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.305 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.305 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.306 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.306 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:36.306 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.665971837, 2024-09-15T00:13:36.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:13:36.306 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.306 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.306 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.306 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.306 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.306 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.306 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:13:36.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.307 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:36.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:36.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:36.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:13:36.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:13:36.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:13:36.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:13:36.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:13:36.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:13:36.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:13:36.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:13:36.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:13:36.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:13:36.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:13:36.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:13:36.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:13:36.310 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.310 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.310 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.311 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.311 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:36.311 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.665954107, 2024-09-15T00:13:36.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:13:36.311 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.311 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.311 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.311 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:13:36.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.312 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:36.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:36.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:36.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:13:36.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:13:36.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:13:36.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:13:36.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:13:36.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:13:36.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:13:36.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:13:36.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:13:36.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:13:36.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:13:36.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:13:36.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:13:36.315 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.315 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.315 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.316 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.316 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:36.316 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.66593694300001, 2024-09-15T00:13:36.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:13:36.316 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.316 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.316 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.316 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.316 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.316 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.316 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:13:36.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.317 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:36.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:36.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:36.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:13:36.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:13:36.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:13:36.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:13:36.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:13:36.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:13:36.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:13:36.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:13:36.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:13:36.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:13:36.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:13:36.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:13:36.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:13:36.320 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.320 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.320 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.320 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.321 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:36.321 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.665915267, 2024-09-15T00:13:36.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:13:36.321 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.321 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.321 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.321 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.321 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.321 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.321 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:13:36.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.321 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:36.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:36.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:36.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:13:36.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:13:36.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:13:36.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:13:36.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:13:36.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:13:36.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:13:36.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:13:36.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:13:36.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:13:36.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:13:36.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:13:36.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:13:36.325 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.325 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.325 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.326 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.326 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:36.326 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.66586914299999, 2024-09-15T00:13:36.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:13:36.326 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.326 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.326 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.326 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.326 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.326 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.327 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:13:36.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.327 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:36.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:36.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:36.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:13:36.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:13:36.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:13:36.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:13:36.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:13:36.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:13:36.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:13:36.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:13:36.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:13:36.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:13:36.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:13:36.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:13:36.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:13:36.330 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.330 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.330 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.331 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.331 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:36.331 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.665839386, 2024-09-15T00:13:36.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:13:36.331 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.331 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.331 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.331 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.331 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.331 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.331 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:13:36.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.332 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:36.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:36.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:13:36.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:13:36.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:13:36.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:13:36.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:13:36.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:36.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:13:36.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:13:36.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:13:36.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:13:36.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:13:36.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:13:36.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:13:36.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:13:36.335 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.335 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.335 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.336 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.336 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:36.336 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.665813067, 2024-09-15T00:13:36.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:13:36.336 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.336 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.336 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.336 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.336 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.336 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.336 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:13:36.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.337 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:36.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:36.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:36.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:36.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:13:36.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:13:36.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:13:36.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:13:36.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:13:36.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:13:36.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:13:36.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:13:36.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:13:36.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:13:36.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:13:36.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:13:36.341 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.341 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.341 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.341 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.341 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:36.341 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.665777205, 2024-09-15T00:13:36.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:13:36.341 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.342 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.342 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.342 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.342 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.342 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.342 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:13:36.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.342 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:36.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:36.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:36.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:13:36.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:13:36.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:13:36.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:13:36.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:13:36.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:13:36.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:13:36.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:13:36.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:13:36.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:13:36.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:13:36.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:13:36.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:13:36.346 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.346 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.346 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.346 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.346 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:36.346 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.665755487, 2024-09-15T00:13:36.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:13:36.346 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.346 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.346 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.347 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:13:36.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.347 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:36.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:36.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:13:36.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:13:36.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:13:36.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:13:36.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:13:36.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:13:36.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:13:36.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:13:36.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:13:36.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:13:36.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:13:36.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:13:36.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:13:36.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:13:36.351 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.351 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.351 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.351 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.352 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:36.352 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.66555902, 2024-09-15T00:13:36.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:13:36.352 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.352 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.352 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.352 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:13:36.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.352 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:36.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:36.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:36.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:13:36.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:13:36.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:13:36.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:13:36.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:13:36.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:13:36.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:13:36.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:13:36.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:13:36.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:13:36.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:13:36.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:13:36.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:13:36.356 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.356 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.357 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.357 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.357 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:36.357 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.665533181, 2024-09-15T00:13:36.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:13:36.357 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.357 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.357 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.358 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.358 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.358 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.358 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:13:36.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.358 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:36.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:36.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:36.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:13:36.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:13:36.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:13:36.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:13:36.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:13:36.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:13:36.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:13:36.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:13:36.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:13:36.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:13:36.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:13:36.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:13:36.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:13:36.362 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.362 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.362 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.362 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.362 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:36.362 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.66547153499999, 2024-09-15T00:13:36.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:13:36.362 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.362 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.362 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.363 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.363 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.363 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.363 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:13:36.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.363 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:36.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:36.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:13:36.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:13:36.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:13:36.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:13:36.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:13:36.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:13:36.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:13:36.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:13:36.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:13:36.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:13:36.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:13:36.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:13:36.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:13:36.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:13:36.366 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.367 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.367 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.367 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.367 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:36.367 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.66543220699999, 2024-09-15T00:13:36.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:13:36.367 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.367 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.367 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.367 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.367 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.368 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.368 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:13:36.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.368 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:36.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:36.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:13:36.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:13:36.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:13:36.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:13:36.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:13:36.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:13:36.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:13:36.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:13:36.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:13:36.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:13:36.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:13:36.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:36.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:13:36.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:13:36.371 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.371 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.372 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.372 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.372 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:36.372 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.663706807, 2024-09-15T00:13:36.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:13:36.372 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.372 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.372 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.372 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.372 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.373 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.373 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:13:36.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.373 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:36.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:36.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:13:36.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:13:36.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:13:36.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:13:36.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:13:36.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:13:36.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:13:36.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:13:36.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:13:36.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:13:36.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:13:36.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:13:36.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:13:36.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:13:36.376 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.377 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.377 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.377 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:36.377 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:36.377 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 111.66368464599999, 2024-09-15T00:13:36.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:13:36.377 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.377 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.377 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:36.378 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.378 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.378 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.378 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:13:36.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.378 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:36.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:36.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:36.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:13:36.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:13:36.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:13:36.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:13:36.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:13:36.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:13:36.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:13:36.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:13:36.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:13:36.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:13:36.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:13:36.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:13:36.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:13:36.382 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.382 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.382 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.382 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:145034 3.6 3:618673df:::rbd_object_map.1139646a7d4e.000000000000002b:head [writefull 0~48] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T00:13:36.382 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:36.382 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.261431041999998, 2024-09-15T00:13:36.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0081079740000001, 2024-09-15T00:13:36.383 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:36.383 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:36.383 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:36.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:36.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:36.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:36.383 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 145034 2024-09-15T00:13:36.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.383 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:36.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:36.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:36.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:36.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:36.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:36.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:36.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017198+0000", 2024-09-15T00:13:36.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0100000000000004e-07 2024-09-15T00:13:36.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:36.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017202+0000", 2024-09-15T00:13:36.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:13:36.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:36.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017212+0000", 2024-09-15T00:13:36.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7189999999999996e-06 2024-09-15T00:13:36.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:36.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017219+0000", 2024-09-15T00:13:36.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7839999999999998e-06 2024-09-15T00:13:36.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017268+0000", 2024-09-15T00:13:36.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:13:36.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:13:36.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017277+0000", 2024-09-15T00:13:36.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1899999999999995e-06 2024-09-15T00:13:36.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:36.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019139+0000", 2024-09-15T00:13:36.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0018626129999999 2024-09-15T00:13:36.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:36.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019236+0000", 2024-09-15T00:13:36.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6513000000000002e-05 2024-09-15T00:13:36.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:36.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019318+0000", 2024-09-15T00:13:36.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1848000000000006e-05 2024-09-15T00:13:36.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:36.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.023151+0000", 2024-09-15T00:13:36.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038332169999999999 2024-09-15T00:13:36.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:36.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025217+0000", 2024-09-15T00:13:36.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020657610000000002 2024-09-15T00:13:36.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:36.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025267+0000", 2024-09-15T00:13:36.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0198000000000003e-05 2024-09-15T00:13:36.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:36.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:36.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:36.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025305+0000", 2024-09-15T00:13:36.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8065999999999998e-05 2024-09-15T00:13:36.390 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.390 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.390 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.390 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:36.390 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:36.390 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:36.392 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:36.392 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:36.390+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:36.492 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:36.491+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:36.496 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:36.496 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:36.496 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:36.496 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:36.506 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:36.513 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:13:36.594 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:36.592+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:36.617 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:36.633 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:36.633 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:36.649 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:36.650 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:36.665 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:36.665 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:36.681 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:36.681 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:36.694 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:36.693+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:36.697 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:36.697 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:36.713 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:36.714 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:36.721 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:36.721 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:36.730 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:36.730 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:36.731 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:36.746 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:36.746 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:36.799 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:36.797+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:36.833 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:36.834 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:36.843 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:36.896 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:36.895+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:36.945 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:36.945 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:36.954 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:36.997 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:36.996+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:37.057 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:37.057 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:37.066 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:37.099 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:37.097+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:37.164 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T00:13:37.171 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:37.171 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:37.181 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:37.182 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T00:13:37.225 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:37.224+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:37.290 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:37.291 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:37.299 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:37.341 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:37.339+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:37.403 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:37.401+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:37.443 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:37.443 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:37.452 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:37.504 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:37.503+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:37.557 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:37.557 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:37.566 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:37.604 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:37.603+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:37.669 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:37.669 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:37.677 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:37.706 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:37.704+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:37.781 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:37.781 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:37.790 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:37.806 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:37.805+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:37.893 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:37.893 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:37.902 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:37.946 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:37.945+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:38.006 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:38.006 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:38.007 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:38.005+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:38.014 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:38.108 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:38.106+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:38.159 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:38.159 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:38.166 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T00:13:38.168 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:38.186 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:13:38.188 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T00:13:38.188 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:13:38.209 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:38.207+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:38.292 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:38.292 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:38.302 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:38.345 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:38.343+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:38.411 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:38.409+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:38.451 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:38.451 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:38.460 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:38.529 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:13:38.531 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:13:38.532 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:38.528+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:38.543 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-15T00:13:38.318371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":968213732,"num_objects":1841,"num_object_clones":543,"num_object_copies":3682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1841,"num_whiteouts":935,"num_read":233703,"num_read_kb":1275638,"num_write":27604,"num_write_kb":587327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4595,"num_bytes_recovered":357152014,"num_keys_recovered":646,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22929,"ondisk_log_size":22929,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":424968,"kb_used_data":51816,"kb_used_omap":9,"kb_used_meta":372982,"kb_avail":565806072,"statfs":{"total":579820584960,"available":579385417728,"internally_reserved":0,"allocated":53059584,"data_stored":627088482,"data_compressed":12597086,"data_compressed_allocated":35569664,"data_compressed_original":616132701,"omap_allocated":9566,"internal_metadata":381934242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-106591566,"num_objects":-203,"num_object_clones":-79,"num_object_copies":-406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-203,"num_whiteouts":-62,"num_read":1634,"num_read_kb":1392,"num_write":235,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000576"},"pg_stats":[{"pgid":"3.1f","version":"146'573","reported_seq":6152,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.159383+0000","last_change":"2024-09-15T00:13:34.159383+0000","last_active":"2024-09-15T00:13:34.159383+0000","last_peered":"2024-09-15T00:13:34.159383+0000","last_clean":"2024-09-15T00:13:34.159383+0000","last_became_active":"2024-09-15T00:12:11.013266+0000","last_became_peered":"2024-09-15T00:12:11.013266+0000","last_unstale":"2024-09-15T00:13:34.159383+0000","last_undegraded":"2024-09-15T00:13:34.159383+0000","last_fullsized":"2024-09-15T00:13:34.159383+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:34:20.058729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016096719999999999,"stat_sum":{"num_bytes":16592310,"num_objects":53,"num_object_clones":15,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":30,"num_read":6709,"num_read_kb":26141,"num_write":660,"num_write_kb":18365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":5475412,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1e","version":"146'560","reported_seq":7325,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.258194+0000","last_change":"2024-09-15T00:13:34.159108+0000","last_active":"2024-09-15T00:13:34.258194+0000","last_peered":"2024-09-15T00:13:34.258194+0000","last_clean":"2024-09-15T00:13:34.258194+0000","last_became_active":"2024-09-15T00:11:19.598640+0000","last_became_peered":"2024-09-15T00:11:19.598640+0000","last_unstale":"2024-09-15T00:13:34.258194+0000","last_undegraded":"2024-09-15T00:13:34.258194+0000","last_fullsized":"2024-09-15T00:13:34.258194+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:48:21.806980+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.019446548000000001,"stat_sum":{"num_bytes":32680534,"num_objects":70,"num_object_clones":24,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":33,"num_read":7531,"num_read_kb":39376,"num_write":712,"num_write_kb":10308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1d","version":"146'670","reported_seq":4669,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.235572+0000","last_change":"2024-09-15T00:13:34.159096+0000","last_active":"2024-09-15T00:13:34.235572+0000","last_peered":"2024-09-15T00:13:34.235572+0000","last_clean":"2024-09-15T00:13:34.235572+0000","last_became_active":"2024-09-15T00:11:19.595954+0000","last_became_peered":"2024-09-15T00:11:19.595954+0000","last_unstale":"2024-09-15T00:13:34.235572+0000","last_undegraded":"2024-09-15T00:13:34.235572+0000","last_fullsized":"2024-09-15T00:13:34.235572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:33:06.414275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011511760000000001,"stat_sum":{"num_bytes":13742710,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":25,"num_read":5581,"num_read_kb":45089,"num_write":767,"num_write_kb":31484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1c","version":"146'608","reported_seq":5467,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.267457+0000","last_change":"2024-09-15T00:13:34.159674+0000","last_active":"2024-09-15T00:13:34.267457+0000","last_peered":"2024-09-15T00:13:34.267457+0000","last_clean":"2024-09-15T00:13:34.267457+0000","last_became_active":"2024-09-15T00:11:19.602583+0000","last_became_peered":"2024-09-15T00:11:19.602583+0000","last_unstale":"2024-09-15T00:13:34.267457+0000","last_undegraded":"2024-09-15T00:13:34.267457+0000","last_fullsized":"2024-09-15T00:13:34.267457+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:38:02.261378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015881549999999999,"stat_sum":{"num_bytes":17465295,"num_objects":51,"num_object_clones":16,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":25,"num_read":5721,"num_read_kb":35056,"num_write":748,"num_write_kb":10221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1b","version":"146'760","reported_seq":6604,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.157822+0000","last_change":"2024-09-15T00:13:34.158090+0000","last_active":"2024-09-15T00:13:34.157822+0000","last_peered":"2024-09-15T00:13:34.157822+0000","last_clean":"2024-09-15T00:13:34.157822+0000","last_became_active":"2024-09-15T00:12:44.038243+0000","last_became_peered":"2024-09-15T00:12:44.038243+0000","last_unstale":"2024-09-15T00:13:34.157822+0000","last_undegraded":"2024-09-15T00:13:34.157822+0000","last_fullsized":"2024-09-15T00:13:34.157822+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:00:48.671663+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010189524,"stat_sum":{"num_bytes":24979401,"num_objects":72,"num_object_clones":26,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":36,"num_read":7633,"num_read_kb":34391,"num_write":883,"num_write_kb":14645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":10954706,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1a","version":"146'658","reported_seq":6741,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.278629+0000","last_change":"2024-09-15T00:13:34.160722+0000","last_active":"2024-09-15T00:13:34.278629+0000","last_peered":"2024-09-15T00:13:34.278629+0000","last_clean":"2024-09-15T00:13:34.278629+0000","last_became_active":"2024-09-15T00:13:27.022226+0000","last_became_peered":"2024-09-15T00:13:27.022226+0000","last_unstale":"2024-09-15T00:13:34.278629+0000","last_undegraded":"2024-09-15T00:13:34.278629+0000","last_fullsized":"2024-09-15T00:13:34.278629+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:50:00.160894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048399400000000001,"stat_sum":{"num_bytes":58087005,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":7944,"num_read_kb":61770,"num_write":758,"num_write_kb":33353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":65965475,"num_keys_recovered":138,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.19","version":"145'618","reported_seq":5030,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.160602+0000","last_change":"2024-09-15T00:13:34.160822+0000","last_active":"2024-09-15T00:13:34.160602+0000","last_peered":"2024-09-15T00:13:34.160602+0000","last_clean":"2024-09-15T00:13:34.160602+0000","last_became_active":"2024-09-15T00:13:26.020430+0000","last_became_peered":"2024-09-15T00:13:26.020430+0000","last_unstale":"2024-09-15T00:13:34.160602+0000","last_undegraded":"2024-09-15T00:13:34.160602+0000","last_fullsized":"2024-09-15T00:13:34.160602+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:23:56.282108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057772900000000002,"stat_sum":{"num_bytes":30917179,"num_objects":60,"num_object_clones":18,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":29,"num_read":4671,"num_read_kb":42435,"num_write":680,"num_write_kb":9827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":6381633,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.18","version":"144'628","reported_seq":7803,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.140207+0000","last_change":"2024-09-15T00:13:34.140771+0000","last_active":"2024-09-15T00:13:34.140207+0000","last_peered":"2024-09-15T00:13:34.140207+0000","last_clean":"2024-09-15T00:13:34.140207+0000","last_became_active":"2024-09-15T00:12:12.020924+0000","last_became_peered":"2024-09-15T00:12:12.020924+0000","last_unstale":"2024-09-15T00:13:34.140207+0000","last_undegraded":"2024-09-15T00:13:34.140207+0000","last_fullsized":"2024-09-15T00:13:34.140207+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:16:24.930926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001283462,"stat_sum":{"num_bytes":22180626,"num_objects":66,"num_object_clones":21,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":31,"num_read":8037,"num_read_kb":40066,"num_write":738,"num_write_kb":15973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4986328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.17","version":"143'593","reported_seq":4811,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.159730+0000","last_change":"2024-09-15T00:13:34.161625+0000","last_active":"2024-09-15T00:13:34.159730+0000","last_peered":"2024-09-15T00:13:34.159730+0000","last_clean":"2024-09-15T00:13:34.159730+0000","last_became_active":"2024-09-15T00:12:11.013261+0000","last_became_peered":"2024-09-15T00:12:11.013261+0000","last_unstale":"2024-09-15T00:13:34.159730+0000","last_undegraded":"2024-09-15T00:13:34.159730+0000","last_fullsized":"2024-09-15T00:13:34.159730+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:46:35.268761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035127549999999998,"stat_sum":{"num_bytes":45142964,"num_objects":70,"num_object_clones":22,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":38,"num_read":5247,"num_read_kb":45406,"num_write":595,"num_write_kb":17982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":12460957,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.16","version":"146'972","reported_seq":7820,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.242587+0000","last_change":"2024-09-15T00:13:34.163590+0000","last_active":"2024-09-15T00:13:34.242587+0000","last_peered":"2024-09-15T00:13:34.242587+0000","last_clean":"2024-09-15T00:13:34.242587+0000","last_became_active":"2024-09-15T00:13:26.026214+0000","last_became_peered":"2024-09-15T00:13:26.026214+0000","last_unstale":"2024-09-15T00:13:34.242587+0000","last_undegraded":"2024-09-15T00:13:34.242587+0000","last_fullsized":"2024-09-15T00:13:34.242587+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":972,"log_dups_size":0,"ondisk_log_size":972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:55:58.681353+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0056989550000000003,"stat_sum":{"num_bytes":40253729,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":25,"num_read":8828,"num_read_kb":39209,"num_write":1229,"num_write_kb":20604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":20681466,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.15","version":"144'616","reported_seq":6654,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.148926+0000","last_change":"2024-09-15T00:13:34.149682+0000","last_active":"2024-09-15T00:13:34.148926+0000","last_peered":"2024-09-15T00:13:34.148926+0000","last_clean":"2024-09-15T00:13:34.148926+0000","last_became_active":"2024-09-15T00:11:53.010039+0000","last_became_peered":"2024-09-15T00:11:53.010039+0000","last_unstale":"2024-09-15T00:13:34.148926+0000","last_undegraded":"2024-09-15T00:13:34.148926+0000","last_fullsized":"2024-09-15T00:13:34.148926+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:17:00.605000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010058892,"stat_sum":{"num_bytes":37464658,"num_objects":53,"num_object_clones":15,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":28,"num_read":7561,"num_read_kb":45608,"num_write":714,"num_write_kb":14831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":9563692,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.14","version":"146'1001","reported_seq":7130,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.297495+0000","last_change":"2024-09-15T00:13:34.156892+0000","last_active":"2024-09-15T00:13:34.297495+0000","last_peered":"2024-09-15T00:13:34.297495+0000","last_clean":"2024-09-15T00:13:34.297495+0000","last_became_active":"2024-09-15T00:12:44.028859+0000","last_became_peered":"2024-09-15T00:12:44.028859+0000","last_unstale":"2024-09-15T00:13:34.297495+0000","last_undegraded":"2024-09-15T00:13:34.297495+0000","last_fullsized":"2024-09-15T00:13:34.297495+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:09:16.438377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080058300000000002,"stat_sum":{"num_bytes":32515177,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":36,"num_read":9634,"num_read_kb":58257,"num_write":1444,"num_write_kb":24998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":3041694,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":284,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.148940+0000","last_change":"2024-09-15T00:13:11.204550+0000","last_active":"2024-09-15T00:13:34.148940+0000","last_peered":"2024-09-15T00:13:34.148940+0000","last_clean":"2024-09-15T00:13:34.148940+0000","last_became_active":"2024-09-15T00:13:11.204211+0000","last_became_peered":"2024-09-15T00:13:11.204211+0000","last_unstale":"2024-09-15T00:13:34.148940+0000","last_undegraded":"2024-09-15T00:13:34.148940+0000","last_fullsized":"2024-09-15T00:13:34.148940+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:16:00.619195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"146'639","reported_seq":7372,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.161464+0000","last_change":"2024-09-15T00:13:34.161464+0000","last_active":"2024-09-15T00:13:34.161464+0000","last_peered":"2024-09-15T00:13:34.161464+0000","last_clean":"2024-09-15T00:13:34.161464+0000","last_became_active":"2024-09-15T00:13:28.029209+0000","last_became_peered":"2024-09-15T00:13:28.029209+0000","last_unstale":"2024-09-15T00:13:34.161464+0000","last_undegraded":"2024-09-15T00:13:34.161464+0000","last_fullsized":"2024-09-15T00:13:34.161464+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:40:20.934298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085075600000000004,"stat_sum":{"num_bytes":41566721,"num_objects":70,"num_object_clones":22,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":41,"num_read":6771,"num_read_kb":42479,"num_write":645,"num_write_kb":25730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":26144067,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":34,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.160450+0000","last_change":"2024-09-15T00:13:22.049830+0000","last_active":"2024-09-15T00:13:34.160450+0000","last_peered":"2024-09-15T00:13:34.160450+0000","last_clean":"2024-09-15T00:13:34.160450+0000","last_became_active":"2024-09-15T00:13:22.049357+0000","last_became_peered":"2024-09-15T00:13:22.049357+0000","last_unstale":"2024-09-15T00:13:34.160450+0000","last_undegraded":"2024-09-15T00:13:34.160450+0000","last_fullsized":"2024-09-15T00:13:34.160450+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:29:10.717532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"146'637","reported_seq":8689,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.295489+0000","last_change":"2024-09-15T00:13:34.148837+0000","last_active":"2024-09-15T00:13:34.295489+0000","last_peered":"2024-09-15T00:13:34.295489+0000","last_clean":"2024-09-15T00:13:34.295489+0000","last_became_active":"2024-09-15T00:12:10.010014+0000","last_became_peered":"2024-09-15T00:12:10.010014+0000","last_unstale":"2024-09-15T00:13:34.295489+0000","last_undegraded":"2024-09-15T00:13:34.295489+0000","last_fullsized":"2024-09-15T00:13:34.295489+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:25:34.159932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0092416619999999994,"stat_sum":{"num_bytes":17161677,"num_objects":69,"num_object_clones":22,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":35,"num_read":9902,"num_read_kb":57043,"num_write":672,"num_write_kb":17736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":11200702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":170,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.147189+0000","last_change":"2024-09-15T00:12:44.035622+0000","last_active":"2024-09-15T00:13:34.147189+0000","last_peered":"2024-09-15T00:13:34.147189+0000","last_clean":"2024-09-15T00:13:34.147189+0000","last_became_active":"2024-09-15T00:12:44.035383+0000","last_became_peered":"2024-09-15T00:12:44.035383+0000","last_unstale":"2024-09-15T00:13:34.147189+0000","last_undegraded":"2024-09-15T00:13:34.147189+0000","last_fullsized":"2024-09-15T00:13:34.147189+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:30:08.250776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"145'678","reported_seq":10826,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.160780+0000","last_change":"2024-09-15T00:13:34.160980+0000","last_active":"2024-09-15T00:13:34.160780+0000","last_peered":"2024-09-15T00:13:34.160780+0000","last_clean":"2024-09-15T00:13:34.160780+0000","last_became_active":"2024-09-15T00:13:26.022113+0000","last_became_peered":"2024-09-15T00:13:26.022113+0000","last_unstale":"2024-09-15T00:13:34.160780+0000","last_undegraded":"2024-09-15T00:13:34.160780+0000","last_fullsized":"2024-09-15T00:13:34.160780+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:23:44.429772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066889599999999999,"stat_sum":{"num_bytes":55407800,"num_objects":64,"num_object_clones":20,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":12152,"num_read_kb":60045,"num_write":725,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":606,"num_bytes_recovered":40146959,"num_keys_recovered":159,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.160392+0000","last_change":"2024-09-15T00:13:22.048567+0000","last_active":"2024-09-15T00:13:34.160392+0000","last_peered":"2024-09-15T00:13:34.160392+0000","last_clean":"2024-09-15T00:13:34.160392+0000","last_became_active":"2024-09-15T00:13:22.048228+0000","last_became_peered":"2024-09-15T00:13:22.048228+0000","last_unstale":"2024-09-15T00:13:34.160392+0000","last_undegraded":"2024-09-15T00:13:34.160392+0000","last_fullsized":"2024-09-15T00:13:34.160392+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:27:06.941012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"144'488","reported_seq":2759,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.161663+0000","last_change":"2024-09-15T00:13:34.161837+0000","last_active":"2024-09-15T00:13:34.161663+0000","last_peered":"2024-09-15T00:13:34.161663+0000","last_clean":"2024-09-15T00:13:34.161663+0000","last_became_active":"2024-09-15T00:11:19.595633+0000","last_became_peered":"2024-09-15T00:11:19.595633+0000","last_unstale":"2024-09-15T00:13:34.161663+0000","last_undegraded":"2024-09-15T00:13:34.161663+0000","last_fullsized":"2024-09-15T00:13:34.161663+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:34:46.379301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003715838,"stat_sum":{"num_bytes":9685942,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":20,"num_read":3263,"num_read_kb":16863,"num_write":506,"num_write_kb":8972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.3","version":"147'664","reported_seq":7192,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-15T00:13:35.155095+0000","last_change":"2024-09-15T00:13:35.041177+0000","last_active":"2024-09-15T00:13:35.155095+0000","last_peered":"2024-09-15T00:13:35.155095+0000","last_clean":"2024-09-15T00:13:35.155095+0000","last_became_active":"2024-09-15T00:12:11.013156+0000","last_became_peered":"2024-09-15T00:12:11.013156+0000","last_unstale":"2024-09-15T00:13:35.155095+0000","last_undegraded":"2024-09-15T00:13:35.155095+0000","last_fullsized":"2024-09-15T00:13:35.155095+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:55:56.506145+0000","scrub_duration":0,"objects_trimmed":7,"snaptrim_duration":0.023321544999999999,"stat_sum":{"num_bytes":43719735,"num_objects":51,"num_object_clones":15,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":27,"num_read":6845,"num_read_kb":42866,"num_write":641,"num_write_kb":26766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":9273484,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"6"}]},{"pgid":"2.2","version":"21'2","reported_seq":289,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.157721+0000","last_change":"2024-09-15T00:13:22.052918+0000","last_active":"2024-09-15T00:13:34.157721+0000","last_peered":"2024-09-15T00:13:34.157721+0000","last_clean":"2024-09-15T00:13:34.157721+0000","last_became_active":"2024-09-15T00:13:22.049795+0000","last_became_peered":"2024-09-15T00:13:22.049795+0000","last_unstale":"2024-09-15T00:13:34.157721+0000","last_undegraded":"2024-09-15T00:13:34.157721+0000","last_fullsized":"2024-09-15T00:13:34.157721+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:50:48.495996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"131'608","reported_seq":6854,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.160865+0000","last_change":"2024-09-15T00:13:34.161058+0000","last_active":"2024-09-15T00:13:34.160865+0000","last_peered":"2024-09-15T00:13:34.160865+0000","last_clean":"2024-09-15T00:13:34.160865+0000","last_became_active":"2024-09-15T00:13:28.027146+0000","last_became_peered":"2024-09-15T00:13:28.027146+0000","last_unstale":"2024-09-15T00:13:34.160865+0000","last_undegraded":"2024-09-15T00:13:34.160865+0000","last_fullsized":"2024-09-15T00:13:34.160865+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:42:05.921781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064049799999999998,"stat_sum":{"num_bytes":18517623,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":29,"num_read":7136,"num_read_kb":33196,"num_write":672,"num_write_kb":14429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":27650205,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.1","version":"0'0","reported_seq":222,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.139346+0000","last_change":"2024-09-15T00:13:11.203976+0000","last_active":"2024-09-15T00:13:34.139346+0000","last_peered":"2024-09-15T00:13:34.139346+0000","last_clean":"2024-09-15T00:13:34.139346+0000","last_became_active":"2024-09-15T00:13:11.203652+0000","last_became_peered":"2024-09-15T00:13:11.203652+0000","last_unstale":"2024-09-15T00:13:34.139346+0000","last_undegraded":"2024-09-15T00:13:34.139346+0000","last_fullsized":"2024-09-15T00:13:34.139346+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:23:34.249491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"146'662","reported_seq":6442,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.157894+0000","last_change":"2024-09-15T00:13:34.158112+0000","last_active":"2024-09-15T00:13:34.157894+0000","last_peered":"2024-09-15T00:13:34.157894+0000","last_clean":"2024-09-15T00:13:34.157894+0000","last_became_active":"2024-09-15T00:12:44.035426+0000","last_became_peered":"2024-09-15T00:12:44.035426+0000","last_unstale":"2024-09-15T00:13:34.157894+0000","last_undegraded":"2024-09-15T00:13:34.157894+0000","last_fullsized":"2024-09-15T00:13:34.157894+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:07:16.407812+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010688496,"stat_sum":{"num_bytes":35149170,"num_objects":52,"num_object_clones":16,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":22,"num_read":6712,"num_read_kb":49044,"num_write":745,"num_write_kb":22019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":6058433,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.160360+0000","last_change":"2024-09-15T00:13:22.049592+0000","last_active":"2024-09-15T00:13:34.160360+0000","last_peered":"2024-09-15T00:13:34.160360+0000","last_clean":"2024-09-15T00:13:34.160360+0000","last_became_active":"2024-09-15T00:13:22.049283+0000","last_became_peered":"2024-09-15T00:13:22.049283+0000","last_unstale":"2024-09-15T00:13:34.160360+0000","last_undegraded":"2024-09-15T00:13:34.160360+0000","last_fullsized":"2024-09-15T00:13:34.160360+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:01:57.390380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"146'562","reported_seq":5096,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.157256+0000","last_change":"2024-09-15T00:13:34.157256+0000","last_active":"2024-09-15T00:13:34.157256+0000","last_peered":"2024-09-15T00:13:34.157256+0000","last_clean":"2024-09-15T00:13:34.157256+0000","last_became_active":"2024-09-15T00:11:19.594528+0000","last_became_peered":"2024-09-15T00:11:19.594528+0000","last_unstale":"2024-09-15T00:13:34.157256+0000","last_undegraded":"2024-09-15T00:13:34.157256+0000","last_fullsized":"2024-09-15T00:13:34.157256+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:24:34.378839+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.017633067999999998,"stat_sum":{"num_bytes":24609760,"num_objects":54,"num_object_clones":17,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":5312,"num_read_kb":42111,"num_write":566,"num_write_kb":17758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.3","version":"19'1","reported_seq":306,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.157628+0000","last_change":"2024-09-15T00:13:11.210218+0000","last_active":"2024-09-15T00:13:34.157628+0000","last_peered":"2024-09-15T00:13:34.157628+0000","last_clean":"2024-09-15T00:13:34.157628+0000","last_became_active":"2024-09-15T00:13:11.204222+0000","last_became_peered":"2024-09-15T00:13:11.204222+0000","last_unstale":"2024-09-15T00:13:34.157628+0000","last_undegraded":"2024-09-15T00:13:34.157628+0000","last_fullsized":"2024-09-15T00:13:34.157628+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:56:26.999234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":401,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.148689+0000","last_change":"2024-09-15T00:12:10.022102+0000","last_active":"2024-09-15T00:13:34.148689+0000","last_peered":"2024-09-15T00:13:34.148689+0000","last_clean":"2024-09-15T00:13:34.148689+0000","last_became_active":"2024-09-15T00:12:10.021831+0000","last_became_peered":"2024-09-15T00:12:10.021831+0000","last_unstale":"2024-09-15T00:13:34.148689+0000","last_undegraded":"2024-09-15T00:13:34.148689+0000","last_fullsized":"2024-09-15T00:13:34.148689+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_clean_scrub_stamp":"2024-09-15T00:11:04.453672+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:54:25.088117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"146'736","reported_seq":7087,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.271579+0000","last_change":"2024-09-15T00:13:34.140095+0000","last_active":"2024-09-15T00:13:34.271579+0000","last_peered":"2024-09-15T00:13:34.271579+0000","last_clean":"2024-09-15T00:13:34.271579+0000","last_became_active":"2024-09-15T00:13:27.022710+0000","last_became_peered":"2024-09-15T00:13:27.022710+0000","last_unstale":"2024-09-15T00:13:34.271579+0000","last_undegraded":"2024-09-15T00:13:34.271579+0000","last_fullsized":"2024-09-15T00:13:34.271579+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:37:25.714982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076078500000000004,"stat_sum":{"num_bytes":33316980,"num_objects":68,"num_object_clones":20,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":31,"num_read":7647,"num_read_kb":57656,"num_write":871,"num_write_kb":24338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":13659332,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.9","version":"146'1401","reported_seq":8471,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.279062+0000","last_change":"2024-09-15T00:13:34.147041+0000","last_active":"2024-09-15T00:13:34.279062+0000","last_peered":"2024-09-15T00:13:34.279062+0000","last_clean":"2024-09-15T00:13:34.279062+0000","last_became_active":"2024-09-15T00:12:23.125462+0000","last_became_peered":"2024-09-15T00:12:23.125462+0000","last_unstale":"2024-09-15T00:13:34.279062+0000","last_undegraded":"2024-09-15T00:13:34.279062+0000","last_fullsized":"2024-09-15T00:13:34.279062+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1401,"log_dups_size":0,"ondisk_log_size":1401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:58:15.251002+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0076933230000000002,"stat_sum":{"num_bytes":27350442,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":25,"num_read":14119,"num_read_kb":34863,"num_write":1985,"num_write_kb":12908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":10512832,"num_keys_recovered":83,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.a","version":"146'641","reported_seq":6214,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.150005+0000","last_change":"2024-09-15T00:13:34.150005+0000","last_active":"2024-09-15T00:13:34.150005+0000","last_peered":"2024-09-15T00:13:34.150005+0000","last_clean":"2024-09-15T00:13:34.150005+0000","last_became_active":"2024-09-15T00:11:19.602625+0000","last_became_peered":"2024-09-15T00:11:19.602625+0000","last_unstale":"2024-09-15T00:13:34.150005+0000","last_undegraded":"2024-09-15T00:13:34.150005+0000","last_fullsized":"2024-09-15T00:13:34.150005+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:20:08.576178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001171481,"stat_sum":{"num_bytes":23225220,"num_objects":55,"num_object_clones":15,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":25,"num_read":6713,"num_read_kb":22421,"num_write":638,"num_write_kb":11611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.b","version":"146'383","reported_seq":4693,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.149929+0000","last_change":"2024-09-15T00:13:34.149929+0000","last_active":"2024-09-15T00:13:34.149929+0000","last_peered":"2024-09-15T00:13:34.149929+0000","last_clean":"2024-09-15T00:13:34.149929+0000","last_became_active":"2024-09-15T00:12:12.020929+0000","last_became_peered":"2024-09-15T00:12:12.020929+0000","last_unstale":"2024-09-15T00:13:34.149929+0000","last_undegraded":"2024-09-15T00:13:34.149929+0000","last_fullsized":"2024-09-15T00:13:34.149929+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:18:08.085312+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.01045074,"stat_sum":{"num_bytes":40600821,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":3878,"num_read_kb":33277,"num_write":463,"num_write_kb":22691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":10092575,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.c","version":"145'852","reported_seq":4537,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.158879+0000","last_change":"2024-09-15T00:13:34.158879+0000","last_active":"2024-09-15T00:13:34.158879+0000","last_peered":"2024-09-15T00:13:34.158879+0000","last_clean":"2024-09-15T00:13:34.158879+0000","last_became_active":"2024-09-15T00:11:19.595088+0000","last_became_peered":"2024-09-15T00:11:19.595088+0000","last_unstale":"2024-09-15T00:13:34.158879+0000","last_undegraded":"2024-09-15T00:13:34.158879+0000","last_fullsized":"2024-09-15T00:13:34.158879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:21:23.113189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097285399999999999,"stat_sum":{"num_bytes":25803235,"num_objects":47,"num_object_clones":14,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":27,"num_read":7414,"num_read_kb":42973,"num_write":1059,"num_write_kb":15363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.d","version":"145'907","reported_seq":5670,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.158262+0000","last_change":"2024-09-15T00:13:34.158566+0000","last_active":"2024-09-15T00:13:34.158262+0000","last_peered":"2024-09-15T00:13:34.158262+0000","last_clean":"2024-09-15T00:13:34.158262+0000","last_became_active":"2024-09-15T00:11:50.008449+0000","last_became_peered":"2024-09-15T00:11:50.008449+0000","last_unstale":"2024-09-15T00:13:34.158262+0000","last_undegraded":"2024-09-15T00:13:34.158262+0000","last_fullsized":"2024-09-15T00:13:34.158262+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:09:36.258943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069127100000000003,"stat_sum":{"num_bytes":35136551,"num_objects":68,"num_object_clones":23,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":34,"num_read":7494,"num_read_kb":20433,"num_write":1242,"num_write_kb":5927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1062007,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.e","version":"146'1342","reported_seq":5739,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.161241+0000","last_change":"2024-09-15T00:13:34.161241+0000","last_active":"2024-09-15T00:13:34.161241+0000","last_peered":"2024-09-15T00:13:34.161241+0000","last_clean":"2024-09-15T00:13:34.161241+0000","last_became_active":"2024-09-15T00:13:28.027590+0000","last_became_peered":"2024-09-15T00:13:28.027590+0000","last_unstale":"2024-09-15T00:13:34.161241+0000","last_undegraded":"2024-09-15T00:13:34.161241+0000","last_fullsized":"2024-09-15T00:13:34.161241+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1342,"log_dups_size":0,"ondisk_log_size":1342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:23:09.609016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074993200000000003,"stat_sum":{"num_bytes":28955093,"num_objects":55,"num_object_clones":17,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":26,"num_read":8826,"num_read_kb":45129,"num_write":2085,"num_write_kb":24436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":17835233,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.f","version":"147'482","reported_seq":6152,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-15T00:13:35.022194+0000","last_change":"2024-09-15T00:13:35.025250+0000","last_active":"2024-09-15T00:13:35.022194+0000","last_peered":"2024-09-15T00:13:35.022194+0000","last_clean":"2024-09-15T00:13:35.022194+0000","last_became_active":"2024-09-15T00:12:11.009246+0000","last_became_peered":"2024-09-15T00:12:11.009246+0000","last_unstale":"2024-09-15T00:13:35.022194+0000","last_undegraded":"2024-09-15T00:13:35.022194+0000","last_fullsized":"2024-09-15T00:13:35.022194+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:54:20.829567+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00774678,"stat_sum":{"num_bytes":30281103,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":29,"num_read":5777,"num_read_kb":31101,"num_write":491,"num_write_kb":15000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":4060758,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"6"}]},{"pgid":"3.10","version":"146'654","reported_seq":6140,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.247738+0000","last_change":"2024-09-15T00:13:34.157663+0000","last_active":"2024-09-15T00:13:34.247738+0000","last_peered":"2024-09-15T00:13:34.247738+0000","last_clean":"2024-09-15T00:13:34.247738+0000","last_became_active":"2024-09-15T00:12:11.013203+0000","last_became_peered":"2024-09-15T00:12:11.013203+0000","last_unstale":"2024-09-15T00:13:34.247738+0000","last_undegraded":"2024-09-15T00:13:34.247738+0000","last_fullsized":"2024-09-15T00:13:34.247738+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:15:31.168563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086653779999999996,"stat_sum":{"num_bytes":32227095,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":26,"num_read":6869,"num_read_kb":31433,"num_write":793,"num_write_kb":14924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":8339863,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.11","version":"146'539","reported_seq":4341,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.158784+0000","last_change":"2024-09-15T00:13:34.158784+0000","last_active":"2024-09-15T00:13:34.158784+0000","last_peered":"2024-09-15T00:13:34.158784+0000","last_clean":"2024-09-15T00:13:34.158784+0000","last_became_active":"2024-09-15T00:12:44.028118+0000","last_became_peered":"2024-09-15T00:12:44.028118+0000","last_unstale":"2024-09-15T00:13:34.158784+0000","last_undegraded":"2024-09-15T00:13:34.158784+0000","last_fullsized":"2024-09-15T00:13:34.158784+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:20:00.972793+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0099657570000000004,"stat_sum":{"num_bytes":32383240,"num_objects":60,"num_object_clones":19,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":33,"num_read":4158,"num_read_kb":15372,"num_write":581,"num_write_kb":11093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":758528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.12","version":"144'636","reported_seq":7244,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.157328+0000","last_change":"2024-09-15T00:13:34.157581+0000","last_active":"2024-09-15T00:13:34.157328+0000","last_peered":"2024-09-15T00:13:34.157328+0000","last_clean":"2024-09-15T00:13:34.157328+0000","last_became_active":"2024-09-15T00:12:11.012806+0000","last_became_peered":"2024-09-15T00:12:11.012806+0000","last_unstale":"2024-09-15T00:13:34.157328+0000","last_undegraded":"2024-09-15T00:13:34.157328+0000","last_fullsized":"2024-09-15T00:13:34.157328+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:50.347806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017881986999999998,"stat_sum":{"num_bytes":19631098,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":35,"num_read":6786,"num_read_kb":50204,"num_write":776,"num_write_kb":25712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":4214788,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.13","version":"145'1041","reported_seq":7854,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.157704+0000","last_change":"2024-09-15T00:13:34.157944+0000","last_active":"2024-09-15T00:13:34.157704+0000","last_peered":"2024-09-15T00:13:34.157704+0000","last_clean":"2024-09-15T00:13:34.157704+0000","last_became_active":"2024-09-15T00:13:14.214678+0000","last_became_peered":"2024-09-15T00:13:14.214678+0000","last_unstale":"2024-09-15T00:13:34.157704+0000","last_undegraded":"2024-09-15T00:13:34.157704+0000","last_fullsized":"2024-09-15T00:13:34.157704+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1041,"log_dups_size":0,"ondisk_log_size":1041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:26:08.268514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0088638620000000001,"stat_sum":{"num_bytes":20872451,"num_objects":60,"num_object_clones":18,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":34,"num_read":10716,"num_read_kb":34225,"num_write":1359,"num_write_kb":19656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":25450111,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1c","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":967623345,"num_objects":1837,"num_object_clones":543,"num_object_copies":3674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1837,"num_whiteouts":935,"num_read":233587,"num_read_kb":1275538,"num_write":27443,"num_write_kb":584909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4585,"num_bytes_recovered":355971240,"num_keys_recovered":646,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45957120,"data_stored":679110527,"data_compressed":13335018,"data_compressed_allocated":38498304,"data_compressed_original":672869815,"omap_allocated":5367,"internal_metadata":0},"log_size":22809,"ondisk_log_size":22809,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":2361472,"data_compressed":24608,"data_compressed_allocated":147456,"data_compressed_original":2342912,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99324,"kb_used_data":9828,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":94272516,"statfs":{"total":96636764160,"available":96535056384,"internally_reserved":0,"allocated":10063872,"data_stored":127997347,"data_compressed":2463294,"data_compressed_allocated":7127040,"data_compressed_original":126139408,"omap_allocated":1595,"internal_metadata":91617733},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542179,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66752,"kb_used_data":9180,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305088,"statfs":{"total":96636764160,"available":96568410112,"internally_reserved":0,"allocated":9400320,"data_stored":116347048,"data_compressed":2174315,"data_compressed_allocated":6373376,"data_compressed_original":114491664,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":0,"seq":60129542178,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59616,"kb_used_data":6212,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94312224,"statfs":{"total":96636764160,"available":96575717376,"internally_reserved":0,"allocated":6361088,"data_stored":60365695,"data_compressed":1259155,"data_compressed_allocated":3809280,"data_compressed_original":58954122,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542176,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65172,"kb_used_data":7616,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94306668,"statfs":{"total":96636764160,"available":96570028032,"internally_reserved":0,"allocated":7798784,"data_stored":84856072,"data_compressed":1837488,"data_compressed_allocated":4997120,"data_compressed_original":83138454,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":463856467978,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65440,"kb_used_data":7884,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94306400,"statfs":{"total":96636764160,"available":96569753600,"internally_reserved":0,"allocated":8073216,"data_stored":93261795,"data_compressed":2018065,"data_compressed_allocated":5230592,"data_compressed_original":91397461,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68664,"kb_used_data":11096,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94303176,"statfs":{"total":96636764160,"available":96566452224,"internally_reserved":0,"allocated":11362304,"data_stored":144260525,"data_compressed":2844769,"data_compressed_allocated":8032256,"data_compressed_original":142011592,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9224192,"data_stored":139149624,"data_compressed":2721214,"data_compressed_allocated":7655424,"data_compressed_original":137751752,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6262784,"data_stored":91550270,"data_compressed":2016730,"data_compressed_allocated":5169152,"data_compressed_original":90520917,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1138688,"data_stored":14688390,"data_compressed":297956,"data_compressed_allocated":970752,"data_compressed_original":14557184,"omap_allocated":5367,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7380992,"data_stored":113346102,"data_compressed":2064456,"data_compressed_allocated":6123520,"data_compressed_original":112349456,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8302592,"data_stored":127149856,"data_compressed":2463294,"data_compressed_allocated":7127040,"data_compressed_original":126139408,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5758976,"data_stored":81905065,"data_compressed":1799033,"data_compressed_allocated":4816896,"data_compressed_original":81127318,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4493312,"data_stored":57985375,"data_compressed":1237878,"data_compressed_allocated":3694592,"data_compressed_original":57414026,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3395584,"data_stored":53335845,"data_compressed":734457,"data_compressed_allocated":2940928,"data_compressed_original":53009754,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:13:38.546 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:13:38.606 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:38.606 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:38.612 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:38.610+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:38.615 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:38.712 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:38.711+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:38.758 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:13:38.764 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:38.764 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:38.773 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:38.814 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:38.813+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:38.876 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:13:38.876 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:13:38.882 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:38.882 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:38.890 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-15T00:13:38.318371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":968213732,"num_objects":1841,"num_object_clones":543,"num_object_copies":3682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1841,"num_whiteouts":935,"num_read":233703,"num_read_kb":1275638,"num_write":27604,"num_write_kb":587327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4595,"num_bytes_recovered":357152014,"num_keys_recovered":646,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22929,"ondisk_log_size":22929,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":424968,"kb_used_data":51816,"kb_used_omap":9,"kb_used_meta":372982,"kb_avail":565806072,"statfs":{"total":579820584960,"available":579385417728,"internally_reserved":0,"allocated":53059584,"data_stored":627088482,"data_compressed":12597086,"data_compressed_allocated":35569664,"data_compressed_original":616132701,"omap_allocated":9566,"internal_metadata":381934242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-106591566,"num_objects":-203,"num_object_clones":-79,"num_object_copies":-406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-203,"num_whiteouts":-62,"num_read":1634,"num_read_kb":1392,"num_write":235,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000576"},"pg_stats":[{"pgid":"3.1f","version":"146'573","reported_seq":6152,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.159383+0000","last_change":"2024-09-15T00:13:34.159383+0000","last_active":"2024-09-15T00:13:34.159383+0000","last_peered":"2024-09-15T00:13:34.159383+0000","last_clean":"2024-09-15T00:13:34.159383+0000","last_became_active":"2024-09-15T00:12:11.013266+0000","last_became_peered":"2024-09-15T00:12:11.013266+0000","last_unstale":"2024-09-15T00:13:34.159383+0000","last_undegraded":"2024-09-15T00:13:34.159383+0000","last_fullsized":"2024-09-15T00:13:34.159383+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:34:20.058729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016096719999999999,"stat_sum":{"num_bytes":16592310,"num_objects":53,"num_object_clones":15,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":30,"num_read":6709,"num_read_kb":26141,"num_write":660,"num_write_kb":18365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":5475412,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1e","version":"146'560","reported_seq":7325,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.258194+0000","last_change":"2024-09-15T00:13:34.159108+0000","last_active":"2024-09-15T00:13:34.258194+0000","last_peered":"2024-09-15T00:13:34.258194+0000","last_clean":"2024-09-15T00:13:34.258194+0000","last_became_active":"2024-09-15T00:11:19.598640+0000","last_became_peered":"2024-09-15T00:11:19.598640+0000","last_unstale":"2024-09-15T00:13:34.258194+0000","last_undegraded":"2024-09-15T00:13:34.258194+0000","last_fullsized":"2024-09-15T00:13:34.258194+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:48:21.806980+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.019446548000000001,"stat_sum":{"num_bytes":32680534,"num_objects":70,"num_object_clones":24,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":33,"num_read":7531,"num_read_kb":39376,"num_write":712,"num_write_kb":10308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1d","version":"146'670","reported_seq":4669,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.235572+0000","last_change":"2024-09-15T00:13:34.159096+0000","last_active":"2024-09-15T00:13:34.235572+0000","last_peered":"2024-09-15T00:13:34.235572+0000","last_clean":"2024-09-15T00:13:34.235572+0000","last_became_active":"2024-09-15T00:11:19.595954+0000","last_became_peered":"2024-09-15T00:11:19.595954+0000","last_unstale":"2024-09-15T00:13:34.235572+0000","last_undegraded":"2024-09-15T00:13:34.235572+0000","last_fullsized":"2024-09-15T00:13:34.235572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:33:06.414275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011511760000000001,"stat_sum":{"num_bytes":13742710,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":25,"num_read":5581,"num_read_kb":45089,"num_write":767,"num_write_kb":31484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1c","version":"146'608","reported_seq":5467,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.267457+0000","last_change":"2024-09-15T00:13:34.159674+0000","last_active":"2024-09-15T00:13:34.267457+0000","last_peered":"2024-09-15T00:13:34.267457+0000","last_clean":"2024-09-15T00:13:34.267457+0000","last_became_active":"2024-09-15T00:11:19.602583+0000","last_became_peered":"2024-09-15T00:11:19.602583+0000","last_unstale":"2024-09-15T00:13:34.267457+0000","last_undegraded":"2024-09-15T00:13:34.267457+0000","last_fullsized":"2024-09-15T00:13:34.267457+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:38:02.261378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015881549999999999,"stat_sum":{"num_bytes":17465295,"num_objects":51,"num_object_clones":16,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":25,"num_read":5721,"num_read_kb":35056,"num_write":748,"num_write_kb":10221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1b","version":"146'760","reported_seq":6604,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.157822+0000","last_change":"2024-09-15T00:13:34.158090+0000","last_active":"2024-09-15T00:13:34.157822+0000","last_peered":"2024-09-15T00:13:34.157822+0000","last_clean":"2024-09-15T00:13:34.157822+0000","last_became_active":"2024-09-15T00:12:44.038243+0000","last_became_peered":"2024-09-15T00:12:44.038243+0000","last_unstale":"2024-09-15T00:13:34.157822+0000","last_undegraded":"2024-09-15T00:13:34.157822+0000","last_fullsized":"2024-09-15T00:13:34.157822+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:00:48.671663+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010189524,"stat_sum":{"num_bytes":24979401,"num_objects":72,"num_object_clones":26,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":36,"num_read":7633,"num_read_kb":34391,"num_write":883,"num_write_kb":14645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":10954706,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1a","version":"146'658","reported_seq":6741,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.278629+0000","last_change":"2024-09-15T00:13:34.160722+0000","last_active":"2024-09-15T00:13:34.278629+0000","last_peered":"2024-09-15T00:13:34.278629+0000","last_clean":"2024-09-15T00:13:34.278629+0000","last_became_active":"2024-09-15T00:13:27.022226+0000","last_became_peered":"2024-09-15T00:13:27.022226+0000","last_unstale":"2024-09-15T00:13:34.278629+0000","last_undegraded":"2024-09-15T00:13:34.278629+0000","last_fullsized":"2024-09-15T00:13:34.278629+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:50:00.160894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048399400000000001,"stat_sum":{"num_bytes":58087005,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":7944,"num_read_kb":61770,"num_write":758,"num_write_kb":33353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":65965475,"num_keys_recovered":138,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.19","version":"145'618","reported_seq":5030,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.160602+0000","last_change":"2024-09-15T00:13:34.160822+0000","last_active":"2024-09-15T00:13:34.160602+0000","last_peered":"2024-09-15T00:13:34.160602+0000","last_clean":"2024-09-15T00:13:34.160602+0000","last_became_active":"2024-09-15T00:13:26.020430+0000","last_became_peered":"2024-09-15T00:13:26.020430+0000","last_unstale":"2024-09-15T00:13:34.160602+0000","last_undegraded":"2024-09-15T00:13:34.160602+0000","last_fullsized":"2024-09-15T00:13:34.160602+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:23:56.282108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057772900000000002,"stat_sum":{"num_bytes":30917179,"num_objects":60,"num_object_clones":18,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":29,"num_read":4671,"num_read_kb":42435,"num_write":680,"num_write_kb":9827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":6381633,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.18","version":"144'628","reported_seq":7803,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.140207+0000","last_change":"2024-09-15T00:13:34.140771+0000","last_active":"2024-09-15T00:13:34.140207+0000","last_peered":"2024-09-15T00:13:34.140207+0000","last_clean":"2024-09-15T00:13:34.140207+0000","last_became_active":"2024-09-15T00:12:12.020924+0000","last_became_peered":"2024-09-15T00:12:12.020924+0000","last_unstale":"2024-09-15T00:13:34.140207+0000","last_undegraded":"2024-09-15T00:13:34.140207+0000","last_fullsized":"2024-09-15T00:13:34.140207+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:16:24.930926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001283462,"stat_sum":{"num_bytes":22180626,"num_objects":66,"num_object_clones":21,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":31,"num_read":8037,"num_read_kb":40066,"num_write":738,"num_write_kb":15973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4986328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.17","version":"143'593","reported_seq":4811,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.159730+0000","last_change":"2024-09-15T00:13:34.161625+0000","last_active":"2024-09-15T00:13:34.159730+0000","last_peered":"2024-09-15T00:13:34.159730+0000","last_clean":"2024-09-15T00:13:34.159730+0000","last_became_active":"2024-09-15T00:12:11.013261+0000","last_became_peered":"2024-09-15T00:12:11.013261+0000","last_unstale":"2024-09-15T00:13:34.159730+0000","last_undegraded":"2024-09-15T00:13:34.159730+0000","last_fullsized":"2024-09-15T00:13:34.159730+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:46:35.268761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035127549999999998,"stat_sum":{"num_bytes":45142964,"num_objects":70,"num_object_clones":22,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":38,"num_read":5247,"num_read_kb":45406,"num_write":595,"num_write_kb":17982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":12460957,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.16","version":"146'972","reported_seq":7820,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.242587+0000","last_change":"2024-09-15T00:13:34.163590+0000","last_active":"2024-09-15T00:13:34.242587+0000","last_peered":"2024-09-15T00:13:34.242587+0000","last_clean":"2024-09-15T00:13:34.242587+0000","last_became_active":"2024-09-15T00:13:26.026214+0000","last_became_peered":"2024-09-15T00:13:26.026214+0000","last_unstale":"2024-09-15T00:13:34.242587+0000","last_undegraded":"2024-09-15T00:13:34.242587+0000","last_fullsized":"2024-09-15T00:13:34.242587+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":972,"log_dups_size":0,"ondisk_log_size":972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:55:58.681353+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0056989550000000003,"stat_sum":{"num_bytes":40253729,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":25,"num_read":8828,"num_read_kb":39209,"num_write":1229,"num_write_kb":20604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":20681466,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.15","version":"144'616","reported_seq":6654,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.148926+0000","last_change":"2024-09-15T00:13:34.149682+0000","last_active":"2024-09-15T00:13:34.148926+0000","last_peered":"2024-09-15T00:13:34.148926+0000","last_clean":"2024-09-15T00:13:34.148926+0000","last_became_active":"2024-09-15T00:11:53.010039+0000","last_became_peered":"2024-09-15T00:11:53.010039+0000","last_unstale":"2024-09-15T00:13:34.148926+0000","last_undegraded":"2024-09-15T00:13:34.148926+0000","last_fullsized":"2024-09-15T00:13:34.148926+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:17:00.605000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010058892,"stat_sum":{"num_bytes":37464658,"num_objects":53,"num_object_clones":15,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":28,"num_read":7561,"num_read_kb":45608,"num_write":714,"num_write_kb":14831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":9563692,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.14","version":"146'1001","reported_seq":7130,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.297495+0000","last_change":"2024-09-15T00:13:34.156892+0000","last_active":"2024-09-15T00:13:34.297495+0000","last_peered":"2024-09-15T00:13:34.297495+0000","last_clean":"2024-09-15T00:13:34.297495+0000","last_became_active":"2024-09-15T00:12:44.028859+0000","last_became_peered":"2024-09-15T00:12:44.028859+0000","last_unstale":"2024-09-15T00:13:34.297495+0000","last_undegraded":"2024-09-15T00:13:34.297495+0000","last_fullsized":"2024-09-15T00:13:34.297495+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:09:16.438377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080058300000000002,"stat_sum":{"num_bytes":32515177,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":36,"num_read":9634,"num_read_kb":58257,"num_write":1444,"num_write_kb":24998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":3041694,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":284,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.148940+0000","last_change":"2024-09-15T00:13:11.204550+0000","last_active":"2024-09-15T00:13:34.148940+0000","last_peered":"2024-09-15T00:13:34.148940+0000","last_clean":"2024-09-15T00:13:34.148940+0000","last_became_active":"2024-09-15T00:13:11.204211+0000","last_became_peered":"2024-09-15T00:13:11.204211+0000","last_unstale":"2024-09-15T00:13:34.148940+0000","last_undegraded":"2024-09-15T00:13:34.148940+0000","last_fullsized":"2024-09-15T00:13:34.148940+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:16:00.619195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"146'639","reported_seq":7372,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.161464+0000","last_change":"2024-09-15T00:13:34.161464+0000","last_active":"2024-09-15T00:13:34.161464+0000","last_peered":"2024-09-15T00:13:34.161464+0000","last_clean":"2024-09-15T00:13:34.161464+0000","last_became_active":"2024-09-15T00:13:28.029209+0000","last_became_peered":"2024-09-15T00:13:28.029209+0000","last_unstale":"2024-09-15T00:13:34.161464+0000","last_undegraded":"2024-09-15T00:13:34.161464+0000","last_fullsized":"2024-09-15T00:13:34.161464+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:40:20.934298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085075600000000004,"stat_sum":{"num_bytes":41566721,"num_objects":70,"num_object_clones":22,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":41,"num_read":6771,"num_read_kb":42479,"num_write":645,"num_write_kb":25730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":26144067,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":34,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.160450+0000","last_change":"2024-09-15T00:13:22.049830+0000","last_active":"2024-09-15T00:13:34.160450+0000","last_peered":"2024-09-15T00:13:34.160450+0000","last_clean":"2024-09-15T00:13:34.160450+0000","last_became_active":"2024-09-15T00:13:22.049357+0000","last_became_peered":"2024-09-15T00:13:22.049357+0000","last_unstale":"2024-09-15T00:13:34.160450+0000","last_undegraded":"2024-09-15T00:13:34.160450+0000","last_fullsized":"2024-09-15T00:13:34.160450+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:29:10.717532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"146'637","reported_seq":8689,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.295489+0000","last_change":"2024-09-15T00:13:34.148837+0000","last_active":"2024-09-15T00:13:34.295489+0000","last_peered":"2024-09-15T00:13:34.295489+0000","last_clean":"2024-09-15T00:13:34.295489+0000","last_became_active":"2024-09-15T00:12:10.010014+0000","last_became_peered":"2024-09-15T00:12:10.010014+0000","last_unstale":"2024-09-15T00:13:34.295489+0000","last_undegraded":"2024-09-15T00:13:34.295489+0000","last_fullsized":"2024-09-15T00:13:34.295489+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:25:34.159932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0092416619999999994,"stat_sum":{"num_bytes":17161677,"num_objects":69,"num_object_clones":22,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":35,"num_read":9902,"num_read_kb":57043,"num_write":672,"num_write_kb":17736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":11200702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":170,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.147189+0000","last_change":"2024-09-15T00:12:44.035622+0000","last_active":"2024-09-15T00:13:34.147189+0000","last_peered":"2024-09-15T00:13:34.147189+0000","last_clean":"2024-09-15T00:13:34.147189+0000","last_became_active":"2024-09-15T00:12:44.035383+0000","last_became_peered":"2024-09-15T00:12:44.035383+0000","last_unstale":"2024-09-15T00:13:34.147189+0000","last_undegraded":"2024-09-15T00:13:34.147189+0000","last_fullsized":"2024-09-15T00:13:34.147189+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:30:08.250776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"145'678","reported_seq":10826,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.160780+0000","last_change":"2024-09-15T00:13:34.160980+0000","last_active":"2024-09-15T00:13:34.160780+0000","last_peered":"2024-09-15T00:13:34.160780+0000","last_clean":"2024-09-15T00:13:34.160780+0000","last_became_active":"2024-09-15T00:13:26.022113+0000","last_became_peered":"2024-09-15T00:13:26.022113+0000","last_unstale":"2024-09-15T00:13:34.160780+0000","last_undegraded":"2024-09-15T00:13:34.160780+0000","last_fullsized":"2024-09-15T00:13:34.160780+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:23:44.429772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066889599999999999,"stat_sum":{"num_bytes":55407800,"num_objects":64,"num_object_clones":20,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":12152,"num_read_kb":60045,"num_write":725,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":606,"num_bytes_recovered":40146959,"num_keys_recovered":159,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.160392+0000","last_change":"2024-09-15T00:13:22.048567+0000","last_active":"2024-09-15T00:13:34.160392+0000","last_peered":"2024-09-15T00:13:34.160392+0000","last_clean":"2024-09-15T00:13:34.160392+0000","last_became_active":"2024-09-15T00:13:22.048228+0000","last_became_peered":"2024-09-15T00:13:22.048228+0000","last_unstale":"2024-09-15T00:13:34.160392+0000","last_undegraded":"2024-09-15T00:13:34.160392+0000","last_fullsized":"2024-09-15T00:13:34.160392+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:27:06.941012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"144'488","reported_seq":2759,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.161663+0000","last_change":"2024-09-15T00:13:34.161837+0000","last_active":"2024-09-15T00:13:34.161663+0000","last_peered":"2024-09-15T00:13:34.161663+0000","last_clean":"2024-09-15T00:13:34.161663+0000","last_became_active":"2024-09-15T00:11:19.595633+0000","last_became_peered":"2024-09-15T00:11:19.595633+0000","last_unstale":"2024-09-15T00:13:34.161663+0000","last_undegraded":"2024-09-15T00:13:34.161663+0000","last_fullsized":"2024-09-15T00:13:34.161663+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:34:46.379301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003715838,"stat_sum":{"num_bytes":9685942,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":20,"num_read":3263,"num_read_kb":16863,"num_write":506,"num_write_kb":8972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.3","version":"147'664","reported_seq":7192,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-15T00:13:35.155095+0000","last_change":"2024-09-15T00:13:35.041177+0000","last_active":"2024-09-15T00:13:35.155095+0000","last_peered":"2024-09-15T00:13:35.155095+0000","last_clean":"2024-09-15T00:13:35.155095+0000","last_became_active":"2024-09-15T00:12:11.013156+0000","last_became_peered":"2024-09-15T00:12:11.013156+0000","last_unstale":"2024-09-15T00:13:35.155095+0000","last_undegraded":"2024-09-15T00:13:35.155095+0000","last_fullsized":"2024-09-15T00:13:35.155095+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:55:56.506145+0000","scrub_duration":0,"objects_trimmed":7,"snaptrim_duration":0.023321544999999999,"stat_sum":{"num_bytes":43719735,"num_objects":51,"num_object_clones":15,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":27,"num_read":6845,"num_read_kb":42866,"num_write":641,"num_write_kb":26766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":9273484,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"6"}]},{"pgid":"2.2","version":"21'2","reported_seq":289,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.157721+0000","last_change":"2024-09-15T00:13:22.052918+0000","last_active":"2024-09-15T00:13:34.157721+0000","last_peered":"2024-09-15T00:13:34.157721+0000","last_clean":"2024-09-15T00:13:34.157721+0000","last_became_active":"2024-09-15T00:13:22.049795+0000","last_became_peered":"2024-09-15T00:13:22.049795+0000","last_unstale":"2024-09-15T00:13:34.157721+0000","last_undegraded":"2024-09-15T00:13:34.157721+0000","last_fullsized":"2024-09-15T00:13:34.157721+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:50:48.495996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"131'608","reported_seq":6854,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.160865+0000","last_change":"2024-09-15T00:13:34.161058+0000","last_active":"2024-09-15T00:13:34.160865+0000","last_peered":"2024-09-15T00:13:34.160865+0000","last_clean":"2024-09-15T00:13:34.160865+0000","last_became_active":"2024-09-15T00:13:28.027146+0000","last_became_peered":"2024-09-15T00:13:28.027146+0000","last_unstale":"2024-09-15T00:13:34.160865+0000","last_undegraded":"2024-09-15T00:13:34.160865+0000","last_fullsized":"2024-09-15T00:13:34.160865+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:42:05.921781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064049799999999998,"stat_sum":{"num_bytes":18517623,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":29,"num_read":7136,"num_read_kb":33196,"num_write":672,"num_write_kb":14429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":27650205,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.1","version":"0'0","reported_seq":222,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.139346+0000","last_change":"2024-09-15T00:13:11.203976+0000","last_active":"2024-09-15T00:13:34.139346+0000","last_peered":"2024-09-15T00:13:34.139346+0000","last_clean":"2024-09-15T00:13:34.139346+0000","last_became_active":"2024-09-15T00:13:11.203652+0000","last_became_peered":"2024-09-15T00:13:11.203652+0000","last_unstale":"2024-09-15T00:13:34.139346+0000","last_undegraded":"2024-09-15T00:13:34.139346+0000","last_fullsized":"2024-09-15T00:13:34.139346+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:23:34.249491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"146'662","reported_seq":6442,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.157894+0000","last_change":"2024-09-15T00:13:34.158112+0000","last_active":"2024-09-15T00:13:34.157894+0000","last_peered":"2024-09-15T00:13:34.157894+0000","last_clean":"2024-09-15T00:13:34.157894+0000","last_became_active":"2024-09-15T00:12:44.035426+0000","last_became_peered":"2024-09-15T00:12:44.035426+0000","last_unstale":"2024-09-15T00:13:34.157894+0000","last_undegraded":"2024-09-15T00:13:34.157894+0000","last_fullsized":"2024-09-15T00:13:34.157894+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:07:16.407812+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010688496,"stat_sum":{"num_bytes":35149170,"num_objects":52,"num_object_clones":16,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":22,"num_read":6712,"num_read_kb":49044,"num_write":745,"num_write_kb":22019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":6058433,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.160360+0000","last_change":"2024-09-15T00:13:22.049592+0000","last_active":"2024-09-15T00:13:34.160360+0000","last_peered":"2024-09-15T00:13:34.160360+0000","last_clean":"2024-09-15T00:13:34.160360+0000","last_became_active":"2024-09-15T00:13:22.049283+0000","last_became_peered":"2024-09-15T00:13:22.049283+0000","last_unstale":"2024-09-15T00:13:34.160360+0000","last_undegraded":"2024-09-15T00:13:34.160360+0000","last_fullsized":"2024-09-15T00:13:34.160360+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:01:57.390380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"146'562","reported_seq":5096,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.157256+0000","last_change":"2024-09-15T00:13:34.157256+0000","last_active":"2024-09-15T00:13:34.157256+0000","last_peered":"2024-09-15T00:13:34.157256+0000","last_clean":"2024-09-15T00:13:34.157256+0000","last_became_active":"2024-09-15T00:11:19.594528+0000","last_became_peered":"2024-09-15T00:11:19.594528+0000","last_unstale":"2024-09-15T00:13:34.157256+0000","last_undegraded":"2024-09-15T00:13:34.157256+0000","last_fullsized":"2024-09-15T00:13:34.157256+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:24:34.378839+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.017633067999999998,"stat_sum":{"num_bytes":24609760,"num_objects":54,"num_object_clones":17,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":5312,"num_read_kb":42111,"num_write":566,"num_write_kb":17758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.3","version":"19'1","reported_seq":306,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.157628+0000","last_change":"2024-09-15T00:13:11.210218+0000","last_active":"2024-09-15T00:13:34.157628+0000","last_peered":"2024-09-15T00:13:34.157628+0000","last_clean":"2024-09-15T00:13:34.157628+0000","last_became_active":"2024-09-15T00:13:11.204222+0000","last_became_peered":"2024-09-15T00:13:11.204222+0000","last_unstale":"2024-09-15T00:13:34.157628+0000","last_undegraded":"2024-09-15T00:13:34.157628+0000","last_fullsized":"2024-09-15T00:13:34.157628+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:56:26.999234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":401,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.148689+0000","last_change":"2024-09-15T00:12:10.022102+0000","last_active":"2024-09-15T00:13:34.148689+0000","last_peered":"2024-09-15T00:13:34.148689+0000","last_clean":"2024-09-15T00:13:34.148689+0000","last_became_active":"2024-09-15T00:12:10.021831+0000","last_became_peered":"2024-09-15T00:12:10.021831+0000","last_unstale":"2024-09-15T00:13:34.148689+0000","last_undegraded":"2024-09-15T00:13:34.148689+0000","last_fullsized":"2024-09-15T00:13:34.148689+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_clean_scrub_stamp":"2024-09-15T00:11:04.453672+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:54:25.088117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"146'736","reported_seq":7087,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.271579+0000","last_change":"2024-09-15T00:13:34.140095+0000","last_active":"2024-09-15T00:13:34.271579+0000","last_peered":"2024-09-15T00:13:34.271579+0000","last_clean":"2024-09-15T00:13:34.271579+0000","last_became_active":"2024-09-15T00:13:27.022710+0000","last_became_peered":"2024-09-15T00:13:27.022710+0000","last_unstale":"2024-09-15T00:13:34.271579+0000","last_undegraded":"2024-09-15T00:13:34.271579+0000","last_fullsized":"2024-09-15T00:13:34.271579+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:37:25.714982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076078500000000004,"stat_sum":{"num_bytes":33316980,"num_objects":68,"num_object_clones":20,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":31,"num_read":7647,"num_read_kb":57656,"num_write":871,"num_write_kb":24338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":13659332,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.9","version":"146'1401","reported_seq":8471,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.279062+0000","last_change":"2024-09-15T00:13:34.147041+0000","last_active":"2024-09-15T00:13:34.279062+0000","last_peered":"2024-09-15T00:13:34.279062+0000","last_clean":"2024-09-15T00:13:34.279062+0000","last_became_active":"2024-09-15T00:12:23.125462+0000","last_became_peered":"2024-09-15T00:12:23.125462+0000","last_unstale":"2024-09-15T00:13:34.279062+0000","last_undegraded":"2024-09-15T00:13:34.279062+0000","last_fullsized":"2024-09-15T00:13:34.279062+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1401,"log_dups_size":0,"ondisk_log_size":1401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:58:15.251002+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0076933230000000002,"stat_sum":{"num_bytes":27350442,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":25,"num_read":14119,"num_read_kb":34863,"num_write":1985,"num_write_kb":12908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":10512832,"num_keys_recovered":83,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.a","version":"146'641","reported_seq":6214,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.150005+0000","last_change":"2024-09-15T00:13:34.150005+0000","last_active":"2024-09-15T00:13:34.150005+0000","last_peered":"2024-09-15T00:13:34.150005+0000","last_clean":"2024-09-15T00:13:34.150005+0000","last_became_active":"2024-09-15T00:11:19.602625+0000","last_became_peered":"2024-09-15T00:11:19.602625+0000","last_unstale":"2024-09-15T00:13:34.150005+0000","last_undegraded":"2024-09-15T00:13:34.150005+0000","last_fullsized":"2024-09-15T00:13:34.150005+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:20:08.576178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001171481,"stat_sum":{"num_bytes":23225220,"num_objects":55,"num_object_clones":15,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":25,"num_read":6713,"num_read_kb":22421,"num_write":638,"num_write_kb":11611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.b","version":"146'383","reported_seq":4693,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.149929+0000","last_change":"2024-09-15T00:13:34.149929+0000","last_active":"2024-09-15T00:13:34.149929+0000","last_peered":"2024-09-15T00:13:34.149929+0000","last_clean":"2024-09-15T00:13:34.149929+0000","last_became_active":"2024-09-15T00:12:12.020929+0000","last_became_peered":"2024-09-15T00:12:12.020929+0000","last_unstale":"2024-09-15T00:13:34.149929+0000","last_undegraded":"2024-09-15T00:13:34.149929+0000","last_fullsized":"2024-09-15T00:13:34.149929+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:18:08.085312+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.01045074,"stat_sum":{"num_bytes":40600821,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":3878,"num_read_kb":33277,"num_write":463,"num_write_kb":22691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":10092575,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.c","version":"145'852","reported_seq":4537,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.158879+0000","last_change":"2024-09-15T00:13:34.158879+0000","last_active":"2024-09-15T00:13:34.158879+0000","last_peered":"2024-09-15T00:13:34.158879+0000","last_clean":"2024-09-15T00:13:34.158879+0000","last_became_active":"2024-09-15T00:11:19.595088+0000","last_became_peered":"2024-09-15T00:11:19.595088+0000","last_unstale":"2024-09-15T00:13:34.158879+0000","last_undegraded":"2024-09-15T00:13:34.158879+0000","last_fullsized":"2024-09-15T00:13:34.158879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:21:23.113189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097285399999999999,"stat_sum":{"num_bytes":25803235,"num_objects":47,"num_object_clones":14,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":27,"num_read":7414,"num_read_kb":42973,"num_write":1059,"num_write_kb":15363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.d","version":"145'907","reported_seq":5670,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.158262+0000","last_change":"2024-09-15T00:13:34.158566+0000","last_active":"2024-09-15T00:13:34.158262+0000","last_peered":"2024-09-15T00:13:34.158262+0000","last_clean":"2024-09-15T00:13:34.158262+0000","last_became_active":"2024-09-15T00:11:50.008449+0000","last_became_peered":"2024-09-15T00:11:50.008449+0000","last_unstale":"2024-09-15T00:13:34.158262+0000","last_undegraded":"2024-09-15T00:13:34.158262+0000","last_fullsized":"2024-09-15T00:13:34.158262+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:09:36.258943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069127100000000003,"stat_sum":{"num_bytes":35136551,"num_objects":68,"num_object_clones":23,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":34,"num_read":7494,"num_read_kb":20433,"num_write":1242,"num_write_kb":5927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1062007,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.e","version":"146'1342","reported_seq":5739,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.161241+0000","last_change":"2024-09-15T00:13:34.161241+0000","last_active":"2024-09-15T00:13:34.161241+0000","last_peered":"2024-09-15T00:13:34.161241+0000","last_clean":"2024-09-15T00:13:34.161241+0000","last_became_active":"2024-09-15T00:13:28.027590+0000","last_became_peered":"2024-09-15T00:13:28.027590+0000","last_unstale":"2024-09-15T00:13:34.161241+0000","last_undegraded":"2024-09-15T00:13:34.161241+0000","last_fullsized":"2024-09-15T00:13:34.161241+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1342,"log_dups_size":0,"ondisk_log_size":1342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:23:09.609016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074993200000000003,"stat_sum":{"num_bytes":28955093,"num_objects":55,"num_object_clones":17,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":26,"num_read":8826,"num_read_kb":45129,"num_write":2085,"num_write_kb":24436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":17835233,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.f","version":"147'482","reported_seq":6152,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-15T00:13:35.022194+0000","last_change":"2024-09-15T00:13:35.025250+0000","last_active":"2024-09-15T00:13:35.022194+0000","last_peered":"2024-09-15T00:13:35.022194+0000","last_clean":"2024-09-15T00:13:35.022194+0000","last_became_active":"2024-09-15T00:12:11.009246+0000","last_became_peered":"2024-09-15T00:12:11.009246+0000","last_unstale":"2024-09-15T00:13:35.022194+0000","last_undegraded":"2024-09-15T00:13:35.022194+0000","last_fullsized":"2024-09-15T00:13:35.022194+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:54:20.829567+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00774678,"stat_sum":{"num_bytes":30281103,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":29,"num_read":5777,"num_read_kb":31101,"num_write":491,"num_write_kb":15000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":4060758,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"6"}]},{"pgid":"3.10","version":"146'654","reported_seq":6140,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.247738+0000","last_change":"2024-09-15T00:13:34.157663+0000","last_active":"2024-09-15T00:13:34.247738+0000","last_peered":"2024-09-15T00:13:34.247738+0000","last_clean":"2024-09-15T00:13:34.247738+0000","last_became_active":"2024-09-15T00:12:11.013203+0000","last_became_peered":"2024-09-15T00:12:11.013203+0000","last_unstale":"2024-09-15T00:13:34.247738+0000","last_undegraded":"2024-09-15T00:13:34.247738+0000","last_fullsized":"2024-09-15T00:13:34.247738+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:15:31.168563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086653779999999996,"stat_sum":{"num_bytes":32227095,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":26,"num_read":6869,"num_read_kb":31433,"num_write":793,"num_write_kb":14924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":8339863,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.11","version":"146'539","reported_seq":4341,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.158784+0000","last_change":"2024-09-15T00:13:34.158784+0000","last_active":"2024-09-15T00:13:34.158784+0000","last_peered":"2024-09-15T00:13:34.158784+0000","last_clean":"2024-09-15T00:13:34.158784+0000","last_became_active":"2024-09-15T00:12:44.028118+0000","last_became_peered":"2024-09-15T00:12:44.028118+0000","last_unstale":"2024-09-15T00:13:34.158784+0000","last_undegraded":"2024-09-15T00:13:34.158784+0000","last_fullsized":"2024-09-15T00:13:34.158784+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:20:00.972793+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0099657570000000004,"stat_sum":{"num_bytes":32383240,"num_objects":60,"num_object_clones":19,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":33,"num_read":4158,"num_read_kb":15372,"num_write":581,"num_write_kb":11093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":758528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.12","version":"144'636","reported_seq":7244,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.157328+0000","last_change":"2024-09-15T00:13:34.157581+0000","last_active":"2024-09-15T00:13:34.157328+0000","last_peered":"2024-09-15T00:13:34.157328+0000","last_clean":"2024-09-15T00:13:34.157328+0000","last_became_active":"2024-09-15T00:12:11.012806+0000","last_became_peered":"2024-09-15T00:12:11.012806+0000","last_unstale":"2024-09-15T00:13:34.157328+0000","last_undegraded":"2024-09-15T00:13:34.157328+0000","last_fullsized":"2024-09-15T00:13:34.157328+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:50.347806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017881986999999998,"stat_sum":{"num_bytes":19631098,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":35,"num_read":6786,"num_read_kb":50204,"num_write":776,"num_write_kb":25712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":4214788,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.13","version":"145'1041","reported_seq":7854,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T00:13:34.157704+0000","last_change":"2024-09-15T00:13:34.157944+0000","last_active":"2024-09-15T00:13:34.157704+0000","last_peered":"2024-09-15T00:13:34.157704+0000","last_clean":"2024-09-15T00:13:34.157704+0000","last_became_active":"2024-09-15T00:13:14.214678+0000","last_became_peered":"2024-09-15T00:13:14.214678+0000","last_unstale":"2024-09-15T00:13:34.157704+0000","last_undegraded":"2024-09-15T00:13:34.157704+0000","last_fullsized":"2024-09-15T00:13:34.157704+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1041,"log_dups_size":0,"ondisk_log_size":1041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:26:08.268514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0088638620000000001,"stat_sum":{"num_bytes":20872451,"num_objects":60,"num_object_clones":18,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":34,"num_read":10716,"num_read_kb":34225,"num_write":1359,"num_write_kb":19656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":25450111,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1c","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":967623345,"num_objects":1837,"num_object_clones":543,"num_object_copies":3674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1837,"num_whiteouts":935,"num_read":233587,"num_read_kb":1275538,"num_write":27443,"num_write_kb":584909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4585,"num_bytes_recovered":355971240,"num_keys_recovered":646,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45957120,"data_stored":679110527,"data_compressed":13335018,"data_compressed_allocated":38498304,"data_compressed_original":672869815,"omap_allocated":5367,"internal_metadata":0},"log_size":22809,"ondisk_log_size":22809,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":2361472,"data_compressed":24608,"data_compressed_allocated":147456,"data_compressed_original":2342912,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99324,"kb_used_data":9828,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":94272516,"statfs":{"total":96636764160,"available":96535056384,"internally_reserved":0,"allocated":10063872,"data_stored":127997347,"data_compressed":2463294,"data_compressed_allocated":7127040,"data_compressed_original":126139408,"omap_allocated":1595,"internal_metadata":91617733},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542179,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66752,"kb_used_data":9180,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305088,"statfs":{"total":96636764160,"available":96568410112,"internally_reserved":0,"allocated":9400320,"data_stored":116347048,"data_compressed":2174315,"data_compressed_allocated":6373376,"data_compressed_original":114491664,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":0,"seq":60129542178,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59616,"kb_used_data":6212,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94312224,"statfs":{"total":96636764160,"available":96575717376,"internally_reserved":0,"allocated":6361088,"data_stored":60365695,"data_compressed":1259155,"data_compressed_allocated":3809280,"data_compressed_original":58954122,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542176,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65172,"kb_used_data":7616,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94306668,"statfs":{"total":96636764160,"available":96570028032,"internally_reserved":0,"allocated":7798784,"data_stored":84856072,"data_compressed":1837488,"data_compressed_allocated":4997120,"data_compressed_original":83138454,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":463856467978,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65440,"kb_used_data":7884,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94306400,"statfs":{"total":96636764160,"available":96569753600,"internally_reserved":0,"allocated":8073216,"data_stored":93261795,"data_compressed":2018065,"data_compressed_allocated":5230592,"data_compressed_original":91397461,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68664,"kb_used_data":11096,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94303176,"statfs":{"total":96636764160,"available":96566452224,"internally_reserved":0,"allocated":11362304,"data_stored":144260525,"data_compressed":2844769,"data_compressed_allocated":8032256,"data_compressed_original":142011592,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5504,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9224192,"data_stored":139149624,"data_compressed":2721214,"data_compressed_allocated":7655424,"data_compressed_original":137751752,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6262784,"data_stored":91550270,"data_compressed":2016730,"data_compressed_allocated":5169152,"data_compressed_original":90520917,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1138688,"data_stored":14688390,"data_compressed":297956,"data_compressed_allocated":970752,"data_compressed_original":14557184,"omap_allocated":5367,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7380992,"data_stored":113346102,"data_compressed":2064456,"data_compressed_allocated":6123520,"data_compressed_original":112349456,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8302592,"data_stored":127149856,"data_compressed":2463294,"data_compressed_allocated":7127040,"data_compressed_original":126139408,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5758976,"data_stored":81905065,"data_compressed":1799033,"data_compressed_allocated":4816896,"data_compressed_original":81127318,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4493312,"data_stored":57985375,"data_compressed":1237878,"data_compressed_allocated":3694592,"data_compressed_original":57414026,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3395584,"data_stored":53335845,"data_compressed":734457,"data_compressed_allocated":2940928,"data_compressed_original":53009754,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:13:38.892 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T00:13:38.892 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:13:38.892 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-15T00:13:38.892 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T00:13:38.900 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:38.901 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:38.914 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:38.912+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:38.917 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:38.917 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:38.922 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:38.922 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:38.931 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:38.934 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:38.934 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:38.951 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:38.951 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:38.968 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:38.968 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:39.000 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:39.000 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:39.015 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:39.013+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:39.017 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:39.017 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:39.055 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:39.055 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:39.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:39.055 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:39.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.055 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:13:39.055 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:39.055 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 110.80960135399999, 2024-09-15T00:13:39.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:13:39.056 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.056 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.056 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:39.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.056 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.056 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:13:39.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.056 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:39.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:39.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:13:39.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:13:39.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:13:39.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:13:39.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:13:39.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:13:39.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:13:39.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:39.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:13:39.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:13:39.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:13:39.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:13:39.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:39.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:13:39.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:13:39.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:13:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:13:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:13:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:13:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:13:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:13:39.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:13:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:13:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58878 3.8 3:12456482:::rbd_data.1139766160da.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:39.061 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 105.76329577, 2024-09-15T00:13:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061956199000000003, 2024-09-15T00:13:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58878 2024-09-15T00:13:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289151+0000", 2024-09-15T00:13:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:13:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289473+0000", 2024-09-15T00:13:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032183800000000002 2024-09-15T00:13:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289487+0000", 2024-09-15T00:13:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-05 2024-09-15T00:13:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289493+0000", 2024-09-15T00:13:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4489999999999996e-06 2024-09-15T00:13:39.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289542+0000", 2024-09-15T00:13:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8844999999999997e-05 2024-09-15T00:13:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289642+0000", 2024-09-15T00:13:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9607999999999999e-05 2024-09-15T00:13:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:13:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.293908+0000", 2024-09-15T00:13:39.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659350000000002 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351022+0000", 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057114535000000001 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351033+0000", 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0263e-05 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351070+0000", 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7666000000000002e-05 2024-09-15T00:13:39.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351106+0000", 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999997e-05 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58879 3.7 3:e169894d:::rbd_data.1139766160da.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 105.760944282, 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043716294000000003, 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.067 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58879 2024-09-15T00:13:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291502+0000", 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291664+0000", 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016216599999999999 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291674+0000", 2024-09-15T00:13:39.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0276e-05 2024-09-15T00:13:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291680+0000", 2024-09-15T00:13:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2580000000000003e-06 2024-09-15T00:13:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291732+0000", 2024-09-15T00:13:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2114999999999998e-05 2024-09-15T00:13:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291866+0000", 2024-09-15T00:13:39.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013416900000000001 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.296426+0000", 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045595779999999999 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335075+0000", 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038649347000000001 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335097+0000", 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2565000000000002e-05 2024-09-15T00:13:39.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335161+0000", 2024-09-15T00:13:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3928000000000002e-05 2024-09-15T00:13:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335217+0000", 2024-09-15T00:13:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6042000000000003e-05 2024-09-15T00:13:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.072 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58880 3.8 3:13d91cfd:::rbd_data.1139766160da.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 105.757552175, 2024-09-15T00:13:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056331463999999998, 2024-09-15T00:13:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58880 2024-09-15T00:13:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294896+0000", 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.356e-06 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295288+0000", 2024-09-15T00:13:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039223200000000001 2024-09-15T00:13:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295304+0000", 2024-09-15T00:13:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5937e-05 2024-09-15T00:13:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295312+0000", 2024-09-15T00:13:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1489999999999994e-06 2024-09-15T00:13:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333162+0000", 2024-09-15T00:13:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037849840000000003 2024-09-15T00:13:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333269+0000", 2024-09-15T00:13:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010749800000000001 2024-09-15T00:13:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:13:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.336198+0000", 2024-09-15T00:13:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029284739999999999 2024-09-15T00:13:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351137+0000", 2024-09-15T00:13:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014939103 2024-09-15T00:13:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351150+0000", 2024-09-15T00:13:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3542e-05 2024-09-15T00:13:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351191+0000", 2024-09-15T00:13:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0791000000000003e-05 2024-09-15T00:13:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351225+0000", 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3541999999999998e-05 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58878 3.8 e56/23 ondisk, result = 0)", 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 105.751022392, 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049689096000000002, 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301425+0000", 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.782e-06 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301433+0000", 2024-09-15T00:13:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689999999999994e-06 2024-09-15T00:13:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301449+0000", 2024-09-15T00:13:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5767e-05 2024-09-15T00:13:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301458+0000", 2024-09-15T00:13:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:13:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351009+0000", 2024-09-15T00:13:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049551523 2024-09-15T00:13:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351017+0000", 2024-09-15T00:13:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8359999999999992e-06 2024-09-15T00:13:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351112+0000", 2024-09-15T00:13:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5173999999999994e-05 2024-09-15T00:13:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.081 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127004 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 6881280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.084116928, 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735720000000003, 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127004 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:13:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968331+0000", 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.849e-06 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968334+0000", 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7689999999999998e-06 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968338+0000", 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2490000000000001e-06 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008987+0000", 2024-09-15T00:13:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040649117999999998 2024-09-15T00:13:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009027+0000", 2024-09-15T00:13:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0357e-05 2024-09-15T00:13:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009064+0000", 2024-09-15T00:13:39.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7036000000000003e-05 2024-09-15T00:13:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127005 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7004160~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.084104375999999, 2024-09-15T00:13:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040824358999999998, 2024-09-15T00:13:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:39.086 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127005 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-15T00:13:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968343+0000", 2024-09-15T00:13:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3430000000000001e-06 2024-09-15T00:13:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968349+0000", 2024-09-15T00:13:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3080000000000004e-06 2024-09-15T00:13:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968355+0000", 2024-09-15T00:13:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9499999999999998e-06 2024-09-15T00:13:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009091+0000", 2024-09-15T00:13:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735645000000001 2024-09-15T00:13:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009130+0000", 2024-09-15T00:13:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8951e-05 2024-09-15T00:13:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009165+0000", 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5926000000000001e-05 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127006 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7127040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.084087318000002, 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040912783000000001, 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:39.090 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127006 2024-09-15T00:13:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:39.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968359+0000", 2024-09-15T00:13:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0419999999999999e-06 2024-09-15T00:13:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968363+0000", 2024-09-15T00:13:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5999999999999998e-06 2024-09-15T00:13:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968366+0000", 2024-09-15T00:13:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5179999999999999e-06 2024-09-15T00:13:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009196+0000", 2024-09-15T00:13:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040830313 2024-09-15T00:13:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009234+0000", 2024-09-15T00:13:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8562000000000001e-05 2024-09-15T00:13:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009271+0000", 2024-09-15T00:13:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6538000000000002e-05 2024-09-15T00:13:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127007 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7249920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.084077119, 2024-09-15T00:13:39.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041019218000000003, 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127007 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968369+0000", 2024-09-15T00:13:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-15T00:13:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968370+0000", 2024-09-15T00:13:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.012e-06 2024-09-15T00:13:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968374+0000", 2024-09-15T00:13:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3839999999999999e-06 2024-09-15T00:13:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968376+0000", 2024-09-15T00:13:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5339999999999998e-06 2024-09-15T00:13:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009292+0000", 2024-09-15T00:13:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040915092 2024-09-15T00:13:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009333+0000", 2024-09-15T00:13:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1105e-05 2024-09-15T00:13:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009388+0000", 2024-09-15T00:13:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4868999999999999e-05 2024-09-15T00:13:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.098 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127008 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7364608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.084065998, 2024-09-15T00:13:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041115290999999998, 2024-09-15T00:13:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127008 2024-09-15T00:13:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968380+0000", 2024-09-15T00:13:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3999999999999998e-07 2024-09-15T00:13:39.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968381+0000", 2024-09-15T00:13:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8700000000000004e-07 2024-09-15T00:13:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968385+0000", 2024-09-15T00:13:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9260000000000002e-06 2024-09-15T00:13:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968389+0000", 2024-09-15T00:13:39.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.685e-06 2024-09-15T00:13:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009419+0000", 2024-09-15T00:13:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041029852999999998 2024-09-15T00:13:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009458+0000", 2024-09-15T00:13:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8538000000000003e-05 2024-09-15T00:13:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009495+0000", 2024-09-15T00:13:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7061999999999999e-05 2024-09-15T00:13:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127009 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7487488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.084031881000001, 2024-09-15T00:13:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041233063, 2024-09-15T00:13:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.103 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127009 2024-09-15T00:13:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7199999999999999e-07 2024-09-15T00:13:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968417+0000", 2024-09-15T00:13:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3810000000000002e-06 2024-09-15T00:13:39.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968423+0000", 2024-09-15T00:13:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8820000000000003e-06 2024-09-15T00:13:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968433+0000", 2024-09-15T00:13:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0005e-05 2024-09-15T00:13:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009527+0000", 2024-09-15T00:13:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041093750999999998 2024-09-15T00:13:39.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009601+0000", 2024-09-15T00:13:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3979000000000004e-05 2024-09-15T00:13:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009647+0000", 2024-09-15T00:13:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5493e-05 2024-09-15T00:13:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.107 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127010 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7610368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.084002877000003, 2024-09-15T00:13:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041325924999999999, 2024-09-15T00:13:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127010 2024-09-15T00:13:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.108 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968445+0000", 2024-09-15T00:13:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:13:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968452+0000", 2024-09-15T00:13:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-15T00:13:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968456+0000", 2024-09-15T00:13:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5210000000000004e-06 2024-09-15T00:13:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009664+0000", 2024-09-15T00:13:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041207404000000003 2024-09-15T00:13:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009723+0000", 2024-09-15T00:13:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9036000000000002e-05 2024-09-15T00:13:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009768+0000", 2024-09-15T00:13:39.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5578000000000003e-05 2024-09-15T00:13:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127011 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7733248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.083984642000004, 2024-09-15T00:13:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041482441000000002, 2024-09-15T00:13:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:39.112 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127011 2024-09-15T00:13:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968466+0000", 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968470+0000", 2024-09-15T00:13:39.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6469999999999997e-06 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968473+0000", 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.977e-06 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009800+0000", 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041326918999999997 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009869+0000", 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8935000000000003e-05 2024-09-15T00:13:39.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009943+0000", 2024-09-15T00:13:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4188000000000002e-05 2024-09-15T00:13:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127012 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7856128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.083969139999994, 2024-09-15T00:13:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041640756000000001, 2024-09-15T00:13:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127012 2024-09-15T00:13:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968477+0000", 2024-09-15T00:13:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968482+0000", 2024-09-15T00:13:39.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6180000000000003e-06 2024-09-15T00:13:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968486+0000", 2024-09-15T00:13:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.501e-06 2024-09-15T00:13:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968489+0000", 2024-09-15T00:13:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1530000000000001e-06 2024-09-15T00:13:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009983+0000", 2024-09-15T00:13:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041493980999999999 2024-09-15T00:13:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010051+0000", 2024-09-15T00:13:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8039000000000004e-05 2024-09-15T00:13:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010117+0000", 2024-09-15T00:13:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6254000000000006e-05 2024-09-15T00:13:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127013 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7979008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.083953320000006, 2024-09-15T00:13:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041763863999999998, 2024-09-15T00:13:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.121 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127013 2024-09-15T00:13:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969767+0000", 2024-09-15T00:13:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001275006 2024-09-15T00:13:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969772+0000", 2024-09-15T00:13:39.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4769999999999997e-06 2024-09-15T00:13:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969781+0000", 2024-09-15T00:13:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1959999999999997e-06 2024-09-15T00:13:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969788+0000", 2024-09-15T00:13:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6359999999999994e-06 2024-09-15T00:13:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010162+0000", 2024-09-15T00:13:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040373291999999998 2024-09-15T00:13:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010212+0000", 2024-09-15T00:13:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0543e-05 2024-09-15T00:13:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010256+0000", 2024-09-15T00:13:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3714000000000002e-05 2024-09-15T00:13:39.125 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127015 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8224768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.082614383999996, 2024-09-15T00:13:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040710186000000002, 2024-09-15T00:13:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.126 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127015 2024-09-15T00:13:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969832+0000", 2024-09-15T00:13:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:13:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969834+0000", 2024-09-15T00:13:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2919999999999998e-06 2024-09-15T00:13:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969843+0000", 2024-09-15T00:13:39.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0480000000000004e-06 2024-09-15T00:13:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969848+0000", 2024-09-15T00:13:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-15T00:13:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010457+0000", 2024-09-15T00:13:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040609054999999998 2024-09-15T00:13:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010501+0000", 2024-09-15T00:13:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3396000000000003e-05 2024-09-15T00:13:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010541+0000", 2024-09-15T00:13:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0599000000000002e-05 2024-09-15T00:13:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127016 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.082563758000006, 2024-09-15T00:13:39.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040766216000000001, 2024-09-15T00:13:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127016 2024-09-15T00:13:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:13:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969886+0000", 2024-09-15T00:13:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8449999999999996e-06 2024-09-15T00:13:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969897+0000", 2024-09-15T00:13:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-15T00:13:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969904+0000", 2024-09-15T00:13:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1169999999999998e-06 2024-09-15T00:13:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010561+0000", 2024-09-15T00:13:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040657470000000001 2024-09-15T00:13:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010607+0000", 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6108e-05 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010648+0000", 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0479000000000003e-05 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127694 3.9 3:9390b205:::rbd_data.11396481e1ec.00000000000000df:head [set-alloc-hint object_size 1048576 write_size 1048576,write 819200~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:39.134 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 76.827844640999999, 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90376419399999997, 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127694 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224602+0000", 2024-09-15T00:13:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5899999999999995e-07 2024-09-15T00:13:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224610+0000", 2024-09-15T00:13:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9130000000000003e-06 2024-09-15T00:13:39.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224620+0000", 2024-09-15T00:13:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0426e-05 2024-09-15T00:13:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224627+0000", 2024-09-15T00:13:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4390000000000003e-06 2024-09-15T00:13:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224675+0000", 2024-09-15T00:13:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7636000000000003e-05 2024-09-15T00:13:39.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224681+0000", 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1460000000000001e-06 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126388+0000", 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90170724999999996 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126526+0000", 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013730599999999999 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126684+0000", 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000157935 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.127932+0000", 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012479240000000001 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128269+0000", 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033761099999999998 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128324+0000", 2024-09-15T00:13:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5244999999999997e-05 2024-09-15T00:13:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128365+0000", 2024-09-15T00:13:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0503999999999997e-05 2024-09-15T00:13:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:134707 3.15 3:af1fbdb5:::rbd_data.113933a4f79a.000000000000001c:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T00:13:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:39.140 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 67.359877526999995, 2024-09-15T00:13:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057328626000000001, 2024-09-15T00:13:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 134707 2024-09-15T00:13:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692569+0000", 2024-09-15T00:13:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.339e-06 2024-09-15T00:13:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692574+0000", 2024-09-15T00:13:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-15T00:13:39.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692584+0000", 2024-09-15T00:13:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0363000000000001e-05 2024-09-15T00:13:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692590+0000", 2024-09-15T00:13:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.818e-06 2024-09-15T00:13:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692633+0000", 2024-09-15T00:13:39.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.265e-05 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692718+0000", 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5279999999999997e-05 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.695940+0000", 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003222052 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749714+0000", 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.053774126999999998 2024-09-15T00:13:39.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749788+0000", 2024-09-15T00:13:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3693000000000001e-05 2024-09-15T00:13:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749847+0000", 2024-09-15T00:13:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175999999999997e-05 2024-09-15T00:13:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749897+0000", 2024-09-15T00:13:39.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9311999999999998e-05 2024-09-15T00:13:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:134707 3.15 e86/55 ondisk, result = 0)", 2024-09-15T00:13:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 67.351316178000005, 2024-09-15T00:13:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048596066, 2024-09-15T00:13:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701131+0000", 2024-09-15T00:13:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3549999999999999e-06 2024-09-15T00:13:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701139+0000", 2024-09-15T00:13:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1650000000000006e-06 2024-09-15T00:13:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701156+0000", 2024-09-15T00:13:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6730000000000001e-05 2024-09-15T00:13:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701167+0000", 2024-09-15T00:13:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1504e-05 2024-09-15T00:13:39.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749629+0000", 2024-09-15T00:13:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048461952000000003 2024-09-15T00:13:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749640+0000", 2024-09-15T00:13:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0874e-05 2024-09-15T00:13:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749725+0000", 2024-09-15T00:13:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5486000000000001e-05 2024-09-15T00:13:39.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.150 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.150 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.150 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.150 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.150 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:39.151 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:39.152 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:39.149+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:39.178 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is unset 2024-09-15T00:13:39.184 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:13:39.184 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:39.184 INFO:teuthology.orchestra.run.smithi171.stdout: "epoch": 151, 2024-09-15T00:13:39.184 INFO:teuthology.orchestra.run.smithi171.stdout: "fsid": "bad2533c-9c09-4af4-ab93-147afbacb705", 2024-09-15T00:13:39.184 INFO:teuthology.orchestra.run.smithi171.stdout: "created": "2024-09-15T00:10:50.203030+0000", 2024-09-15T00:13:39.184 INFO:teuthology.orchestra.run.smithi171.stdout: "modified": "2024-09-15T00:13:39.171403+0000", 2024-09-15T00:13:39.184 INFO:teuthology.orchestra.run.smithi171.stdout: "last_up_change": "2024-09-15T00:12:55.092989+0000", 2024-09-15T00:13:39.184 INFO:teuthology.orchestra.run.smithi171.stdout: "last_in_change": "2024-09-15T00:13:21.033818+0000", 2024-09-15T00:13:39.184 INFO:teuthology.orchestra.run.smithi171.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T00:13:39.184 INFO:teuthology.orchestra.run.smithi171.stdout: "flags_num": 5799936, 2024-09-15T00:13:39.184 INFO:teuthology.orchestra.run.smithi171.stdout: "flags_set": [ 2024-09-15T00:13:39.184 INFO:teuthology.orchestra.run.smithi171.stdout: "pglog_hardlimit", 2024-09-15T00:13:39.185 INFO:teuthology.orchestra.run.smithi171.stdout: "purged_snapdirs", 2024-09-15T00:13:39.185 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_deletes", 2024-09-15T00:13:39.185 INFO:teuthology.orchestra.run.smithi171.stdout: "sortbitwise" 2024-09-15T00:13:39.185 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:13:39.185 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_version": 6, 2024-09-15T00:13:39.185 INFO:teuthology.orchestra.run.smithi171.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T00:13:39.185 INFO:teuthology.orchestra.run.smithi171.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T00:13:39.185 INFO:teuthology.orchestra.run.smithi171.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T00:13:39.185 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_snapshot": "", 2024-09-15T00:13:39.185 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_max": 3, 2024-09-15T00:13:39.185 INFO:teuthology.orchestra.run.smithi171.stdout: "max_osd": 8, 2024-09-15T00:13:39.185 INFO:teuthology.orchestra.run.smithi171.stdout: "require_min_compat_client": "luminous", 2024-09-15T00:13:39.185 INFO:teuthology.orchestra.run.smithi171.stdout: "min_compat_client": "luminous", 2024-09-15T00:13:39.185 INFO:teuthology.orchestra.run.smithi171.stdout: "require_osd_release": "squid", 2024-09-15T00:13:39.186 INFO:teuthology.orchestra.run.smithi171.stdout: "allow_crimson": false, 2024-09-15T00:13:39.186 INFO:teuthology.orchestra.run.smithi171.stdout: "pools": [ 2024-09-15T00:13:39.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.186 INFO:teuthology.orchestra.run.smithi171.stdout: "pool": 1, 2024-09-15T00:13:39.186 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_name": ".mgr", 2024-09-15T00:13:39.186 INFO:teuthology.orchestra.run.smithi171.stdout: "create_time": "2024-09-15T00:11:04.326356+0000", 2024-09-15T00:13:39.186 INFO:teuthology.orchestra.run.smithi171.stdout: "flags": 1, 2024-09-15T00:13:39.186 INFO:teuthology.orchestra.run.smithi171.stdout: "flags_names": "hashpspool", 2024-09-15T00:13:39.186 INFO:teuthology.orchestra.run.smithi171.stdout: "type": 1, 2024-09-15T00:13:39.186 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 2, 2024-09-15T00:13:39.186 INFO:teuthology.orchestra.run.smithi171.stdout: "min_size": 1, 2024-09-15T00:13:39.186 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_rule": 0, 2024-09-15T00:13:39.186 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:13:39.186 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:13:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:13:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:13:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: "is_stretch_pool": false, 2024-09-15T00:13:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: "object_hash": 2, 2024-09-15T00:13:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:13:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num": 1, 2024-09-15T00:13:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num": 1, 2024-09-15T00:13:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num_target": 1, 2024-09-15T00:13:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_target": 1, 2024-09-15T00:13:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_pending": 1, 2024-09-15T00:13:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: "last_pg_merge_meta": { 2024-09-15T00:13:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: "source_pgid": "0.0", 2024-09-15T00:13:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: "ready_epoch": 0, 2024-09-15T00:13:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_started": 0, 2024-09-15T00:13:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_clean": 0, 2024-09-15T00:13:39.188 INFO:teuthology.orchestra.run.smithi171.stdout: "source_version": "0'0", 2024-09-15T00:13:39.188 INFO:teuthology.orchestra.run.smithi171.stdout: "target_version": "0'0" 2024-09-15T00:13:39.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.188 INFO:teuthology.orchestra.run.smithi171.stdout: "last_change": "17", 2024-09-15T00:13:39.188 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend": "0", 2024-09-15T00:13:39.188 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:13:39.188 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:13:39.188 INFO:teuthology.orchestra.run.smithi171.stdout: "auid": 0, 2024-09-15T00:13:39.188 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:13:39.188 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_seq": 0, 2024-09-15T00:13:39.188 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_epoch": 0, 2024-09-15T00:13:39.188 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_snaps": [], 2024-09-15T00:13:39.188 INFO:teuthology.orchestra.run.smithi171.stdout: "removed_snaps": "[]", 2024-09-15T00:13:39.189 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_bytes": 0, 2024-09-15T00:13:39.189 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_objects": 0, 2024-09-15T00:13:39.189 INFO:teuthology.orchestra.run.smithi171.stdout: "tiers": [], 2024-09-15T00:13:39.189 INFO:teuthology.orchestra.run.smithi171.stdout: "tier_of": -1, 2024-09-15T00:13:39.189 INFO:teuthology.orchestra.run.smithi171.stdout: "read_tier": -1, 2024-09-15T00:13:39.189 INFO:teuthology.orchestra.run.smithi171.stdout: "write_tier": -1, 2024-09-15T00:13:39.189 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_mode": "none", 2024-09-15T00:13:39.189 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_bytes": 0, 2024-09-15T00:13:39.189 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_objects": 0, 2024-09-15T00:13:39.189 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:13:39.189 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:13:39.189 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:13:39.189 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_flush_age": 0, 2024-09-15T00:13:39.190 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_evict_age": 0, 2024-09-15T00:13:39.190 INFO:teuthology.orchestra.run.smithi171.stdout: "erasure_code_profile": "", 2024-09-15T00:13:39.190 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_params": { 2024-09-15T00:13:39.190 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "none" 2024-09-15T00:13:39.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.190 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_period": 0, 2024-09-15T00:13:39.190 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_count": 0, 2024-09-15T00:13:39.190 INFO:teuthology.orchestra.run.smithi171.stdout: "use_gmt_hitset": true, 2024-09-15T00:13:39.190 INFO:teuthology.orchestra.run.smithi171.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:13:39.190 INFO:teuthology.orchestra.run.smithi171.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:13:39.190 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:13:39.190 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:13:39.190 INFO:teuthology.orchestra.run.smithi171.stdout: "grade_table": [], 2024-09-15T00:13:39.190 INFO:teuthology.orchestra.run.smithi171.stdout: "stripe_width": 0, 2024-09-15T00:13:39.191 INFO:teuthology.orchestra.run.smithi171.stdout: "expected_num_objects": 0, 2024-09-15T00:13:39.191 INFO:teuthology.orchestra.run.smithi171.stdout: "fast_read": false, 2024-09-15T00:13:39.191 INFO:teuthology.orchestra.run.smithi171.stdout: "options": { 2024-09-15T00:13:39.191 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_max": 32, 2024-09-15T00:13:39.191 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_min": 1 2024-09-15T00:13:39.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.191 INFO:teuthology.orchestra.run.smithi171.stdout: "application_metadata": { 2024-09-15T00:13:39.191 INFO:teuthology.orchestra.run.smithi171.stdout: "mgr": {} 2024-09-15T00:13:39.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.191 INFO:teuthology.orchestra.run.smithi171.stdout: "read_balance": { 2024-09-15T00:13:39.191 INFO:teuthology.orchestra.run.smithi171.stdout: "score_type": "Fair distribution", 2024-09-15T00:13:39.191 INFO:teuthology.orchestra.run.smithi171.stdout: "score_acting": 6.059999942779541, 2024-09-15T00:13:39.191 INFO:teuthology.orchestra.run.smithi171.stdout: "score_stable": 6.059999942779541, 2024-09-15T00:13:39.191 INFO:teuthology.orchestra.run.smithi171.stdout: "optimal_score": 0.33000001311302185, 2024-09-15T00:13:39.192 INFO:teuthology.orchestra.run.smithi171.stdout: "raw_score_acting": 2, 2024-09-15T00:13:39.192 INFO:teuthology.orchestra.run.smithi171.stdout: "raw_score_stable": 2, 2024-09-15T00:13:39.192 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity_weighted": 0.75, 2024-09-15T00:13:39.192 INFO:teuthology.orchestra.run.smithi171.stdout: "average_primary_affinity": 1, 2024-09-15T00:13:39.192 INFO:teuthology.orchestra.run.smithi171.stdout: "average_primary_affinity_weighted": 0.75 2024-09-15T00:13:39.192 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.192 INFO:teuthology.orchestra.run.smithi171.stdout: "pool": 2, 2024-09-15T00:13:39.192 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_name": "rbd", 2024-09-15T00:13:39.192 INFO:teuthology.orchestra.run.smithi171.stdout: "create_time": "2024-09-15T00:11:08.005967+0000", 2024-09-15T00:13:39.192 INFO:teuthology.orchestra.run.smithi171.stdout: "flags": 8193, 2024-09-15T00:13:39.193 INFO:teuthology.orchestra.run.smithi171.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:13:39.193 INFO:teuthology.orchestra.run.smithi171.stdout: "type": 1, 2024-09-15T00:13:39.193 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 2, 2024-09-15T00:13:39.193 INFO:teuthology.orchestra.run.smithi171.stdout: "min_size": 1, 2024-09-15T00:13:39.193 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_rule": 0, 2024-09-15T00:13:39.193 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:13:39.193 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:13:39.193 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:13:39.193 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:13:39.193 INFO:teuthology.orchestra.run.smithi171.stdout: "is_stretch_pool": false, 2024-09-15T00:13:39.193 INFO:teuthology.orchestra.run.smithi171.stdout: "object_hash": 2, 2024-09-15T00:13:39.193 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:13:39.193 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num": 8, 2024-09-15T00:13:39.193 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num": 8, 2024-09-15T00:13:39.194 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num_target": 8, 2024-09-15T00:13:39.194 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_target": 8, 2024-09-15T00:13:39.194 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_pending": 8, 2024-09-15T00:13:39.194 INFO:teuthology.orchestra.run.smithi171.stdout: "last_pg_merge_meta": { 2024-09-15T00:13:39.194 INFO:teuthology.orchestra.run.smithi171.stdout: "source_pgid": "0.0", 2024-09-15T00:13:39.194 INFO:teuthology.orchestra.run.smithi171.stdout: "ready_epoch": 0, 2024-09-15T00:13:39.194 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_started": 0, 2024-09-15T00:13:39.194 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_clean": 0, 2024-09-15T00:13:39.194 INFO:teuthology.orchestra.run.smithi171.stdout: "source_version": "0'0", 2024-09-15T00:13:39.194 INFO:teuthology.orchestra.run.smithi171.stdout: "target_version": "0'0" 2024-09-15T00:13:39.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.194 INFO:teuthology.orchestra.run.smithi171.stdout: "last_change": "21", 2024-09-15T00:13:39.194 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend": "0", 2024-09-15T00:13:39.195 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:13:39.195 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:13:39.195 INFO:teuthology.orchestra.run.smithi171.stdout: "auid": 0, 2024-09-15T00:13:39.195 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:13:39.195 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_seq": 2, 2024-09-15T00:13:39.195 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_epoch": 21, 2024-09-15T00:13:39.195 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_snaps": [], 2024-09-15T00:13:39.195 INFO:teuthology.orchestra.run.smithi171.stdout: "removed_snaps": "[]", 2024-09-15T00:13:39.195 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_bytes": 0, 2024-09-15T00:13:39.195 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_objects": 0, 2024-09-15T00:13:39.195 INFO:teuthology.orchestra.run.smithi171.stdout: "tiers": [], 2024-09-15T00:13:39.195 INFO:teuthology.orchestra.run.smithi171.stdout: "tier_of": -1, 2024-09-15T00:13:39.195 INFO:teuthology.orchestra.run.smithi171.stdout: "read_tier": -1, 2024-09-15T00:13:39.195 INFO:teuthology.orchestra.run.smithi171.stdout: "write_tier": -1, 2024-09-15T00:13:39.196 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_mode": "none", 2024-09-15T00:13:39.196 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_bytes": 0, 2024-09-15T00:13:39.196 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_objects": 0, 2024-09-15T00:13:39.196 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:13:39.196 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:13:39.196 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:13:39.196 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_flush_age": 0, 2024-09-15T00:13:39.196 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_evict_age": 0, 2024-09-15T00:13:39.196 INFO:teuthology.orchestra.run.smithi171.stdout: "erasure_code_profile": "", 2024-09-15T00:13:39.196 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_params": { 2024-09-15T00:13:39.196 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "none" 2024-09-15T00:13:39.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.196 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_period": 0, 2024-09-15T00:13:39.196 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_count": 0, 2024-09-15T00:13:39.197 INFO:teuthology.orchestra.run.smithi171.stdout: "use_gmt_hitset": true, 2024-09-15T00:13:39.197 INFO:teuthology.orchestra.run.smithi171.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:13:39.197 INFO:teuthology.orchestra.run.smithi171.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:13:39.197 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:13:39.197 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:13:39.197 INFO:teuthology.orchestra.run.smithi171.stdout: "grade_table": [], 2024-09-15T00:13:39.197 INFO:teuthology.orchestra.run.smithi171.stdout: "stripe_width": 0, 2024-09-15T00:13:39.197 INFO:teuthology.orchestra.run.smithi171.stdout: "expected_num_objects": 0, 2024-09-15T00:13:39.197 INFO:teuthology.orchestra.run.smithi171.stdout: "fast_read": false, 2024-09-15T00:13:39.197 INFO:teuthology.orchestra.run.smithi171.stdout: "options": {}, 2024-09-15T00:13:39.197 INFO:teuthology.orchestra.run.smithi171.stdout: "application_metadata": { 2024-09-15T00:13:39.197 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd": {} 2024-09-15T00:13:39.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.197 INFO:teuthology.orchestra.run.smithi171.stdout: "read_balance": { 2024-09-15T00:13:39.198 INFO:teuthology.orchestra.run.smithi171.stdout: "error_message": "pool 2 has some PGs where all OSDs are with primary_affinity 0 (at least pg 2.5), read balance score may not be reliable", 2024-09-15T00:13:39.198 INFO:teuthology.orchestra.run.smithi171.stdout: "score_acting": 0, 2024-09-15T00:13:39.198 INFO:teuthology.orchestra.run.smithi171.stdout: "score_stable": 0 2024-09-15T00:13:39.198 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.198 INFO:teuthology.orchestra.run.smithi171.stdout: "pool": 3, 2024-09-15T00:13:39.198 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_name": "pool_client.0", 2024-09-15T00:13:39.198 INFO:teuthology.orchestra.run.smithi171.stdout: "create_time": "2024-09-15T00:11:17.676102+0000", 2024-09-15T00:13:39.198 INFO:teuthology.orchestra.run.smithi171.stdout: "flags": 8193, 2024-09-15T00:13:39.198 INFO:teuthology.orchestra.run.smithi171.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:13:39.198 INFO:teuthology.orchestra.run.smithi171.stdout: "type": 1, 2024-09-15T00:13:39.198 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 2, 2024-09-15T00:13:39.198 INFO:teuthology.orchestra.run.smithi171.stdout: "min_size": 1, 2024-09-15T00:13:39.199 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_rule": 0, 2024-09-15T00:13:39.199 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:13:39.199 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:13:39.199 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:13:39.199 INFO:teuthology.orchestra.run.smithi171.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:13:39.199 INFO:teuthology.orchestra.run.smithi171.stdout: "is_stretch_pool": false, 2024-09-15T00:13:39.199 INFO:teuthology.orchestra.run.smithi171.stdout: "object_hash": 2, 2024-09-15T00:13:39.199 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:13:39.199 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num": 32, 2024-09-15T00:13:39.199 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num": 32, 2024-09-15T00:13:39.199 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_placement_num_target": 32, 2024-09-15T00:13:39.199 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_target": 32, 2024-09-15T00:13:39.199 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_num_pending": 32, 2024-09-15T00:13:39.199 INFO:teuthology.orchestra.run.smithi171.stdout: "last_pg_merge_meta": { 2024-09-15T00:13:39.200 INFO:teuthology.orchestra.run.smithi171.stdout: "source_pgid": "0.0", 2024-09-15T00:13:39.200 INFO:teuthology.orchestra.run.smithi171.stdout: "ready_epoch": 0, 2024-09-15T00:13:39.200 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_started": 0, 2024-09-15T00:13:39.200 INFO:teuthology.orchestra.run.smithi171.stdout: "last_epoch_clean": 0, 2024-09-15T00:13:39.200 INFO:teuthology.orchestra.run.smithi171.stdout: "source_version": "0'0", 2024-09-15T00:13:39.200 INFO:teuthology.orchestra.run.smithi171.stdout: "target_version": "0'0" 2024-09-15T00:13:39.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.200 INFO:teuthology.orchestra.run.smithi171.stdout: "last_change": "151", 2024-09-15T00:13:39.200 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend": "0", 2024-09-15T00:13:39.200 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:13:39.200 INFO:teuthology.orchestra.run.smithi171.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:13:39.200 INFO:teuthology.orchestra.run.smithi171.stdout: "auid": 0, 2024-09-15T00:13:39.200 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:13:39.200 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_seq": 53, 2024-09-15T00:13:39.201 INFO:teuthology.orchestra.run.smithi171.stdout: "snap_epoch": 151, 2024-09-15T00:13:39.201 INFO:teuthology.orchestra.run.smithi171.stdout: "pool_snaps": [], 2024-09-15T00:13:39.201 INFO:teuthology.orchestra.run.smithi171.stdout: "removed_snaps": "[]", 2024-09-15T00:13:39.201 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_bytes": 0, 2024-09-15T00:13:39.201 INFO:teuthology.orchestra.run.smithi171.stdout: "quota_max_objects": 0, 2024-09-15T00:13:39.201 INFO:teuthology.orchestra.run.smithi171.stdout: "tiers": [], 2024-09-15T00:13:39.201 INFO:teuthology.orchestra.run.smithi171.stdout: "tier_of": -1, 2024-09-15T00:13:39.201 INFO:teuthology.orchestra.run.smithi171.stdout: "read_tier": -1, 2024-09-15T00:13:39.201 INFO:teuthology.orchestra.run.smithi171.stdout: "write_tier": -1, 2024-09-15T00:13:39.201 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_mode": "none", 2024-09-15T00:13:39.201 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_bytes": 0, 2024-09-15T00:13:39.201 INFO:teuthology.orchestra.run.smithi171.stdout: "target_max_objects": 0, 2024-09-15T00:13:39.201 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:13:39.202 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:13:39.202 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:13:39.202 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_flush_age": 0, 2024-09-15T00:13:39.202 INFO:teuthology.orchestra.run.smithi171.stdout: "cache_min_evict_age": 0, 2024-09-15T00:13:39.202 INFO:teuthology.orchestra.run.smithi171.stdout: "erasure_code_profile": "", 2024-09-15T00:13:39.202 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_params": { 2024-09-15T00:13:39.202 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "none" 2024-09-15T00:13:39.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.202 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_period": 0, 2024-09-15T00:13:39.202 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_count": 0, 2024-09-15T00:13:39.202 INFO:teuthology.orchestra.run.smithi171.stdout: "use_gmt_hitset": true, 2024-09-15T00:13:39.202 INFO:teuthology.orchestra.run.smithi171.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:13:39.202 INFO:teuthology.orchestra.run.smithi171.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:13:39.203 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:13:39.203 INFO:teuthology.orchestra.run.smithi171.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:13:39.203 INFO:teuthology.orchestra.run.smithi171.stdout: "grade_table": [], 2024-09-15T00:13:39.203 INFO:teuthology.orchestra.run.smithi171.stdout: "stripe_width": 0, 2024-09-15T00:13:39.203 INFO:teuthology.orchestra.run.smithi171.stdout: "expected_num_objects": 0, 2024-09-15T00:13:39.203 INFO:teuthology.orchestra.run.smithi171.stdout: "fast_read": false, 2024-09-15T00:13:39.203 INFO:teuthology.orchestra.run.smithi171.stdout: "options": {}, 2024-09-15T00:13:39.203 INFO:teuthology.orchestra.run.smithi171.stdout: "application_metadata": { 2024-09-15T00:13:39.203 INFO:teuthology.orchestra.run.smithi171.stdout: "rbd": {} 2024-09-15T00:13:39.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.203 INFO:teuthology.orchestra.run.smithi171.stdout: "read_balance": { 2024-09-15T00:13:39.203 INFO:teuthology.orchestra.run.smithi171.stdout: "error_message": "pool 3 has some PGs where all OSDs are with primary_affinity 0 (at least pg 3.1), read balance score may not be reliable", 2024-09-15T00:13:39.203 INFO:teuthology.orchestra.run.smithi171.stdout: "score_acting": 0, 2024-09-15T00:13:39.203 INFO:teuthology.orchestra.run.smithi171.stdout: "score_stable": 0 2024-09-15T00:13:39.204 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.204 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.204 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:13:39.204 INFO:teuthology.orchestra.run.smithi171.stdout: "osds": [ 2024-09-15T00:13:39.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.204 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 0, 2024-09-15T00:13:39.204 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "7337111f-0e02-490d-976b-3814a03371f0", 2024-09-15T00:13:39.204 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:13:39.204 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:13:39.204 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:13:39.204 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 0, 2024-09-15T00:13:39.204 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:13:39.204 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:13:39.205 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:13:39.205 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 138, 2024-09-15T00:13:39.205 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:13:39.205 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:13:39.205 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:13:39.205 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.205 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.205 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6824", 2024-09-15T00:13:39.205 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:13:39.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.206 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.206 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6825", 2024-09-15T00:13:39.206 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:13:39.206 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.206 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.206 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:13:39.206 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.206 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.206 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6826", 2024-09-15T00:13:39.206 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:13:39.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.207 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.207 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6827", 2024-09-15T00:13:39.207 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:13:39.207 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.207 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.207 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:13:39.207 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.207 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.207 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6830", 2024-09-15T00:13:39.207 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:13:39.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.207 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.208 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6831", 2024-09-15T00:13:39.208 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:13:39.208 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.208 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.208 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:13:39.208 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.208 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.208 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6828", 2024-09-15T00:13:39.208 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:13:39.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.209 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.209 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6829", 2024-09-15T00:13:39.209 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2906515165 2024-09-15T00:13:39.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.209 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.209 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.171:6825/2906515165", 2024-09-15T00:13:39.209 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.171:6827/2906515165", 2024-09-15T00:13:39.209 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.171:6831/2906515165", 2024-09-15T00:13:39.209 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.171:6829/2906515165", 2024-09-15T00:13:39.209 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:13:39.209 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:13:39.209 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:13:39.209 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.210 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 1, 2024-09-15T00:13:39.210 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "64412633-4dd4-4b8d-b3f1-ed3c664ddf48", 2024-09-15T00:13:39.210 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:13:39.210 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:13:39.210 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:13:39.210 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:13:39.210 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:13:39.210 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:13:39.210 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:13:39.210 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 139, 2024-09-15T00:13:39.210 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:13:39.210 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:13:39.210 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:13:39.210 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.211 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.211 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6816", 2024-09-15T00:13:39.211 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:13:39.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.211 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.211 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6817", 2024-09-15T00:13:39.211 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:13:39.211 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.211 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.211 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:13:39.211 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.212 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.212 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6818", 2024-09-15T00:13:39.212 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:13:39.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.212 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.212 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6819", 2024-09-15T00:13:39.212 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:13:39.212 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.212 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.212 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:13:39.212 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.213 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.213 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6822", 2024-09-15T00:13:39.213 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:13:39.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.213 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.213 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6823", 2024-09-15T00:13:39.213 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:13:39.213 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.213 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.213 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:13:39.213 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.214 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.214 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6820", 2024-09-15T00:13:39.214 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:13:39.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.214 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.214 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6821", 2024-09-15T00:13:39.214 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3039831960 2024-09-15T00:13:39.214 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.214 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.214 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.171:6817/3039831960", 2024-09-15T00:13:39.214 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.171:6819/3039831960", 2024-09-15T00:13:39.214 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.171:6823/3039831960", 2024-09-15T00:13:39.215 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.171:6821/3039831960", 2024-09-15T00:13:39.215 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:13:39.215 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:13:39.215 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:13:39.215 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.215 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 2, 2024-09-15T00:13:39.215 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "0300adee-5ce3-43ce-997b-2c0e970edbc7", 2024-09-15T00:13:39.215 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:13:39.215 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 0, 2024-09-15T00:13:39.215 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 0, 2024-09-15T00:13:39.215 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:13:39.215 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 14, 2024-09-15T00:13:39.215 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 79, 2024-09-15T00:13:39.216 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 108, 2024-09-15T00:13:39.216 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 124, 2024-09-15T00:13:39.216 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 81, 2024-09-15T00:13:39.216 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:13:39.216 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:13:39.216 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.216 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.216 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6808", 2024-09-15T00:13:39.216 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 213053077 2024-09-15T00:13:39.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.216 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.216 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6809", 2024-09-15T00:13:39.217 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 213053077 2024-09-15T00:13:39.217 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.217 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.217 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:13:39.217 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.217 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.217 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6810", 2024-09-15T00:13:39.217 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 213053077 2024-09-15T00:13:39.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.217 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.217 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6811", 2024-09-15T00:13:39.218 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 213053077 2024-09-15T00:13:39.218 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.218 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.218 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:13:39.218 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.218 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.218 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6814", 2024-09-15T00:13:39.218 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 213053077 2024-09-15T00:13:39.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.218 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.219 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6815", 2024-09-15T00:13:39.219 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 213053077 2024-09-15T00:13:39.219 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.219 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.219 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:13:39.219 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.219 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.219 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6812", 2024-09-15T00:13:39.219 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 213053077 2024-09-15T00:13:39.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.219 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.219 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6813", 2024-09-15T00:13:39.220 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 213053077 2024-09-15T00:13:39.220 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.220 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.220 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.171:6809/213053077", 2024-09-15T00:13:39.220 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.171:6811/213053077", 2024-09-15T00:13:39.220 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.171:6815/213053077", 2024-09-15T00:13:39.220 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.171:6813/213053077", 2024-09-15T00:13:39.220 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:13:39.221 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:13:39.221 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:13:39.221 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.221 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 3, 2024-09-15T00:13:39.221 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "ec668801-9b8f-401a-829c-0b1f8d8f08fb", 2024-09-15T00:13:39.221 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:13:39.221 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:13:39.221 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:13:39.221 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:13:39.221 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:13:39.221 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:13:39.222 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:13:39.222 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 138, 2024-09-15T00:13:39.222 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:13:39.222 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:13:39.222 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:13:39.222 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.222 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.222 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6800", 2024-09-15T00:13:39.222 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:13:39.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.222 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.222 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6801", 2024-09-15T00:13:39.222 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:13:39.223 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.223 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.223 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:13:39.223 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.223 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.223 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6802", 2024-09-15T00:13:39.223 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:13:39.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.223 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.223 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6803", 2024-09-15T00:13:39.223 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:13:39.224 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.224 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.224 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:13:39.224 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.224 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.224 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6806", 2024-09-15T00:13:39.224 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:13:39.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.224 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.224 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6807", 2024-09-15T00:13:39.225 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:13:39.225 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.225 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.225 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:13:39.225 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.225 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.225 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6804", 2024-09-15T00:13:39.225 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:13:39.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.225 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.225 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:6805", 2024-09-15T00:13:39.226 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1819098683 2024-09-15T00:13:39.226 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.226 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.226 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.171:6801/1819098683", 2024-09-15T00:13:39.226 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.171:6803/1819098683", 2024-09-15T00:13:39.226 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.171:6807/1819098683", 2024-09-15T00:13:39.226 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.171:6805/1819098683", 2024-09-15T00:13:39.226 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:13:39.226 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:13:39.226 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:13:39.226 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.227 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 4, 2024-09-15T00:13:39.227 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "9cc48936-5644-44c6-84bb-06510e933e8d", 2024-09-15T00:13:39.227 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:13:39.227 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:13:39.227 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:13:39.227 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 0, 2024-09-15T00:13:39.227 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:13:39.227 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:13:39.227 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:13:39.227 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 125, 2024-09-15T00:13:39.227 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:13:39.227 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:13:39.227 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:13:39.228 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.228 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.228 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6800", 2024-09-15T00:13:39.228 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:13:39.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.228 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.228 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6801", 2024-09-15T00:13:39.228 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:13:39.228 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.228 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.229 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:13:39.229 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.229 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.229 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6802", 2024-09-15T00:13:39.229 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:13:39.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.229 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.229 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6803", 2024-09-15T00:13:39.229 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:13:39.229 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.229 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.230 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:13:39.230 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.230 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.230 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6806", 2024-09-15T00:13:39.230 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:13:39.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.230 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.230 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6807", 2024-09-15T00:13:39.230 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:13:39.230 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.230 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.231 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:13:39.231 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.231 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.231 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6804", 2024-09-15T00:13:39.231 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:13:39.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.231 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.231 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6805", 2024-09-15T00:13:39.231 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2801626173 2024-09-15T00:13:39.231 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.231 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.232 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.184:6801/2801626173", 2024-09-15T00:13:39.232 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.184:6803/2801626173", 2024-09-15T00:13:39.232 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.184:6807/2801626173", 2024-09-15T00:13:39.232 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.184:6805/2801626173", 2024-09-15T00:13:39.232 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:13:39.232 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:13:39.232 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:13:39.232 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.232 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 5, 2024-09-15T00:13:39.232 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "f6c53297-9b00-476f-9be8-e76ff762b044", 2024-09-15T00:13:39.232 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:13:39.232 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:13:39.232 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:13:39.233 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:13:39.233 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:13:39.233 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:13:39.233 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:13:39.233 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 137, 2024-09-15T00:13:39.233 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:13:39.233 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:13:39.233 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:13:39.233 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.233 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.233 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6808", 2024-09-15T00:13:39.233 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:13:39.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6809", 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6810", 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.234 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.235 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6811", 2024-09-15T00:13:39.235 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:13:39.235 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.235 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.235 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:13:39.235 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.235 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.235 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6814", 2024-09-15T00:13:39.235 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:13:39.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.235 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.235 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6815", 2024-09-15T00:13:39.236 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:13:39.236 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.236 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.236 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:13:39.236 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.236 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.236 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6812", 2024-09-15T00:13:39.236 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:13:39.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.236 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.236 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6813", 2024-09-15T00:13:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3271066302 2024-09-15T00:13:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.184:6809/3271066302", 2024-09-15T00:13:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.184:6811/3271066302", 2024-09-15T00:13:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.184:6815/3271066302", 2024-09-15T00:13:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.184:6813/3271066302", 2024-09-15T00:13:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:13:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:13:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:13:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 6, 2024-09-15T00:13:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "737f08a6-2223-46b3-b9fe-6ae18b067ec3", 2024-09-15T00:13:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:13:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 1, 2024-09-15T00:13:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 1, 2024-09-15T00:13:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:13:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:13:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:13:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:13:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 125, 2024-09-15T00:13:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:13:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:13:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:13:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6816", 2024-09-15T00:13:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:13:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6817", 2024-09-15T00:13:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:13:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:13:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6818", 2024-09-15T00:13:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:13:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6819", 2024-09-15T00:13:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:13:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:13:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6822", 2024-09-15T00:13:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:13:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6823", 2024-09-15T00:13:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:13:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:13:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6820", 2024-09-15T00:13:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:13:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6821", 2024-09-15T00:13:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1903994256 2024-09-15T00:13:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.184:6817/1903994256", 2024-09-15T00:13:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.184:6819/1903994256", 2024-09-15T00:13:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.184:6823/1903994256", 2024-09-15T00:13:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.184:6821/1903994256", 2024-09-15T00:13:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:13:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:13:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:13:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 7, 2024-09-15T00:13:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "uuid": "5a5fba64-9249-426d-814b-fd251e8aa910", 2024-09-15T00:13:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "up": 1, 2024-09-15T00:13:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "in": 0, 2024-09-15T00:13:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "weight": 0, 2024-09-15T00:13:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_affinity": 1, 2024-09-15T00:13:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_begin": 0, 2024-09-15T00:13:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "last_clean_end": 0, 2024-09-15T00:13:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "up_from": 14, 2024-09-15T00:13:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "up_thru": 31, 2024-09-15T00:13:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "down_at": 0, 2024-09-15T00:13:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "lost_at": 0, 2024-09-15T00:13:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addrs": { 2024-09-15T00:13:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6824", 2024-09-15T00:13:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:13:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6825", 2024-09-15T00:13:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:13:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addrs": { 2024-09-15T00:13:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6826", 2024-09-15T00:13:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:13:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6827", 2024-09-15T00:13:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:13:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addrs": { 2024-09-15T00:13:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6830", 2024-09-15T00:13:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:13:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6831", 2024-09-15T00:13:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:13:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addrs": { 2024-09-15T00:13:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "addrvec": [ 2024-09-15T00:13:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v2", 2024-09-15T00:13:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6828", 2024-09-15T00:13:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:13:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-15T00:13:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.184:6829", 2024-09-15T00:13:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 978565004 2024-09-15T00:13:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "public_addr": "172.21.15.184:6825/978565004", 2024-09-15T00:13:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "cluster_addr": "172.21.15.184:6827/978565004", 2024-09-15T00:13:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_back_addr": "172.21.15.184:6831/978565004", 2024-09-15T00:13:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "heartbeat_front_addr": "172.21.15.184:6829/978565004", 2024-09-15T00:13:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "state": [ 2024-09-15T00:13:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "exists", 2024-09-15T00:13:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "up" 2024-09-15T00:13:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:13:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_xinfo": [ 2024-09-15T00:13:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 0, 2024-09-15T00:13:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:13:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:13:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:13:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:13:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:13:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:59.070337+0000", 2024-09-15T00:13:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:13:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 1, 2024-09-15T00:13:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:13:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:13:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:13:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:13:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:13:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.884683+0000", 2024-09-15T00:13:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:13:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 2, 2024-09-15T00:13:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:13:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:13:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:13:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:13:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 65536, 2024-09-15T00:13:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.998882+0000", 2024-09-15T00:13:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 81 2024-09-15T00:13:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 3, 2024-09-15T00:13:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:13:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:13:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:13:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:13:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:13:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.656630+0000", 2024-09-15T00:13:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:13:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 4, 2024-09-15T00:13:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:13:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:13:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:13:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:13:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:13:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.802744+0000", 2024-09-15T00:13:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:13:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 5, 2024-09-15T00:13:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:13:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:13:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:13:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:13:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:13:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.567865+0000", 2024-09-15T00:13:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:13:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 6, 2024-09-15T00:13:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:13:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:13:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:13:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:13:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 0, 2024-09-15T00:13:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:58.739964+0000", 2024-09-15T00:13:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:13:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "osd": 7, 2024-09-15T00:13:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "down_stamp": "0.000000", 2024-09-15T00:13:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_probability": 0, 2024-09-15T00:13:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "laggy_interval": 0, 2024-09-15T00:13:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "features": 4540701547738038271, 2024-09-15T00:13:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "old_weight": 65536, 2024-09-15T00:13:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "last_purged_snaps_scrub": "2024-09-15T00:10:59.565310+0000", 2024-09-15T00:13:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: "dead_epoch": 0 2024-09-15T00:13:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:13:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_upmap": [ 2024-09-15T00:13:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: "pgid": "3.9", 2024-09-15T00:13:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: "osds": [ 2024-09-15T00:13:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: 3, 2024-09-15T00:13:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: 0 2024-09-15T00:13:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:13:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_upmap_items": [], 2024-09-15T00:13:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_upmap_primaries": [], 2024-09-15T00:13:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "pg_temp": [], 2024-09-15T00:13:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "primary_temp": [], 2024-09-15T00:13:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "blocklist": {}, 2024-09-15T00:13:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "range_blocklist": {}, 2024-09-15T00:13:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "erasure_code_profiles": { 2024-09-15T00:13:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "default": { 2024-09-15T00:13:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "crush-failure-domain": "osd", 2024-09-15T00:13:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "k": "2", 2024-09-15T00:13:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "m": "1", 2024-09-15T00:13:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "plugin": "jerasure", 2024-09-15T00:13:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "technique": "reed_sol_van" 2024-09-15T00:13:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "removed_snaps_queue": [ 2024-09-15T00:13:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "pool": 3, 2024-09-15T00:13:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "snaps": [ 2024-09-15T00:13:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "begin": 23, 2024-09-15T00:13:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "length": 10 2024-09-15T00:13:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:13:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "new_removed_snaps": [ 2024-09-15T00:13:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "pool": 3, 2024-09-15T00:13:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "snaps": [ 2024-09-15T00:13:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "begin": 23, 2024-09-15T00:13:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "length": 1 2024-09-15T00:13:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-15T00:13:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "new_purged_snaps": [], 2024-09-15T00:13:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "crush_node_flags": {}, 2024-09-15T00:13:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "device_class_flags": {}, 2024-09-15T00:13:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "stretch_mode": { 2024-09-15T00:13:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "stretch_mode_enabled": false, 2024-09-15T00:13:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "stretch_bucket_count": 0, 2024-09-15T00:13:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "degraded_stretch_mode": 0, 2024-09-15T00:13:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: "recovering_stretch_mode": 0, 2024-09-15T00:13:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: "stretch_mode_bucket": 0 2024-09-15T00:13:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:39.259 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:39.260 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:39.258+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:39.262 INFO:tasks.thrashosds.thrasher:j is {'epoch': 151, 'fsid': 'bad2533c-9c09-4af4-ab93-147afbacb705', 'created': '2024-09-15T00:10:50.203030+0000', 'modified': '2024-09-15T00:13:39.171403+0000', 'last_up_change': '2024-09-15T00:12:55.092989+0000', 'last_in_change': '2024-09-15T00:13:21.033818+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T00:11:04.326356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T00:11:08.005967+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'error_message': 'pool 2 has some PGs where all OSDs are with primary_affinity 0 (at least pg 2.5), read balance score may not be reliable', 'score_acting': 0, 'score_stable': 0}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-15T00:11:17.676102+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 53, 'snap_epoch': 151, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'error_message': 'pool 3 has some PGs where all OSDs are with primary_affinity 0 (at least pg 3.1), read balance score may not be reliable', 'score_acting': 0, 'score_stable': 0}}], 'osds': [{'osd': 0, 'uuid': '7337111f-0e02-490d-976b-3814a03371f0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 138, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6824', 'nonce': 2906515165}, {'type': 'v1', 'addr': '172.21.15.171:6825', 'nonce': 2906515165}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6826', 'nonce': 2906515165}, {'type': 'v1', 'addr': '172.21.15.171:6827', 'nonce': 2906515165}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6830', 'nonce': 2906515165}, {'type': 'v1', 'addr': '172.21.15.171:6831', 'nonce': 2906515165}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6828', 'nonce': 2906515165}, {'type': 'v1', 'addr': '172.21.15.171:6829', 'nonce': 2906515165}]}, 'public_addr': '172.21.15.171:6825/2906515165', 'cluster_addr': '172.21.15.171:6827/2906515165', 'heartbeat_back_addr': '172.21.15.171:6831/2906515165', 'heartbeat_front_addr': '172.21.15.171:6829/2906515165', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '64412633-4dd4-4b8d-b3f1-ed3c664ddf48', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 139, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6816', 'nonce': 3039831960}, {'type': 'v1', 'addr': '172.21.15.171:6817', 'nonce': 3039831960}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6818', 'nonce': 3039831960}, {'type': 'v1', 'addr': '172.21.15.171:6819', 'nonce': 3039831960}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6822', 'nonce': 3039831960}, {'type': 'v1', 'addr': '172.21.15.171:6823', 'nonce': 3039831960}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6820', 'nonce': 3039831960}, {'type': 'v1', 'addr': '172.21.15.171:6821', 'nonce': 3039831960}]}, 'public_addr': '172.21.15.171:6817/3039831960', 'cluster_addr': '172.21.15.171:6819/3039831960', 'heartbeat_back_addr': '172.21.15.171:6823/3039831960', 'heartbeat_front_addr': '172.21.15.171:6821/3039831960', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0300adee-5ce3-43ce-997b-2c0e970edbc7', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 79, 'up_from': 108, 'up_thru': 124, 'down_at': 81, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6808', 'nonce': 213053077}, {'type': 'v1', 'addr': '172.21.15.171:6809', 'nonce': 213053077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6810', 'nonce': 213053077}, {'type': 'v1', 'addr': '172.21.15.171:6811', 'nonce': 213053077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6814', 'nonce': 213053077}, {'type': 'v1', 'addr': '172.21.15.171:6815', 'nonce': 213053077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6812', 'nonce': 213053077}, {'type': 'v1', 'addr': '172.21.15.171:6813', 'nonce': 213053077}]}, 'public_addr': '172.21.15.171:6809/213053077', 'cluster_addr': '172.21.15.171:6811/213053077', 'heartbeat_back_addr': '172.21.15.171:6815/213053077', 'heartbeat_front_addr': '172.21.15.171:6813/213053077', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ec668801-9b8f-401a-829c-0b1f8d8f08fb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 138, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6800', 'nonce': 1819098683}, {'type': 'v1', 'addr': '172.21.15.171:6801', 'nonce': 1819098683}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6802', 'nonce': 1819098683}, {'type': 'v1', 'addr': '172.21.15.171:6803', 'nonce': 1819098683}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6806', 'nonce': 1819098683}, {'type': 'v1', 'addr': '172.21.15.171:6807', 'nonce': 1819098683}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6804', 'nonce': 1819098683}, {'type': 'v1', 'addr': '172.21.15.171:6805', 'nonce': 1819098683}]}, 'public_addr': '172.21.15.171:6801/1819098683', 'cluster_addr': '172.21.15.171:6803/1819098683', 'heartbeat_back_addr': '172.21.15.171:6807/1819098683', 'heartbeat_front_addr': '172.21.15.171:6805/1819098683', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '9cc48936-5644-44c6-84bb-06510e933e8d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 125, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6800', 'nonce': 2801626173}, {'type': 'v1', 'addr': '172.21.15.184:6801', 'nonce': 2801626173}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6802', 'nonce': 2801626173}, {'type': 'v1', 'addr': '172.21.15.184:6803', 'nonce': 2801626173}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6806', 'nonce': 2801626173}, {'type': 'v1', 'addr': '172.21.15.184:6807', 'nonce': 2801626173}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6804', 'nonce': 2801626173}, {'type': 'v1', 'addr': '172.21.15.184:6805', 'nonce': 2801626173}]}, 'public_addr': '172.21.15.184:6801/2801626173', 'cluster_addr': '172.21.15.184:6803/2801626173', 'heartbeat_back_addr': '172.21.15.184:6807/2801626173', 'heartbeat_front_addr': '172.21.15.184:6805/2801626173', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f6c53297-9b00-476f-9be8-e76ff762b044', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 137, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6808', 'nonce': 3271066302}, {'type': 'v1', 'addr': '172.21.15.184:6809', 'nonce': 3271066302}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6810', 'nonce': 3271066302}, {'type': 'v1', 'addr': '172.21.15.184:6811', 'nonce': 3271066302}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6814', 'nonce': 3271066302}, {'type': 'v1', 'addr': '172.21.15.184:6815', 'nonce': 3271066302}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6812', 'nonce': 3271066302}, {'type': 'v1', 'addr': '172.21.15.184:6813', 'nonce': 3271066302}]}, 'public_addr': '172.21.15.184:6809/3271066302', 'cluster_addr': '172.21.15.184:6811/3271066302', 'heartbeat_back_addr': '172.21.15.184:6815/3271066302', 'heartbeat_front_addr': '172.21.15.184:6813/3271066302', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '737f08a6-2223-46b3-b9fe-6ae18b067ec3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 125, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6816', 'nonce': 1903994256}, {'type': 'v1', 'addr': '172.21.15.184:6817', 'nonce': 1903994256}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6818', 'nonce': 1903994256}, {'type': 'v1', 'addr': '172.21.15.184:6819', 'nonce': 1903994256}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6822', 'nonce': 1903994256}, {'type': 'v1', 'addr': '172.21.15.184:6823', 'nonce': 1903994256}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6820', 'nonce': 1903994256}, {'type': 'v1', 'addr': '172.21.15.184:6821', 'nonce': 1903994256}]}, 'public_addr': '172.21.15.184:6817/1903994256', 'cluster_addr': '172.21.15.184:6819/1903994256', 'heartbeat_back_addr': '172.21.15.184:6823/1903994256', 'heartbeat_front_addr': '172.21.15.184:6821/1903994256', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '5a5fba64-9249-426d-814b-fd251e8aa910', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6824', 'nonce': 978565004}, {'type': 'v1', 'addr': '172.21.15.184:6825', 'nonce': 978565004}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6826', 'nonce': 978565004}, {'type': 'v1', 'addr': '172.21.15.184:6827', 'nonce': 978565004}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6830', 'nonce': 978565004}, {'type': 'v1', 'addr': '172.21.15.184:6831', 'nonce': 978565004}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6828', 'nonce': 978565004}, {'type': 'v1', 'addr': '172.21.15.184:6829', 'nonce': 978565004}]}, 'public_addr': '172.21.15.184:6825/978565004', 'cluster_addr': '172.21.15.184:6827/978565004', 'heartbeat_back_addr': '172.21.15.184:6831/978565004', 'heartbeat_front_addr': '172.21.15.184:6829/978565004', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:59.070337+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.884683+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.998882+0000', 'dead_epoch': 81}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.656630+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.802744+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.567865+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:10:58.739964+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-15T00:10:59.565310+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.9', 'osds': [3, 0]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 23, 'length': 10}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 23, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T00:13:39.262 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-15T00:13:39.278 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:39.279 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:39.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.279 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:176677 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=28b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e151)", 2024-09-15T00:13:39.279 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:13:39.277742+0000", 2024-09-15T00:13:39.279 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.3739999999999996e-05, 2024-09-15T00:13:39.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9326000000000006e-05, 2024-09-15T00:13:39.279 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.279 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.279 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:13:39.279 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.279 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.280 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.280 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 176677 2024-09-15T00:13:39.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.280 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:39.277742+0000", 2024-09-15T00:13:39.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:39.277742+0000", 2024-09-15T00:13:39.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:39.277743+0000", 2024-09-15T00:13:39.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5400000000000001e-07 2024-09-15T00:13:39.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:39.277746+0000", 2024-09-15T00:13:39.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.875e-06 2024-09-15T00:13:39.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:39.277752+0000", 2024-09-15T00:13:39.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.207e-06 2024-09-15T00:13:39.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:39.277757+0000", 2024-09-15T00:13:39.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9960000000000001e-06 2024-09-15T00:13:39.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:39.277792+0000", 2024-09-15T00:13:39.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5259999999999998e-05 2024-09-15T00:13:39.282 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.282 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.282 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.282 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.283 INFO:teuthology.orchestra.run.smithi184.stdout: ], 2024-09-15T00:13:39.283 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 1 2024-09-15T00:13:39.283 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:39.288 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:39.362 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:39.360+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:39.432 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:39.432 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:39.432 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:39.432 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:39.432 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:39.441 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:39.462 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:39.461+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:39.544 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:39.544 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.96993889699999, 2024-09-15T00:13:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:13:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.545 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:13:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:39.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:13:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:13:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:13:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:13:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:13:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:13:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:13:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:13:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:13:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:13:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:13:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:13:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:13:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:13:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:13:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:13:39.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:13:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:13:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:13:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:13:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:13:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:13:39.550 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.96990420500001, 2024-09-15T00:13:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:13:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.551 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:13:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:13:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:13:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:13:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:13:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:13:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:13:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:13:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:13:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:13:39.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:13:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:13:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:13:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:13:39.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:13:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:13:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:13:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:13:39.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:13:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:13:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:13:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:13:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.92923051699999, 2024-09-15T00:13:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:13:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:13:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.558 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:13:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:13:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:13:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:13:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:13:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:13:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:13:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:13:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:13:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:13:39.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:13:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:13:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:13:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.92921398599999, 2024-09-15T00:13:39.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:13:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:13:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:13:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:13:39.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:13:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:13:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:13:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:13:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:13:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:13:39.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:13:39.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:13:39.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:13:39.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:13:39.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:13:39.566 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.566 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.567 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.567 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.567 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:39.567 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.929197207, 2024-09-15T00:13:39.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:13:39.567 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.568 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.568 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.568 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.568 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.568 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.568 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:13:39.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.568 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:39.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:39.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:39.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:13:39.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:13:39.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:13:39.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:13:39.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:13:39.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:13:39.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:13:39.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:13:39.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:13:39.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:13:39.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:13:39.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:13:39.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:13:39.573 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.573 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.573 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.573 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.573 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:39.573 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.92917947700001, 2024-09-15T00:13:39.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:13:39.573 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:13:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:39.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:39.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:13:39.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:13:39.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:13:39.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:13:39.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:13:39.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:13:39.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:13:39.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:13:39.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:13:39.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:13:39.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:13:39.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:13:39.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:13:39.577 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.577 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.577 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.578 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.578 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:39.578 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.92916231300001, 2024-09-15T00:13:39.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:13:39.578 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.578 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.578 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.578 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.578 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.578 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.578 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:13:39.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.579 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:39.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:39.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:39.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:13:39.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:13:39.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:13:39.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:13:39.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:13:39.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:13:39.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:13:39.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:13:39.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:13:39.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:13:39.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:13:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:13:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:13:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.929140637, 2024-09-15T00:13:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:13:39.583 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.583 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.583 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.583 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.583 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.583 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.583 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:13:39.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.583 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:39.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:39.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:39.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:13:39.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:13:39.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:13:39.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:13:39.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:13:39.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:13:39.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:13:39.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:13:39.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:13:39.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:13:39.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:13:39.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:13:39.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:13:39.587 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.587 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.587 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.587 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.587 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:39.587 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.929094513, 2024-09-15T00:13:39.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:13:39.587 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.587 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.587 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.587 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.587 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.588 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.588 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:13:39.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.588 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:39.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:39.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:39.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:13:39.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:13:39.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:13:39.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:13:39.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:13:39.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:13:39.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:13:39.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:13:39.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:13:39.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:13:39.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:13:39.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:13:39.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:13:39.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.591 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.592 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.592 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:39.592 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.929064756, 2024-09-15T00:13:39.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:13:39.592 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.592 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.592 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.592 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:13:39.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.592 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:39.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:39.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:13:39.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:13:39.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:13:39.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:13:39.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:13:39.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:39.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:13:39.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:13:39.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:13:39.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:13:39.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:13:39.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:13:39.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:13:39.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:13:39.596 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.596 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.596 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.596 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.596 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:39.596 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.929038437, 2024-09-15T00:13:39.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:13:39.596 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.596 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.597 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.597 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.597 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.597 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.597 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:13:39.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.597 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:39.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:39.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:39.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:39.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:13:39.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:13:39.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:13:39.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:13:39.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:13:39.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:13:39.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:13:39.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:13:39.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:13:39.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:13:39.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:13:39.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:13:39.600 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.600 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.600 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.601 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.601 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:39.601 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.929002575, 2024-09-15T00:13:39.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:13:39.601 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.601 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.601 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.601 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.601 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.601 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.601 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:13:39.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.602 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:39.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:39.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:39.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:13:39.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:13:39.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:13:39.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:13:39.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:13:39.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:13:39.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:13:39.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:13:39.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:13:39.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:13:39.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:13:39.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:13:39.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:13:39.605 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.605 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.605 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.605 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.605 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:39.605 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.928980857, 2024-09-15T00:13:39.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:13:39.606 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.606 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.606 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.606 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:13:39.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.606 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:39.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:39.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:13:39.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:13:39.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:13:39.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:13:39.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:13:39.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:13:39.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:13:39.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:13:39.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:13:39.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:13:39.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:13:39.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:13:39.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:13:39.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:13:39.610 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.610 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.610 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.610 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.610 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:39.610 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.92878439, 2024-09-15T00:13:39.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:13:39.610 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.610 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.610 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.610 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.611 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.611 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.611 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:13:39.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.611 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:39.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:39.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:39.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:13:39.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:13:39.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:13:39.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:13:39.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:13:39.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:13:39.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:13:39.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:13:39.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:13:39.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:13:39.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:13:39.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:13:39.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:13:39.614 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.614 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.614 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.928758551, 2024-09-15T00:13:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:13:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:13:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:39.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:39.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:39.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:13:39.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:13:39.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:13:39.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:13:39.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:13:39.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:13:39.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:13:39.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:13:39.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:13:39.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:13:39.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:13:39.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:13:39.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:13:39.619 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.619 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.619 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.619 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.619 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:39.619 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.928696905, 2024-09-15T00:13:39.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:13:39.619 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.619 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.619 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.620 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.620 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.620 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.620 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:13:39.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.620 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:39.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:39.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:13:39.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:13:39.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:13:39.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:13:39.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:13:39.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:13:39.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:13:39.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:13:39.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:13:39.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:13:39.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:13:39.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:13:39.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:13:39.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:13:39.623 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.623 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.623 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.624 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.624 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:39.624 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.928657577, 2024-09-15T00:13:39.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:13:39.624 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.624 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.624 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.624 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.624 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.624 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.624 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:13:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:13:39.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:13:39.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:13:39.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:13:39.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:13:39.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:13:39.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:13:39.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:13:39.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:13:39.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:13:39.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:13:39.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:39.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:13:39.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:13:39.628 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.628 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.628 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.628 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.628 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:39.628 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.926932177, 2024-09-15T00:13:39.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:13:39.629 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.629 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.629 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.629 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.629 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.629 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.629 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:13:39.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.629 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:39.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:39.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:13:39.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:13:39.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:13:39.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:13:39.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:13:39.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:13:39.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:13:39.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:13:39.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:13:39.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:13:39.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:13:39.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:13:39.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:13:39.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:13:39.632 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.632 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.633 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.633 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:39.633 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:39.633 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 114.92691001599999, 2024-09-15T00:13:39.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:13:39.633 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.633 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.633 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:39.633 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.633 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.633 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.634 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:13:39.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.634 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:39.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:39.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:13:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:13:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:13:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:13:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:13:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:13:39.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:13:39.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:13:39.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:13:39.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:13:39.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:13:39.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:13:39.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:13:39.637 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.637 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.637 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.637 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:145034 3.6 3:618673df:::rbd_object_map.1139646a7d4e.000000000000002b:head [writefull 0~48] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T00:13:39.637 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:39.637 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.524656411999999, 2024-09-15T00:13:39.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0081079740000001, 2024-09-15T00:13:39.638 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:39.638 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:39.638 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:39.638 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:39.638 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:39.638 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:39.638 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 145034 2024-09-15T00:13:39.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.638 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:39.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:39.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:39.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:39.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:39.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:39.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:39.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017198+0000", 2024-09-15T00:13:39.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0100000000000004e-07 2024-09-15T00:13:39.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:39.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017202+0000", 2024-09-15T00:13:39.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:13:39.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:39.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017212+0000", 2024-09-15T00:13:39.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7189999999999996e-06 2024-09-15T00:13:39.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:39.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017219+0000", 2024-09-15T00:13:39.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7839999999999998e-06 2024-09-15T00:13:39.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017268+0000", 2024-09-15T00:13:39.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:13:39.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:13:39.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017277+0000", 2024-09-15T00:13:39.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1899999999999995e-06 2024-09-15T00:13:39.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:39.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019139+0000", 2024-09-15T00:13:39.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0018626129999999 2024-09-15T00:13:39.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:39.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019236+0000", 2024-09-15T00:13:39.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6513000000000002e-05 2024-09-15T00:13:39.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:39.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019318+0000", 2024-09-15T00:13:39.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1848000000000006e-05 2024-09-15T00:13:39.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:39.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.023151+0000", 2024-09-15T00:13:39.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038332169999999999 2024-09-15T00:13:39.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:39.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025217+0000", 2024-09-15T00:13:39.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020657610000000002 2024-09-15T00:13:39.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:39.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025267+0000", 2024-09-15T00:13:39.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0198000000000003e-05 2024-09-15T00:13:39.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:39.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:39.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:39.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025305+0000", 2024-09-15T00:13:39.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8065999999999998e-05 2024-09-15T00:13:39.644 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.644 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.644 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.644 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:39.644 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:39.644 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:39.646 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:39.646 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:39.644+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:39.747 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:39.745+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:39.789 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:39.789 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:39.789 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:39.798 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:39.847 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:39.846+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:39.901 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:39.902 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:39.902 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:39.902 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:39.902 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:39.911 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:39.955 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:39.953+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:40.016 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 115.404428619, 2024-09-15T00:13:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:13:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.017 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:13:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:13:40.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:13:40.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:13:40.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:13:40.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:13:40.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:13:40.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:13:40.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:13:40.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:13:40.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:13:40.021 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.021 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.021 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.021 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:40.022 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:40.022 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 115.40440739, 2024-09-15T00:13:40.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:13:40.022 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.022 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.022 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.022 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.022 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.022 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.022 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:13:40.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.022 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:40.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:40.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:13:40.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:13:40.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:13:40.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:40.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:13:40.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:13:40.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:13:40.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:13:40.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:13:40.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:13:40.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:13:40.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:13:40.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:13:40.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:13:40.026 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.026 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.026 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.026 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:40.026 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:40.026 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 115.404381849, 2024-09-15T00:13:40.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:13:40.026 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.027 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.027 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.027 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.027 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.027 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.027 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:13:40.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.027 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:40.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:40.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:40.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:40.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:13:40.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:13:40.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:13:40.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:13:40.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:13:40.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:13:40.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:13:40.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:13:40.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:13:40.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:13:40.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:13:40.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:13:40.030 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.030 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.030 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.031 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:40.031 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:40.031 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 115.40435937399999, 2024-09-15T00:13:40.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:13:40.031 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.031 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.031 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.031 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.031 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.031 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.031 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:13:40.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.032 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:40.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:40.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:13:40.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:13:40.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:13:40.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:13:40.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:13:40.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:13:40.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:13:40.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:13:40.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:13:40.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:13:40.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:13:40.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:13:40.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:13:40.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:13:40.036 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.036 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.036 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.036 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:40.036 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:40.036 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 115.404338864, 2024-09-15T00:13:40.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:13:40.037 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.037 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.037 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.037 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.037 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.037 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.037 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:13:40.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.037 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:40.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:40.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:40.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:13:40.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:13:40.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:13:40.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:13:40.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:13:40.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:13:40.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:13:40.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:13:40.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:13:40.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:13:40.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:13:40.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:13:40.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:13:40.040 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.040 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.040 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.041 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:40.041 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:40.041 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 115.404318549, 2024-09-15T00:13:40.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:13:40.041 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.041 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.041 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.041 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.041 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.041 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.041 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:13:40.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.042 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:40.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:40.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:13:40.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:13:40.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:13:40.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:13:40.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:13:40.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:13:40.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:13:40.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:13:40.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:13:40.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:13:40.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:13:40.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:13:40.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:13:40.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:13:40.045 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.045 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.045 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.046 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:40.046 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:40.046 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 115.40429377300001, 2024-09-15T00:13:40.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:13:40.046 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.046 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.046 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.046 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.046 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.046 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.046 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:13:40.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.046 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:40.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:40.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:40.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:13:40.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:13:40.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:13:40.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:13:40.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:13:40.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:13:40.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:13:40.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:13:40.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:13:40.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:13:40.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:13:40.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:13:40.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:13:40.050 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.050 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.050 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.050 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:40.050 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:40.050 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 115.224952386, 2024-09-15T00:13:40.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:40.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:13:40.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:13:40.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:13:40.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:13:40.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:13:40.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:13:40.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:13:40.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:13:40.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:13:40.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:13:40.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:13:40.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:13:40.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:40.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:13:40.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:13:40.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:40.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:13:40.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:13:40.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:40.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:13:40.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:13:40.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:40.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:13:40.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:13:40.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:13:40.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:13:40.055 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.055 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.056 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.056 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:13:40.056 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:40.056 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 115.207998934, 2024-09-15T00:13:40.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:13:40.056 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.056 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.056 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.056 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:40.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:40.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:13:40.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:13:40.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:13:40.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:13:40.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:13:40.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:13:40.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:13:40.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:13:40.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:13:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 101.913568027, 2024-09-15T00:13:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:13:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:13:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:40.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:40.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:13:40.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:13:40.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:13:40.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:13:40.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:13:40.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:13:40.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:13:40.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:13:40.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:13:40.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:13:40.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:13:40.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:13:40.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:40.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:13:40.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:13:40.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:40.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:13:40.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:13:40.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:40.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:13:40.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:13:40.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:40.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:13:40.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:13:40.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:13:40.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:13:40.071 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.072 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.072 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.072 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:13:40.072 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:40.072 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 101.901112213, 2024-09-15T00:13:40.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:13:40.072 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.072 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.073 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.073 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:40.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:40.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:13:40.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:13:40.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:13:40.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:13:40.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:13:40.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:13:40.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:13:40.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:13:40.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:13:40.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:13:40.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:13:40.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:13:40.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:13:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:13:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:146935 3.16 3:6c79e092:::rbd_data.11394501364.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.260332695000002, 2024-09-15T00:13:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.058987488999999997, 2024-09-15T00:13:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:40.077 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.077 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.077 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.077 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 146935 2024-09-15T00:13:40.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.077 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:40.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753644+0000", 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6199999999999997e-07 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753648+0000", 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753659+0000", 2024-09-15T00:13:40.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0703000000000001e-05 2024-09-15T00:13:40.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753664+0000", 2024-09-15T00:13:40.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.04e-06 2024-09-15T00:13:40.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753709+0000", 2024-09-15T00:13:40.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4728000000000003e-05 2024-09-15T00:13:40.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753756+0000", 2024-09-15T00:13:40.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7808e-05 2024-09-15T00:13:40.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:40.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.757051+0000", 2024-09-15T00:13:40.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294249 2024-09-15T00:13:40.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:40.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812553+0000", 2024-09-15T00:13:40.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055501807 2024-09-15T00:13:40.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:40.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812564+0000", 2024-09-15T00:13:40.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1467999999999999e-05 2024-09-15T00:13:40.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:40.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812598+0000", 2024-09-15T00:13:40.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3553000000000002e-05 2024-09-15T00:13:40.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812630+0000", 2024-09-15T00:13:40.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2895000000000003e-05 2024-09-15T00:13:40.081 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.081 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.082 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.082 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:146935 3.16 e98/95 ondisk, result = 0)", 2024-09-15T00:13:40.082 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:40.082 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.249529999000003, 2024-09-15T00:13:40.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048111830000000001, 2024-09-15T00:13:40.082 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.082 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.082 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.082 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:40.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:40.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764447+0000", 2024-09-15T00:13:40.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0500000000000002e-07 2024-09-15T00:13:40.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764450+0000", 2024-09-15T00:13:40.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4400000000000001e-06 2024-09-15T00:13:40.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764462+0000", 2024-09-15T00:13:40.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1603000000000001e-05 2024-09-15T00:13:40.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764469+0000", 2024-09-15T00:13:40.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5089999999999996e-06 2024-09-15T00:13:40.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812541+0000", 2024-09-15T00:13:40.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048071585 2024-09-15T00:13:40.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812548+0000", 2024-09-15T00:13:40.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9979999999999997e-06 2024-09-15T00:13:40.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812557+0000", 2024-09-15T00:13:40.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7899999999999994e-06 2024-09-15T00:13:40.085 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.085 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.086 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.086 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:152505 3.17 3:ea3f268c:::rbd_data.113958cb0e55.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:40.086 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:40.086 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 50.347424037000003, 2024-09-15T00:13:40.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.060817959999999997, 2024-09-15T00:13:40.086 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.086 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.086 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:40.086 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.086 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.086 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.086 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 152505 2024-09-15T00:13:40.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.087 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:40.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:40.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666553+0000", 2024-09-15T00:13:40.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-15T00:13:40.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666562+0000", 2024-09-15T00:13:40.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0909999999999999e-06 2024-09-15T00:13:40.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666572+0000", 2024-09-15T00:13:40.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0214000000000001e-05 2024-09-15T00:13:40.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666577+0000", 2024-09-15T00:13:40.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3800000000000002e-06 2024-09-15T00:13:40.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666617+0000", 2024-09-15T00:13:40.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9304000000000002e-05 2024-09-15T00:13:40.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666672+0000", 2024-09-15T00:13:40.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5206e-05 2024-09-15T00:13:40.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:40.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.670821+0000", 2024-09-15T00:13:40.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041493169999999996 2024-09-15T00:13:40.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:40.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727290+0000", 2024-09-15T00:13:40.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.056468580999999997 2024-09-15T00:13:40.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:40.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727302+0000", 2024-09-15T00:13:40.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2605e-05 2024-09-15T00:13:40.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:40.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727337+0000", 2024-09-15T00:13:40.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4783999999999998e-05 2024-09-15T00:13:40.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727370+0000", 2024-09-15T00:13:40.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2366000000000001e-05 2024-09-15T00:13:40.091 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.091 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.092 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.092 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:152505 3.17 e103/68 ondisk, result = 0)", 2024-09-15T00:13:40.092 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:40.092 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 50.335345601, 2024-09-15T00:13:40.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048665938999999998, 2024-09-15T00:13:40.092 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.092 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.092 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.092 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:40.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:40.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678631+0000", 2024-09-15T00:13:40.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5099999999999998e-07 2024-09-15T00:13:40.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678634+0000", 2024-09-15T00:13:40.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.478e-06 2024-09-15T00:13:40.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678647+0000", 2024-09-15T00:13:40.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2085e-05 2024-09-15T00:13:40.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678653+0000", 2024-09-15T00:13:40.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2099999999999998e-06 2024-09-15T00:13:40.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727277+0000", 2024-09-15T00:13:40.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048624405000000002 2024-09-15T00:13:40.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727285+0000", 2024-09-15T00:13:40.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2060000000000007e-06 2024-09-15T00:13:40.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727296+0000", 2024-09-15T00:13:40.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0604e-05 2024-09-15T00:13:40.096 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.096 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.096 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.097 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153959 3.c 3:326b8454:::rbd_data.113958cb0e55.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:40.097 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:40.097 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 49.924117655000003, 2024-09-15T00:13:40.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.068102041000000002, 2024-09-15T00:13:40.097 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.097 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.097 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:40.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.097 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153959 2024-09-15T00:13:40.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.097 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:40.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:40.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089859+0000", 2024-09-15T00:13:40.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1800000000000004e-07 2024-09-15T00:13:40.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089864+0000", 2024-09-15T00:13:40.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2079999999999999e-06 2024-09-15T00:13:40.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089880+0000", 2024-09-15T00:13:40.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5695000000000001e-05 2024-09-15T00:13:40.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089889+0000", 2024-09-15T00:13:40.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.935e-06 2024-09-15T00:13:40.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089942+0000", 2024-09-15T00:13:40.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3219e-05 2024-09-15T00:13:40.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.090066+0000", 2024-09-15T00:13:40.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012385099999999999 2024-09-15T00:13:40.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:40.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.095262+0000", 2024-09-15T00:13:40.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051963950000000004 2024-09-15T00:13:40.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:40.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157865+0000", 2024-09-15T00:13:40.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.062602865999999993 2024-09-15T00:13:40.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:40.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157891+0000", 2024-09-15T00:13:40.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5778e-05 2024-09-15T00:13:40.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:40.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157924+0000", 2024-09-15T00:13:40.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3133000000000003e-05 2024-09-15T00:13:40.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157960+0000", 2024-09-15T00:13:40.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6043e-05 2024-09-15T00:13:40.102 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.102 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.102 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.102 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:153959 3.c e103/23 ondisk, result = 0)", 2024-09-15T00:13:40.102 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 49.911461997000004, 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055370211000000003, 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102515+0000", 2024-09-15T00:13:40.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.206e-06 2024-09-15T00:13:40.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102520+0000", 2024-09-15T00:13:40.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1209999999999998e-06 2024-09-15T00:13:40.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102537+0000", 2024-09-15T00:13:40.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:40.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102546+0000", 2024-09-15T00:13:40.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2329999999999995e-06 2024-09-15T00:13:40.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157847+0000", 2024-09-15T00:13:40.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055300798999999998 2024-09-15T00:13:40.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157857+0000", 2024-09-15T00:13:40.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.046e-05 2024-09-15T00:13:40.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157884+0000", 2024-09-15T00:13:40.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.671e-05 2024-09-15T00:13:40.106 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.106 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.106 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.107 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163346 3.1d 3:baa43c12:::rbd_data.11395d771336.0000000000000065:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:13:40.107 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:40.107 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.242847423999997, 2024-09-15T00:13:40.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047558544000000001, 2024-09-15T00:13:40.107 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.107 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.107 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:40.107 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.107 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.107 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.107 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163346 2024-09-15T00:13:40.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.107 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:40.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:40.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771130+0000", 2024-09-15T00:13:40.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5939999999999999e-06 2024-09-15T00:13:40.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771141+0000", 2024-09-15T00:13:40.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1001e-05 2024-09-15T00:13:40.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771154+0000", 2024-09-15T00:13:40.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2921e-05 2024-09-15T00:13:40.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771160+0000", 2024-09-15T00:13:40.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3849999999999999e-06 2024-09-15T00:13:40.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771207+0000", 2024-09-15T00:13:40.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6888999999999998e-05 2024-09-15T00:13:40.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771306+0000", 2024-09-15T00:13:40.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9259000000000006e-05 2024-09-15T00:13:40.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:40.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.774692+0000", 2024-09-15T00:13:40.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033854850000000001 2024-09-15T00:13:40.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:40.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818600+0000", 2024-09-15T00:13:40.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043908006999999999 2024-09-15T00:13:40.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:40.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818616+0000", 2024-09-15T00:13:40.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6688e-05 2024-09-15T00:13:40.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:40.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818651+0000", 2024-09-15T00:13:40.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4885000000000002e-05 2024-09-15T00:13:40.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818687+0000", 2024-09-15T00:13:40.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5429999999999998e-05 2024-09-15T00:13:40.112 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.113 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.113 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.113 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163578 3.1c 3:3efd71ab:::rbd_data.11395d771336.0000000000000060:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-15T00:13:40.113 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:40.113 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.843424675000001, 2024-09-15T00:13:40.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.085154081000000006, 2024-09-15T00:13:40.113 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.113 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.113 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:40.113 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.113 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.114 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.114 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163578 2024-09-15T00:13:40.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.114 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:40.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:40.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170552+0000", 2024-09-15T00:13:40.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8599999999999997e-07 2024-09-15T00:13:40.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170557+0000", 2024-09-15T00:13:40.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-15T00:13:40.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170567+0000", 2024-09-15T00:13:40.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0628999999999999e-05 2024-09-15T00:13:40.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170573+0000", 2024-09-15T00:13:40.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1649999999999997e-06 2024-09-15T00:13:40.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170619+0000", 2024-09-15T00:13:40.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6405e-05 2024-09-15T00:13:40.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170666+0000", 2024-09-15T00:13:40.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7259000000000003e-05 2024-09-15T00:13:40.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:40.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.175060+0000", 2024-09-15T00:13:40.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043940170000000001 2024-09-15T00:13:40.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:40.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255559+0000", 2024-09-15T00:13:40.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080499065999999994 2024-09-15T00:13:40.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:40.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255616+0000", 2024-09-15T00:13:40.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6422000000000003e-05 2024-09-15T00:13:40.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:40.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255655+0000", 2024-09-15T00:13:40.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9632999999999998e-05 2024-09-15T00:13:40.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255705+0000", 2024-09-15T00:13:40.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.973e-05 2024-09-15T00:13:40.120 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.120 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.120 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.120 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:163578 3.1c e107/23 ondisk, result = 0)", 2024-09-15T00:13:40.121 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:40.121 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.828202584000003, 2024-09-15T00:13:40.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069939984999999996, 2024-09-15T00:13:40.121 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.121 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.121 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.121 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:40.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185774+0000", 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.093e-06 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185779+0000", 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9470000000000002e-06 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185794+0000", 2024-09-15T00:13:40.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4450000000000001e-05 2024-09-15T00:13:40.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185801+0000", 2024-09-15T00:13:40.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9189999999999999e-06 2024-09-15T00:13:40.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255604+0000", 2024-09-15T00:13:40.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069802201999999994 2024-09-15T00:13:40.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255611+0000", 2024-09-15T00:13:40.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7740000000000004e-06 2024-09-15T00:13:40.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255713+0000", 2024-09-15T00:13:40.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0001016 2024-09-15T00:13:40.124 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.124 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.124 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.124 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.124 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.124 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:40.126 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:40.126 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:40.124+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:40.226 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:40.225+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:40.270 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:40.270 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:40.271 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:40.271 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:40.279 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:40.364 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:40.362+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:40.423 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:40.423 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:40.423 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:40.423 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:40.423 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:40.427 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:40.426+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:40.432 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:40.528 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:40.527+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:40.576 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:40.576 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:40.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:40.576 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:40.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.576 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:40.577 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:40.577 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 115.906292264, 2024-09-15T00:13:40.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:13:40.577 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.577 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.577 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:40.577 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.577 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.577 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.578 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:13:40.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.578 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:40.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:40.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:13:40.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:13:40.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:13:40.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:13:40.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:13:40.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:13:40.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:13:40.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:13:40.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:13:40.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:13:40.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:13:40.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:13:40.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:40.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:13:40.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:13:40.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:40.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:13:40.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:13:40.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:40.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:13:40.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:13:40.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:40.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:13:40.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:13:40.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:13:40.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:13:40.584 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.584 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.584 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.585 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125618 3.10 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:40.585 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:40.585 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.735153182000005, 2024-09-15T00:13:40.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030860942999999998, 2024-09-15T00:13:40.585 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.585 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.585 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:40.585 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.585 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.585 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.586 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125618 2024-09-15T00:13:40.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.586 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:40.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:40.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838499+0000", 2024-09-15T00:13:40.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6189999999999998e-06 2024-09-15T00:13:40.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839599+0000", 2024-09-15T00:13:40.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001099764 2024-09-15T00:13:40.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839620+0000", 2024-09-15T00:13:40.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0764e-05 2024-09-15T00:13:40.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839631+0000", 2024-09-15T00:13:40.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1335000000000001e-05 2024-09-15T00:13:40.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839725+0000", 2024-09-15T00:13:40.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4217999999999998e-05 2024-09-15T00:13:40.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839910+0000", 2024-09-15T00:13:40.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000184298 2024-09-15T00:13:40.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:40.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843171+0000", 2024-09-15T00:13:40.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032613540000000002 2024-09-15T00:13:40.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:40.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869187+0000", 2024-09-15T00:13:40.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026015467 2024-09-15T00:13:40.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:40.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869240+0000", 2024-09-15T00:13:40.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3733000000000002e-05 2024-09-15T00:13:40.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:40.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869309+0000", 2024-09-15T00:13:40.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8381999999999997e-05 2024-09-15T00:13:40.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869358+0000", 2024-09-15T00:13:40.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-15T00:13:40.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.591 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.591 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125619 3.a 3:507b29bf:::rbd_data.11396481e1ec.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:40.591 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:40.591 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.733991052999997, 2024-09-15T00:13:40.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045768870000000003, 2024-09-15T00:13:40.591 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.591 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.591 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:40.591 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.592 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125619 2024-09-15T00:13:40.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.592 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:40.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:40.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839660+0000", 2024-09-15T00:13:40.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.22e-06 2024-09-15T00:13:40.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840511+0000", 2024-09-15T00:13:40.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000851284 2024-09-15T00:13:40.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840522+0000", 2024-09-15T00:13:40.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0569e-05 2024-09-15T00:13:40.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840527+0000", 2024-09-15T00:13:40.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.412e-06 2024-09-15T00:13:40.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840573+0000", 2024-09-15T00:13:40.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5247000000000002e-05 2024-09-15T00:13:40.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840682+0000", 2024-09-15T00:13:40.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109368 2024-09-15T00:13:40.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:40.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843976+0000", 2024-09-15T00:13:40.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294445 2024-09-15T00:13:40.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:40.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870833+0000", 2024-09-15T00:13:40.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026856944000000001 2024-09-15T00:13:40.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:40.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885344+0000", 2024-09-15T00:13:40.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014510744000000001 2024-09-15T00:13:40.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:40.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885382+0000", 2024-09-15T00:13:40.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7821000000000003e-05 2024-09-15T00:13:40.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885428+0000", 2024-09-15T00:13:40.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5816000000000003e-05 2024-09-15T00:13:40.597 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.597 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.597 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.597 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125621 3.a 3:53594db9:::rbd_data.11396481e1ec.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:40.597 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:40.597 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.720467674000005, 2024-09-15T00:13:40.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032481797, 2024-09-15T00:13:40.597 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.597 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.597 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:40.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.598 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125621 2024-09-15T00:13:40.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.598 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:40.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:40.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853185+0000", 2024-09-15T00:13:40.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.43e-06 2024-09-15T00:13:40.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854125+0000", 2024-09-15T00:13:40.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000940539 2024-09-15T00:13:40.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854141+0000", 2024-09-15T00:13:40.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5568000000000001e-05 2024-09-15T00:13:40.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854148+0000", 2024-09-15T00:13:40.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6529999999999994e-06 2024-09-15T00:13:40.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870872+0000", 2024-09-15T00:13:40.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016723135 2024-09-15T00:13:40.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870975+0000", 2024-09-15T00:13:40.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010314099999999999 2024-09-15T00:13:40.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:40.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.873082+0000", 2024-09-15T00:13:40.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021069859999999999 2024-09-15T00:13:40.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:40.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885439+0000", 2024-09-15T00:13:40.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012357343999999999 2024-09-15T00:13:40.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:40.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885569+0000", 2024-09-15T00:13:40.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012995199999999999 2024-09-15T00:13:40.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:40.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885624+0000", 2024-09-15T00:13:40.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5281999999999997e-05 2024-09-15T00:13:40.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885664+0000", 2024-09-15T00:13:40.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9767000000000001e-05 2024-09-15T00:13:40.602 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.602 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.603 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.603 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:147075 3.0 3:06ab39f2:::rbd_data.11394501364.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:40.603 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:40.603 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.624569293, 2024-09-15T00:13:40.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046402906000000001, 2024-09-15T00:13:40.603 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.603 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.603 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:40.603 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.603 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.603 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.603 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 147075 2024-09-15T00:13:40.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.604 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:40.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:40.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:40.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4399999999999999e-07 2024-09-15T00:13:40.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949086+0000", 2024-09-15T00:13:40.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.138e-06 2024-09-15T00:13:40.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949096+0000", 2024-09-15T00:13:40.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0438999999999999e-05 2024-09-15T00:13:40.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949101+0000", 2024-09-15T00:13:40.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8999999999999997e-06 2024-09-15T00:13:40.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949145+0000", 2024-09-15T00:13:40.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4141000000000003e-05 2024-09-15T00:13:40.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949195+0000", 2024-09-15T00:13:40.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9657999999999998e-05 2024-09-15T00:13:40.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:40.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.950923+0000", 2024-09-15T00:13:40.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001728753 2024-09-15T00:13:40.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:40.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995390+0000", 2024-09-15T00:13:40.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044466335000000003 2024-09-15T00:13:40.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:40.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995401+0000", 2024-09-15T00:13:40.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1671000000000001e-05 2024-09-15T00:13:40.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:40.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995432+0000", 2024-09-15T00:13:40.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0704999999999998e-05 2024-09-15T00:13:40.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995484+0000", 2024-09-15T00:13:40.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1422000000000003e-05 2024-09-15T00:13:40.612 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.612 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.612 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.612 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:147075 3.0 e98/94 ondisk, result = 0)", 2024-09-15T00:13:40.613 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:40.613 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.616769572000003, 2024-09-15T00:13:40.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038514459000000001, 2024-09-15T00:13:40.613 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.613 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.613 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.614 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:40.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:40.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956881+0000", 2024-09-15T00:13:40.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0599999999999999e-07 2024-09-15T00:13:40.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956885+0000", 2024-09-15T00:13:40.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3500000000000001e-06 2024-09-15T00:13:40.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956897+0000", 2024-09-15T00:13:40.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2303999999999999e-05 2024-09-15T00:13:40.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956902+0000", 2024-09-15T00:13:40.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5169999999999999e-06 2024-09-15T00:13:40.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995377+0000", 2024-09-15T00:13:40.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038474544999999999 2024-09-15T00:13:40.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995385+0000", 2024-09-15T00:13:40.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6240000000000002e-06 2024-09-15T00:13:40.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995395+0000", 2024-09-15T00:13:40.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0213e-05 2024-09-15T00:13:40.617 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.617 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.617 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.618 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153172 3.14 3:2a7843e3:::rbd_data.113958cb0e55.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 983040~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:40.618 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:40.618 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 50.795880508000003, 2024-09-15T00:13:40.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.033232668, 2024-09-15T00:13:40.618 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.618 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.618 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:40.618 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.618 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.618 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.618 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153172 2024-09-15T00:13:40.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.618 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:40.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:40.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777770+0000", 2024-09-15T00:13:40.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.017e-06 2024-09-15T00:13:40.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788009+0000", 2024-09-15T00:13:40.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010238146 2024-09-15T00:13:40.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788024+0000", 2024-09-15T00:13:40.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5582000000000001e-05 2024-09-15T00:13:40.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788048+0000", 2024-09-15T00:13:40.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4173e-05 2024-09-15T00:13:40.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788097+0000", 2024-09-15T00:13:40.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8498999999999998e-05 2024-09-15T00:13:40.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788201+0000", 2024-09-15T00:13:40.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010398799999999999 2024-09-15T00:13:40.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:40.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788393+0000", 2024-09-15T00:13:40.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000192337 2024-09-15T00:13:40.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:40.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810637+0000", 2024-09-15T00:13:40.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.022244202000000001 2024-09-15T00:13:40.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:40.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810903+0000", 2024-09-15T00:13:40.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00026574600000000003 2024-09-15T00:13:40.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:40.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810953+0000", 2024-09-15T00:13:40.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9509000000000003e-05 2024-09-15T00:13:40.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.811002+0000", 2024-09-15T00:13:40.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9468999999999997e-05 2024-09-15T00:13:40.624 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.624 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.625 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.625 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170014 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:40.625 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:40.625 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.726585401999998, 2024-09-15T00:13:40.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030291116999999999, 2024-09-15T00:13:40.625 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.625 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.625 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.625 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.626 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.626 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.626 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170014 2024-09-15T00:13:40.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.626 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:40.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:40.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:40.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:40.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847073+0000", 2024-09-15T00:13:40.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9589999999999995e-06 2024-09-15T00:13:40.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847080+0000", 2024-09-15T00:13:40.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2930000000000003e-06 2024-09-15T00:13:40.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847086+0000", 2024-09-15T00:13:40.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.631e-06 2024-09-15T00:13:40.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877213+0000", 2024-09-15T00:13:40.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030127017999999998 2024-09-15T00:13:40.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877281+0000", 2024-09-15T00:13:40.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7916999999999994e-05 2024-09-15T00:13:40.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877356+0000", 2024-09-15T00:13:40.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4796999999999996e-05 2024-09-15T00:13:40.630 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.630 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.630 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.630 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170017 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1310720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:40.630 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:40.630 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.726556750999997, 2024-09-15T00:13:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030437623, 2024-09-15T00:13:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170017 2024-09-15T00:13:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847094+0000", 2024-09-15T00:13:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:13:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847096+0000", 2024-09-15T00:13:40.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.136e-06 2024-09-15T00:13:40.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847104+0000", 2024-09-15T00:13:40.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7190000000000001e-06 2024-09-15T00:13:40.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847109+0000", 2024-09-15T00:13:40.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3279999999999996e-06 2024-09-15T00:13:40.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877384+0000", 2024-09-15T00:13:40.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030275348000000001 2024-09-15T00:13:40.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877451+0000", 2024-09-15T00:13:40.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6509999999999998e-05 2024-09-15T00:13:40.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877531+0000", 2024-09-15T00:13:40.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9993000000000005e-05 2024-09-15T00:13:40.635 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.635 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.635 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.635 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170018 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1351680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:40.635 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:40.635 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.726534405999999, 2024-09-15T00:13:40.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030591884999999999, 2024-09-15T00:13:40.635 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.635 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.635 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.635 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.636 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.636 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.636 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170018 2024-09-15T00:13:40.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.636 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:40.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:40.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:40.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:13:40.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847118+0000", 2024-09-15T00:13:40.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.03e-06 2024-09-15T00:13:40.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847124+0000", 2024-09-15T00:13:40.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:40.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847129+0000", 2024-09-15T00:13:40.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8600000000000001e-06 2024-09-15T00:13:40.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877560+0000", 2024-09-15T00:13:40.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030431145999999999 2024-09-15T00:13:40.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877629+0000", 2024-09-15T00:13:40.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8793000000000004e-05 2024-09-15T00:13:40.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877707+0000", 2024-09-15T00:13:40.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8139999999999994e-05 2024-09-15T00:13:40.641 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.641 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.641 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.641 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170019 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1392640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:40.641 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:40.641 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.726513658000002, 2024-09-15T00:13:40.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030769023, 2024-09-15T00:13:40.641 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.641 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.641 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.642 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.642 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.642 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.642 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170019 2024-09-15T00:13:40.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.642 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:40.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:40.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847137+0000", 2024-09-15T00:13:40.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:13:40.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847139+0000", 2024-09-15T00:13:40.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.142e-06 2024-09-15T00:13:40.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847146+0000", 2024-09-15T00:13:40.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8120000000000002e-06 2024-09-15T00:13:40.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847157+0000", 2024-09-15T00:13:40.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1093e-05 2024-09-15T00:13:40.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877735+0000", 2024-09-15T00:13:40.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578195999999998 2024-09-15T00:13:40.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877810+0000", 2024-09-15T00:13:40.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5563999999999997e-05 2024-09-15T00:13:40.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877905+0000", 2024-09-15T00:13:40.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4809000000000007e-05 2024-09-15T00:13:40.645 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.645 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.646 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.646 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170020 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:40.646 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:40.646 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.726486418, 2024-09-15T00:13:40.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030917150000000001, 2024-09-15T00:13:40.646 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.646 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.646 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.646 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.646 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.646 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170020 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:40.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:13:40.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847166+0000", 2024-09-15T00:13:40.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2790000000000001e-06 2024-09-15T00:13:40.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847173+0000", 2024-09-15T00:13:40.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3969999999999999e-06 2024-09-15T00:13:40.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847177+0000", 2024-09-15T00:13:40.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8620000000000002e-06 2024-09-15T00:13:40.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877934+0000", 2024-09-15T00:13:40.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030756143999999999 2024-09-15T00:13:40.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878002+0000", 2024-09-15T00:13:40.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7951e-05 2024-09-15T00:13:40.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878081+0000", 2024-09-15T00:13:40.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9087999999999999e-05 2024-09-15T00:13:40.650 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.650 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.650 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.650 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170094 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:40.650 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:40.650 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.726229169, 2024-09-15T00:13:40.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030836142, 2024-09-15T00:13:40.650 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.651 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.651 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.651 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.651 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.651 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.651 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170094 2024-09-15T00:13:40.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.651 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:40.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.652 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:40.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.652 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:40.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3499999999999996e-07 2024-09-15T00:13:40.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.652 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847424+0000", 2024-09-15T00:13:40.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2850000000000001e-06 2024-09-15T00:13:40.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847430+0000", 2024-09-15T00:13:40.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3260000000000001e-06 2024-09-15T00:13:40.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847435+0000", 2024-09-15T00:13:40.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.356e-06 2024-09-15T00:13:40.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878110+0000", 2024-09-15T00:13:40.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030675021 2024-09-15T00:13:40.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878177+0000", 2024-09-15T00:13:40.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6506000000000003e-05 2024-09-15T00:13:40.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878257+0000", 2024-09-15T00:13:40.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0012999999999994e-05 2024-09-15T00:13:40.654 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.654 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.654 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.655 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170096 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 458752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:40.655 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:40.655 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.726187213000003, 2024-09-15T00:13:40.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030977030999999999, 2024-09-15T00:13:40.655 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.655 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.655 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.655 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.655 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.655 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.655 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170096 2024-09-15T00:13:40.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.655 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:40.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:40.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:40.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:40.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847465+0000", 2024-09-15T00:13:40.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0559999999999999e-06 2024-09-15T00:13:40.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847472+0000", 2024-09-15T00:13:40.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3130000000000004e-06 2024-09-15T00:13:40.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847477+0000", 2024-09-15T00:13:40.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9930000000000003e-06 2024-09-15T00:13:40.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878288+0000", 2024-09-15T00:13:40.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030811609 2024-09-15T00:13:40.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878358+0000", 2024-09-15T00:13:40.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0215000000000005e-05 2024-09-15T00:13:40.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878440+0000", 2024-09-15T00:13:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1366999999999995e-05 2024-09-15T00:13:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170097 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.726167128, 2024-09-15T00:13:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031134757999999998, 2024-09-15T00:13:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170097 2024-09-15T00:13:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:40.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:40.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:13:40.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847485+0000", 2024-09-15T00:13:40.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9190000000000002e-06 2024-09-15T00:13:40.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847492+0000", 2024-09-15T00:13:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7109999999999998e-06 2024-09-15T00:13:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847497+0000", 2024-09-15T00:13:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0259999999999998e-06 2024-09-15T00:13:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878469+0000", 2024-09-15T00:13:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030972115000000001 2024-09-15T00:13:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878538+0000", 2024-09-15T00:13:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8669000000000003e-05 2024-09-15T00:13:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878618+0000", 2024-09-15T00:13:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9874000000000001e-05 2024-09-15T00:13:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170099 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 532480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:40.664 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.726115565999997, 2024-09-15T00:13:40.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031262183999999998, 2024-09-15T00:13:40.664 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.664 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.664 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.664 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.664 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.664 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.664 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170099 2024-09-15T00:13:40.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.664 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:40.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:40.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847535+0000", 2024-09-15T00:13:40.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:13:40.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847537+0000", 2024-09-15T00:13:40.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:40.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847544+0000", 2024-09-15T00:13:40.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9630000000000001e-06 2024-09-15T00:13:40.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847549+0000", 2024-09-15T00:13:40.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8910000000000003e-06 2024-09-15T00:13:40.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878646+0000", 2024-09-15T00:13:40.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031097400000000001 2024-09-15T00:13:40.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878717+0000", 2024-09-15T00:13:40.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0993000000000003e-05 2024-09-15T00:13:40.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878797+0000", 2024-09-15T00:13:40.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9425000000000006e-05 2024-09-15T00:13:40.669 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.670 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.670 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.670 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170101 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:40.670 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:40.671 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.726074187000002, 2024-09-15T00:13:40.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031674568, 2024-09-15T00:13:40.671 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.671 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.671 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.671 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.671 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.672 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.672 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170101 2024-09-15T00:13:40.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.672 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.673 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:40.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.673 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.673 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.673 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.673 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:40.674 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.674 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.674 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:40.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:13:40.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847578+0000", 2024-09-15T00:13:40.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0389999999999999e-06 2024-09-15T00:13:40.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847585+0000", 2024-09-15T00:13:40.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.336e-06 2024-09-15T00:13:40.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847596+0000", 2024-09-15T00:13:40.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015999999999999e-05 2024-09-15T00:13:40.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878825+0000", 2024-09-15T00:13:40.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031229848000000001 2024-09-15T00:13:40.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878899+0000", 2024-09-15T00:13:40.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:40.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879250+0000", 2024-09-15T00:13:40.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035110000000000002 2024-09-15T00:13:40.677 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.678 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.678 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.678 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170102 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 614400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:40.678 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:40.678 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.726046889999999, 2024-09-15T00:13:40.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031577371999999999, 2024-09-15T00:13:40.678 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.678 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.678 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.678 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.678 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.678 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.678 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170102 2024-09-15T00:13:40.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.679 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:40.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:40.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:40.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:13:40.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847610+0000", 2024-09-15T00:13:40.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6329999999999999e-06 2024-09-15T00:13:40.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847618+0000", 2024-09-15T00:13:40.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4599999999999997e-06 2024-09-15T00:13:40.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847623+0000", 2024-09-15T00:13:40.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2909999999999998e-06 2024-09-15T00:13:40.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879016+0000", 2024-09-15T00:13:40.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031393227000000003 2024-09-15T00:13:40.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879091+0000", 2024-09-15T00:13:40.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5384999999999997e-05 2024-09-15T00:13:40.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879180+0000", 2024-09-15T00:13:40.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8941000000000001e-05 2024-09-15T00:13:40.682 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.682 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.682 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.682 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170132 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:40.682 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:40.683 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.725999510000001, 2024-09-15T00:13:40.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031722579000000001, 2024-09-15T00:13:40.683 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.683 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.683 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.683 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.683 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.683 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.683 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170132 2024-09-15T00:13:40.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.683 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:40.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.684 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.684 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:40.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.684 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.684 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847651+0000", 2024-09-15T00:13:40.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1900000000000003e-07 2024-09-15T00:13:40.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.685 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847653+0000", 2024-09-15T00:13:40.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0329999999999998e-06 2024-09-15T00:13:40.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.685 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847659+0000", 2024-09-15T00:13:40.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4169999999999997e-06 2024-09-15T00:13:40.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847670+0000", 2024-09-15T00:13:40.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1031e-05 2024-09-15T00:13:40.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879205+0000", 2024-09-15T00:13:40.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031534982000000003 2024-09-15T00:13:40.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879286+0000", 2024-09-15T00:13:40.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0675999999999998e-05 2024-09-15T00:13:40.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879373+0000", 2024-09-15T00:13:40.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6921000000000003e-05 2024-09-15T00:13:40.687 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.687 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.687 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.687 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170133 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:40.687 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:40.687 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.725972357000003, 2024-09-15T00:13:40.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031894817999999998, 2024-09-15T00:13:40.688 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:40.688 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:40.688 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:40.688 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:40.688 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:40.688 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:40.688 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170133 2024-09-15T00:13:40.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.688 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:40.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.688 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:40.688 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:40.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:40.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:40.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:40.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.689 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:40.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853540+0000", 2024-09-15T00:13:40.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058619409999999999 2024-09-15T00:13:40.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.689 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:40.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853545+0000", 2024-09-15T00:13:40.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3430000000000003e-06 2024-09-15T00:13:40.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.690 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:40.690 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853554+0000", 2024-09-15T00:13:40.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4069999999999999e-06 2024-09-15T00:13:40.690 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.690 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:40.690 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853561+0000", 2024-09-15T00:13:40.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5529999999999998e-06 2024-09-15T00:13:40.690 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.690 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:40.690 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879399+0000", 2024-09-15T00:13:40.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025837941 2024-09-15T00:13:40.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.691 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:40.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879477+0000", 2024-09-15T00:13:40.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8367000000000004e-05 2024-09-15T00:13:40.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:40.691 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:40.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:40.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879572+0000", 2024-09-15T00:13:40.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5266000000000006e-05 2024-09-15T00:13:40.691 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.691 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.691 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.691 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:40.691 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:40.692 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:40.693 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:40.694 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:40.691+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:40.794 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:40.792+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:40.837 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:40.838 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:40.838 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:40.838 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:40.847 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:40.895 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:40.893+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:40.950 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:40.950 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:40.950 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:40.950 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:40.950 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:40.958 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:41.002 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:41.001+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:41.029 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:13:41.070 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:41.070 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:41.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:41.070 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:41.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.070 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:41.070 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:41.070 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 134.98888983000001, 2024-09-15T00:13:41.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:13:41.070 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.071 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.071 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:41.071 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.071 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.071 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.071 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:13:41.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.071 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:41.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:41.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:13:41.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:13:41.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:13:41.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:13:41.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:13:41.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:13:41.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:13:41.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:13:41.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:13:41.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:13:41.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:41.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:13:41.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:13:41.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:41.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:13:41.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:13:41.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:13:41.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:13:41.076 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.076 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.076 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.076 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:13:41.076 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:41.076 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 134.97614928799999, 2024-09-15T00:13:41.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:13:41.076 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.076 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.076 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:41.076 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:41.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:41.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:13:41.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:13:41.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:13:41.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:13:41.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:13:41.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:13:41.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:13:41.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:13:41.079 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.079 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.080 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.080 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:41.080 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:41.080 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 134.84379352900001, 2024-09-15T00:13:41.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:13:41.080 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.080 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.080 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:41.080 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.080 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.080 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.080 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:13:41.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.081 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:41.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:41.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:13:41.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:13:41.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:13:41.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:13:41.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:13:41.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:13:41.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:13:41.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:13:41.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:13:41.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:13:41.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:13:41.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:13:41.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:41.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:13:41.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:13:41.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:41.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:13:41.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:13:41.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:41.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:13:41.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:13:41.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:41.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:13:41.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:13:41.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:13:41.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:13:41.085 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.085 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.085 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.085 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:13:41.086 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:41.086 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 134.830331884, 2024-09-15T00:13:41.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:13:41.086 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.086 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.086 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:41.086 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:41.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:41.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:13:41.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:13:41.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:13:41.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:13:41.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:13:41.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:13:41.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:13:41.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:13:41.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:13:41.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:13:41.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:13:41.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:13:41.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:13:41.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:13:41.089 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.089 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.089 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.089 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:13:41.090 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:41.090 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 125.645330401, 2024-09-15T00:13:41.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:13:41.090 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.090 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.090 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:41.090 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.090 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.090 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.092 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:13:41.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.092 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:41.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:41.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:13:41.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:13:41.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:13:41.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:13:41.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:13:41.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:13:41.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:13:41.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:13:41.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:13:41.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:13:41.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:13:41.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:13:41.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:41.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:13:41.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:13:41.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:41.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:13:41.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:13:41.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:41.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:13:41.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:13:41.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:41.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:13:41.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:13:41.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:13:41.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:13:41.096 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.096 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.096 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.096 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:13:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 125.632454134, 2024-09-15T00:13:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:13:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:41.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:13:41.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:13:41.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:13:41.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:13:41.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:13:41.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:13:41.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:13:41.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:13:41.100 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.100 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.100 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.100 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:13:41.100 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:41.101 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.17393899, 2024-09-15T00:13:41.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:13:41.101 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.101 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.101 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:41.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.101 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:13:41.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.101 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:41.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:41.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:13:41.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:13:41.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:13:41.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:13:41.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:13:41.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:13:41.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:13:41.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:13:41.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:13:41.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:13:41.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:13:41.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:13:41.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:41.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:13:41.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:13:41.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:41.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:13:41.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:13:41.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:41.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:13:41.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:13:41.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:41.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:13:41.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:13:41.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:13:41.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:13:41.106 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.106 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.106 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.106 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:41.106 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:41.106 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.45642843, 2024-09-15T00:13:41.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:13:41.106 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.106 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.106 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:41.106 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:13:41.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:13:41.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:13:41.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:13:41.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:13:41.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:13:41.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:13:41.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:13:41.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:13:41.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:13:41.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:13:41.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:13:41.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:13:41.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:13:41.110 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.110 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.110 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.111 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:41.111 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:41.111 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.45640982099999, 2024-09-15T00:13:41.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:13:41.111 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.111 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.111 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:41.111 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.111 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.111 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.111 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:13:41.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.111 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:41.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:41.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:41.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:13:41.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:13:41.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:13:41.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:13:41.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:13:41.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:13:41.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:13:41.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:13:41.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:13:41.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:13:41.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:13:41.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:13:41.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:13:41.115 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.115 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.115 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.115 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:41.115 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:41.115 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.456384497, 2024-09-15T00:13:41.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:13:41.115 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.115 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.115 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:41.115 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.115 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.116 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.116 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:13:41.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.116 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:41.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:41.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:41.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:13:41.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:13:41.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:13:41.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:13:41.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:13:41.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:13:41.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:13:41.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:13:41.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:13:41.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:13:41.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:13:41.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:13:41.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:13:41.119 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.119 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.119 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.119 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:41.119 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:41.119 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.45636617300001, 2024-09-15T00:13:41.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:13:41.119 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.120 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.120 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:41.120 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.120 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.120 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.120 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:13:41.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.120 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:41.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:13:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:13:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:13:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:13:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:13:41.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:13:41.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:13:41.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:13:41.123 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.123 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.123 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.123 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:41.123 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:41.123 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.4561836, 2024-09-15T00:13:41.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:13:41.124 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.124 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.124 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:41.124 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.124 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.124 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.124 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:13:41.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.124 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:41.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:41.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:41.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:13:41.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:13:41.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:13:41.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:13:41.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:13:41.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:13:41.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:13:41.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:13:41.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:13:41.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:13:41.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:13:41.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:13:41.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:13:41.127 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.127 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.127 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.128 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:41.128 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:41.128 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.456154759, 2024-09-15T00:13:41.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:13:41.128 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.128 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.128 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:41.128 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.128 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.128 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.128 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:13:41.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.128 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:41.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:41.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:41.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:13:41.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:13:41.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:13:41.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:13:41.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:13:41.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:13:41.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:13:41.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:13:41.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:13:41.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:13:41.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:13:41.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:13:41.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:13:41.132 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.132 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.132 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.132 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:41.132 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:41.132 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.456132548, 2024-09-15T00:13:41.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:13:41.132 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.132 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.132 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:41.132 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.133 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.133 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.133 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:13:41.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.133 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:41.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:41.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:13:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:13:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:13:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:13:41.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:13:41.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:13:41.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:13:41.136 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.136 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.136 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.136 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:41.136 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:41.136 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.456111389, 2024-09-15T00:13:41.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:13:41.136 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.137 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.137 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:41.137 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.137 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.137 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.137 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:13:41.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.137 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:41.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:41.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:41.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:13:41.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:13:41.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:13:41.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:13:41.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:13:41.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:13:41.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:13:41.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:13:41.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:13:41.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:13:41.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:13:41.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:13:41.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:13:41.140 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.140 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.140 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.140 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:41.140 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:41.140 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.45608144099999, 2024-09-15T00:13:41.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:13:41.141 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.141 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.141 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:41.141 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.141 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.141 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.141 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:13:41.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.141 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:13:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:13:41.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:13:41.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:13:41.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:13:41.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:13:41.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:13:41.144 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.144 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.144 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.145 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:41.145 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:41.145 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.456059477, 2024-09-15T00:13:41.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:13:41.145 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.145 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.145 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:41.145 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.145 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.145 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.145 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:13:41.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.145 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:41.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:41.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:41.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:13:41.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:13:41.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:13:41.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:13:41.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:13:41.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:13:41.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:13:41.148 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.148 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.149 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.149 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:41.149 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:41.149 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.45604204999999, 2024-09-15T00:13:41.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:13:41.149 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.149 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.149 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:41.149 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.149 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.149 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.149 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:13:41.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.150 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:41.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:41.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:13:41.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:13:41.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:13:41.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:13:41.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:13:41.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:41.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:13:41.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:13:41.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:13:41.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:13:41.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:13:41.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:13:41.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:13:41.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:13:41.153 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.153 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.153 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.153 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:41.153 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:41.153 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.456020251, 2024-09-15T00:13:41.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:13:41.153 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.153 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.154 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:41.154 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.154 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.154 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.154 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:13:41.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.154 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:41.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:41.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:13:41.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:13:41.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:13:41.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:13:41.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:13:41.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:13:41.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:13:41.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:13:41.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:13:41.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:13:41.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:13:41.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:13:41.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:13:41.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:13:41.157 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.157 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.157 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.157 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:41.157 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:41.158 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.406791892, 2024-09-15T00:13:41.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:13:41.158 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:41.158 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:41.158 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:41.158 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:41.158 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:41.158 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:41.158 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:13:41.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.158 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:41.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:41.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:41.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:41.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:41.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:41.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:41.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:13:41.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:13:41.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:41.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:13:41.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:13:41.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:41.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:13:41.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:13:41.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:41.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:13:41.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:13:41.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:41.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:13:41.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:13:41.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:41.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:13:41.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:13:41.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:41.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:13:41.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:13:41.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:41.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:41.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:41.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:13:41.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:13:41.163 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.163 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.163 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.163 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:41.163 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:41.163 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:41.164 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:41.165 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:41.163+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:41.167 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:41.167 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:41.187 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:41.187 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:41.203 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:41.204 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:41.220 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:41.220 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:41.237 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:41.237 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:41.254 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:41.254 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:41.261 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:13:41.265 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:41.263+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:41.271 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:41.271 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:41.272 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:41.272 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:41.281 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:41.289 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:41.290 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:41.366 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:41.364+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:41.396 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:41.396 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:41.406 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:41.467 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:41.466+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:41.550 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:41.550 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:41.559 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:41.603 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:41.601+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:41.662 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:41.663 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:41.669 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:41.667+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:41.671 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:41.769 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:41.768+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:41.816 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:41.816 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:41.825 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:41.870 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:41.869+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:41.969 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:41.969 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:41.971 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:41.970+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:41.978 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:42.072 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:42.070+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:42.123 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:42.123 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:42.131 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:42.173 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:42.171+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:42.181 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is set 2024-09-15T00:13:42.274 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:42.272+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:42.300 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:42.300 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:42.310 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:42.374 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:42.373+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:42.414 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:42.414 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:42.423 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:42.507 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:42.506+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:42.566 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:42.567 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:42.575 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:42.616 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:42.614+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:42.677 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:42.676+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:42.719 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:42.719 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:42.729 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:42.778 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:42.777+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:42.873 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:42.873 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:42.879 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:42.878+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:42.882 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:42.981 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:42.979+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:42.993 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:42.994 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:43.002 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:43.081 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:43.080+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:43.147 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:43.147 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:43.155 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:43.199 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:43.197+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:43.284 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:43.284 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:43.282+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:43.306 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:13:43.385 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:43.383+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:43.426 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:43.427 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:43.427 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:43.427 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:43.436 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:43.443 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:43.443 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:43.459 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:43.459 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:43.475 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:43.475 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:43.485 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:43.483+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:43.492 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:43.492 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:43.509 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:43.509 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:43.526 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:43.526 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:43.542 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:43.542 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:43.582 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:43.582 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:43.582 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:43.582 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:43.582 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:43.592 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:43.594 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:43.593+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:43.687 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:43.686+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:43.696 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:43.697 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:43.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:43.697 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:43.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.697 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:43.697 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:43.697 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 119.084616735, 2024-09-15T00:13:43.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:13:43.697 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.697 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.697 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:43.697 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:43.698 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:43.698 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:43.698 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:13:43.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.698 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:43.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:43.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:43.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:13:43.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:13:43.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:13:43.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:13:43.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:13:43.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:13:43.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:13:43.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:13:43.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:13:43.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:13:43.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:13:43.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:13:43.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:13:43.701 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.701 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.701 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.701 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:43.702 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:43.702 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 119.084595506, 2024-09-15T00:13:43.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:13:43.702 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.702 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.702 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:43.702 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:43.702 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:43.702 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:43.702 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:13:43.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.702 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:43.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:43.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:13:43.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:13:43.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:13:43.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:43.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:13:43.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:13:43.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:13:43.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:13:43.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:13:43.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:13:43.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:13:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:13:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:13:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:13:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 119.084569965, 2024-09-15T00:13:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:13:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:13:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:13:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:13:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:13:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:13:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:13:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:13:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:13:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:13:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:13:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:13:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:13:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:13:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 119.08454749000001, 2024-09-15T00:13:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:13:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:13:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:13:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:13:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:13:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:13:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:13:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:13:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:13:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:13:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:13:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:13:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 119.08452698000001, 2024-09-15T00:13:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:13:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:13:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:13:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:13:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:13:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:13:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:13:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:13:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:13:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:13:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:13:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:13:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:13:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:13:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:13:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 119.08450666500001, 2024-09-15T00:13:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:13:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:13:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:13:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:13:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:13:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:13:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:13:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:13:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:13:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:13:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:13:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:13:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:13:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:13:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:13:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:13:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 119.084481889, 2024-09-15T00:13:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:13:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:13:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:13:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:13:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:13:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:13:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:13:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:13:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:13:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:13:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:13:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:13:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:13:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:13:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:13:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 118.90514050199999, 2024-09-15T00:13:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:13:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:13:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:13:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:13:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:13:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:13:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:13:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:13:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:13:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:13:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:13:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:13:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:13:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:13:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:13:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:13:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:13:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:13:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:13:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:13:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:13:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:13:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:13:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:13:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:13:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 118.88818705, 2024-09-15T00:13:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:13:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:13:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:13:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:13:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:13:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:13:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:13:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:13:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:13:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:13:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:13:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:13:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:13:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:13:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:13:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:13:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.59375614299999, 2024-09-15T00:13:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:13:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:13:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:13:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:13:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:13:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:13:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:13:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:13:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:13:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:13:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:13:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:13:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:13:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:13:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:13:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:13:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:13:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:13:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:13:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:13:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:13:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:13:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:13:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:13:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:13:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 105.581300329, 2024-09-15T00:13:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:13:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:13:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:13:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:13:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:13:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:13:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:13:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:13:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:13:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:13:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:13:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.749 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.749 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:13:43.749 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:13:43.749 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.749 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.752 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.752 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:13:43.752 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:13:43.752 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.752 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.752 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.752 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.752 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.752 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:146935 3.16 3:6c79e092:::rbd_data.11394501364.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:43.752 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:43.753 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 58.940520810999999, 2024-09-15T00:13:43.753 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.058987488999999997, 2024-09-15T00:13:43.753 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.753 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.753 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:43.753 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:43.753 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:43.753 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:43.753 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 146935 2024-09-15T00:13:43.753 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.753 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.753 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.753 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.754 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:43.754 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.754 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.754 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.754 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.754 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:43.754 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.754 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.754 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.754 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.754 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753644+0000", 2024-09-15T00:13:43.754 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6199999999999997e-07 2024-09-15T00:13:43.754 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.754 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.754 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.755 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753648+0000", 2024-09-15T00:13:43.755 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-15T00:13:43.755 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.755 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.755 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.755 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753659+0000", 2024-09-15T00:13:43.755 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0703000000000001e-05 2024-09-15T00:13:43.755 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.755 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.755 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.755 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753664+0000", 2024-09-15T00:13:43.755 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.04e-06 2024-09-15T00:13:43.755 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.755 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.755 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.756 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753709+0000", 2024-09-15T00:13:43.756 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4728000000000003e-05 2024-09-15T00:13:43.756 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.756 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.756 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.756 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753756+0000", 2024-09-15T00:13:43.756 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7808e-05 2024-09-15T00:13:43.756 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.756 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.756 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:43.756 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.757051+0000", 2024-09-15T00:13:43.756 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294249 2024-09-15T00:13:43.756 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.757 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.757 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:43.757 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812553+0000", 2024-09-15T00:13:43.757 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055501807 2024-09-15T00:13:43.757 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.757 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.757 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:43.757 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812564+0000", 2024-09-15T00:13:43.757 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1467999999999999e-05 2024-09-15T00:13:43.757 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.757 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.757 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:43.757 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812598+0000", 2024-09-15T00:13:43.757 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3553000000000002e-05 2024-09-15T00:13:43.758 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.758 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.758 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.758 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812630+0000", 2024-09-15T00:13:43.758 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2895000000000003e-05 2024-09-15T00:13:43.758 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.758 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.758 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.758 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.758 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.758 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:146935 3.16 e98/95 ondisk, result = 0)", 2024-09-15T00:13:43.758 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:43.758 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 58.929718115, 2024-09-15T00:13:43.758 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048111830000000001, 2024-09-15T00:13:43.759 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.759 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.759 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:43.759 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.759 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.759 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.759 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:43.759 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.759 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.759 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.759 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.759 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:43.759 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.759 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.759 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.760 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.760 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764447+0000", 2024-09-15T00:13:43.760 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0500000000000002e-07 2024-09-15T00:13:43.760 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.760 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.760 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.760 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764450+0000", 2024-09-15T00:13:43.760 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4400000000000001e-06 2024-09-15T00:13:43.760 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.760 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.760 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.760 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764462+0000", 2024-09-15T00:13:43.760 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1603000000000001e-05 2024-09-15T00:13:43.760 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.760 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.761 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.761 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764469+0000", 2024-09-15T00:13:43.761 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5089999999999996e-06 2024-09-15T00:13:43.761 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.761 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.761 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.761 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812541+0000", 2024-09-15T00:13:43.761 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048071585 2024-09-15T00:13:43.761 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.761 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.761 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.761 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812548+0000", 2024-09-15T00:13:43.761 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9979999999999997e-06 2024-09-15T00:13:43.761 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.762 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.762 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.762 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812557+0000", 2024-09-15T00:13:43.762 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7899999999999994e-06 2024-09-15T00:13:43.762 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.762 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.762 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.762 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.762 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.762 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:152505 3.17 3:ea3f268c:::rbd_data.113958cb0e55.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:43.762 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:43.762 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 54.027612153, 2024-09-15T00:13:43.762 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.060817959999999997, 2024-09-15T00:13:43.762 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.762 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.763 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:43.763 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:43.763 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:43.763 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:43.763 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 152505 2024-09-15T00:13:43.763 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.763 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.763 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.763 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.763 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:43.763 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.763 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.763 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.763 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.764 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:43.764 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.764 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.764 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.764 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.764 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666553+0000", 2024-09-15T00:13:43.764 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-15T00:13:43.764 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.764 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.764 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.764 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666562+0000", 2024-09-15T00:13:43.764 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0909999999999999e-06 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666572+0000", 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0214000000000001e-05 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666577+0000", 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3800000000000002e-06 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666617+0000", 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9304000000000002e-05 2024-09-15T00:13:43.765 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.766 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.766 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.766 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666672+0000", 2024-09-15T00:13:43.766 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5206e-05 2024-09-15T00:13:43.766 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.766 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.766 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:43.766 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.670821+0000", 2024-09-15T00:13:43.766 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041493169999999996 2024-09-15T00:13:43.766 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.766 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.766 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:43.766 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727290+0000", 2024-09-15T00:13:43.766 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.056468580999999997 2024-09-15T00:13:43.767 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.767 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.767 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:43.767 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727302+0000", 2024-09-15T00:13:43.767 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2605e-05 2024-09-15T00:13:43.767 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.767 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.767 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:43.767 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727337+0000", 2024-09-15T00:13:43.767 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4783999999999998e-05 2024-09-15T00:13:43.767 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.767 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.767 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.767 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727370+0000", 2024-09-15T00:13:43.767 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2366000000000001e-05 2024-09-15T00:13:43.768 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.768 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.768 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.768 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.768 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.768 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:152505 3.17 e103/68 ondisk, result = 0)", 2024-09-15T00:13:43.768 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:43.768 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 54.015533716999997, 2024-09-15T00:13:43.768 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048665938999999998, 2024-09-15T00:13:43.768 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.768 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.768 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:43.768 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.768 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.769 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.769 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:43.769 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.769 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.769 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.769 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.769 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:43.769 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.769 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.769 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.769 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.769 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678631+0000", 2024-09-15T00:13:43.769 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5099999999999998e-07 2024-09-15T00:13:43.769 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.769 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.770 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.770 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678634+0000", 2024-09-15T00:13:43.770 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.478e-06 2024-09-15T00:13:43.770 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.770 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.770 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.770 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678647+0000", 2024-09-15T00:13:43.770 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2085e-05 2024-09-15T00:13:43.770 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.770 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.770 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.770 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678653+0000", 2024-09-15T00:13:43.771 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2099999999999998e-06 2024-09-15T00:13:43.771 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.771 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.771 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.771 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727277+0000", 2024-09-15T00:13:43.771 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048624405000000002 2024-09-15T00:13:43.771 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.771 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.771 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.771 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727285+0000", 2024-09-15T00:13:43.772 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2060000000000007e-06 2024-09-15T00:13:43.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.772 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.772 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.772 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727296+0000", 2024-09-15T00:13:43.772 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0604e-05 2024-09-15T00:13:43.772 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.772 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.772 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.772 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.772 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153959 3.c 3:326b8454:::rbd_data.113958cb0e55.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:43.772 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:43.773 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 53.604305771, 2024-09-15T00:13:43.773 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.068102041000000002, 2024-09-15T00:13:43.773 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.773 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.773 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:43.773 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:43.773 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:43.773 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:43.773 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153959 2024-09-15T00:13:43.773 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.773 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.773 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.773 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.773 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:43.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.774 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:43.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.774 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089859+0000", 2024-09-15T00:13:43.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1800000000000004e-07 2024-09-15T00:13:43.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.775 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.775 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089864+0000", 2024-09-15T00:13:43.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2079999999999999e-06 2024-09-15T00:13:43.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.775 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.775 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.775 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089880+0000", 2024-09-15T00:13:43.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5695000000000001e-05 2024-09-15T00:13:43.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.775 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.775 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.775 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089889+0000", 2024-09-15T00:13:43.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.935e-06 2024-09-15T00:13:43.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089942+0000", 2024-09-15T00:13:43.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3219e-05 2024-09-15T00:13:43.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.090066+0000", 2024-09-15T00:13:43.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012385099999999999 2024-09-15T00:13:43.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:43.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.095262+0000", 2024-09-15T00:13:43.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051963950000000004 2024-09-15T00:13:43.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:43.777 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157865+0000", 2024-09-15T00:13:43.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.062602865999999993 2024-09-15T00:13:43.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:43.777 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157891+0000", 2024-09-15T00:13:43.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5778e-05 2024-09-15T00:13:43.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:43.777 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157924+0000", 2024-09-15T00:13:43.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3133000000000003e-05 2024-09-15T00:13:43.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157960+0000", 2024-09-15T00:13:43.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6043e-05 2024-09-15T00:13:43.778 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.778 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.778 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.778 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:153959 3.c e103/23 ondisk, result = 0)", 2024-09-15T00:13:43.778 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:43.778 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 53.591650113, 2024-09-15T00:13:43.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055370211000000003, 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.779 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.780 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102515+0000", 2024-09-15T00:13:43.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.206e-06 2024-09-15T00:13:43.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.780 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.780 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102520+0000", 2024-09-15T00:13:43.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1209999999999998e-06 2024-09-15T00:13:43.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.780 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.780 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102537+0000", 2024-09-15T00:13:43.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:43.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102546+0000", 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2329999999999995e-06 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157847+0000", 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055300798999999998 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157857+0000", 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.046e-05 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.782 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157884+0000", 2024-09-15T00:13:43.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.671e-05 2024-09-15T00:13:43.782 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.782 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.782 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.782 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163346 3.1d 3:baa43c12:::rbd_data.11395d771336.0000000000000065:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:13:43.782 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:43.782 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 49.923035540000001, 2024-09-15T00:13:43.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047558544000000001, 2024-09-15T00:13:43.782 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.782 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.782 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:43.782 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:43.783 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:43.783 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:43.783 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163346 2024-09-15T00:13:43.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.783 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:43.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:43.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771130+0000", 2024-09-15T00:13:43.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5939999999999999e-06 2024-09-15T00:13:43.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771141+0000", 2024-09-15T00:13:43.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1001e-05 2024-09-15T00:13:43.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771154+0000", 2024-09-15T00:13:43.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2921e-05 2024-09-15T00:13:43.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771160+0000", 2024-09-15T00:13:43.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3849999999999999e-06 2024-09-15T00:13:43.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771207+0000", 2024-09-15T00:13:43.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6888999999999998e-05 2024-09-15T00:13:43.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.786 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771306+0000", 2024-09-15T00:13:43.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9259000000000006e-05 2024-09-15T00:13:43.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.786 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:43.786 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.774692+0000", 2024-09-15T00:13:43.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033854850000000001 2024-09-15T00:13:43.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.786 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:43.786 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818600+0000", 2024-09-15T00:13:43.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043908006999999999 2024-09-15T00:13:43.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.786 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:43.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818616+0000", 2024-09-15T00:13:43.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6688e-05 2024-09-15T00:13:43.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:43.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818651+0000", 2024-09-15T00:13:43.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4885000000000002e-05 2024-09-15T00:13:43.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818687+0000", 2024-09-15T00:13:43.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5429999999999998e-05 2024-09-15T00:13:43.787 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.787 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.787 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.788 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163578 3.1c 3:3efd71ab:::rbd_data.11395d771336.0000000000000060:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-15T00:13:43.788 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:43.788 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 49.523612790999998, 2024-09-15T00:13:43.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.085154081000000006, 2024-09-15T00:13:43.788 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.788 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.788 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:43.788 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:43.788 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:43.788 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:43.788 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163578 2024-09-15T00:13:43.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.788 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:43.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:43.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170552+0000", 2024-09-15T00:13:43.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8599999999999997e-07 2024-09-15T00:13:43.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170557+0000", 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170567+0000", 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0628999999999999e-05 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170573+0000", 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1649999999999997e-06 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.791 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.791 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170619+0000", 2024-09-15T00:13:43.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6405e-05 2024-09-15T00:13:43.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.791 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.791 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170666+0000", 2024-09-15T00:13:43.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7259000000000003e-05 2024-09-15T00:13:43.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.791 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:43.791 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.175060+0000", 2024-09-15T00:13:43.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043940170000000001 2024-09-15T00:13:43.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:43.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255559+0000", 2024-09-15T00:13:43.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080499065999999994 2024-09-15T00:13:43.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:43.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255616+0000", 2024-09-15T00:13:43.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6422000000000003e-05 2024-09-15T00:13:43.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:43.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255655+0000", 2024-09-15T00:13:43.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9632999999999998e-05 2024-09-15T00:13:43.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255705+0000", 2024-09-15T00:13:43.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.973e-05 2024-09-15T00:13:43.793 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.793 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.793 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.793 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:163578 3.1c e107/23 ondisk, result = 0)", 2024-09-15T00:13:43.793 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:43.793 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 49.5083907, 2024-09-15T00:13:43.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069939984999999996, 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:43.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185774+0000", 2024-09-15T00:13:43.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.093e-06 2024-09-15T00:13:43.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:43.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185779+0000", 2024-09-15T00:13:43.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9470000000000002e-06 2024-09-15T00:13:43.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:43.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185794+0000", 2024-09-15T00:13:43.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4450000000000001e-05 2024-09-15T00:13:43.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:43.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185801+0000", 2024-09-15T00:13:43.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9189999999999999e-06 2024-09-15T00:13:43.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:43.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255604+0000", 2024-09-15T00:13:43.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069802201999999994 2024-09-15T00:13:43.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:43.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255611+0000", 2024-09-15T00:13:43.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7740000000000004e-06 2024-09-15T00:13:43.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:43.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:43.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:43.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255713+0000", 2024-09-15T00:13:43.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0001016 2024-09-15T00:13:43.797 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.797 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.797 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.797 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:43.797 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:43.797 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:43.798 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:43.799 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:43.796+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:43.899 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:43.898+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:43.902 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:43.902 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:43.902 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:43.902 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:43.912 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:44.000 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:43.999+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:44.015 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:44.015 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:44.015 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:44.015 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:44.015 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:44.025 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:44.102 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:44.100+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:44.171 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:44.171 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:44.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:44.171 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:44.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.171 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:44.172 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:44.172 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 119.501195858, 2024-09-15T00:13:44.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:13:44.172 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.172 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.172 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:44.172 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.172 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.172 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.172 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:13:44.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.172 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:44.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:44.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:13:44.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:13:44.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:13:44.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:13:44.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:13:44.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:13:44.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:13:44.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:13:44.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:13:44.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:13:44.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:13:44.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:13:44.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:44.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:13:44.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:13:44.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:44.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:13:44.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:13:44.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:44.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:13:44.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:13:44.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:44.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:13:44.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:13:44.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:13:44.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:13:44.177 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.177 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.177 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.178 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125618 3.10 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:44.178 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:44.178 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.330056776000006, 2024-09-15T00:13:44.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030860942999999998, 2024-09-15T00:13:44.178 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.178 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.178 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:44.178 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.178 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.178 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.178 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125618 2024-09-15T00:13:44.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.178 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:44.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:44.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838499+0000", 2024-09-15T00:13:44.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6189999999999998e-06 2024-09-15T00:13:44.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839599+0000", 2024-09-15T00:13:44.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001099764 2024-09-15T00:13:44.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839620+0000", 2024-09-15T00:13:44.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0764e-05 2024-09-15T00:13:44.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839631+0000", 2024-09-15T00:13:44.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1335000000000001e-05 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839725+0000", 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4217999999999998e-05 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839910+0000", 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000184298 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843171+0000", 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032613540000000002 2024-09-15T00:13:44.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869187+0000", 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026015467 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869240+0000", 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3733000000000002e-05 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869309+0000", 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8381999999999997e-05 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.183 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869358+0000", 2024-09-15T00:13:44.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-15T00:13:44.183 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.183 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.183 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.183 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125619 3.a 3:507b29bf:::rbd_data.11396481e1ec.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:44.183 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:44.183 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.328894646999998, 2024-09-15T00:13:44.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045768870000000003, 2024-09-15T00:13:44.183 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.183 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.183 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:44.184 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.184 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.184 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.184 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125619 2024-09-15T00:13:44.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.184 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:44.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:44.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839660+0000", 2024-09-15T00:13:44.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.22e-06 2024-09-15T00:13:44.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840511+0000", 2024-09-15T00:13:44.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000851284 2024-09-15T00:13:44.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840522+0000", 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0569e-05 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840527+0000", 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.412e-06 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840573+0000", 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5247000000000002e-05 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840682+0000", 2024-09-15T00:13:44.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109368 2024-09-15T00:13:44.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.187 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:44.187 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843976+0000", 2024-09-15T00:13:44.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294445 2024-09-15T00:13:44.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.187 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:44.187 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870833+0000", 2024-09-15T00:13:44.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026856944000000001 2024-09-15T00:13:44.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.187 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:44.187 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885344+0000", 2024-09-15T00:13:44.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014510744000000001 2024-09-15T00:13:44.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:44.188 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885382+0000", 2024-09-15T00:13:44.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7821000000000003e-05 2024-09-15T00:13:44.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.188 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885428+0000", 2024-09-15T00:13:44.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5816000000000003e-05 2024-09-15T00:13:44.188 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.188 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.188 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.189 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125621 3.a 3:53594db9:::rbd_data.11396481e1ec.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:44.189 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:44.189 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 82.315371268000007, 2024-09-15T00:13:44.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032481797, 2024-09-15T00:13:44.189 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.189 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.189 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:44.189 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.189 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.189 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.189 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125621 2024-09-15T00:13:44.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.189 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:44.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:44.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853185+0000", 2024-09-15T00:13:44.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.43e-06 2024-09-15T00:13:44.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854125+0000", 2024-09-15T00:13:44.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000940539 2024-09-15T00:13:44.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854141+0000", 2024-09-15T00:13:44.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5568000000000001e-05 2024-09-15T00:13:44.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854148+0000", 2024-09-15T00:13:44.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6529999999999994e-06 2024-09-15T00:13:44.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.192 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.192 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870872+0000", 2024-09-15T00:13:44.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016723135 2024-09-15T00:13:44.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.192 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.192 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870975+0000", 2024-09-15T00:13:44.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010314099999999999 2024-09-15T00:13:44.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.192 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:44.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.873082+0000", 2024-09-15T00:13:44.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021069859999999999 2024-09-15T00:13:44.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.193 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:44.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885439+0000", 2024-09-15T00:13:44.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012357343999999999 2024-09-15T00:13:44.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.193 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:44.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885569+0000", 2024-09-15T00:13:44.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012995199999999999 2024-09-15T00:13:44.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:44.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885624+0000", 2024-09-15T00:13:44.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5281999999999997e-05 2024-09-15T00:13:44.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885664+0000", 2024-09-15T00:13:44.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9767000000000001e-05 2024-09-15T00:13:44.194 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.194 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.194 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.194 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:147075 3.0 3:06ab39f2:::rbd_data.11394501364.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:44.195 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:44.195 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.219472887000002, 2024-09-15T00:13:44.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046402906000000001, 2024-09-15T00:13:44.195 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.195 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.195 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:44.195 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.195 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.195 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.195 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 147075 2024-09-15T00:13:44.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.195 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:44.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:44.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:44.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4399999999999999e-07 2024-09-15T00:13:44.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949086+0000", 2024-09-15T00:13:44.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.138e-06 2024-09-15T00:13:44.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949096+0000", 2024-09-15T00:13:44.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0438999999999999e-05 2024-09-15T00:13:44.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949101+0000", 2024-09-15T00:13:44.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8999999999999997e-06 2024-09-15T00:13:44.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949145+0000", 2024-09-15T00:13:44.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4141000000000003e-05 2024-09-15T00:13:44.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949195+0000", 2024-09-15T00:13:44.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9657999999999998e-05 2024-09-15T00:13:44.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:44.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.950923+0000", 2024-09-15T00:13:44.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001728753 2024-09-15T00:13:44.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:44.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995390+0000", 2024-09-15T00:13:44.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044466335000000003 2024-09-15T00:13:44.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:44.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995401+0000", 2024-09-15T00:13:44.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1671000000000001e-05 2024-09-15T00:13:44.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:44.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995432+0000", 2024-09-15T00:13:44.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0704999999999998e-05 2024-09-15T00:13:44.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995484+0000", 2024-09-15T00:13:44.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1422000000000003e-05 2024-09-15T00:13:44.200 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.200 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.200 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.200 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:147075 3.0 e98/94 ondisk, result = 0)", 2024-09-15T00:13:44.200 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:44.200 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.211673165999997, 2024-09-15T00:13:44.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038514459000000001, 2024-09-15T00:13:44.201 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.201 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.201 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.201 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:44.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:44.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956881+0000", 2024-09-15T00:13:44.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0599999999999999e-07 2024-09-15T00:13:44.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956885+0000", 2024-09-15T00:13:44.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3500000000000001e-06 2024-09-15T00:13:44.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956897+0000", 2024-09-15T00:13:44.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2303999999999999e-05 2024-09-15T00:13:44.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956902+0000", 2024-09-15T00:13:44.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5169999999999999e-06 2024-09-15T00:13:44.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995377+0000", 2024-09-15T00:13:44.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038474544999999999 2024-09-15T00:13:44.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995385+0000", 2024-09-15T00:13:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6240000000000002e-06 2024-09-15T00:13:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995395+0000", 2024-09-15T00:13:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0213e-05 2024-09-15T00:13:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153172 3.14 3:2a7843e3:::rbd_data.113958cb0e55.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 983040~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:44.205 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 54.390784101999998, 2024-09-15T00:13:44.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.033232668, 2024-09-15T00:13:44.205 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.205 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.205 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:44.205 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.205 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.205 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.205 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153172 2024-09-15T00:13:44.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.205 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:44.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:44.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777770+0000", 2024-09-15T00:13:44.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.017e-06 2024-09-15T00:13:44.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788009+0000", 2024-09-15T00:13:44.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010238146 2024-09-15T00:13:44.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788024+0000", 2024-09-15T00:13:44.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5582000000000001e-05 2024-09-15T00:13:44.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788048+0000", 2024-09-15T00:13:44.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4173e-05 2024-09-15T00:13:44.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788097+0000", 2024-09-15T00:13:44.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8498999999999998e-05 2024-09-15T00:13:44.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788201+0000", 2024-09-15T00:13:44.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010398799999999999 2024-09-15T00:13:44.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:44.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788393+0000", 2024-09-15T00:13:44.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000192337 2024-09-15T00:13:44.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:44.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810637+0000", 2024-09-15T00:13:44.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.022244202000000001 2024-09-15T00:13:44.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:44.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810903+0000", 2024-09-15T00:13:44.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00026574600000000003 2024-09-15T00:13:44.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:44.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810953+0000", 2024-09-15T00:13:44.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9509000000000003e-05 2024-09-15T00:13:44.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.811002+0000", 2024-09-15T00:13:44.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9468999999999997e-05 2024-09-15T00:13:44.210 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.210 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.210 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.210 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170014 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:44.210 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:44.210 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.321488995999999, 2024-09-15T00:13:44.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030291116999999999, 2024-09-15T00:13:44.210 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.210 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.210 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.211 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.211 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.211 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.211 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170014 2024-09-15T00:13:44.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.211 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:44.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:44.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:44.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:44.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847073+0000", 2024-09-15T00:13:44.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9589999999999995e-06 2024-09-15T00:13:44.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847080+0000", 2024-09-15T00:13:44.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2930000000000003e-06 2024-09-15T00:13:44.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847086+0000", 2024-09-15T00:13:44.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.631e-06 2024-09-15T00:13:44.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877213+0000", 2024-09-15T00:13:44.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030127017999999998 2024-09-15T00:13:44.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877281+0000", 2024-09-15T00:13:44.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7916999999999994e-05 2024-09-15T00:13:44.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877356+0000", 2024-09-15T00:13:44.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4796999999999996e-05 2024-09-15T00:13:44.214 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.214 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.214 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.214 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170017 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1310720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:44.215 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:44.215 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.321460344999998, 2024-09-15T00:13:44.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030437623, 2024-09-15T00:13:44.215 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.215 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.215 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.215 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.215 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.215 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.215 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170017 2024-09-15T00:13:44.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.215 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:44.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:44.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847094+0000", 2024-09-15T00:13:44.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:13:44.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847096+0000", 2024-09-15T00:13:44.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.136e-06 2024-09-15T00:13:44.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847104+0000", 2024-09-15T00:13:44.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7190000000000001e-06 2024-09-15T00:13:44.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847109+0000", 2024-09-15T00:13:44.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3279999999999996e-06 2024-09-15T00:13:44.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877384+0000", 2024-09-15T00:13:44.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030275348000000001 2024-09-15T00:13:44.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877451+0000", 2024-09-15T00:13:44.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6509999999999998e-05 2024-09-15T00:13:44.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877531+0000", 2024-09-15T00:13:44.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9993000000000005e-05 2024-09-15T00:13:44.219 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.219 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.219 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.219 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170018 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1351680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:44.219 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:44.219 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.321438000000001, 2024-09-15T00:13:44.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030591884999999999, 2024-09-15T00:13:44.219 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.219 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.219 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.219 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.219 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.219 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.220 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170018 2024-09-15T00:13:44.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.220 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:44.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:44.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:44.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:13:44.221 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847118+0000", 2024-09-15T00:13:44.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.03e-06 2024-09-15T00:13:44.221 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847124+0000", 2024-09-15T00:13:44.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:44.221 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847129+0000", 2024-09-15T00:13:44.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8600000000000001e-06 2024-09-15T00:13:44.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877560+0000", 2024-09-15T00:13:44.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030431145999999999 2024-09-15T00:13:44.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877629+0000", 2024-09-15T00:13:44.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8793000000000004e-05 2024-09-15T00:13:44.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877707+0000", 2024-09-15T00:13:44.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8139999999999994e-05 2024-09-15T00:13:44.223 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.223 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.223 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.223 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170019 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1392640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:44.223 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:44.223 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.321417252000003, 2024-09-15T00:13:44.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030769023, 2024-09-15T00:13:44.224 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.224 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.224 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.224 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.224 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.224 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.224 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170019 2024-09-15T00:13:44.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.224 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:44.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.225 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.225 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.225 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:44.225 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.225 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.225 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.225 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847137+0000", 2024-09-15T00:13:44.225 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:13:44.225 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.225 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847139+0000", 2024-09-15T00:13:44.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.142e-06 2024-09-15T00:13:44.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847146+0000", 2024-09-15T00:13:44.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8120000000000002e-06 2024-09-15T00:13:44.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847157+0000", 2024-09-15T00:13:44.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1093e-05 2024-09-15T00:13:44.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877735+0000", 2024-09-15T00:13:44.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578195999999998 2024-09-15T00:13:44.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877810+0000", 2024-09-15T00:13:44.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5563999999999997e-05 2024-09-15T00:13:44.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877905+0000", 2024-09-15T00:13:44.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4809000000000007e-05 2024-09-15T00:13:44.228 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.228 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.228 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.228 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170020 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:44.228 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:44.228 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.321390012000002, 2024-09-15T00:13:44.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030917150000000001, 2024-09-15T00:13:44.228 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.228 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.228 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.228 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.229 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.229 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.229 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170020 2024-09-15T00:13:44.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.229 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:44.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:44.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:44.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:13:44.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847166+0000", 2024-09-15T00:13:44.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2790000000000001e-06 2024-09-15T00:13:44.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847173+0000", 2024-09-15T00:13:44.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3969999999999999e-06 2024-09-15T00:13:44.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847177+0000", 2024-09-15T00:13:44.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8620000000000002e-06 2024-09-15T00:13:44.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877934+0000", 2024-09-15T00:13:44.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030756143999999999 2024-09-15T00:13:44.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878002+0000", 2024-09-15T00:13:44.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7951e-05 2024-09-15T00:13:44.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878081+0000", 2024-09-15T00:13:44.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9087999999999999e-05 2024-09-15T00:13:44.232 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.232 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.232 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.233 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170094 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:44.233 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:44.233 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.321132763000001, 2024-09-15T00:13:44.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030836142, 2024-09-15T00:13:44.233 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.233 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.233 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.233 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.233 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.233 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.233 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170094 2024-09-15T00:13:44.233 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.233 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3499999999999996e-07 2024-09-15T00:13:44.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847424+0000", 2024-09-15T00:13:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2850000000000001e-06 2024-09-15T00:13:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847430+0000", 2024-09-15T00:13:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3260000000000001e-06 2024-09-15T00:13:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847435+0000", 2024-09-15T00:13:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.356e-06 2024-09-15T00:13:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878110+0000", 2024-09-15T00:13:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030675021 2024-09-15T00:13:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878177+0000", 2024-09-15T00:13:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6506000000000003e-05 2024-09-15T00:13:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878257+0000", 2024-09-15T00:13:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0012999999999994e-05 2024-09-15T00:13:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170096 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 458752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.321090806999997, 2024-09-15T00:13:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030977030999999999, 2024-09-15T00:13:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.237 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170096 2024-09-15T00:13:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847465+0000", 2024-09-15T00:13:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0559999999999999e-06 2024-09-15T00:13:44.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847472+0000", 2024-09-15T00:13:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3130000000000004e-06 2024-09-15T00:13:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847477+0000", 2024-09-15T00:13:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9930000000000003e-06 2024-09-15T00:13:44.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878288+0000", 2024-09-15T00:13:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030811609 2024-09-15T00:13:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878358+0000", 2024-09-15T00:13:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0215000000000005e-05 2024-09-15T00:13:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.241 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878440+0000", 2024-09-15T00:13:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1366999999999995e-05 2024-09-15T00:13:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170097 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.321070722000002, 2024-09-15T00:13:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031134757999999998, 2024-09-15T00:13:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.242 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170097 2024-09-15T00:13:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:13:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847485+0000", 2024-09-15T00:13:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9190000000000002e-06 2024-09-15T00:13:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847492+0000", 2024-09-15T00:13:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7109999999999998e-06 2024-09-15T00:13:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847497+0000", 2024-09-15T00:13:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0259999999999998e-06 2024-09-15T00:13:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878469+0000", 2024-09-15T00:13:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030972115000000001 2024-09-15T00:13:44.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878538+0000", 2024-09-15T00:13:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8669000000000003e-05 2024-09-15T00:13:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878618+0000", 2024-09-15T00:13:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9874000000000001e-05 2024-09-15T00:13:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170099 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 532480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.321019159999999, 2024-09-15T00:13:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031262183999999998, 2024-09-15T00:13:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170099 2024-09-15T00:13:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.247 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847535+0000", 2024-09-15T00:13:44.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:13:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847537+0000", 2024-09-15T00:13:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847544+0000", 2024-09-15T00:13:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9630000000000001e-06 2024-09-15T00:13:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847549+0000", 2024-09-15T00:13:44.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8910000000000003e-06 2024-09-15T00:13:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878646+0000", 2024-09-15T00:13:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031097400000000001 2024-09-15T00:13:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878717+0000", 2024-09-15T00:13:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0993000000000003e-05 2024-09-15T00:13:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878797+0000", 2024-09-15T00:13:44.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9425000000000006e-05 2024-09-15T00:13:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170101 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.320977781000003, 2024-09-15T00:13:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031674568, 2024-09-15T00:13:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.251 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170101 2024-09-15T00:13:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:44.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:13:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847578+0000", 2024-09-15T00:13:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0389999999999999e-06 2024-09-15T00:13:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847585+0000", 2024-09-15T00:13:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.336e-06 2024-09-15T00:13:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847596+0000", 2024-09-15T00:13:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015999999999999e-05 2024-09-15T00:13:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878825+0000", 2024-09-15T00:13:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031229848000000001 2024-09-15T00:13:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878899+0000", 2024-09-15T00:13:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879250+0000", 2024-09-15T00:13:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035110000000000002 2024-09-15T00:13:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170102 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 614400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.320950484000001, 2024-09-15T00:13:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031577371999999999, 2024-09-15T00:13:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170102 2024-09-15T00:13:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:13:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847610+0000", 2024-09-15T00:13:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6329999999999999e-06 2024-09-15T00:13:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847618+0000", 2024-09-15T00:13:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4599999999999997e-06 2024-09-15T00:13:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847623+0000", 2024-09-15T00:13:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2909999999999998e-06 2024-09-15T00:13:44.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879016+0000", 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031393227000000003 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879091+0000", 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5384999999999997e-05 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879180+0000", 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8941000000000001e-05 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.259 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170132 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.320903104000003, 2024-09-15T00:13:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031722579000000001, 2024-09-15T00:13:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.260 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170132 2024-09-15T00:13:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847651+0000", 2024-09-15T00:13:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1900000000000003e-07 2024-09-15T00:13:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847653+0000", 2024-09-15T00:13:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0329999999999998e-06 2024-09-15T00:13:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847659+0000", 2024-09-15T00:13:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4169999999999997e-06 2024-09-15T00:13:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847670+0000", 2024-09-15T00:13:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1031e-05 2024-09-15T00:13:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879205+0000", 2024-09-15T00:13:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031534982000000003 2024-09-15T00:13:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879286+0000", 2024-09-15T00:13:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0675999999999998e-05 2024-09-15T00:13:44.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879373+0000", 2024-09-15T00:13:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6921000000000003e-05 2024-09-15T00:13:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170133 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.320875950999998, 2024-09-15T00:13:44.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031894817999999998, 2024-09-15T00:13:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170133 2024-09-15T00:13:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853540+0000", 2024-09-15T00:13:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058619409999999999 2024-09-15T00:13:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853545+0000", 2024-09-15T00:13:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3430000000000003e-06 2024-09-15T00:13:44.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853554+0000", 2024-09-15T00:13:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4069999999999999e-06 2024-09-15T00:13:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853561+0000", 2024-09-15T00:13:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5529999999999998e-06 2024-09-15T00:13:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879399+0000", 2024-09-15T00:13:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025837941 2024-09-15T00:13:44.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879477+0000", 2024-09-15T00:13:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8367000000000004e-05 2024-09-15T00:13:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879572+0000", 2024-09-15T00:13:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5266000000000006e-05 2024-09-15T00:13:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.268 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.269 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:44.269 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:13:44.270 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 0, 1] out_osds: [7, 2] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-15T00:13:44.270 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:13:44.270 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-09-15T00:13:44.270 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 7 2024-09-15T00:13:44.271 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:44.268+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:44.272 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:44.371 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:44.369+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:44.379 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:44.379 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:44.379 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:44.379 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:44.389 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:44.472 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:44.470+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:44.533 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:44.533 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:44.533 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:44.533 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:44.533 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:44.541 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:44.573 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:44.572+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:44.574 INFO:tasks.ceph.mon.a.smithi171.stderr:2024-09-15T00:13:44.572+0000 7f0425d35640 -1 mon.a@0(leader).osd e156 definitely_dead 0 2024-09-15T00:13:44.645 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:44.645 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:44.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:44.646 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:44.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.646 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:44.646 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:44.646 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 138.56431412399999, 2024-09-15T00:13:44.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:13:44.646 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.646 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.646 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:44.646 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.646 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.646 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.647 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:13:44.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.647 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:44.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:44.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:13:44.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:13:44.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:13:44.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:13:44.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:13:44.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:13:44.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:13:44.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:13:44.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:13:44.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:13:44.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:13:44.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:13:44.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:44.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:13:44.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:13:44.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:44.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:13:44.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:13:44.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:44.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:13:44.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:13:44.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:44.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:13:44.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:13:44.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:13:44.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:13:44.652 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.652 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.652 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.652 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:13:44.652 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:44.652 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 138.551573582, 2024-09-15T00:13:44.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:13:44.652 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.652 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.652 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.652 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:44.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:44.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:13:44.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:13:44.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:13:44.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:13:44.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:13:44.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:13:44.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:13:44.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:13:44.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:13:44.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:13:44.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:13:44.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:13:44.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:13:44.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:13:44.656 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.656 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.656 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.656 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:44.656 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:44.656 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 138.419217823, 2024-09-15T00:13:44.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:13:44.656 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.656 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.656 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:44.657 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.657 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.657 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.657 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:13:44.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.657 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:44.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:44.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:13:44.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:13:44.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:13:44.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:13:44.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:13:44.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:13:44.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:13:44.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:13:44.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:13:44.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:13:44.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:13:44.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:13:44.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:44.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:13:44.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:13:44.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:44.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:13:44.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:13:44.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:44.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:13:44.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:13:44.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:44.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:13:44.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:13:44.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:13:44.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:13:44.662 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.662 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.662 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.662 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:13:44.662 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:44.662 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 138.40575617799999, 2024-09-15T00:13:44.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:13:44.662 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.662 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.662 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.663 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:44.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:44.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:13:44.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:13:44.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:13:44.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:13:44.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:13:44.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:13:44.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:13:44.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:13:44.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:13:44.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:13:44.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:13:44.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:13:44.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:13:44.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:13:44.666 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.666 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.666 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.667 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:13:44.667 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:44.667 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.22075469500001, 2024-09-15T00:13:44.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:13:44.667 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.667 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.667 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:44.667 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.667 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.667 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.667 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:13:44.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.667 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:44.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:44.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:13:44.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:13:44.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:13:44.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:13:44.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:13:44.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:13:44.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:13:44.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:13:44.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:13:44.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:13:44.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:13:44.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:13:44.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:44.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:13:44.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:13:44.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:44.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:13:44.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:13:44.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:44.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:13:44.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:13:44.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:44.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:13:44.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:13:44.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:13:44.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:13:44.672 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.672 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.673 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.673 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.673 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.673 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:13:44.673 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:44.673 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.20787842799999, 2024-09-15T00:13:44.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:13:44.673 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.673 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.673 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.673 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.673 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.673 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.673 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:44.674 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.674 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.674 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:44.674 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.674 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.674 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:13:44.674 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:13:44.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.674 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:13:44.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:13:44.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:13:44.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:13:44.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:13:44.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:13:44.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:13:44.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:13:44.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:13:44.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:13:44.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:13:44.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:13:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:13:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.749363284, 2024-09-15T00:13:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:13:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.677 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.678 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.678 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:13:44.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.678 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:44.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:44.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:13:44.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:13:44.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:13:44.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:13:44.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:13:44.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:13:44.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:13:44.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:13:44.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:13:44.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:13:44.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:13:44.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:13:44.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:44.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:13:44.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:13:44.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:44.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:13:44.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:13:44.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:44.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:13:44.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:13:44.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:44.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:13:44.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:13:44.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:13:44.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:13:44.683 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.683 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.683 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.683 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:44.683 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:44.683 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.031852724, 2024-09-15T00:13:44.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:13:44.684 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.684 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.684 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.684 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.684 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.684 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.684 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:13:44.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.684 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.684 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.684 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:44.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.685 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:44.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.685 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:13:44.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:13:44.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.685 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:13:44.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:13:44.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:13:44.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:13:44.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:13:44.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:13:44.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:13:44.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:13:44.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:13:44.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:13:44.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.688 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:13:44.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:13:44.688 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.688 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.688 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.688 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:44.688 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:44.688 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.031834115, 2024-09-15T00:13:44.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:13:44.688 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.689 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.689 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.689 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.689 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.689 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.689 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:13:44.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.689 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.689 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:44.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.690 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.690 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:44.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.690 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.690 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.690 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:44.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:13:44.690 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.690 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.690 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:13:44.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:13:44.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.691 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:13:44.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:13:44.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.691 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:13:44.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:13:44.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.691 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:13:44.692 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:13:44.692 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.692 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:13:44.692 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:13:44.692 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.692 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:13:44.692 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:13:44.692 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.692 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.693 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.693 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.693 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.693 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:44.693 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:44.693 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.031808791, 2024-09-15T00:13:44.693 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:13:44.693 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.693 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.693 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.693 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.693 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.693 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.694 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:13:44.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.694 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.694 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:44.694 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.694 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:44.694 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:44.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:13:44.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:13:44.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:13:44.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:13:44.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:13:44.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:13:44.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:13:44.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.696 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:13:44.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:13:44.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.696 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:13:44.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:13:44.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.697 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.697 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:13:44.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:13:44.697 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.697 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.697 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.697 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.697 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:44.697 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:44.697 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.03179046699999, 2024-09-15T00:13:44.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:13:44.697 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.697 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.697 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.698 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.698 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.698 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.698 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:13:44.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.698 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:44.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:44.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:13:44.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:13:44.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:13:44.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:13:44.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:13:44.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:44.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:13:44.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:13:44.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:13:44.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:13:44.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:13:44.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:13:44.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:13:44.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:13:44.701 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.701 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.701 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.702 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:44.702 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:44.702 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.031607894, 2024-09-15T00:13:44.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:13:44.702 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.702 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.702 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.702 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.702 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.702 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.702 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:13:44.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.702 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:44.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:44.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:44.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:13:44.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:13:44.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:13:44.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:13:44.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:13:44.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:13:44.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:13:44.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:13:44.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:13:44.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:13:44.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:13:44.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:13:44.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:13:44.706 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.706 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.706 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.706 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:44.706 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:44.706 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.031579053, 2024-09-15T00:13:44.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:13:44.706 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.706 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.706 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.707 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.707 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.707 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.707 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:13:44.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.707 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:44.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:44.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:44.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:13:44.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:13:44.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:13:44.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.709 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:13:44.709 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:13:44.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.709 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.709 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.709 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:13:44.709 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:13:44.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.709 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.709 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.709 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:13:44.709 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:13:44.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.709 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:13:44.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:13:44.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:13:44.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:13:44.710 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.710 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.710 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.710 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:44.711 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:44.711 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.031556842, 2024-09-15T00:13:44.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:13:44.711 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.711 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.711 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.711 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.711 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.711 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.711 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:13:44.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.711 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:44.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:44.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:44.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:13:44.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.713 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.713 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:13:44.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:13:44.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.713 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.713 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.713 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:13:44.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:13:44.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.713 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.713 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.713 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:13:44.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:13:44.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.714 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.714 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:13:44.714 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:13:44.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.714 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.714 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:13:44.714 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:13:44.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.714 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.714 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:13:44.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:13:44.715 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.715 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.715 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.715 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:44.715 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:44.715 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.031535683, 2024-09-15T00:13:44.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:13:44.715 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.715 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.715 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.715 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.716 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.716 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.716 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:13:44.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.716 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:44.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:44.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:44.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:13:44.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:13:44.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:13:44.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.718 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.718 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:13:44.718 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:13:44.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.718 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.718 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.718 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:13:44.718 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:13:44.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.718 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.718 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.718 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:13:44.718 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:13:44.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.718 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:13:44.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:13:44.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:13:44.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:13:44.719 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.719 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.719 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.719 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:44.720 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:44.720 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.031505735, 2024-09-15T00:13:44.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:13:44.720 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.720 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.720 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.720 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.720 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.720 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.720 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:13:44.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.720 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.720 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:44.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:44.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:44.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:13:44.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.722 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.722 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:13:44.722 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:13:44.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.722 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.722 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:13:44.722 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:13:44.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.722 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.722 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:13:44.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:13:44.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.723 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:13:44.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:13:44.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.723 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:13:44.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:13:44.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:13:44.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:13:44.724 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.724 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.724 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.724 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:44.724 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:44.724 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.031483771, 2024-09-15T00:13:44.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:13:44.725 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.725 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.725 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.725 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.725 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.725 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.725 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:13:44.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.725 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.725 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.725 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:44.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:44.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:44.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:13:44.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:13:44.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:13:44.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:13:44.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:13:44.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:13:44.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:13:44.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:13:44.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:13:44.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:13:44.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:13:44.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:13:44.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:13:44.729 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.729 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.729 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.729 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:44.729 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:44.729 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.03146634399999, 2024-09-15T00:13:44.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:13:44.729 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.729 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.729 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.729 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:13:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:13:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:13:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:13:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:13:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:13:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:13:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:13:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:13:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:13:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:13:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:13:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:13:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:13:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.031444545, 2024-09-15T00:13:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:13:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:13:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:13:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:13:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:13:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:13:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:13:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:13:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:13:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:13:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:13:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:13:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:13:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:13:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:13:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:13:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 119.982216186, 2024-09-15T00:13:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:13:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:13:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:13:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:13:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:13:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:13:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:13:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:13:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:13:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:13:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:13:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:13:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:13:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:13:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:13:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:13:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:13:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:13:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:13:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:13:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:44.744 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:13:44.744 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:13:44.744 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:44.744 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:44.744 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:44.744 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:13:44.744 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:13:44.744 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.744 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.744 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.744 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:44.744 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:44.744 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:44.746 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:44.747 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:44.744+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:44.846 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:44.845+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:44.849 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:44.849 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:44.850 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:44.850 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:44.858 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:44.947 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:44.946+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:45.003 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:45.003 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:45.003 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:45.003 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:45.003 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:45.012 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:45.022 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is set 2024-09-15T00:13:45.022 INFO:teuthology.orchestra.run.smithi171.stderr:marked in osd.7. 2024-09-15T00:13:45.046 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-09-15T00:13:45.047 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:45.046+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:45.127 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:45.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9, version: 37'47, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9@37'47, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.270210419000001, 2024-09-15T00:13:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054905559999999997, 2024-09-15T00:13:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853793+0000", 2024-09-15T00:13:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9599999999999999e-07 2024-09-15T00:13:45.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854798+0000", 2024-09-15T00:13:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010050770000000001 2024-09-15T00:13:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854814+0000", 2024-09-15T00:13:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6478999999999999e-05 2024-09-15T00:13:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854825+0000", 2024-09-15T00:13:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0492999999999999e-05 2024-09-15T00:13:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854889+0000", 2024-09-15T00:13:45.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4134999999999995e-05 2024-09-15T00:13:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854898+0000", 2024-09-15T00:13:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6260000000000001e-06 2024-09-15T00:13:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.859283+0000", 2024-09-15T00:13:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043852500000000003 2024-09-15T00:13:45.132 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15, version: 59'205, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15@59'205, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.143989963999999, 2024-09-15T00:13:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059047939999999997, 2024-09-15T00:13:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.133 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.44e-07 2024-09-15T00:13:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980977+0000", 2024-09-15T00:13:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00096402300000000005 2024-09-15T00:13:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980995+0000", 2024-09-15T00:13:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7214000000000002e-05 2024-09-15T00:13:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981007+0000", 2024-09-15T00:13:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2028000000000001e-05 2024-09-15T00:13:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981069+0000", 2024-09-15T00:13:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1977000000000006e-05 2024-09-15T00:13:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981077+0000", 2024-09-15T00:13:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2120000000000003e-06 2024-09-15T00:13:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.985918+0000", 2024-09-15T00:13:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048406960000000002 2024-09-15T00:13:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head, version: 62'221, data_included: [0~262144,393216~786432], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head@62'221, size: 1572864, copy_subset: [(0, 1572864)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.121721698999998, 2024-09-15T00:13:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054170770000000002, 2024-09-15T00:13:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002282+0000", 2024-09-15T00:13:45.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4499999999999997e-07 2024-09-15T00:13:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003196+0000", 2024-09-15T00:13:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091476399999999999 2024-09-15T00:13:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003213+0000", 2024-09-15T00:13:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6554000000000001e-05 2024-09-15T00:13:45.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003230+0000", 2024-09-15T00:13:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6617999999999999e-05 2024-09-15T00:13:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003292+0000", 2024-09-15T00:13:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2193999999999994e-05 2024-09-15T00:13:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003300+0000", 2024-09-15T00:13:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1429999999999998e-06 2024-09-15T00:13:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.007698+0000", 2024-09-15T00:13:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043981589999999996 2024-09-15T00:13:45.143 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.070991618999997, 2024-09-15T00:13:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067633270000000004, 2024-09-15T00:13:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.144 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053012+0000", 2024-09-15T00:13:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1659999999999999e-06 2024-09-15T00:13:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054086+0000", 2024-09-15T00:13:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010732789999999999 2024-09-15T00:13:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054100+0000", 2024-09-15T00:13:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4123e-05 2024-09-15T00:13:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054114+0000", 2024-09-15T00:13:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4442e-05 2024-09-15T00:13:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054192+0000", 2024-09-15T00:13:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8288000000000007e-05 2024-09-15T00:13:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054200+0000", 2024-09-15T00:13:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-15T00:13:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.059774+0000", 2024-09-15T00:13:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055741230000000003 2024-09-15T00:13:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.062674291, 2024-09-15T00:13:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049777950000000001, 2024-09-15T00:13:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061330+0000", 2024-09-15T00:13:45.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.11e-06 2024-09-15T00:13:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062350+0000", 2024-09-15T00:13:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010207969999999999 2024-09-15T00:13:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062365+0000", 2024-09-15T00:13:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4157e-05 2024-09-15T00:13:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062380+0000", 2024-09-15T00:13:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5129e-05 2024-09-15T00:13:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062467+0000", 2024-09-15T00:13:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7247e-05 2024-09-15T00:13:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062475+0000", 2024-09-15T00:13:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4300000000000006e-06 2024-09-15T00:13:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.066306+0000", 2024-09-15T00:13:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038309250000000002 2024-09-15T00:13:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.056791333, 2024-09-15T00:13:45.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052454889999999999, 2024-09-15T00:13:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067212+0000", 2024-09-15T00:13:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-07 2024-09-15T00:13:45.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068119+0000", 2024-09-15T00:13:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090671199999999995 2024-09-15T00:13:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068135+0000", 2024-09-15T00:13:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5739e-05 2024-09-15T00:13:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068150+0000", 2024-09-15T00:13:45.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5693e-05 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068236+0000", 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6100000000000006e-05 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068245+0000", 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1960000000000008e-06 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.072457+0000", 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042123719999999998 2024-09-15T00:13:45.158 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.050812983999997, 2024-09-15T00:13:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049532020000000003, 2024-09-15T00:13:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:13:45.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074164+0000", 2024-09-15T00:13:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097354400000000002 2024-09-15T00:13:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074178+0000", 2024-09-15T00:13:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4134999999999999e-05 2024-09-15T00:13:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074192+0000", 2024-09-15T00:13:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.451e-05 2024-09-15T00:13:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074272+0000", 2024-09-15T00:13:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9989999999999998e-05 2024-09-15T00:13:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074280+0000", 2024-09-15T00:13:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9990000000000008e-06 2024-09-15T00:13:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078143+0000", 2024-09-15T00:13:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00386258 2024-09-15T00:13:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [4194304~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.045102172, 2024-09-15T00:13:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049408730000000001, 2024-09-15T00:13:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.163 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:45.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2399999999999999e-07 2024-09-15T00:13:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079875+0000", 2024-09-15T00:13:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097349100000000005 2024-09-15T00:13:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079889+0000", 2024-09-15T00:13:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4148000000000001e-05 2024-09-15T00:13:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079904+0000", 2024-09-15T00:13:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5159e-05 2024-09-15T00:13:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079964+0000", 2024-09-15T00:13:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0204999999999997e-05 2024-09-15T00:13:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079971+0000", 2024-09-15T00:13:45.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2490000000000004e-06 2024-09-15T00:13:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.083841+0000", 2024-09-15T00:13:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038701970000000001 2024-09-15T00:13:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [5242880~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 6291456, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 38.039397192000003, 2024-09-15T00:13:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052525740000000003, 2024-09-15T00:13:45.167 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:13:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085579+0000", 2024-09-15T00:13:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097307600000000002 2024-09-15T00:13:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085592+0000", 2024-09-15T00:13:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3277e-05 2024-09-15T00:13:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085608+0000", 2024-09-15T00:13:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5773e-05 2024-09-15T00:13:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085687+0000", 2024-09-15T00:13:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8956999999999996e-05 2024-09-15T00:13:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085694+0000", 2024-09-15T00:13:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.956e-06 2024-09-15T00:13:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.089858+0000", 2024-09-15T00:13:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041641159999999998 2024-09-15T00:13:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:172552 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-15T00:13:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:45.172 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.934939884000002, 2024-09-15T00:13:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.997031471, 2024-09-15T00:13:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 172552 2024-09-15T00:13:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189064+0000", 2024-09-15T00:13:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.198e-06 2024-09-15T00:13:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189069+0000", 2024-09-15T00:13:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9069999999999998e-06 2024-09-15T00:13:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189080+0000", 2024-09-15T00:13:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:13:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189086+0000", 2024-09-15T00:13:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5429999999999999e-06 2024-09-15T00:13:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189128+0000", 2024-09-15T00:13:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2255000000000001e-05 2024-09-15T00:13:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189135+0000", 2024-09-15T00:13:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6200000000000001e-06 2024-09-15T00:13:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185787+0000", 2024-09-15T00:13:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99665157199999999 2024-09-15T00:13:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185926+0000", 2024-09-15T00:13:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013970399999999999 2024-09-15T00:13:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.186094+0000", 2024-09-15T00:13:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016813099999999999 2024-09-15T00:13:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.177 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9, version: 37'60, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9@37'60, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.876079525000002, 2024-09-15T00:13:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049454060000000003, 2024-09-15T00:13:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247924+0000", 2024-09-15T00:13:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:13:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248394+0000", 2024-09-15T00:13:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047063399999999999 2024-09-15T00:13:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248411+0000", 2024-09-15T00:13:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6384000000000001e-05 2024-09-15T00:13:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248441+0000", 2024-09-15T00:13:45.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0559999999999999e-05 2024-09-15T00:13:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248499+0000", 2024-09-15T00:13:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7274000000000001e-05 2024-09-15T00:13:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248507+0000", 2024-09-15T00:13:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2390000000000002e-06 2024-09-15T00:13:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.252869+0000", 2024-09-15T00:13:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043617200000000004 2024-09-15T00:13:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head, version: 59'393, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head@59'393, size: 5701632, copy_subset: [(0, 5701632)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.746631258000001, 2024-09-15T00:13:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004807469, 2024-09-15T00:13:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4799999999999998e-07 2024-09-15T00:13:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377830+0000", 2024-09-15T00:13:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000457954 2024-09-15T00:13:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377847+0000", 2024-09-15T00:13:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6741000000000001e-05 2024-09-15T00:13:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377858+0000", 2024-09-15T00:13:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0951999999999999e-05 2024-09-15T00:13:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377929+0000", 2024-09-15T00:13:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0697000000000004e-05 2024-09-15T00:13:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377941+0000", 2024-09-15T00:13:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2529e-05 2024-09-15T00:13:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.382179+0000", 2024-09-15T00:13:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042379480000000001 2024-09-15T00:13:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.188 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head, version: 88'473, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head@88'473, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.664511824999998, 2024-09-15T00:13:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005823272, 2024-09-15T00:13:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459492+0000", 2024-09-15T00:13:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9800000000000003e-07 2024-09-15T00:13:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459940+0000", 2024-09-15T00:13:45.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044854699999999999 2024-09-15T00:13:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459957+0000", 2024-09-15T00:13:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6586e-05 2024-09-15T00:13:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459968+0000", 2024-09-15T00:13:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.145e-05 2024-09-15T00:13:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460043+0000", 2024-09-15T00:13:45.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4451999999999998e-05 2024-09-15T00:13:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460057+0000", 2024-09-15T00:13:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.417e-05 2024-09-15T00:13:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.465314+0000", 2024-09-15T00:13:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052574689999999999 2024-09-15T00:13:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.429335643999998, 2024-09-15T00:13:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051811399999999999, 2024-09-15T00:13:45.193 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694668+0000", 2024-09-15T00:13:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4600000000000005e-07 2024-09-15T00:13:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695136+0000", 2024-09-15T00:13:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00046864399999999999 2024-09-15T00:13:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695153+0000", 2024-09-15T00:13:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6218e-05 2024-09-15T00:13:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695189+0000", 2024-09-15T00:13:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6393000000000002e-05 2024-09-15T00:13:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695263+0000", 2024-09-15T00:13:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4175e-05 2024-09-15T00:13:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695271+0000", 2024-09-15T00:13:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2800000000000003e-06 2024-09-15T00:13:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.699848+0000", 2024-09-15T00:13:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045768839999999998 2024-09-15T00:13:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:45.197 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.421237435999998, 2024-09-15T00:13:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048716510000000003, 2024-09-15T00:13:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702783+0000", 2024-09-15T00:13:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7268000000000001e-05 2024-09-15T00:13:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703221+0000", 2024-09-15T00:13:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00043866899999999998 2024-09-15T00:13:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703237+0000", 2024-09-15T00:13:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5553000000000002e-05 2024-09-15T00:13:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703247+0000", 2024-09-15T00:13:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0392e-05 2024-09-15T00:13:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703358+0000", 2024-09-15T00:13:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110676 2024-09-15T00:13:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703369+0000", 2024-09-15T00:13:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0716e-05 2024-09-15T00:13:45.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.707637+0000", 2024-09-15T00:13:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042683770000000003 2024-09-15T00:13:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18, version: 62'274, data_included: [0~786432], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18@62'274, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.375156353999998, 2024-09-15T00:13:45.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005453014, 2024-09-15T00:13:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748847+0000", 2024-09-15T00:13:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8299999999999997e-07 2024-09-15T00:13:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749010+0000", 2024-09-15T00:13:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000163144 2024-09-15T00:13:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749027+0000", 2024-09-15T00:13:45.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6518999999999999e-05 2024-09-15T00:13:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749037+0000", 2024-09-15T00:13:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0438000000000001e-05 2024-09-15T00:13:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749112+0000", 2024-09-15T00:13:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4946999999999999e-05 2024-09-15T00:13:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749123+0000", 2024-09-15T00:13:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1014e-05 2024-09-15T00:13:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.754299+0000", 2024-09-15T00:13:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051763690000000001 2024-09-15T00:13:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.206 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20, version: 80'334, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20@80'334, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 20=[]:{20: [20]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.320519310000002, 2024-09-15T00:13:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048933570000000001, 2024-09-15T00:13:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803484+0000", 2024-09-15T00:13:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:13:45.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803938+0000", 2024-09-15T00:13:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045364599999999999 2024-09-15T00:13:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803954+0000", 2024-09-15T00:13:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6456999999999998e-05 2024-09-15T00:13:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803965+0000", 2024-09-15T00:13:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.076e-05 2024-09-15T00:13:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804023+0000", 2024-09-15T00:13:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7787000000000001e-05 2024-09-15T00:13:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804031+0000", 2024-09-15T00:13:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2160000000000006e-06 2024-09-15T00:13:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.808377+0000", 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043458969999999996 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [0~131072,196608~786432,1048576~131072], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.283732714000003, 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047737029999999998, 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:45.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:45.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840271+0000", 2024-09-15T00:13:45.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:13:45.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840721+0000", 2024-09-15T00:13:45.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045029800000000002 2024-09-15T00:13:45.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840738+0000", 2024-09-15T00:13:45.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6671999999999999e-05 2024-09-15T00:13:45.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840749+0000", 2024-09-15T00:13:45.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1251e-05 2024-09-15T00:13:45.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840817+0000", 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8046000000000005e-05 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840825+0000", 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2519999999999999e-06 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.845044+0000", 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042185900000000004 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.214 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.215 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [1179648~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2228224, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:45.215 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:45.215 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.275708667000004, 2024-09-15T00:13:45.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053056819999999999, 2024-09-15T00:13:45.215 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.215 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.215 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.215 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848295+0000", 2024-09-15T00:13:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4000000000000001e-07 2024-09-15T00:13:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848728+0000", 2024-09-15T00:13:45.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000433709 2024-09-15T00:13:45.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848745+0000", 2024-09-15T00:13:45.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6167999999999998e-05 2024-09-15T00:13:45.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848788+0000", 2024-09-15T00:13:45.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3269000000000002e-05 2024-09-15T00:13:45.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848857+0000", 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9453000000000001e-05 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848866+0000", 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2779999999999993e-06 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.853600+0000", 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047341650000000002 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head, version: 93'455, data_included: [0~98304,114688~16384,147456~589824,753664~65536,835584~16384,868352~262144], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head@93'455, size: 2342912, copy_subset: [(0, 2342912)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1130496, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.216574641000001, 2024-09-15T00:13:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063250900000000002, 2024-09-15T00:13:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:45.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907429+0000", 2024-09-15T00:13:45.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.85e-07 2024-09-15T00:13:45.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908008+0000", 2024-09-15T00:13:45.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00057949599999999998 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908026+0000", 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7700999999999999e-05 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908038+0000", 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2131e-05 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908101+0000", 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2596999999999997e-05 2024-09-15T00:13:45.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908110+0000", 2024-09-15T00:13:45.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8540000000000003e-06 2024-09-15T00:13:45.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.913753+0000", 2024-09-15T00:13:45.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056435260000000003 2024-09-15T00:13:45.222 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.222 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.222 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.222 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.222 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.223 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:45.224 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:45.225 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:45.223+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:45.324 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:45.323+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:45.369 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:45.369 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:45.369 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:45.377 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:45.425 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:45.424+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:45.521 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:45.521 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:45.521 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:45.521 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:45.521 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:45.525 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:45.524+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:45.530 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:45.554 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:13:45.627 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:45.626+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:45.678 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:45.678 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133834 3.1b e85/69 3:da9a78a3:::rbd_data.11395dfcf2b8.0000000000000000:head v 85'356, mlcod=85'354)", 2024-09-15T00:13:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.112210451999999, 2024-09-15T00:13:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026079054000000001, 2024-09-15T00:13:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563338+0000", 2024-09-15T00:13:45.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.418e-06 2024-09-15T00:13:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569385+0000", 2024-09-15T00:13:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006047335 2024-09-15T00:13:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569418+0000", 2024-09-15T00:13:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2957999999999999e-05 2024-09-15T00:13:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569432+0000", 2024-09-15T00:13:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3718e-05 2024-09-15T00:13:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569519+0000", 2024-09-15T00:13:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7022999999999997e-05 2024-09-15T00:13:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569540+0000", 2024-09-15T00:13:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1426000000000001e-05 2024-09-15T00:13:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589362+0000", 2024-09-15T00:13:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019821398 2024-09-15T00:13:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589415+0000", 2024-09-15T00:13:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3777999999999999e-05 2024-09-15T00:13:45.683 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133836 3.1 e85/67 3:849f9a98:::rbd_data.11395dfcf2b8.0000000000000002:head v 85'304, mlcod=84'301)", 2024-09-15T00:13:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 75.104515542000001, 2024-09-15T00:13:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023231600000000002, 2024-09-15T00:13:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.684 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:45.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571032+0000", 2024-09-15T00:13:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-15T00:13:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576635+0000", 2024-09-15T00:13:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056028700000000002 2024-09-15T00:13:45.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576650+0000", 2024-09-15T00:13:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.505e-05 2024-09-15T00:13:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576658+0000", 2024-09-15T00:13:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6879999999999999e-06 2024-09-15T00:13:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576717+0000", 2024-09-15T00:13:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175000000000002e-05 2024-09-15T00:13:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576742+0000", 2024-09-15T00:13:45.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4734000000000001e-05 2024-09-15T00:13:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594234+0000", 2024-09-15T00:13:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017492527000000001 2024-09-15T00:13:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594263+0000", 2024-09-15T00:13:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8629e-05 2024-09-15T00:13:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.689 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146609 3.b e98/69 3:d39b42ba:::rbd_data.11394501364.0000000000000001:head v 98'283, mlcod=93'281)", 2024-09-15T00:13:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 61.194797426999997, 2024-09-15T00:13:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020892945, 2024-09-15T00:13:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.690 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480751+0000", 2024-09-15T00:13:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6509999999999999e-06 2024-09-15T00:13:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482738+0000", 2024-09-15T00:13:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019867410000000002 2024-09-15T00:13:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482754+0000", 2024-09-15T00:13:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6171000000000002e-05 2024-09-15T00:13:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482769+0000", 2024-09-15T00:13:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4688000000000001e-05 2024-09-15T00:13:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482835+0000", 2024-09-15T00:13:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6526000000000006e-05 2024-09-15T00:13:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482858+0000", 2024-09-15T00:13:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2959000000000002e-05 2024-09-15T00:13:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501587+0000", 2024-09-15T00:13:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018728471 2024-09-15T00:13:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501642+0000", 2024-09-15T00:13:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5738000000000001e-05 2024-09-15T00:13:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146835 3.12 e98/68 3:4eee39d6:::rbd_data.11394501364.0000000000000000:head v 98'474, mlcod=93'472)", 2024-09-15T00:13:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 61.086170156999998, 2024-09-15T00:13:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020080434000000001, 2024-09-15T00:13:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.696 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589378+0000", 2024-09-15T00:13:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7770000000000001e-06 2024-09-15T00:13:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590678+0000", 2024-09-15T00:13:45.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012991750000000001 2024-09-15T00:13:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590699+0000", 2024-09-15T00:13:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1353e-05 2024-09-15T00:13:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590710+0000", 2024-09-15T00:13:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1245e-05 2024-09-15T00:13:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590800+0000", 2024-09-15T00:13:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9963000000000006e-05 2024-09-15T00:13:45.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590823+0000", 2024-09-15T00:13:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3289e-05 2024-09-15T00:13:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609368+0000", 2024-09-15T00:13:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018545078 2024-09-15T00:13:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609457+0000", 2024-09-15T00:13:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8554000000000006e-05 2024-09-15T00:13:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146837 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7201319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 61.075390693999999, 2024-09-15T00:13:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024751038, 2024-09-15T00:13:45.702 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146837 2024-09-15T00:13:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600157+0000", 2024-09-15T00:13:45.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:13:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601965+0000", 2024-09-15T00:13:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001807613 2024-09-15T00:13:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601985+0000", 2024-09-15T00:13:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0679e-05 2024-09-15T00:13:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601998+0000", 2024-09-15T00:13:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2955999999999999e-05 2024-09-15T00:13:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602070+0000", 2024-09-15T00:13:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1958000000000005e-05 2024-09-15T00:13:45.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602237+0000", 2024-09-15T00:13:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016621299999999999 2024-09-15T00:13:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602799+0000", 2024-09-15T00:13:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000562723 2024-09-15T00:13:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.620811+0000", 2024-09-15T00:13:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018011207000000001 2024-09-15T00:13:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624793+0000", 2024-09-15T00:13:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0039828770000000001 2024-09-15T00:13:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624855+0000", 2024-09-15T00:13:45.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.143e-05 2024-09-15T00:13:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624907+0000", 2024-09-15T00:13:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2262000000000002e-05 2024-09-15T00:13:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.709 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146936 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.921877375000001, 2024-09-15T00:13:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056847126999999997, 2024-09-15T00:13:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146936 2024-09-15T00:13:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.710 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753671+0000", 2024-09-15T00:13:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2839999999999999e-06 2024-09-15T00:13:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753677+0000", 2024-09-15T00:13:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0260000000000004e-06 2024-09-15T00:13:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753691+0000", 2024-09-15T00:13:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3891e-05 2024-09-15T00:13:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753698+0000", 2024-09-15T00:13:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8480000000000001e-06 2024-09-15T00:13:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753750+0000", 2024-09-15T00:13:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1338000000000002e-05 2024-09-15T00:13:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753814+0000", 2024-09-15T00:13:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4642999999999998e-05 2024-09-15T00:13:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.756359+0000", 2024-09-15T00:13:45.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025450619999999998 2024-09-15T00:13:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810435+0000", 2024-09-15T00:13:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.054075397999999997 2024-09-15T00:13:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810446+0000", 2024-09-15T00:13:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1005000000000001e-05 2024-09-15T00:13:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810482+0000", 2024-09-15T00:13:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6494e-05 2024-09-15T00:13:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810517+0000", 2024-09-15T00:13:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4137999999999997e-05 2024-09-15T00:13:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:146936 3.6 e98/97 ondisk, result = 0)", 2024-09-15T00:13:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.912094320999998, 2024-09-15T00:13:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046987205999999997, 2024-09-15T00:13:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763453+0000", 2024-09-15T00:13:45.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5199999999999996e-07 2024-09-15T00:13:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763457+0000", 2024-09-15T00:13:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8569999999999997e-06 2024-09-15T00:13:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763469+0000", 2024-09-15T00:13:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1977999999999999e-05 2024-09-15T00:13:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763476+0000", 2024-09-15T00:13:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:13:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810423+0000", 2024-09-15T00:13:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946993999999999 2024-09-15T00:13:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810431+0000", 2024-09-15T00:13:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0339999999999997e-06 2024-09-15T00:13:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810440+0000", 2024-09-15T00:13:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9989999999999997e-06 2024-09-15T00:13:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:151036 3.1f e103/68 3:f84a1a09:::rbd_data.113958cb0e55.000000000000000c:head v 103'452, mlcod=100'450)", 2024-09-15T00:13:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.338748093, 2024-09-15T00:13:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020618753, 2024-09-15T00:13:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:45.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:13:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340692+0000", 2024-09-15T00:13:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038928890000000001 2024-09-15T00:13:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340704+0000", 2024-09-15T00:13:45.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1497e-05 2024-09-15T00:13:45.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340709+0000", 2024-09-15T00:13:45.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1529999999999996e-06 2024-09-15T00:13:45.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340789+0000", 2024-09-15T00:13:45.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9907999999999995e-05 2024-09-15T00:13:45.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340810+0000", 2024-09-15T00:13:45.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1837000000000001e-05 2024-09-15T00:13:45.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357368+0000", 2024-09-15T00:13:45.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016557199000000002 2024-09-15T00:13:45.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357417+0000", 2024-09-15T00:13:45.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9783000000000001e-05 2024-09-15T00:13:45.727 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.727 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.727 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.727 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:151042 3.6 3:6419cde4:::rbd_data.113958cb0e55.0000000000000031:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:45.727 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:45.727 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.309669974999998, 2024-09-15T00:13:45.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025627375000000001, 2024-09-15T00:13:45.727 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.727 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.728 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.728 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:45.728 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:45.728 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:45.728 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 151042 2024-09-15T00:13:45.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.728 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:45.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:45.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:45.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7899999999999998e-07 2024-09-15T00:13:45.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365881+0000", 2024-09-15T00:13:45.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9770000000000002e-06 2024-09-15T00:13:45.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365890+0000", 2024-09-15T00:13:45.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5920000000000001e-06 2024-09-15T00:13:45.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365894+0000", 2024-09-15T00:13:45.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4569999999999998e-06 2024-09-15T00:13:45.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365927+0000", 2024-09-15T00:13:45.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2036000000000003e-05 2024-09-15T00:13:45.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365970+0000", 2024-09-15T00:13:45.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3197e-05 2024-09-15T00:13:45.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:45.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.367432+0000", 2024-09-15T00:13:45.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014621720000000001 2024-09-15T00:13:45.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:45.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391358+0000", 2024-09-15T00:13:45.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023925921999999999 2024-09-15T00:13:45.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:45.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391415+0000", 2024-09-15T00:13:45.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7253999999999998e-05 2024-09-15T00:13:45.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391456+0000", 2024-09-15T00:13:45.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0497999999999997e-05 2024-09-15T00:13:45.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391504+0000", 2024-09-15T00:13:45.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8591000000000003e-05 2024-09-15T00:13:45.734 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.734 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.734 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.734 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:151042 3.6 e103/97 ondisk, result = 0)", 2024-09-15T00:13:45.734 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:45.734 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.303926961999998, 2024-09-15T00:13:45.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019892730000000001, 2024-09-15T00:13:45.734 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.735 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.735 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:45.735 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:45.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:45.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:45.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8100000000000003e-07 2024-09-15T00:13:45.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371623+0000", 2024-09-15T00:13:45.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6290000000000002e-06 2024-09-15T00:13:45.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371632+0000", 2024-09-15T00:13:45.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6379999999999993e-06 2024-09-15T00:13:45.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371636+0000", 2024-09-15T00:13:45.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:13:45.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391403+0000", 2024-09-15T00:13:45.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019766982999999998 2024-09-15T00:13:45.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391411+0000", 2024-09-15T00:13:45.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7309999999999993e-06 2024-09-15T00:13:45.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391513+0000", 2024-09-15T00:13:45.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101648 2024-09-15T00:13:45.739 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.739 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.739 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.739 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152025 3.3 e103/68 3:c0279197:::rbd_data.113958cb0e55.0000000000000007:head v 103'464, mlcod=102'461)", 2024-09-15T00:13:45.739 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:45.739 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.159439366000001, 2024-09-15T00:13:45.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021264476000000001, 2024-09-15T00:13:45.739 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.739 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.739 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.740 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:45.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:45.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516109+0000", 2024-09-15T00:13:45.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.666e-06 2024-09-15T00:13:45.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.519997+0000", 2024-09-15T00:13:45.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003888449 2024-09-15T00:13:45.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520016+0000", 2024-09-15T00:13:45.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8059000000000001e-05 2024-09-15T00:13:45.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520026+0000", 2024-09-15T00:13:45.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9599999999999995e-06 2024-09-15T00:13:45.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520089+0000", 2024-09-15T00:13:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3074e-05 2024-09-15T00:13:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520108+0000", 2024-09-15T00:13:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8981999999999999e-05 2024-09-15T00:13:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537326+0000", 2024-09-15T00:13:45.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017218102999999998 2024-09-15T00:13:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537372+0000", 2024-09-15T00:13:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6183000000000002e-05 2024-09-15T00:13:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152284 3.f e103/68 3:f4272b5d:::rbd_data.113958cb0e55.0000000000000006:head v 103'370, mlcod=103'368)", 2024-09-15T00:13:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:45.744 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.110962835000002, 2024-09-15T00:13:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020145026999999999, 2024-09-15T00:13:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564585+0000", 2024-09-15T00:13:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7000000000000003e-07 2024-09-15T00:13:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568352+0000", 2024-09-15T00:13:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037675909999999998 2024-09-15T00:13:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568370+0000", 2024-09-15T00:13:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7158000000000001e-05 2024-09-15T00:13:45.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568378+0000", 2024-09-15T00:13:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6449999999999994e-06 2024-09-15T00:13:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568454+0000", 2024-09-15T00:13:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5285999999999997e-05 2024-09-15T00:13:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568472+0000", 2024-09-15T00:13:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8447000000000001e-05 2024-09-15T00:13:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584693+0000", 2024-09-15T00:13:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016221467999999999 2024-09-15T00:13:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584729+0000", 2024-09-15T00:13:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5462e-05 2024-09-15T00:13:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152500 3.17 e103/68 3:edff5adf:::rbd_data.113958cb0e55.000000000000000d:head v 103'461, mlcod=103'459)", 2024-09-15T00:13:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.033976250000002, 2024-09-15T00:13:45.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020217809, 2024-09-15T00:13:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641572+0000", 2024-09-15T00:13:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-15T00:13:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644961+0000", 2024-09-15T00:13:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033882610000000001 2024-09-15T00:13:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644979+0000", 2024-09-15T00:13:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8179e-05 2024-09-15T00:13:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644987+0000", 2024-09-15T00:13:45.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8860000000000004e-06 2024-09-15T00:13:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645044+0000", 2024-09-15T00:13:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7141000000000001e-05 2024-09-15T00:13:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645062+0000", 2024-09-15T00:13:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8258e-05 2024-09-15T00:13:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661749+0000", 2024-09-15T00:13:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016686553999999999 2024-09-15T00:13:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661788+0000", 2024-09-15T00:13:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9690000000000001e-05 2024-09-15T00:13:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153171 3.18 e103/69 3:1db186e8:::rbd_data.113958cb0e55.0000000000000002:head v 103'520, mlcod=103'518)", 2024-09-15T00:13:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.895491344, 2024-09-15T00:13:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022266838000000001, 2024-09-15T00:13:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780057+0000", 2024-09-15T00:13:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5880000000000001e-06 2024-09-15T00:13:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781273+0000", 2024-09-15T00:13:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001216087 2024-09-15T00:13:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781311+0000", 2024-09-15T00:13:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7656e-05 2024-09-15T00:13:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781325+0000", 2024-09-15T00:13:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4508999999999999e-05 2024-09-15T00:13:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781403+0000", 2024-09-15T00:13:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7927000000000001e-05 2024-09-15T00:13:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781422+0000", 2024-09-15T00:13:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8575000000000001e-05 2024-09-15T00:13:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802249+0000", 2024-09-15T00:13:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020827577 2024-09-15T00:13:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802322+0000", 2024-09-15T00:13:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2918999999999998e-05 2024-09-15T00:13:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.760 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153772 3.12 e103/68 3:4911a4f6:::rbd_data.113958cb0e55.0000000000000001:head v 103'490, mlcod=103'488)", 2024-09-15T00:13:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.792620716999998, 2024-09-15T00:13:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021181004, 2024-09-15T00:13:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882927+0000", 2024-09-15T00:13:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.426e-06 2024-09-15T00:13:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883966+0000", 2024-09-15T00:13:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010384610000000001 2024-09-15T00:13:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883985+0000", 2024-09-15T00:13:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9276e-05 2024-09-15T00:13:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883994+0000", 2024-09-15T00:13:45.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9490000000000002e-06 2024-09-15T00:13:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884054+0000", 2024-09-15T00:13:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9419999999999997e-05 2024-09-15T00:13:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884073+0000", 2024-09-15T00:13:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9137e-05 2024-09-15T00:13:45.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904038+0000", 2024-09-15T00:13:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019965092 2024-09-15T00:13:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904107+0000", 2024-09-15T00:13:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9243000000000001e-05 2024-09-15T00:13:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.765 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163112 3.3 e106/68 3:c668f3d5:::rbd_data.11395d771336.0000000000000001:head v 106'489, mlcod=105'487)", 2024-09-15T00:13:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 52.210723448000003, 2024-09-15T00:13:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019317183000000002, 2024-09-15T00:13:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464824+0000", 2024-09-15T00:13:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1100000000000005e-07 2024-09-15T00:13:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466874+0000", 2024-09-15T00:13:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002049497 2024-09-15T00:13:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466884+0000", 2024-09-15T00:13:45.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0424999999999999e-05 2024-09-15T00:13:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466890+0000", 2024-09-15T00:13:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-15T00:13:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466944+0000", 2024-09-15T00:13:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4592999999999997e-05 2024-09-15T00:13:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466958+0000", 2024-09-15T00:13:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3993e-05 2024-09-15T00:13:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484068+0000", 2024-09-15T00:13:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017109994 2024-09-15T00:13:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484140+0000", 2024-09-15T00:13:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2224000000000005e-05 2024-09-15T00:13:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163119 3.0 e106/94 3:06118613:::rbd_data.11395d771336.0000000000000004:head v 106'549, mlcod=105'547)", 2024-09-15T00:13:45.771 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 52.199108244999998, 2024-09-15T00:13:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017175867000000001, 2024-09-15T00:13:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6700000000000002e-07 2024-09-15T00:13:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477786+0000", 2024-09-15T00:13:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013468149999999999 2024-09-15T00:13:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477797+0000", 2024-09-15T00:13:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0635000000000001e-05 2024-09-15T00:13:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477805+0000", 2024-09-15T00:13:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2989999999999997e-06 2024-09-15T00:13:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477859+0000", 2024-09-15T00:13:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3699000000000002e-05 2024-09-15T00:13:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477878+0000", 2024-09-15T00:13:45.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9335e-05 2024-09-15T00:13:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493560+0000", 2024-09-15T00:13:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015681748999999998 2024-09-15T00:13:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493614+0000", 2024-09-15T00:13:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4367999999999999e-05 2024-09-15T00:13:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.776 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.5.0:5005 3.17 e110/68 3:eb0359e8:::rbd_data.113964deed84.0000000000000251:35 v 110'537, mlcod=109'534)", 2024-09-15T00:13:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.539257872999997, 2024-09-15T00:13:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015446896999999999, 2024-09-15T00:13:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136290+0000", 2024-09-15T00:13:45.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7400000000000003e-07 2024-09-15T00:13:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136294+0000", 2024-09-15T00:13:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4989999999999996e-06 2024-09-15T00:13:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136304+0000", 2024-09-15T00:13:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0319999999999999e-05 2024-09-15T00:13:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136310+0000", 2024-09-15T00:13:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.981e-06 2024-09-15T00:13:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136350+0000", 2024-09-15T00:13:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0088e-05 2024-09-15T00:13:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136364+0000", 2024-09-15T00:13:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3954999999999999e-05 2024-09-15T00:13:45.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151681+0000", 2024-09-15T00:13:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015316969999999999 2024-09-15T00:13:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151736+0000", 2024-09-15T00:13:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4410000000000003e-05 2024-09-15T00:13:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1758 3.10 e110/68 3:08149147:::rbd_data.113964deed84.0000000000000055:35 v 110'594, mlcod=109'591)", 2024-09-15T00:13:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.534959647000001, 2024-09-15T00:13:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016278652000000001, 2024-09-15T00:13:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:45.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140599+0000", 2024-09-15T00:13:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1109e-05 2024-09-15T00:13:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140605+0000", 2024-09-15T00:13:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-15T00:13:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140612+0000", 2024-09-15T00:13:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6089999999999997e-06 2024-09-15T00:13:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140656+0000", 2024-09-15T00:13:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3754999999999997e-05 2024-09-15T00:13:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140674+0000", 2024-09-15T00:13:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7640999999999999e-05 2024-09-15T00:13:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156823+0000", 2024-09-15T00:13:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016149047999999999 2024-09-15T00:13:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156866+0000", 2024-09-15T00:13:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3205999999999999e-05 2024-09-15T00:13:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.786 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.4.0:5722 3.e e110/95 3:74a58daf:::rbd_data.113964deed84.00000000000000c1:35 v 110'1287, mlcod=109'1284)", 2024-09-15T00:13:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 47.525885291000002, 2024-09-15T00:13:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01830555, 2024-09-15T00:13:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149662+0000", 2024-09-15T00:13:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:13:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149666+0000", 2024-09-15T00:13:45.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2220000000000004e-06 2024-09-15T00:13:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149672+0000", 2024-09-15T00:13:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3410000000000002e-06 2024-09-15T00:13:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149676+0000", 2024-09-15T00:13:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.566e-06 2024-09-15T00:13:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149709+0000", 2024-09-15T00:13:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2403000000000001e-05 2024-09-15T00:13:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149719+0000", 2024-09-15T00:13:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0608000000000001e-05 2024-09-15T00:13:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167944+0000", 2024-09-15T00:13:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018225183999999998 2024-09-15T00:13:45.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167967+0000", 2024-09-15T00:13:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2705e-05 2024-09-15T00:13:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:45.791 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:45.791 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:45.793 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:45.794 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:45.794 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:45.794 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:45.795 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:45.795 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:45.795 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:45.795 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:45.796 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:45.796 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:45.796 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:45.796 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:45.797 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:45.797 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:45.797 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:45.798 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:45.791+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:45.808 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:45.808 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:45.893 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:45.892+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:45.912 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:45.912 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:45.921 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:45.995 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:45.993+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:46.071 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:46.071 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:46.079 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:46.096 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:46.094+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:46.184 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:46.184 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:46.193 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:46.196 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:46.194+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:46.297 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:46.297 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:46.298 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:46.295+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:46.306 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:46.399 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:46.398+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:46.449 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:46.450 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:46.458 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:46.502 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:46.501+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:46.562 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:46.562 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:46.570 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:46.616 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:46.614+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:46.693 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:46.693 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:46.702 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:46.700+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:46.703 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:46.803 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:46.801+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:46.853 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:46.853 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:46.862 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:46.904 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:46.903+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:46.973 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:47.017 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:47.016+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:47.047 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:13:47.084 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:47.084 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:47.092 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:47.134 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:47.132+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:47.207 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:47.205+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:47.237 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:47.237 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:47.246 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:47.330 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:47.328+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:47.351 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:13:47.350+0000 7fa9e77fe640 0 --2- 172.21.15.171:0/4015995700 >> v2:172.21.15.171:3300/0 conn(0x7fa9e8071070 0x7fa9e813c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T00:13:47.390 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:47.390 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:47.398 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:47.408 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:47.406+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:47.509 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:47.508+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:47.556 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-15T00:13:47.555+0000 7fa9e77fe640 0 --2- 172.21.15.171:0/3838681631 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9e81434e0 0x7fa9e8143960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T00:13:47.570 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:47.570 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:47.581 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:47.622 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:47.621+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:47.711 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:47.710+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:47.732 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:47.732 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:47.742 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:47.820 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:13:47.825 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:47.824+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:47.894 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:47.894 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:47.904 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:47.944 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:47.945 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:47.946 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:47.944+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:47.964 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:47.965 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:47.983 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:47.983 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:48.002 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:48.002 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:48.013 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:48.012+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:48.022 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:48.022 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:48.041 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:48.058 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:48.058 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:48.059 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:48.059 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:48.068 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:48.076 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:48.076 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:48.129 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:48.127+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:48.185 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:48.186 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:48.186 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:48.186 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:48.186 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:48.196 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:48.207 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is unset 2024-09-15T00:13:48.239 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:48.238+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:48.316 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:48.314+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:48.325 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:48.325 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:48.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:48.326 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:48.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.326 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:48.326 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:48.326 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 142.24382055699999, 2024-09-15T00:13:48.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:13:48.326 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.326 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.326 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:48.326 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.326 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.327 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.327 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:13:48.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.327 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:48.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:13:48.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:13:48.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:13:48.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:13:48.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:13:48.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:13:48.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:13:48.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:13:48.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:13:48.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:13:48.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:13:48.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:13:48.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:13:48.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:48.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:13:48.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:13:48.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:48.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:13:48.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:13:48.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:48.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:13:48.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:13:48.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:48.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:13:48.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:13:48.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:13:48.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:13:48.332 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.332 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.333 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.333 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:13:48.333 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:48.333 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 142.231080015, 2024-09-15T00:13:48.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:13:48.333 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.333 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.333 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:48.333 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:48.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:13:48.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:13:48.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:13:48.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:13:48.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:13:48.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:13:48.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:13:48.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:13:48.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:13:48.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:13:48.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:13:48.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:13:48.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:13:48.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:13:48.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:13:48.337 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.337 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.337 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.338 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:48.338 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:48.338 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 142.098724256, 2024-09-15T00:13:48.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:13:48.338 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.338 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.338 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:48.338 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.338 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.338 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.339 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:13:48.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.339 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:48.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:13:48.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:13:48.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:13:48.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:13:48.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:13:48.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:13:48.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:13:48.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:13:48.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:13:48.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:13:48.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:13:48.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:13:48.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:13:48.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:48.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:13:48.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:13:48.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:48.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:13:48.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:13:48.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:48.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:13:48.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:13:48.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:48.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:13:48.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:13:48.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:13:48.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:13:48.345 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.345 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.345 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.345 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:13:48.345 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:48.345 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 142.08526261099999, 2024-09-15T00:13:48.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:13:48.345 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.345 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.345 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:48.346 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:48.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:13:48.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:13:48.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:13:48.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:13:48.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:13:48.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:13:48.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:13:48.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:13:48.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:13:48.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:13:48.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:13:48.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:13:48.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:13:48.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:13:48.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:13:48.349 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.350 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.350 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.350 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:13:48.350 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:48.350 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 132.90026112800001, 2024-09-15T00:13:48.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:13:48.350 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.350 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.350 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:48.351 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.351 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.351 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.351 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:13:48.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.351 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:48.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:13:48.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:13:48.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:13:48.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:13:48.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:13:48.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:13:48.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:13:48.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:13:48.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:13:48.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:13:48.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:13:48.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:13:48.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:13:48.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:48.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:13:48.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:13:48.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:48.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:13:48.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:13:48.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:48.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:13:48.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:13:48.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:48.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:13:48.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:13:48.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:13:48.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:13:48.356 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.356 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.357 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.357 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:13:48.357 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:48.357 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 132.88738486099999, 2024-09-15T00:13:48.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:13:48.357 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.357 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.357 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:48.357 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:48.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:13:48.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:13:48.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:13:48.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:13:48.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:13:48.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:13:48.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:13:48.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:13:48.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:13:48.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:13:48.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:13:48.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:13:48.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:13:48.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:13:48.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:13:48.361 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.361 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.361 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.361 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:13:48.362 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:48.362 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 127.428869717, 2024-09-15T00:13:48.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:13:48.362 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.362 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.362 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:48.362 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.362 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.362 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.362 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:13:48.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.362 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:48.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:13:48.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:13:48.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:13:48.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:13:48.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:13:48.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:13:48.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:13:48.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:13:48.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:13:48.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:13:48.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:13:48.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:13:48.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:13:48.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:48.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:13:48.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:13:48.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:48.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:13:48.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:13:48.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:48.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:13:48.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:13:48.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:48.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:13:48.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:13:48.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:13:48.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:13:48.368 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.368 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.368 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.368 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48992 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:48.368 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:48.368 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.711359157, 2024-09-15T00:13:48.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028055819999999999, 2024-09-15T00:13:48.368 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.369 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.369 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:48.369 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.369 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.369 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.369 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48992 2024-09-15T00:13:48.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.369 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:48.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610981+0000", 2024-09-15T00:13:48.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610982+0000", 2024-09-15T00:13:48.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:13:48.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610983+0000", 2024-09-15T00:13:48.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-15T00:13:48.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610989+0000", 2024-09-15T00:13:48.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9680000000000003e-06 2024-09-15T00:13:48.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.610994+0000", 2024-09-15T00:13:48.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:13:48.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638851+0000", 2024-09-15T00:13:48.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027857417999999998 2024-09-15T00:13:48.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.638943+0000", 2024-09-15T00:13:48.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1360000000000006e-05 2024-09-15T00:13:48.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639037+0000", 2024-09-15T00:13:48.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3974999999999994e-05 2024-09-15T00:13:48.373 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.373 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.373 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.373 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48993 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:48.373 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:48.373 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.711340548, 2024-09-15T00:13:48.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028262233000000001, 2024-09-15T00:13:48.373 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.373 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.374 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:48.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.374 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48993 2024-09-15T00:13:48.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.374 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:48.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:48.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611000+0000", 2024-09-15T00:13:48.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:13:48.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611006+0000", 2024-09-15T00:13:48.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.144e-06 2024-09-15T00:13:48.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611012+0000", 2024-09-15T00:13:48.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3319999999999997e-06 2024-09-15T00:13:48.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611018+0000", 2024-09-15T00:13:48.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4480000000000002e-06 2024-09-15T00:13:48.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639086+0000", 2024-09-15T00:13:48.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028068263 2024-09-15T00:13:48.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639174+0000", 2024-09-15T00:13:48.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7911999999999994e-05 2024-09-15T00:13:48.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639262+0000", 2024-09-15T00:13:48.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7783999999999998e-05 2024-09-15T00:13:48.378 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.378 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.378 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.379 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48994 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:48.379 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:48.379 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.711315224, 2024-09-15T00:13:48.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028428841999999999, 2024-09-15T00:13:48.379 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.379 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.379 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:48.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.379 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48994 2024-09-15T00:13:48.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.380 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:48.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:48.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611025+0000", 2024-09-15T00:13:48.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:13:48.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611027+0000", 2024-09-15T00:13:48.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:13:48.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611033+0000", 2024-09-15T00:13:48.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8889999999999999e-06 2024-09-15T00:13:48.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611038+0000", 2024-09-15T00:13:48.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6070000000000001e-06 2024-09-15T00:13:48.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639292+0000", 2024-09-15T00:13:48.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028254532999999998 2024-09-15T00:13:48.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639380+0000", 2024-09-15T00:13:48.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7268999999999994e-05 2024-09-15T00:13:48.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639454+0000", 2024-09-15T00:13:48.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4138000000000001e-05 2024-09-15T00:13:48.384 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.384 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.384 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.384 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:48.384 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:48.384 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.71129689999999, 2024-09-15T00:13:48.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:13:48.384 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.384 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.384 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:48.384 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.385 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.385 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.385 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:13:48.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.385 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:48.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:13:48.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:13:48.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:13:48.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:13:48.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:13:48.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:13:48.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:48.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:13:48.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:13:48.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:13:48.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:13:48.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:13:48.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:13:48.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:13:48.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:13:48.389 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.389 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.389 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.389 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:48.389 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:48.389 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.711114327, 2024-09-15T00:13:48.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:13:48.389 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.390 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.390 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:48.390 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.390 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.390 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.390 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:13:48.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.390 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:48.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:48.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:13:48.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:13:48.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:13:48.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:13:48.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:13:48.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:13:48.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:13:48.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:13:48.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:13:48.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:13:48.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:13:48.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:13:48.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:13:48.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:13:48.394 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.394 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.394 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.395 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:48.395 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:48.395 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.711085486, 2024-09-15T00:13:48.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:13:48.395 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.395 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.395 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:48.395 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.395 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.395 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.395 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:13:48.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.396 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:48.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:48.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:13:48.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:13:48.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:13:48.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:13:48.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:13:48.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:13:48.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:13:48.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:13:48.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:13:48.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:13:48.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:13:48.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:13:48.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:13:48.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:13:48.399 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.399 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.400 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.400 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:48.400 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:48.400 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.711063275, 2024-09-15T00:13:48.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:13:48.400 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.400 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.400 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:48.400 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.400 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.401 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.401 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:13:48.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.401 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:48.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:48.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:13:48.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:13:48.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:13:48.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:13:48.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:13:48.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:13:48.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:13:48.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:13:48.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:13:48.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:13:48.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:13:48.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:13:48.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:13:48.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:13:48.405 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.405 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.405 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.405 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:48.405 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:48.405 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.711042116, 2024-09-15T00:13:48.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:13:48.405 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.406 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.406 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:48.406 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.406 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.406 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.406 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:13:48.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.406 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:48.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:48.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:13:48.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:13:48.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:13:48.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:13:48.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:13:48.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:13:48.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:13:48.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:13:48.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:13:48.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:13:48.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:13:48.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:13:48.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:13:48.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:13:48.410 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.410 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.411 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.411 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:48.411 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:48.411 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.711012168, 2024-09-15T00:13:48.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:13:48.411 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.411 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.411 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:48.411 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.412 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.412 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.412 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:13:48.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.412 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:48.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:48.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:13:48.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:13:48.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:13:48.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:13:48.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:13:48.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:13:48.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:13:48.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:13:48.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:13:48.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:13:48.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:13:48.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:13:48.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:13:48.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:13:48.416 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.416 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.416 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.417 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:48.417 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:48.417 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.710990204, 2024-09-15T00:13:48.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:13:48.417 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.417 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.417 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:48.417 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.417 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.417 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.417 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:13:48.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.418 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:48.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:48.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:13:48.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:13:48.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:13:48.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:13:48.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:13:48.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:13:48.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:13:48.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:13:48.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:13:48.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:13:48.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:13:48.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:13:48.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:13:48.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:13:48.422 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.422 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.422 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.422 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:48.422 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:48.422 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.71097277699999, 2024-09-15T00:13:48.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:13:48.422 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.422 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.422 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:48.423 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.423 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.423 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.423 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:13:48.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.423 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:48.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:13:48.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:13:48.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:13:48.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:13:48.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:13:48.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:13:48.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:13:48.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:13:48.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:13:48.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:13:48.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:13:48.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:13:48.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:13:48.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:13:48.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:13:48.427 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.427 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.427 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.427 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:48.427 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:48.427 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.710950978, 2024-09-15T00:13:48.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:13:48.428 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.428 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.428 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:48.428 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.428 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.428 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.428 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:13:48.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.428 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:48.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:13:48.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:13:48.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:13:48.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:13:48.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:13:48.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:13:48.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:13:48.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:13:48.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:13:48.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:13:48.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:13:48.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:13:48.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:13:48.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:13:48.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:13:48.432 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.432 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.433 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.433 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:48.433 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:48.433 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.661722619, 2024-09-15T00:13:48.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:13:48.433 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:48.433 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:48.433 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:48.433 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:48.433 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:48.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.434 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:13:48.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.434 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:48.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:48.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:48.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:48.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:13:48.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:48.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:48.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:13:48.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:13:48.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:48.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:13:48.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:13:48.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:48.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:13:48.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:13:48.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:13:48.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:13:48.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:48.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:13:48.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:13:48.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:48.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:13:48.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:13:48.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:48.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:13:48.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:13:48.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:48.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:13:48.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:13:48.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:48.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:13:48.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:13:48.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:48.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:13:48.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:13:48.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:48.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:48.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:48.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:13:48.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:13:48.439 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.439 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.440 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.440 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:48.440 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:48.440 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:48.441 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:48.442 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:48.440+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:48.542 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:48.540+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:48.547 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:48.547 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:48.547 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:48.547 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:48.556 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:48.643 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:48.642+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:48.702 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:48.702 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:48.702 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:48.703 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:48.703 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:48.711 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:48.752 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:48.751+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:48.846 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:48.844+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:48.864 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9, version: 37'47, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9@37'47, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.006309424000001, 2024-09-15T00:13:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054905559999999997, 2024-09-15T00:13:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:13:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853793+0000", 2024-09-15T00:13:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9599999999999999e-07 2024-09-15T00:13:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854798+0000", 2024-09-15T00:13:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010050770000000001 2024-09-15T00:13:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854814+0000", 2024-09-15T00:13:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6478999999999999e-05 2024-09-15T00:13:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854825+0000", 2024-09-15T00:13:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0492999999999999e-05 2024-09-15T00:13:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854889+0000", 2024-09-15T00:13:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4134999999999995e-05 2024-09-15T00:13:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854898+0000", 2024-09-15T00:13:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6260000000000001e-06 2024-09-15T00:13:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.859283+0000", 2024-09-15T00:13:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043852500000000003 2024-09-15T00:13:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15, version: 59'205, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15@59'205, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.880088968999999, 2024-09-15T00:13:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059047939999999997, 2024-09-15T00:13:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:13:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.44e-07 2024-09-15T00:13:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980977+0000", 2024-09-15T00:13:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00096402300000000005 2024-09-15T00:13:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980995+0000", 2024-09-15T00:13:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7214000000000002e-05 2024-09-15T00:13:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981007+0000", 2024-09-15T00:13:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2028000000000001e-05 2024-09-15T00:13:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981069+0000", 2024-09-15T00:13:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1977000000000006e-05 2024-09-15T00:13:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981077+0000", 2024-09-15T00:13:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2120000000000003e-06 2024-09-15T00:13:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.985918+0000", 2024-09-15T00:13:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048406960000000002 2024-09-15T00:13:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head, version: 62'221, data_included: [0~262144,393216~786432], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head@62'221, size: 1572864, copy_subset: [(0, 1572864)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.857820703999998, 2024-09-15T00:13:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054170770000000002, 2024-09-15T00:13:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:13:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002282+0000", 2024-09-15T00:13:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4499999999999997e-07 2024-09-15T00:13:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003196+0000", 2024-09-15T00:13:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091476399999999999 2024-09-15T00:13:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003213+0000", 2024-09-15T00:13:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6554000000000001e-05 2024-09-15T00:13:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003230+0000", 2024-09-15T00:13:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6617999999999999e-05 2024-09-15T00:13:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003292+0000", 2024-09-15T00:13:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2193999999999994e-05 2024-09-15T00:13:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003300+0000", 2024-09-15T00:13:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1429999999999998e-06 2024-09-15T00:13:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.007698+0000", 2024-09-15T00:13:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043981589999999996 2024-09-15T00:13:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.807090623999997, 2024-09-15T00:13:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067633270000000004, 2024-09-15T00:13:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:13:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053012+0000", 2024-09-15T00:13:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1659999999999999e-06 2024-09-15T00:13:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054086+0000", 2024-09-15T00:13:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010732789999999999 2024-09-15T00:13:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054100+0000", 2024-09-15T00:13:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4123e-05 2024-09-15T00:13:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054114+0000", 2024-09-15T00:13:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4442e-05 2024-09-15T00:13:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054192+0000", 2024-09-15T00:13:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8288000000000007e-05 2024-09-15T00:13:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054200+0000", 2024-09-15T00:13:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-15T00:13:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.059774+0000", 2024-09-15T00:13:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055741230000000003 2024-09-15T00:13:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.798773296, 2024-09-15T00:13:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049777950000000001, 2024-09-15T00:13:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:13:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061330+0000", 2024-09-15T00:13:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.11e-06 2024-09-15T00:13:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062350+0000", 2024-09-15T00:13:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010207969999999999 2024-09-15T00:13:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062365+0000", 2024-09-15T00:13:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4157e-05 2024-09-15T00:13:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062380+0000", 2024-09-15T00:13:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5129e-05 2024-09-15T00:13:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062467+0000", 2024-09-15T00:13:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7247e-05 2024-09-15T00:13:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062475+0000", 2024-09-15T00:13:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4300000000000006e-06 2024-09-15T00:13:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.066306+0000", 2024-09-15T00:13:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038309250000000002 2024-09-15T00:13:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.792890337999999, 2024-09-15T00:13:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052454889999999999, 2024-09-15T00:13:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:13:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067212+0000", 2024-09-15T00:13:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-07 2024-09-15T00:13:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068119+0000", 2024-09-15T00:13:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090671199999999995 2024-09-15T00:13:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068135+0000", 2024-09-15T00:13:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5739e-05 2024-09-15T00:13:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068150+0000", 2024-09-15T00:13:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5693e-05 2024-09-15T00:13:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068236+0000", 2024-09-15T00:13:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6100000000000006e-05 2024-09-15T00:13:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068245+0000", 2024-09-15T00:13:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1960000000000008e-06 2024-09-15T00:13:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.072457+0000", 2024-09-15T00:13:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042123719999999998 2024-09-15T00:13:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.786911989000004, 2024-09-15T00:13:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049532020000000003, 2024-09-15T00:13:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:13:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:13:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074164+0000", 2024-09-15T00:13:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097354400000000002 2024-09-15T00:13:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074178+0000", 2024-09-15T00:13:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4134999999999999e-05 2024-09-15T00:13:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074192+0000", 2024-09-15T00:13:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.451e-05 2024-09-15T00:13:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074272+0000", 2024-09-15T00:13:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9989999999999998e-05 2024-09-15T00:13:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074280+0000", 2024-09-15T00:13:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9990000000000008e-06 2024-09-15T00:13:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078143+0000", 2024-09-15T00:13:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00386258 2024-09-15T00:13:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [4194304~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.781201177, 2024-09-15T00:13:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049408730000000001, 2024-09-15T00:13:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:48.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:13:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2399999999999999e-07 2024-09-15T00:13:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079875+0000", 2024-09-15T00:13:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097349100000000005 2024-09-15T00:13:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079889+0000", 2024-09-15T00:13:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4148000000000001e-05 2024-09-15T00:13:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079904+0000", 2024-09-15T00:13:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5159e-05 2024-09-15T00:13:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079964+0000", 2024-09-15T00:13:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0204999999999997e-05 2024-09-15T00:13:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079971+0000", 2024-09-15T00:13:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2490000000000004e-06 2024-09-15T00:13:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.083841+0000", 2024-09-15T00:13:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038701970000000001 2024-09-15T00:13:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [5242880~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 6291456, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.775496197000002, 2024-09-15T00:13:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052525740000000003, 2024-09-15T00:13:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:13:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:13:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085579+0000", 2024-09-15T00:13:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097307600000000002 2024-09-15T00:13:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085592+0000", 2024-09-15T00:13:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3277e-05 2024-09-15T00:13:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085608+0000", 2024-09-15T00:13:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5773e-05 2024-09-15T00:13:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085687+0000", 2024-09-15T00:13:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8956999999999996e-05 2024-09-15T00:13:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085694+0000", 2024-09-15T00:13:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.956e-06 2024-09-15T00:13:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.089858+0000", 2024-09-15T00:13:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041641159999999998 2024-09-15T00:13:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:172552 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-15T00:13:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.671038889000002, 2024-09-15T00:13:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.997031471, 2024-09-15T00:13:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 172552 2024-09-15T00:13:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:13:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189064+0000", 2024-09-15T00:13:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.198e-06 2024-09-15T00:13:48.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189069+0000", 2024-09-15T00:13:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9069999999999998e-06 2024-09-15T00:13:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189080+0000", 2024-09-15T00:13:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:13:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189086+0000", 2024-09-15T00:13:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5429999999999999e-06 2024-09-15T00:13:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189128+0000", 2024-09-15T00:13:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2255000000000001e-05 2024-09-15T00:13:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189135+0000", 2024-09-15T00:13:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6200000000000001e-06 2024-09-15T00:13:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185787+0000", 2024-09-15T00:13:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99665157199999999 2024-09-15T00:13:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185926+0000", 2024-09-15T00:13:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013970399999999999 2024-09-15T00:13:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.186094+0000", 2024-09-15T00:13:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016813099999999999 2024-09-15T00:13:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9, version: 37'60, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9@37'60, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.612178530000001, 2024-09-15T00:13:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049454060000000003, 2024-09-15T00:13:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:13:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247924+0000", 2024-09-15T00:13:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:13:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248394+0000", 2024-09-15T00:13:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047063399999999999 2024-09-15T00:13:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248411+0000", 2024-09-15T00:13:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6384000000000001e-05 2024-09-15T00:13:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248441+0000", 2024-09-15T00:13:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0559999999999999e-05 2024-09-15T00:13:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248499+0000", 2024-09-15T00:13:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7274000000000001e-05 2024-09-15T00:13:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248507+0000", 2024-09-15T00:13:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2390000000000002e-06 2024-09-15T00:13:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.252869+0000", 2024-09-15T00:13:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043617200000000004 2024-09-15T00:13:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head, version: 59'393, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head@59'393, size: 5701632, copy_subset: [(0, 5701632)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.482730263000001, 2024-09-15T00:13:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004807469, 2024-09-15T00:13:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:13:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4799999999999998e-07 2024-09-15T00:13:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377830+0000", 2024-09-15T00:13:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000457954 2024-09-15T00:13:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377847+0000", 2024-09-15T00:13:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6741000000000001e-05 2024-09-15T00:13:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377858+0000", 2024-09-15T00:13:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0951999999999999e-05 2024-09-15T00:13:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377929+0000", 2024-09-15T00:13:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0697000000000004e-05 2024-09-15T00:13:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377941+0000", 2024-09-15T00:13:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2529e-05 2024-09-15T00:13:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.382179+0000", 2024-09-15T00:13:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042379480000000001 2024-09-15T00:13:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head, version: 88'473, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head@88'473, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.400610829999998, 2024-09-15T00:13:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005823272, 2024-09-15T00:13:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:13:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459492+0000", 2024-09-15T00:13:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9800000000000003e-07 2024-09-15T00:13:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459940+0000", 2024-09-15T00:13:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044854699999999999 2024-09-15T00:13:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459957+0000", 2024-09-15T00:13:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6586e-05 2024-09-15T00:13:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459968+0000", 2024-09-15T00:13:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.145e-05 2024-09-15T00:13:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460043+0000", 2024-09-15T00:13:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4451999999999998e-05 2024-09-15T00:13:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460057+0000", 2024-09-15T00:13:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.417e-05 2024-09-15T00:13:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.465314+0000", 2024-09-15T00:13:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052574689999999999 2024-09-15T00:13:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.165434648999998, 2024-09-15T00:13:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051811399999999999, 2024-09-15T00:13:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:13:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694668+0000", 2024-09-15T00:13:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4600000000000005e-07 2024-09-15T00:13:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695136+0000", 2024-09-15T00:13:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00046864399999999999 2024-09-15T00:13:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695153+0000", 2024-09-15T00:13:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6218e-05 2024-09-15T00:13:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695189+0000", 2024-09-15T00:13:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6393000000000002e-05 2024-09-15T00:13:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695263+0000", 2024-09-15T00:13:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4175e-05 2024-09-15T00:13:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695271+0000", 2024-09-15T00:13:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2800000000000003e-06 2024-09-15T00:13:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.699848+0000", 2024-09-15T00:13:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045768839999999998 2024-09-15T00:13:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.157336440999998, 2024-09-15T00:13:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048716510000000003, 2024-09-15T00:13:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:13:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702783+0000", 2024-09-15T00:13:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7268000000000001e-05 2024-09-15T00:13:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703221+0000", 2024-09-15T00:13:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00043866899999999998 2024-09-15T00:13:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703237+0000", 2024-09-15T00:13:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5553000000000002e-05 2024-09-15T00:13:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703247+0000", 2024-09-15T00:13:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0392e-05 2024-09-15T00:13:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703358+0000", 2024-09-15T00:13:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110676 2024-09-15T00:13:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703369+0000", 2024-09-15T00:13:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0716e-05 2024-09-15T00:13:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.707637+0000", 2024-09-15T00:13:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042683770000000003 2024-09-15T00:13:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18, version: 62'274, data_included: [0~786432], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18@62'274, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.111255358999998, 2024-09-15T00:13:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005453014, 2024-09-15T00:13:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:13:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748847+0000", 2024-09-15T00:13:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8299999999999997e-07 2024-09-15T00:13:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749010+0000", 2024-09-15T00:13:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000163144 2024-09-15T00:13:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749027+0000", 2024-09-15T00:13:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6518999999999999e-05 2024-09-15T00:13:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749037+0000", 2024-09-15T00:13:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0438000000000001e-05 2024-09-15T00:13:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749112+0000", 2024-09-15T00:13:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4946999999999999e-05 2024-09-15T00:13:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749123+0000", 2024-09-15T00:13:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1014e-05 2024-09-15T00:13:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.754299+0000", 2024-09-15T00:13:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051763690000000001 2024-09-15T00:13:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20, version: 80'334, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20@80'334, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 20=[]:{20: [20]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.056618315000001, 2024-09-15T00:13:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048933570000000001, 2024-09-15T00:13:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:13:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803484+0000", 2024-09-15T00:13:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:13:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803938+0000", 2024-09-15T00:13:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045364599999999999 2024-09-15T00:13:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803954+0000", 2024-09-15T00:13:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6456999999999998e-05 2024-09-15T00:13:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803965+0000", 2024-09-15T00:13:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.076e-05 2024-09-15T00:13:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804023+0000", 2024-09-15T00:13:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7787000000000001e-05 2024-09-15T00:13:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804031+0000", 2024-09-15T00:13:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2160000000000006e-06 2024-09-15T00:13:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.808377+0000", 2024-09-15T00:13:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043458969999999996 2024-09-15T00:13:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [0~131072,196608~786432,1048576~131072], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.019831719000003, 2024-09-15T00:13:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047737029999999998, 2024-09-15T00:13:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840270+0000", 2024-09-15T00:13:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840271+0000", 2024-09-15T00:13:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:13:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840721+0000", 2024-09-15T00:13:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045029800000000002 2024-09-15T00:13:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840738+0000", 2024-09-15T00:13:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6671999999999999e-05 2024-09-15T00:13:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840749+0000", 2024-09-15T00:13:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1251e-05 2024-09-15T00:13:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840817+0000", 2024-09-15T00:13:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8046000000000005e-05 2024-09-15T00:13:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.840825+0000", 2024-09-15T00:13:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2519999999999999e-06 2024-09-15T00:13:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.845044+0000", 2024-09-15T00:13:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042185900000000004 2024-09-15T00:13:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [1179648~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2228224, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:13:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.011807672000003, 2024-09-15T00:13:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053056819999999999, 2024-09-15T00:13:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:13:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848295+0000", 2024-09-15T00:13:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4000000000000001e-07 2024-09-15T00:13:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848728+0000", 2024-09-15T00:13:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000433709 2024-09-15T00:13:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848745+0000", 2024-09-15T00:13:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6167999999999998e-05 2024-09-15T00:13:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848788+0000", 2024-09-15T00:13:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3269000000000002e-05 2024-09-15T00:13:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848857+0000", 2024-09-15T00:13:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9453000000000001e-05 2024-09-15T00:13:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848866+0000", 2024-09-15T00:13:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2779999999999993e-06 2024-09-15T00:13:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.853600+0000", 2024-09-15T00:13:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047341650000000002 2024-09-15T00:13:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head, version: 93'455, data_included: [0~98304,114688~16384,147456~589824,753664~65536,835584~16384,868352~262144], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head@93'455, size: 2342912, copy_subset: [(0, 2342912)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1130496, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:13:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.952673646000001, 2024-09-15T00:13:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063250900000000002, 2024-09-15T00:13:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:13:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907429+0000", 2024-09-15T00:13:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.85e-07 2024-09-15T00:13:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908008+0000", 2024-09-15T00:13:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00057949599999999998 2024-09-15T00:13:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908026+0000", 2024-09-15T00:13:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7700999999999999e-05 2024-09-15T00:13:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908038+0000", 2024-09-15T00:13:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2131e-05 2024-09-15T00:13:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908101+0000", 2024-09-15T00:13:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2596999999999997e-05 2024-09-15T00:13:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908110+0000", 2024-09-15T00:13:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8540000000000003e-06 2024-09-15T00:13:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.913753+0000", 2024-09-15T00:13:48.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056435260000000003 2024-09-15T00:13:48.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.950 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:48.950 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:48.950 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:48.951 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:48.952 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:48.949+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:49.051 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:49.050+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:49.106 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:49.106 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:49.106 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:49.106 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:49.115 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:49.157 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:49.155+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:49.253 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:49.252+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:49.266 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:49.266 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:49.266 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:49.275 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:49.354 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:49.352+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:49.421 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133834 3.1b e85/69 3:da9a78a3:::rbd_data.11395dfcf2b8.0000000000000000:head v 85'356, mlcod=85'354)", 2024-09-15T00:13:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.854920686, 2024-09-15T00:13:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026079054000000001, 2024-09-15T00:13:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.421 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563338+0000", 2024-09-15T00:13:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.418e-06 2024-09-15T00:13:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569385+0000", 2024-09-15T00:13:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006047335 2024-09-15T00:13:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569418+0000", 2024-09-15T00:13:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2957999999999999e-05 2024-09-15T00:13:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569432+0000", 2024-09-15T00:13:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3718e-05 2024-09-15T00:13:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569519+0000", 2024-09-15T00:13:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7022999999999997e-05 2024-09-15T00:13:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569540+0000", 2024-09-15T00:13:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1426000000000001e-05 2024-09-15T00:13:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589362+0000", 2024-09-15T00:13:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019821398 2024-09-15T00:13:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589415+0000", 2024-09-15T00:13:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3777999999999999e-05 2024-09-15T00:13:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.425 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133836 3.1 e85/67 3:849f9a98:::rbd_data.11395dfcf2b8.0000000000000002:head v 85'304, mlcod=84'301)", 2024-09-15T00:13:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.847225776000002, 2024-09-15T00:13:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023231600000000002, 2024-09-15T00:13:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571032+0000", 2024-09-15T00:13:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-15T00:13:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576635+0000", 2024-09-15T00:13:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056028700000000002 2024-09-15T00:13:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576650+0000", 2024-09-15T00:13:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.505e-05 2024-09-15T00:13:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576658+0000", 2024-09-15T00:13:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6879999999999999e-06 2024-09-15T00:13:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576717+0000", 2024-09-15T00:13:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175000000000002e-05 2024-09-15T00:13:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576742+0000", 2024-09-15T00:13:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4734000000000001e-05 2024-09-15T00:13:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594234+0000", 2024-09-15T00:13:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017492527000000001 2024-09-15T00:13:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594263+0000", 2024-09-15T00:13:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8629e-05 2024-09-15T00:13:49.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146609 3.b e98/69 3:d39b42ba:::rbd_data.11394501364.0000000000000001:head v 98'283, mlcod=93'281)", 2024-09-15T00:13:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.937507660999998, 2024-09-15T00:13:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020892945, 2024-09-15T00:13:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480751+0000", 2024-09-15T00:13:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6509999999999999e-06 2024-09-15T00:13:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482738+0000", 2024-09-15T00:13:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019867410000000002 2024-09-15T00:13:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482754+0000", 2024-09-15T00:13:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6171000000000002e-05 2024-09-15T00:13:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482769+0000", 2024-09-15T00:13:49.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4688000000000001e-05 2024-09-15T00:13:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482835+0000", 2024-09-15T00:13:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6526000000000006e-05 2024-09-15T00:13:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482858+0000", 2024-09-15T00:13:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2959000000000002e-05 2024-09-15T00:13:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501587+0000", 2024-09-15T00:13:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018728471 2024-09-15T00:13:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501642+0000", 2024-09-15T00:13:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5738000000000001e-05 2024-09-15T00:13:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146835 3.12 e98/68 3:4eee39d6:::rbd_data.11394501364.0000000000000000:head v 98'474, mlcod=93'472)", 2024-09-15T00:13:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:49.434 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.828880390999998, 2024-09-15T00:13:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020080434000000001, 2024-09-15T00:13:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589378+0000", 2024-09-15T00:13:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7770000000000001e-06 2024-09-15T00:13:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590678+0000", 2024-09-15T00:13:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012991750000000001 2024-09-15T00:13:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590699+0000", 2024-09-15T00:13:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1353e-05 2024-09-15T00:13:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590710+0000", 2024-09-15T00:13:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1245e-05 2024-09-15T00:13:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590800+0000", 2024-09-15T00:13:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9963000000000006e-05 2024-09-15T00:13:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590823+0000", 2024-09-15T00:13:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3289e-05 2024-09-15T00:13:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609368+0000", 2024-09-15T00:13:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018545078 2024-09-15T00:13:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609457+0000", 2024-09-15T00:13:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8554000000000006e-05 2024-09-15T00:13:49.438 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146837 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7201319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.818100928000007, 2024-09-15T00:13:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024751038, 2024-09-15T00:13:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146837 2024-09-15T00:13:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600157+0000", 2024-09-15T00:13:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:13:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601965+0000", 2024-09-15T00:13:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001807613 2024-09-15T00:13:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601985+0000", 2024-09-15T00:13:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0679e-05 2024-09-15T00:13:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601998+0000", 2024-09-15T00:13:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2955999999999999e-05 2024-09-15T00:13:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602070+0000", 2024-09-15T00:13:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1958000000000005e-05 2024-09-15T00:13:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602237+0000", 2024-09-15T00:13:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016621299999999999 2024-09-15T00:13:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602799+0000", 2024-09-15T00:13:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000562723 2024-09-15T00:13:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.620811+0000", 2024-09-15T00:13:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018011207000000001 2024-09-15T00:13:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624793+0000", 2024-09-15T00:13:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0039828770000000001 2024-09-15T00:13:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624855+0000", 2024-09-15T00:13:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.143e-05 2024-09-15T00:13:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624907+0000", 2024-09-15T00:13:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2262000000000002e-05 2024-09-15T00:13:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.444 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146936 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.664587608999994, 2024-09-15T00:13:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056847126999999997, 2024-09-15T00:13:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.445 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146936 2024-09-15T00:13:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753671+0000", 2024-09-15T00:13:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2839999999999999e-06 2024-09-15T00:13:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753677+0000", 2024-09-15T00:13:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0260000000000004e-06 2024-09-15T00:13:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753691+0000", 2024-09-15T00:13:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3891e-05 2024-09-15T00:13:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753698+0000", 2024-09-15T00:13:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8480000000000001e-06 2024-09-15T00:13:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753750+0000", 2024-09-15T00:13:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1338000000000002e-05 2024-09-15T00:13:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753814+0000", 2024-09-15T00:13:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4642999999999998e-05 2024-09-15T00:13:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.756359+0000", 2024-09-15T00:13:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025450619999999998 2024-09-15T00:13:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810435+0000", 2024-09-15T00:13:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.054075397999999997 2024-09-15T00:13:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810446+0000", 2024-09-15T00:13:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1005000000000001e-05 2024-09-15T00:13:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810482+0000", 2024-09-15T00:13:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6494e-05 2024-09-15T00:13:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810517+0000", 2024-09-15T00:13:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4137999999999997e-05 2024-09-15T00:13:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:146936 3.6 e98/97 ondisk, result = 0)", 2024-09-15T00:13:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.654804554999998, 2024-09-15T00:13:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046987205999999997, 2024-09-15T00:13:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763453+0000", 2024-09-15T00:13:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5199999999999996e-07 2024-09-15T00:13:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763457+0000", 2024-09-15T00:13:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8569999999999997e-06 2024-09-15T00:13:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763469+0000", 2024-09-15T00:13:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1977999999999999e-05 2024-09-15T00:13:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763476+0000", 2024-09-15T00:13:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:13:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810423+0000", 2024-09-15T00:13:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946993999999999 2024-09-15T00:13:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810431+0000", 2024-09-15T00:13:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0339999999999997e-06 2024-09-15T00:13:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810440+0000", 2024-09-15T00:13:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9989999999999997e-06 2024-09-15T00:13:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:151036 3.1f e103/68 3:f84a1a09:::rbd_data.113958cb0e55.000000000000000c:head v 103'452, mlcod=100'450)", 2024-09-15T00:13:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.081458327, 2024-09-15T00:13:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020618753, 2024-09-15T00:13:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:13:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340692+0000", 2024-09-15T00:13:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038928890000000001 2024-09-15T00:13:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340704+0000", 2024-09-15T00:13:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1497e-05 2024-09-15T00:13:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340709+0000", 2024-09-15T00:13:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1529999999999996e-06 2024-09-15T00:13:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340789+0000", 2024-09-15T00:13:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9907999999999995e-05 2024-09-15T00:13:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340810+0000", 2024-09-15T00:13:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1837000000000001e-05 2024-09-15T00:13:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357368+0000", 2024-09-15T00:13:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016557199000000002 2024-09-15T00:13:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357417+0000", 2024-09-15T00:13:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9783000000000001e-05 2024-09-15T00:13:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:151042 3.6 3:6419cde4:::rbd_data.113958cb0e55.0000000000000031:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.052380208999999, 2024-09-15T00:13:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025627375000000001, 2024-09-15T00:13:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 151042 2024-09-15T00:13:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7899999999999998e-07 2024-09-15T00:13:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365881+0000", 2024-09-15T00:13:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9770000000000002e-06 2024-09-15T00:13:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365890+0000", 2024-09-15T00:13:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5920000000000001e-06 2024-09-15T00:13:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365894+0000", 2024-09-15T00:13:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4569999999999998e-06 2024-09-15T00:13:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365927+0000", 2024-09-15T00:13:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2036000000000003e-05 2024-09-15T00:13:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365970+0000", 2024-09-15T00:13:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3197e-05 2024-09-15T00:13:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.367432+0000", 2024-09-15T00:13:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014621720000000001 2024-09-15T00:13:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391358+0000", 2024-09-15T00:13:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023925921999999999 2024-09-15T00:13:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391415+0000", 2024-09-15T00:13:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7253999999999998e-05 2024-09-15T00:13:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391456+0000", 2024-09-15T00:13:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0497999999999997e-05 2024-09-15T00:13:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391504+0000", 2024-09-15T00:13:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8591000000000003e-05 2024-09-15T00:13:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:151042 3.6 e103/97 ondisk, result = 0)", 2024-09-15T00:13:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.046637195999999, 2024-09-15T00:13:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019892730000000001, 2024-09-15T00:13:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8100000000000003e-07 2024-09-15T00:13:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371623+0000", 2024-09-15T00:13:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6290000000000002e-06 2024-09-15T00:13:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371632+0000", 2024-09-15T00:13:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6379999999999993e-06 2024-09-15T00:13:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371636+0000", 2024-09-15T00:13:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:13:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391403+0000", 2024-09-15T00:13:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019766982999999998 2024-09-15T00:13:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391411+0000", 2024-09-15T00:13:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7309999999999993e-06 2024-09-15T00:13:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391513+0000", 2024-09-15T00:13:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101648 2024-09-15T00:13:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152025 3.3 e103/68 3:c0279197:::rbd_data.113958cb0e55.0000000000000007:head v 103'464, mlcod=102'461)", 2024-09-15T00:13:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.902149600000001, 2024-09-15T00:13:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021264476000000001, 2024-09-15T00:13:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516109+0000", 2024-09-15T00:13:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.666e-06 2024-09-15T00:13:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.519997+0000", 2024-09-15T00:13:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003888449 2024-09-15T00:13:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520016+0000", 2024-09-15T00:13:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8059000000000001e-05 2024-09-15T00:13:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520026+0000", 2024-09-15T00:13:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9599999999999995e-06 2024-09-15T00:13:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520089+0000", 2024-09-15T00:13:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3074e-05 2024-09-15T00:13:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520108+0000", 2024-09-15T00:13:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8981999999999999e-05 2024-09-15T00:13:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537326+0000", 2024-09-15T00:13:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017218102999999998 2024-09-15T00:13:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537372+0000", 2024-09-15T00:13:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6183000000000002e-05 2024-09-15T00:13:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152284 3.f e103/68 3:f4272b5d:::rbd_data.113958cb0e55.0000000000000006:head v 103'370, mlcod=103'368)", 2024-09-15T00:13:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.853673069000003, 2024-09-15T00:13:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020145026999999999, 2024-09-15T00:13:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564585+0000", 2024-09-15T00:13:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7000000000000003e-07 2024-09-15T00:13:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568352+0000", 2024-09-15T00:13:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037675909999999998 2024-09-15T00:13:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568370+0000", 2024-09-15T00:13:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7158000000000001e-05 2024-09-15T00:13:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568378+0000", 2024-09-15T00:13:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6449999999999994e-06 2024-09-15T00:13:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568454+0000", 2024-09-15T00:13:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5285999999999997e-05 2024-09-15T00:13:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568472+0000", 2024-09-15T00:13:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8447000000000001e-05 2024-09-15T00:13:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584693+0000", 2024-09-15T00:13:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016221467999999999 2024-09-15T00:13:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584729+0000", 2024-09-15T00:13:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5462e-05 2024-09-15T00:13:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152500 3.17 e103/68 3:edff5adf:::rbd_data.113958cb0e55.000000000000000d:head v 103'461, mlcod=103'459)", 2024-09-15T00:13:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.776686484000003, 2024-09-15T00:13:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020217809, 2024-09-15T00:13:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641572+0000", 2024-09-15T00:13:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-15T00:13:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644961+0000", 2024-09-15T00:13:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033882610000000001 2024-09-15T00:13:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644979+0000", 2024-09-15T00:13:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8179e-05 2024-09-15T00:13:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644987+0000", 2024-09-15T00:13:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8860000000000004e-06 2024-09-15T00:13:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645044+0000", 2024-09-15T00:13:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7141000000000001e-05 2024-09-15T00:13:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645062+0000", 2024-09-15T00:13:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8258e-05 2024-09-15T00:13:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661749+0000", 2024-09-15T00:13:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016686553999999999 2024-09-15T00:13:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661788+0000", 2024-09-15T00:13:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9690000000000001e-05 2024-09-15T00:13:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153171 3.18 e103/69 3:1db186e8:::rbd_data.113958cb0e55.0000000000000002:head v 103'520, mlcod=103'518)", 2024-09-15T00:13:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.638201578, 2024-09-15T00:13:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022266838000000001, 2024-09-15T00:13:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780057+0000", 2024-09-15T00:13:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5880000000000001e-06 2024-09-15T00:13:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781273+0000", 2024-09-15T00:13:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001216087 2024-09-15T00:13:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781311+0000", 2024-09-15T00:13:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7656e-05 2024-09-15T00:13:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781325+0000", 2024-09-15T00:13:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4508999999999999e-05 2024-09-15T00:13:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781403+0000", 2024-09-15T00:13:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7927000000000001e-05 2024-09-15T00:13:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781422+0000", 2024-09-15T00:13:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8575000000000001e-05 2024-09-15T00:13:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802249+0000", 2024-09-15T00:13:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020827577 2024-09-15T00:13:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802322+0000", 2024-09-15T00:13:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2918999999999998e-05 2024-09-15T00:13:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153772 3.12 e103/68 3:4911a4f6:::rbd_data.113958cb0e55.0000000000000001:head v 103'490, mlcod=103'488)", 2024-09-15T00:13:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.535330950999999, 2024-09-15T00:13:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021181004, 2024-09-15T00:13:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882927+0000", 2024-09-15T00:13:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.426e-06 2024-09-15T00:13:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883966+0000", 2024-09-15T00:13:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010384610000000001 2024-09-15T00:13:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883985+0000", 2024-09-15T00:13:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9276e-05 2024-09-15T00:13:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883994+0000", 2024-09-15T00:13:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9490000000000002e-06 2024-09-15T00:13:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884054+0000", 2024-09-15T00:13:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9419999999999997e-05 2024-09-15T00:13:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884073+0000", 2024-09-15T00:13:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9137e-05 2024-09-15T00:13:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904038+0000", 2024-09-15T00:13:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019965092 2024-09-15T00:13:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904107+0000", 2024-09-15T00:13:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9243000000000001e-05 2024-09-15T00:13:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163112 3.3 e106/68 3:c668f3d5:::rbd_data.11395d771336.0000000000000001:head v 106'489, mlcod=105'487)", 2024-09-15T00:13:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.953433681999996, 2024-09-15T00:13:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019317183000000002, 2024-09-15T00:13:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464824+0000", 2024-09-15T00:13:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1100000000000005e-07 2024-09-15T00:13:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466874+0000", 2024-09-15T00:13:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002049497 2024-09-15T00:13:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466884+0000", 2024-09-15T00:13:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0424999999999999e-05 2024-09-15T00:13:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466890+0000", 2024-09-15T00:13:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-15T00:13:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466944+0000", 2024-09-15T00:13:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4592999999999997e-05 2024-09-15T00:13:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466958+0000", 2024-09-15T00:13:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3993e-05 2024-09-15T00:13:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484068+0000", 2024-09-15T00:13:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017109994 2024-09-15T00:13:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484140+0000", 2024-09-15T00:13:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2224000000000005e-05 2024-09-15T00:13:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163119 3.0 e106/94 3:06118613:::rbd_data.11395d771336.0000000000000004:head v 106'549, mlcod=105'547)", 2024-09-15T00:13:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.941818478999998, 2024-09-15T00:13:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017175867000000001, 2024-09-15T00:13:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6700000000000002e-07 2024-09-15T00:13:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477786+0000", 2024-09-15T00:13:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013468149999999999 2024-09-15T00:13:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477797+0000", 2024-09-15T00:13:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0635000000000001e-05 2024-09-15T00:13:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477805+0000", 2024-09-15T00:13:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2989999999999997e-06 2024-09-15T00:13:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477859+0000", 2024-09-15T00:13:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3699000000000002e-05 2024-09-15T00:13:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477878+0000", 2024-09-15T00:13:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9335e-05 2024-09-15T00:13:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493560+0000", 2024-09-15T00:13:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015681748999999998 2024-09-15T00:13:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493614+0000", 2024-09-15T00:13:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4367999999999999e-05 2024-09-15T00:13:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.5.0:5005 3.17 e110/68 3:eb0359e8:::rbd_data.113964deed84.0000000000000251:35 v 110'537, mlcod=109'534)", 2024-09-15T00:13:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.281968106999997, 2024-09-15T00:13:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015446896999999999, 2024-09-15T00:13:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136290+0000", 2024-09-15T00:13:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7400000000000003e-07 2024-09-15T00:13:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136294+0000", 2024-09-15T00:13:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4989999999999996e-06 2024-09-15T00:13:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136304+0000", 2024-09-15T00:13:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0319999999999999e-05 2024-09-15T00:13:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136310+0000", 2024-09-15T00:13:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.981e-06 2024-09-15T00:13:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136350+0000", 2024-09-15T00:13:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0088e-05 2024-09-15T00:13:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136364+0000", 2024-09-15T00:13:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3954999999999999e-05 2024-09-15T00:13:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151681+0000", 2024-09-15T00:13:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015316969999999999 2024-09-15T00:13:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151736+0000", 2024-09-15T00:13:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4410000000000003e-05 2024-09-15T00:13:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1758 3.10 e110/68 3:08149147:::rbd_data.113964deed84.0000000000000055:35 v 110'594, mlcod=109'591)", 2024-09-15T00:13:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.277669881000001, 2024-09-15T00:13:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016278652000000001, 2024-09-15T00:13:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140599+0000", 2024-09-15T00:13:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1109e-05 2024-09-15T00:13:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140605+0000", 2024-09-15T00:13:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-15T00:13:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140612+0000", 2024-09-15T00:13:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6089999999999997e-06 2024-09-15T00:13:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140656+0000", 2024-09-15T00:13:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3754999999999997e-05 2024-09-15T00:13:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140674+0000", 2024-09-15T00:13:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7640999999999999e-05 2024-09-15T00:13:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156823+0000", 2024-09-15T00:13:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016149047999999999 2024-09-15T00:13:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156866+0000", 2024-09-15T00:13:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3205999999999999e-05 2024-09-15T00:13:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.4.0:5722 3.e e110/95 3:74a58daf:::rbd_data.113964deed84.00000000000000c1:35 v 110'1287, mlcod=109'1284)", 2024-09-15T00:13:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 51.268595525000002, 2024-09-15T00:13:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01830555, 2024-09-15T00:13:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149662+0000", 2024-09-15T00:13:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:13:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149666+0000", 2024-09-15T00:13:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2220000000000004e-06 2024-09-15T00:13:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149672+0000", 2024-09-15T00:13:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3410000000000002e-06 2024-09-15T00:13:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149676+0000", 2024-09-15T00:13:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.566e-06 2024-09-15T00:13:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149709+0000", 2024-09-15T00:13:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2403000000000001e-05 2024-09-15T00:13:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149719+0000", 2024-09-15T00:13:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0608000000000001e-05 2024-09-15T00:13:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167944+0000", 2024-09-15T00:13:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018225183999999998 2024-09-15T00:13:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167967+0000", 2024-09-15T00:13:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2705e-05 2024-09-15T00:13:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.517 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:49.519 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:49.519 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:49.517+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:49.620 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:49.619+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:49.622 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:49.622 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:49.623 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:49.623 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:49.631 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:49.722 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:49.720+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:49.734 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:49.735 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:49.744 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:49.822 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:49.821+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:49.848 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 143.666315156, 2024-09-15T00:13:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:13:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.849 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:13:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:13:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:13:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:13:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:13:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:13:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:13:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:13:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:13:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:13:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:13:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:13:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:13:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:13:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:13:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:13:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:13:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:13:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:13:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:13:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:13:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:13:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:13:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.854 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:13:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 133.93647397699999, 2024-09-15T00:13:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:13:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:13:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:13:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:13:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:13:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:13:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:13:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:13:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:13:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:13:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:13:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:13:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:13:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:13:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:13:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:13:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:13:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:13:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:13:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:13:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:13:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:13:49.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:13:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:13:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:13:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 133.929515614, 2024-09-15T00:13:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:13:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.860 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:13:49.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:13:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:13:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:13:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:13:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:13:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:13:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:13:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:13:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:13:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:13:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:13:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:13:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:13:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 125.182326777, 2024-09-15T00:13:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:13:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:13:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:13:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:13:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:13:49.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:13:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:13:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:13:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:13:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:13:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:13:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:13:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:13:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:13:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:13:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:13:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:13:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:13:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:13:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:13:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:13:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:13:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:13:49.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:13:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:13:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97493 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 100.992744057, 2024-09-15T00:13:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033259141999999998, 2024-09-15T00:13:49.870 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97493 2024-09-15T00:13:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-15T00:13:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853401+0000", 2024-09-15T00:13:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:13:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853407+0000", 2024-09-15T00:13:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-15T00:13:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853411+0000", 2024-09-15T00:13:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.481e-06 2024-09-15T00:13:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886544+0000", 2024-09-15T00:13:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033132861999999999 2024-09-15T00:13:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886587+0000", 2024-09-15T00:13:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2942000000000003e-05 2024-09-15T00:13:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886655+0000", 2024-09-15T00:13:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7711000000000003e-05 2024-09-15T00:13:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97494 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 100.992722934, 2024-09-15T00:13:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033392390000000001, 2024-09-15T00:13:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.875 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97494 2024-09-15T00:13:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-15T00:13:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853419+0000", 2024-09-15T00:13:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.917e-06 2024-09-15T00:13:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853425+0000", 2024-09-15T00:13:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:13:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853433+0000", 2024-09-15T00:13:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8120000000000008e-06 2024-09-15T00:13:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886680+0000", 2024-09-15T00:13:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033247207000000001 2024-09-15T00:13:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886726+0000", 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5902000000000002e-05 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886809+0000", 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3475999999999998e-05 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97495 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 100.992701981, 2024-09-15T00:13:49.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033640503000000002, 2024-09-15T00:13:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97495 2024-09-15T00:13:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:13:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853440+0000", 2024-09-15T00:13:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.804e-06 2024-09-15T00:13:49.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853446+0000", 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5559999999999998e-06 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853450+0000", 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9519999999999996e-06 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886904+0000", 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033454207999999999 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887007+0000", 2024-09-15T00:13:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102904 2024-09-15T00:13:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887078+0000", 2024-09-15T00:13:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1719999999999998e-05 2024-09-15T00:13:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97496 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 100.992684883, 2024-09-15T00:13:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033802431000000001, 2024-09-15T00:13:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97496 2024-09-15T00:13:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.53e-07 2024-09-15T00:13:49.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853462+0000", 2024-09-15T00:13:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-15T00:13:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853468+0000", 2024-09-15T00:13:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3609999999999997e-06 2024-09-15T00:13:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853473+0000", 2024-09-15T00:13:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-15T00:13:49.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887111+0000", 2024-09-15T00:13:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033638376999999997 2024-09-15T00:13:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887178+0000", 2024-09-15T00:13:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6877000000000004e-05 2024-09-15T00:13:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887257+0000", 2024-09-15T00:13:49.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9616999999999993e-05 2024-09-15T00:13:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97587 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 100.992661649, 2024-09-15T00:13:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033973642999999998, 2024-09-15T00:13:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.888 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97587 2024-09-15T00:13:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853479+0000", 2024-09-15T00:13:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:13:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853481+0000", 2024-09-15T00:13:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.136e-06 2024-09-15T00:13:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853487+0000", 2024-09-15T00:13:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2439999999999998e-06 2024-09-15T00:13:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853492+0000", 2024-09-15T00:13:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:13:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887284+0000", 2024-09-15T00:13:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033791827000000003 2024-09-15T00:13:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887361+0000", 2024-09-15T00:13:49.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6983000000000005e-05 2024-09-15T00:13:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887452+0000", 2024-09-15T00:13:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1236999999999994e-05 2024-09-15T00:13:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97588 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:49.892 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 100.992641492, 2024-09-15T00:13:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034134617999999999, 2024-09-15T00:13:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97588 2024-09-15T00:13:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:49.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853499+0000", 2024-09-15T00:13:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:13:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853501+0000", 2024-09-15T00:13:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.26e-06 2024-09-15T00:13:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853507+0000", 2024-09-15T00:13:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2899999999999999e-06 2024-09-15T00:13:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853512+0000", 2024-09-15T00:13:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8330000000000001e-06 2024-09-15T00:13:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887495+0000", 2024-09-15T00:13:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033982529999999997 2024-09-15T00:13:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887548+0000", 2024-09-15T00:13:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3445000000000001e-05 2024-09-15T00:13:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887633+0000", 2024-09-15T00:13:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4890999999999997e-05 2024-09-15T00:13:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.896 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126866 3.6 3.973c9fa6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.889319546999999, 2024-09-15T00:13:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153301000000003, 2024-09-15T00:13:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126866 2024-09-15T00:13:49.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956821+0000", 2024-09-15T00:13:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.91e-07 2024-09-15T00:13:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956822+0000", 2024-09-15T00:13:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.705e-06 2024-09-15T00:13:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956828+0000", 2024-09-15T00:13:49.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6760000000000004e-06 2024-09-15T00:13:49.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956833+0000", 2024-09-15T00:13:49.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5379999999999996e-06 2024-09-15T00:13:49.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989970+0000", 2024-09-15T00:13:49.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033137560000000003 2024-09-15T00:13:49.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989974+0000", 2024-09-15T00:13:49.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-15T00:13:49.901 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.901 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.901 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.902 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126952 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:49.902 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:49.902 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.889176875000004, 2024-09-15T00:13:49.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033156066999999997, 2024-09-15T00:13:49.902 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.902 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.902 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:49.902 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.902 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.902 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.902 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126952 2024-09-15T00:13:49.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.902 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:49.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:49.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:49.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:13:49.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956965+0000", 2024-09-15T00:13:49.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3379999999999999e-06 2024-09-15T00:13:49.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956968+0000", 2024-09-15T00:13:49.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0419999999999998e-06 2024-09-15T00:13:49.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956970+0000", 2024-09-15T00:13:49.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4899999999999999e-06 2024-09-15T00:13:49.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990113+0000", 2024-09-15T00:13:49.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033142642 2024-09-15T00:13:49.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990119+0000", 2024-09-15T00:13:49.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2979999999999997e-06 2024-09-15T00:13:49.905 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.906 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.906 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.906 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126954 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:49.906 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:49.906 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.889131379999995, 2024-09-15T00:13:49.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033158449, 2024-09-15T00:13:49.906 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.906 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.906 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:49.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.906 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126954 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:49.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-15T00:13:49.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957010+0000", 2024-09-15T00:13:49.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3659999999999999e-06 2024-09-15T00:13:49.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957013+0000", 2024-09-15T00:13:49.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1140000000000002e-06 2024-09-15T00:13:49.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957016+0000", 2024-09-15T00:13:49.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1209999999999998e-06 2024-09-15T00:13:49.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990163+0000", 2024-09-15T00:13:49.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033146451 2024-09-15T00:13:49.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990167+0000", 2024-09-15T00:13:49.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0579999999999999e-06 2024-09-15T00:13:49.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.909 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126955 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.889120165999998, 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033160617000000003, 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126955 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:49.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:49.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:49.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-15T00:13:49.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957021+0000", 2024-09-15T00:13:49.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-15T00:13:49.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957024+0000", 2024-09-15T00:13:49.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1599999999999998e-06 2024-09-15T00:13:49.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957027+0000", 2024-09-15T00:13:49.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000001e-06 2024-09-15T00:13:49.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990177+0000", 2024-09-15T00:13:49.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033149741000000003 2024-09-15T00:13:49.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990180+0000", 2024-09-15T00:13:49.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4910000000000001e-06 2024-09-15T00:13:49.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.913 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.914 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126956 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:49.914 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:49.914 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.889110044999995, 2024-09-15T00:13:49.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033163633999999997, 2024-09-15T00:13:49.914 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.914 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.914 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:49.914 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.914 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.914 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.914 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126956 2024-09-15T00:13:49.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.914 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957031+0000", 2024-09-15T00:13:49.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-06 2024-09-15T00:13:49.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957034+0000", 2024-09-15T00:13:49.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9960000000000002e-06 2024-09-15T00:13:49.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957037+0000", 2024-09-15T00:13:49.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6419999999999999e-06 2024-09-15T00:13:49.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990190+0000", 2024-09-15T00:13:49.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153152999999998 2024-09-15T00:13:49.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990193+0000", 2024-09-15T00:13:49.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6069999999999999e-06 2024-09-15T00:13:49.917 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.917 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.917 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.918 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126957 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:49.918 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:49.918 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.889100395, 2024-09-15T00:13:49.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033166442999999997, 2024-09-15T00:13:49.918 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.918 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.918 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:49.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.918 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126957 2024-09-15T00:13:49.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.919 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:49.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:49.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957040+0000", 2024-09-15T00:13:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:13:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957041+0000", 2024-09-15T00:13:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0440000000000001e-06 2024-09-15T00:13:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957044+0000", 2024-09-15T00:13:49.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0170000000000001e-06 2024-09-15T00:13:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957048+0000", 2024-09-15T00:13:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0319999999999997e-06 2024-09-15T00:13:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990202+0000", 2024-09-15T00:13:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033154640999999999 2024-09-15T00:13:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990206+0000", 2024-09-15T00:13:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.507e-06 2024-09-15T00:13:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126958 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.889089165000001, 2024-09-15T00:13:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033167613999999998, 2024-09-15T00:13:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.922 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126958 2024-09-15T00:13:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:13:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957053+0000", 2024-09-15T00:13:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.756e-06 2024-09-15T00:13:49.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957059+0000", 2024-09-15T00:13:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2770000000000002e-06 2024-09-15T00:13:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957063+0000", 2024-09-15T00:13:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2459999999999997e-06 2024-09-15T00:13:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990215+0000", 2024-09-15T00:13:49.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033151509000000003 2024-09-15T00:13:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990218+0000", 2024-09-15T00:13:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5099999999999999e-06 2024-09-15T00:13:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126959 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.889070161000006, 2024-09-15T00:13:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033161673000000003, 2024-09-15T00:13:49.926 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126959 2024-09-15T00:13:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:13:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957072+0000", 2024-09-15T00:13:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2239999999999998e-06 2024-09-15T00:13:49.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957079+0000", 2024-09-15T00:13:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.19e-06 2024-09-15T00:13:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957083+0000", 2024-09-15T00:13:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:13:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990228+0000", 2024-09-15T00:13:49.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033144818 2024-09-15T00:13:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990231+0000", 2024-09-15T00:13:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:13:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126960 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 87.889051354000003, 2024-09-15T00:13:49.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033155166, 2024-09-15T00:13:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126960 2024-09-15T00:13:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:13:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957091+0000", 2024-09-15T00:13:49.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:13:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957097+0000", 2024-09-15T00:13:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6050000000000003e-06 2024-09-15T00:13:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957102+0000", 2024-09-15T00:13:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5539999999999999e-06 2024-09-15T00:13:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990240+0000", 2024-09-15T00:13:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033138289000000001 2024-09-15T00:13:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990244+0000", 2024-09-15T00:13:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4740000000000001e-06 2024-09-15T00:13:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.934 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:174575 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138)", 2024-09-15T00:13:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:26.073788+0000", 2024-09-15T00:13:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.772352039000001, 2024-09-15T00:13:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.95024539600000002, 2024-09-15T00:13:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 174575 2024-09-15T00:13:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073788+0000", 2024-09-15T00:13:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073788+0000", 2024-09-15T00:13:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073789+0000", 2024-09-15T00:13:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3999999999999999e-06 2024-09-15T00:13:49.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073796+0000", 2024-09-15T00:13:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5529999999999998e-06 2024-09-15T00:13:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073807+0000", 2024-09-15T00:13:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0825000000000001e-05 2024-09-15T00:13:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073812+0000", 2024-09-15T00:13:49.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6690000000000003e-06 2024-09-15T00:13:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073870+0000", 2024-09-15T00:13:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7241999999999998e-05 2024-09-15T00:13:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073878+0000", 2024-09-15T00:13:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0760000000000003e-06 2024-09-15T00:13:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:27.023756+0000", 2024-09-15T00:13:49.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94987846899999995 2024-09-15T00:13:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:27.023891+0000", 2024-09-15T00:13:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000134785 2024-09-15T00:13:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:27.024033+0000", 2024-09-15T00:13:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014237700000000001 2024-09-15T00:13:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:49.940 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:49.940 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:49.941 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:49.942 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:49.940+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:50.042 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:50.040+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:50.048 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 0, 1, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-15T00:13:50.048 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:13:50.048 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-15T00:13:50.048 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-15T00:13:50.088 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:13:50.094 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:50.094 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:50.094 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:50.095 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:50.105 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:50.143 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:50.142+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:50.220 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:50.220 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:50.226 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:13:50.237 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:50.238 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:50.244 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:50.242+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:50.255 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:50.255 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:50.257 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:50.258 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:50.267 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:50.291 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:50.291 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:50.325 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:50.325 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:50.345 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:50.343+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:50.359 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:50.360 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:50.394 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:50.394 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:50.400 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:50.400 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:50.410 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:50.428 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:50.428 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:50.443 INFO:tasks.ceph.mon.a.smithi171.stderr:2024-09-15T00:13:50.442+0000 7f0425d35640 -1 mon.a@0(leader).osd e162 definitely_dead 0 2024-09-15T00:13:50.445 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:50.444+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:50.516 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:50.517 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:50.523 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:50.567 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:50.565+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:50.626 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:50.626 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:50.635 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:50.676 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:50.674+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:50.748 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:50.747+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:50.778 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:50.778 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:50.787 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:50.849 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:50.847+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:50.889 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:50.889 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:50.897 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:50.951 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:50.949+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:51.019 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is unset 2024-09-15T00:13:51.026 INFO:teuthology.orchestra.run.smithi171.stderr:marked in osd.2. 2024-09-15T00:13:51.045 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-15T00:13:51.050 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:51.050 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:51.051 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:51.049+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:51.060 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:51.152 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:51.150+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:51.202 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:51.202 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:51.211 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:51.252 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:51.251+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:51.314 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:51.314 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:51.322 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:51.366 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:51.364+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:51.425 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:51.425 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:51.434 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:51.475 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:51.473+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:51.555 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:51.554+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:51.587 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:51.656 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:51.654+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:51.689 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:51.689 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:51.698 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:51.756 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:51.755+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:51.801 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:51.801 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:51.810 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:51.894 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:51.892+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:51.953 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:51.953 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:51.958 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:51.957+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:51.961 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:52.060 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:52.058+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:52.080 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:52.080 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:52.089 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:52.160 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:52.158+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:52.193 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:52.193 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:52.201 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:52.261 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:52.259+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:52.347 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:52.347 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:52.357 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:52.401 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:52.399+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:52.439 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:13:52.462 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:52.463 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:52.463 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:52.461+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:52.472 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:52.564 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:52.565 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:52.565 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:52.563+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:52.581 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:52.581 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:52.597 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:52.598 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:52.614 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:52.614 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:52.619 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:52.619 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:52.619 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:52.619 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:52.619 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:52.628 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:52.631 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:52.631 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:52.648 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:52.648 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:52.665 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:52.666 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:52.666 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:52.664+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:52.684 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:52.684 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:52.737 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:52.737 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:52.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:52.738 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:52.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.738 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133834 3.1b e85/69 3:da9a78a3:::rbd_data.11395dfcf2b8.0000000000000000:head v 85'356, mlcod=85'354)", 2024-09-15T00:13:52.738 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:52.738 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.171509205000007, 2024-09-15T00:13:52.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026079054000000001, 2024-09-15T00:13:52.738 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.738 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.738 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.738 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:52.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:13:52.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563338+0000", 2024-09-15T00:13:52.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.418e-06 2024-09-15T00:13:52.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569385+0000", 2024-09-15T00:13:52.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006047335 2024-09-15T00:13:52.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569418+0000", 2024-09-15T00:13:52.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2957999999999999e-05 2024-09-15T00:13:52.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569432+0000", 2024-09-15T00:13:52.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3718e-05 2024-09-15T00:13:52.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569519+0000", 2024-09-15T00:13:52.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7022999999999997e-05 2024-09-15T00:13:52.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569540+0000", 2024-09-15T00:13:52.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1426000000000001e-05 2024-09-15T00:13:52.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589362+0000", 2024-09-15T00:13:52.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019821398 2024-09-15T00:13:52.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589415+0000", 2024-09-15T00:13:52.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3777999999999999e-05 2024-09-15T00:13:52.742 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.742 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.742 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.742 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133836 3.1 e85/67 3:849f9a98:::rbd_data.11395dfcf2b8.0000000000000002:head v 85'304, mlcod=84'301)", 2024-09-15T00:13:52.742 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:52.743 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.163814294999995, 2024-09-15T00:13:52.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023231600000000002, 2024-09-15T00:13:52.743 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.743 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.743 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.743 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:52.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:13:52.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571032+0000", 2024-09-15T00:13:52.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-15T00:13:52.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576635+0000", 2024-09-15T00:13:52.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056028700000000002 2024-09-15T00:13:52.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576650+0000", 2024-09-15T00:13:52.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.505e-05 2024-09-15T00:13:52.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576658+0000", 2024-09-15T00:13:52.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6879999999999999e-06 2024-09-15T00:13:52.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576717+0000", 2024-09-15T00:13:52.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175000000000002e-05 2024-09-15T00:13:52.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576742+0000", 2024-09-15T00:13:52.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4734000000000001e-05 2024-09-15T00:13:52.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594234+0000", 2024-09-15T00:13:52.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017492527000000001 2024-09-15T00:13:52.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594263+0000", 2024-09-15T00:13:52.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8629e-05 2024-09-15T00:13:52.747 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.747 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.747 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.747 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146609 3.b e98/69 3:d39b42ba:::rbd_data.11394501364.0000000000000001:head v 98'283, mlcod=93'281)", 2024-09-15T00:13:52.747 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:52.747 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.254096180000005, 2024-09-15T00:13:52.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020892945, 2024-09-15T00:13:52.747 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.747 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.747 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.747 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:52.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:13:52.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480751+0000", 2024-09-15T00:13:52.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6509999999999999e-06 2024-09-15T00:13:52.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482738+0000", 2024-09-15T00:13:52.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019867410000000002 2024-09-15T00:13:52.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482754+0000", 2024-09-15T00:13:52.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6171000000000002e-05 2024-09-15T00:13:52.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482769+0000", 2024-09-15T00:13:52.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4688000000000001e-05 2024-09-15T00:13:52.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482835+0000", 2024-09-15T00:13:52.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6526000000000006e-05 2024-09-15T00:13:52.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482858+0000", 2024-09-15T00:13:52.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2959000000000002e-05 2024-09-15T00:13:52.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501587+0000", 2024-09-15T00:13:52.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018728471 2024-09-15T00:13:52.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501642+0000", 2024-09-15T00:13:52.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5738000000000001e-05 2024-09-15T00:13:52.751 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.751 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.751 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.751 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146835 3.12 e98/68 3:4eee39d6:::rbd_data.11394501364.0000000000000000:head v 98'474, mlcod=93'472)", 2024-09-15T00:13:52.751 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:52.751 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.145468910000005, 2024-09-15T00:13:52.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020080434000000001, 2024-09-15T00:13:52.752 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.752 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.752 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.752 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:52.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:13:52.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589378+0000", 2024-09-15T00:13:52.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7770000000000001e-06 2024-09-15T00:13:52.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590678+0000", 2024-09-15T00:13:52.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012991750000000001 2024-09-15T00:13:52.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590699+0000", 2024-09-15T00:13:52.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1353e-05 2024-09-15T00:13:52.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590710+0000", 2024-09-15T00:13:52.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1245e-05 2024-09-15T00:13:52.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590800+0000", 2024-09-15T00:13:52.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9963000000000006e-05 2024-09-15T00:13:52.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590823+0000", 2024-09-15T00:13:52.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3289e-05 2024-09-15T00:13:52.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609368+0000", 2024-09-15T00:13:52.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018545078 2024-09-15T00:13:52.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609457+0000", 2024-09-15T00:13:52.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8554000000000006e-05 2024-09-15T00:13:52.756 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.756 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.756 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.756 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146837 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7201319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:52.756 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:52.756 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.134689447, 2024-09-15T00:13:52.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024751038, 2024-09-15T00:13:52.757 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.757 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.757 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.757 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:52.757 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:52.757 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:52.757 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146837 2024-09-15T00:13:52.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.757 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:52.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:13:52.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600157+0000", 2024-09-15T00:13:52.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:13:52.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601965+0000", 2024-09-15T00:13:52.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001807613 2024-09-15T00:13:52.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601985+0000", 2024-09-15T00:13:52.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0679e-05 2024-09-15T00:13:52.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601998+0000", 2024-09-15T00:13:52.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2955999999999999e-05 2024-09-15T00:13:52.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602070+0000", 2024-09-15T00:13:52.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1958000000000005e-05 2024-09-15T00:13:52.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602237+0000", 2024-09-15T00:13:52.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016621299999999999 2024-09-15T00:13:52.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:52.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602799+0000", 2024-09-15T00:13:52.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000562723 2024-09-15T00:13:52.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:52.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.620811+0000", 2024-09-15T00:13:52.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018011207000000001 2024-09-15T00:13:52.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:52.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624793+0000", 2024-09-15T00:13:52.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0039828770000000001 2024-09-15T00:13:52.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624855+0000", 2024-09-15T00:13:52.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.143e-05 2024-09-15T00:13:52.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624907+0000", 2024-09-15T00:13:52.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2262000000000002e-05 2024-09-15T00:13:52.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.762 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.762 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146936 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:52.762 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:52.762 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 67.981176128000001, 2024-09-15T00:13:52.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056847126999999997, 2024-09-15T00:13:52.763 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.763 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.763 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:52.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:52.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:52.763 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146936 2024-09-15T00:13:52.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.763 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:52.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:13:52.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753671+0000", 2024-09-15T00:13:52.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2839999999999999e-06 2024-09-15T00:13:52.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753677+0000", 2024-09-15T00:13:52.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0260000000000004e-06 2024-09-15T00:13:52.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753691+0000", 2024-09-15T00:13:52.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3891e-05 2024-09-15T00:13:52.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753698+0000", 2024-09-15T00:13:52.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8480000000000001e-06 2024-09-15T00:13:52.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753750+0000", 2024-09-15T00:13:52.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1338000000000002e-05 2024-09-15T00:13:52.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753814+0000", 2024-09-15T00:13:52.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4642999999999998e-05 2024-09-15T00:13:52.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:52.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.756359+0000", 2024-09-15T00:13:52.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025450619999999998 2024-09-15T00:13:52.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:52.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810435+0000", 2024-09-15T00:13:52.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.054075397999999997 2024-09-15T00:13:52.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:52.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810446+0000", 2024-09-15T00:13:52.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1005000000000001e-05 2024-09-15T00:13:52.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810482+0000", 2024-09-15T00:13:52.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6494e-05 2024-09-15T00:13:52.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810517+0000", 2024-09-15T00:13:52.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4137999999999997e-05 2024-09-15T00:13:52.768 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.768 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.768 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.768 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:146936 3.6 e98/97 ondisk, result = 0)", 2024-09-15T00:13:52.768 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:52.768 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 67.971393074000005, 2024-09-15T00:13:52.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046987205999999997, 2024-09-15T00:13:52.768 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.768 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.768 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:52.768 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:52.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:13:52.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763453+0000", 2024-09-15T00:13:52.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5199999999999996e-07 2024-09-15T00:13:52.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763457+0000", 2024-09-15T00:13:52.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8569999999999997e-06 2024-09-15T00:13:52.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763469+0000", 2024-09-15T00:13:52.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1977999999999999e-05 2024-09-15T00:13:52.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763476+0000", 2024-09-15T00:13:52.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:13:52.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810423+0000", 2024-09-15T00:13:52.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946993999999999 2024-09-15T00:13:52.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810431+0000", 2024-09-15T00:13:52.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0339999999999997e-06 2024-09-15T00:13:52.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810440+0000", 2024-09-15T00:13:52.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9989999999999997e-06 2024-09-15T00:13:52.772 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.772 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.772 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.772 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:151036 3.1f e103/68 3:f84a1a09:::rbd_data.113958cb0e55.000000000000000c:head v 103'452, mlcod=100'450)", 2024-09-15T00:13:52.772 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:52.772 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.398046846, 2024-09-15T00:13:52.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020618753, 2024-09-15T00:13:52.772 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.773 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.773 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.773 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:52.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:52.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:13:52.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:13:52.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340692+0000", 2024-09-15T00:13:52.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038928890000000001 2024-09-15T00:13:52.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340704+0000", 2024-09-15T00:13:52.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1497e-05 2024-09-15T00:13:52.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340709+0000", 2024-09-15T00:13:52.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1529999999999996e-06 2024-09-15T00:13:52.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340789+0000", 2024-09-15T00:13:52.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9907999999999995e-05 2024-09-15T00:13:52.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340810+0000", 2024-09-15T00:13:52.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1837000000000001e-05 2024-09-15T00:13:52.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357368+0000", 2024-09-15T00:13:52.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016557199000000002 2024-09-15T00:13:52.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357417+0000", 2024-09-15T00:13:52.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9783000000000001e-05 2024-09-15T00:13:52.777 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.777 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.777 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.778 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:151042 3.6 3:6419cde4:::rbd_data.113958cb0e55.0000000000000031:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:52.778 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:52.778 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.368968727999999, 2024-09-15T00:13:52.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025627375000000001, 2024-09-15T00:13:52.778 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.778 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.778 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.778 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:52.778 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:52.778 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:52.778 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 151042 2024-09-15T00:13:52.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.778 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:52.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:52.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:13:52.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7899999999999998e-07 2024-09-15T00:13:52.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365881+0000", 2024-09-15T00:13:52.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9770000000000002e-06 2024-09-15T00:13:52.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365890+0000", 2024-09-15T00:13:52.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5920000000000001e-06 2024-09-15T00:13:52.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365894+0000", 2024-09-15T00:13:52.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4569999999999998e-06 2024-09-15T00:13:52.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365927+0000", 2024-09-15T00:13:52.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2036000000000003e-05 2024-09-15T00:13:52.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365970+0000", 2024-09-15T00:13:52.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3197e-05 2024-09-15T00:13:52.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:52.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.367432+0000", 2024-09-15T00:13:52.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014621720000000001 2024-09-15T00:13:52.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:52.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391358+0000", 2024-09-15T00:13:52.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023925921999999999 2024-09-15T00:13:52.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:52.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391415+0000", 2024-09-15T00:13:52.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7253999999999998e-05 2024-09-15T00:13:52.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391456+0000", 2024-09-15T00:13:52.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0497999999999997e-05 2024-09-15T00:13:52.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391504+0000", 2024-09-15T00:13:52.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8591000000000003e-05 2024-09-15T00:13:52.783 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.783 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.783 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.784 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:151042 3.6 e103/97 ondisk, result = 0)", 2024-09-15T00:13:52.784 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:52.784 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.363225714999999, 2024-09-15T00:13:52.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019892730000000001, 2024-09-15T00:13:52.784 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.784 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.784 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:52.784 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:52.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:52.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:13:52.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8100000000000003e-07 2024-09-15T00:13:52.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371623+0000", 2024-09-15T00:13:52.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6290000000000002e-06 2024-09-15T00:13:52.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371632+0000", 2024-09-15T00:13:52.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6379999999999993e-06 2024-09-15T00:13:52.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371636+0000", 2024-09-15T00:13:52.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:13:52.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391403+0000", 2024-09-15T00:13:52.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019766982999999998 2024-09-15T00:13:52.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391411+0000", 2024-09-15T00:13:52.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7309999999999993e-06 2024-09-15T00:13:52.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391513+0000", 2024-09-15T00:13:52.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101648 2024-09-15T00:13:52.787 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152025 3.3 e103/68 3:c0279197:::rbd_data.113958cb0e55.0000000000000007:head v 103'464, mlcod=102'461)", 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.218738119000001, 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021264476000000001, 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:52.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:13:52.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516109+0000", 2024-09-15T00:13:52.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.666e-06 2024-09-15T00:13:52.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.519997+0000", 2024-09-15T00:13:52.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003888449 2024-09-15T00:13:52.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520016+0000", 2024-09-15T00:13:52.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8059000000000001e-05 2024-09-15T00:13:52.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520026+0000", 2024-09-15T00:13:52.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9599999999999995e-06 2024-09-15T00:13:52.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520089+0000", 2024-09-15T00:13:52.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3074e-05 2024-09-15T00:13:52.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520108+0000", 2024-09-15T00:13:52.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8981999999999999e-05 2024-09-15T00:13:52.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537326+0000", 2024-09-15T00:13:52.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017218102999999998 2024-09-15T00:13:52.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537372+0000", 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6183000000000002e-05 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152284 3.f e103/68 3:f4272b5d:::rbd_data.113958cb0e55.0000000000000006:head v 103'370, mlcod=103'368)", 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.170261588000002, 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020145026999999999, 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.792 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:52.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:13:52.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564585+0000", 2024-09-15T00:13:52.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7000000000000003e-07 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568352+0000", 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037675909999999998 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568370+0000", 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7158000000000001e-05 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568378+0000", 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6449999999999994e-06 2024-09-15T00:13:52.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568454+0000", 2024-09-15T00:13:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5285999999999997e-05 2024-09-15T00:13:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568472+0000", 2024-09-15T00:13:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8447000000000001e-05 2024-09-15T00:13:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584693+0000", 2024-09-15T00:13:52.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016221467999999999 2024-09-15T00:13:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584729+0000", 2024-09-15T00:13:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5462e-05 2024-09-15T00:13:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152500 3.17 e103/68 3:edff5adf:::rbd_data.113958cb0e55.000000000000000d:head v 103'461, mlcod=103'459)", 2024-09-15T00:13:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.093275003000002, 2024-09-15T00:13:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020217809, 2024-09-15T00:13:52.796 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:13:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641572+0000", 2024-09-15T00:13:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-15T00:13:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644961+0000", 2024-09-15T00:13:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033882610000000001 2024-09-15T00:13:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644979+0000", 2024-09-15T00:13:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8179e-05 2024-09-15T00:13:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644987+0000", 2024-09-15T00:13:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8860000000000004e-06 2024-09-15T00:13:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645044+0000", 2024-09-15T00:13:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7141000000000001e-05 2024-09-15T00:13:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645062+0000", 2024-09-15T00:13:52.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8258e-05 2024-09-15T00:13:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661749+0000", 2024-09-15T00:13:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016686553999999999 2024-09-15T00:13:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661788+0000", 2024-09-15T00:13:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9690000000000001e-05 2024-09-15T00:13:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.800 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153171 3.18 e103/69 3:1db186e8:::rbd_data.113958cb0e55.0000000000000002:head v 103'520, mlcod=103'518)", 2024-09-15T00:13:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 62.954790097, 2024-09-15T00:13:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022266838000000001, 2024-09-15T00:13:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:13:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780057+0000", 2024-09-15T00:13:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5880000000000001e-06 2024-09-15T00:13:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781273+0000", 2024-09-15T00:13:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001216087 2024-09-15T00:13:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781311+0000", 2024-09-15T00:13:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7656e-05 2024-09-15T00:13:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781325+0000", 2024-09-15T00:13:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4508999999999999e-05 2024-09-15T00:13:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781403+0000", 2024-09-15T00:13:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7927000000000001e-05 2024-09-15T00:13:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781422+0000", 2024-09-15T00:13:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8575000000000001e-05 2024-09-15T00:13:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802249+0000", 2024-09-15T00:13:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020827577 2024-09-15T00:13:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802322+0000", 2024-09-15T00:13:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2918999999999998e-05 2024-09-15T00:13:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153772 3.12 e103/68 3:4911a4f6:::rbd_data.113958cb0e55.0000000000000001:head v 103'490, mlcod=103'488)", 2024-09-15T00:13:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 62.851919469999999, 2024-09-15T00:13:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021181004, 2024-09-15T00:13:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.805 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:13:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882927+0000", 2024-09-15T00:13:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.426e-06 2024-09-15T00:13:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883966+0000", 2024-09-15T00:13:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010384610000000001 2024-09-15T00:13:52.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883985+0000", 2024-09-15T00:13:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9276e-05 2024-09-15T00:13:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883994+0000", 2024-09-15T00:13:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9490000000000002e-06 2024-09-15T00:13:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884054+0000", 2024-09-15T00:13:52.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9419999999999997e-05 2024-09-15T00:13:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884073+0000", 2024-09-15T00:13:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9137e-05 2024-09-15T00:13:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904038+0000", 2024-09-15T00:13:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019965092 2024-09-15T00:13:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904107+0000", 2024-09-15T00:13:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9243000000000001e-05 2024-09-15T00:13:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163112 3.3 e106/68 3:c668f3d5:::rbd_data.11395d771336.0000000000000001:head v 106'489, mlcod=105'487)", 2024-09-15T00:13:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.270022201000003, 2024-09-15T00:13:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019317183000000002, 2024-09-15T00:13:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.810 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:13:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464824+0000", 2024-09-15T00:13:52.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1100000000000005e-07 2024-09-15T00:13:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466874+0000", 2024-09-15T00:13:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002049497 2024-09-15T00:13:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466884+0000", 2024-09-15T00:13:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0424999999999999e-05 2024-09-15T00:13:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466890+0000", 2024-09-15T00:13:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-15T00:13:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466944+0000", 2024-09-15T00:13:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4592999999999997e-05 2024-09-15T00:13:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466958+0000", 2024-09-15T00:13:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3993e-05 2024-09-15T00:13:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484068+0000", 2024-09-15T00:13:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017109994 2024-09-15T00:13:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484140+0000", 2024-09-15T00:13:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2224000000000005e-05 2024-09-15T00:13:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163119 3.0 e106/94 3:06118613:::rbd_data.11395d771336.0000000000000004:head v 106'549, mlcod=105'547)", 2024-09-15T00:13:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.258406997999998, 2024-09-15T00:13:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017175867000000001, 2024-09-15T00:13:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:13:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6700000000000002e-07 2024-09-15T00:13:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477786+0000", 2024-09-15T00:13:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013468149999999999 2024-09-15T00:13:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477797+0000", 2024-09-15T00:13:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0635000000000001e-05 2024-09-15T00:13:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477805+0000", 2024-09-15T00:13:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2989999999999997e-06 2024-09-15T00:13:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477859+0000", 2024-09-15T00:13:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3699000000000002e-05 2024-09-15T00:13:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477878+0000", 2024-09-15T00:13:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9335e-05 2024-09-15T00:13:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493560+0000", 2024-09-15T00:13:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015681748999999998 2024-09-15T00:13:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493614+0000", 2024-09-15T00:13:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4367999999999999e-05 2024-09-15T00:13:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.818 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.5.0:5005 3.17 e110/68 3:eb0359e8:::rbd_data.113964deed84.0000000000000251:35 v 110'537, mlcod=109'534)", 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 54.598556625999997, 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015446896999999999, 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:13:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136290+0000", 2024-09-15T00:13:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7400000000000003e-07 2024-09-15T00:13:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136294+0000", 2024-09-15T00:13:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4989999999999996e-06 2024-09-15T00:13:52.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136304+0000", 2024-09-15T00:13:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0319999999999999e-05 2024-09-15T00:13:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136310+0000", 2024-09-15T00:13:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.981e-06 2024-09-15T00:13:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136350+0000", 2024-09-15T00:13:52.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0088e-05 2024-09-15T00:13:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136364+0000", 2024-09-15T00:13:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3954999999999999e-05 2024-09-15T00:13:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151681+0000", 2024-09-15T00:13:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015316969999999999 2024-09-15T00:13:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151736+0000", 2024-09-15T00:13:52.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4410000000000003e-05 2024-09-15T00:13:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1758 3.10 e110/68 3:08149147:::rbd_data.113964deed84.0000000000000055:35 v 110'594, mlcod=109'591)", 2024-09-15T00:13:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 54.594258400000001, 2024-09-15T00:13:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016278652000000001, 2024-09-15T00:13:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.823 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:13:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140599+0000", 2024-09-15T00:13:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1109e-05 2024-09-15T00:13:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140605+0000", 2024-09-15T00:13:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-15T00:13:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140612+0000", 2024-09-15T00:13:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6089999999999997e-06 2024-09-15T00:13:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140656+0000", 2024-09-15T00:13:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3754999999999997e-05 2024-09-15T00:13:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140674+0000", 2024-09-15T00:13:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7640999999999999e-05 2024-09-15T00:13:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156823+0000", 2024-09-15T00:13:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016149047999999999 2024-09-15T00:13:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156866+0000", 2024-09-15T00:13:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3205999999999999e-05 2024-09-15T00:13:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.4.0:5722 3.e e110/95 3:74a58daf:::rbd_data.113964deed84.00000000000000c1:35 v 110'1287, mlcod=109'1284)", 2024-09-15T00:13:52.827 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 54.585184044000002, 2024-09-15T00:13:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01830555, 2024-09-15T00:13:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:13:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149662+0000", 2024-09-15T00:13:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:13:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149666+0000", 2024-09-15T00:13:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2220000000000004e-06 2024-09-15T00:13:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149672+0000", 2024-09-15T00:13:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3410000000000002e-06 2024-09-15T00:13:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149676+0000", 2024-09-15T00:13:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.566e-06 2024-09-15T00:13:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149709+0000", 2024-09-15T00:13:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2403000000000001e-05 2024-09-15T00:13:52.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149719+0000", 2024-09-15T00:13:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0608000000000001e-05 2024-09-15T00:13:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167944+0000", 2024-09-15T00:13:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018225183999999998 2024-09-15T00:13:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167967+0000", 2024-09-15T00:13:52.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2705e-05 2024-09-15T00:13:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:52.832 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:52.832 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:52.833 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:52.834 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:52.832+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:52.934 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:52.933+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:52.977 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:52.977 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:52.977 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:52.977 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:52.986 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:53.041 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:13:53.070 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:53.068+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:53.135 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:53.134+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:53.159 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:53.159 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:53.159 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:53.159 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:53.159 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:53.170 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:53.253 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:53.252+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:53.331 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:53.332 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:53.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:53.332 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:53.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.332 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:13:53.332 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:53.332 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 147.14899366200001, 2024-09-15T00:13:53.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:13:53.332 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.333 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.333 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:53.333 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.333 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.333 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.333 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:13:53.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.333 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:53.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:13:53.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:13:53.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:13:53.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:13:53.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:13:53.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:13:53.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:13:53.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:53.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:13:53.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:13:53.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:53.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:13:53.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:13:53.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:53.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:13:53.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:13:53.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:53.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:13:53.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:13:53.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:13:53.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:13:53.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.338 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.338 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:13:53.338 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:53.338 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 137.419152483, 2024-09-15T00:13:53.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:13:53.339 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.339 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.339 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:53.339 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.339 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.339 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.339 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:13:53.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.339 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:53.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:13:53.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:13:53.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:13:53.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:13:53.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:13:53.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:13:53.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:13:53.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:13:53.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:53.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:13:53.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:13:53.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:53.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:13:53.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:13:53.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:53.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:13:53.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:13:53.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:53.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:13:53.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:13:53.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:13:53.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:13:53.344 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.344 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.344 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.344 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:13:53.344 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:53.345 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 137.41219412000001, 2024-09-15T00:13:53.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:13:53.345 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.345 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.345 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.345 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:53.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:13:53.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:13:53.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:13:53.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:13:53.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:13:53.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:13:53.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:13:53.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:13:53.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:13:53.348 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.348 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.348 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.348 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:53.348 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:53.349 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.665005283, 2024-09-15T00:13:53.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:13:53.349 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.349 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.349 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:53.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.349 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:13:53.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.349 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:53.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:13:53.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:13:53.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:13:53.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:13:53.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:13:53.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:13:53.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:13:53.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:13:53.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:13:53.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:13:53.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:13:53.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:13:53.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:13:53.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:13:53.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:13:53.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:13:53.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:53.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:13:53.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:13:53.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:53.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:13:53.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:13:53.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:53.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:13:53.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:13:53.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:13:53.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:13:53.354 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.354 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.354 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.354 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97493 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:53.354 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:53.354 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 104.475422563, 2024-09-15T00:13:53.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033259141999999998, 2024-09-15T00:13:53.355 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.355 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.355 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.355 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.355 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.355 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.355 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97493 2024-09-15T00:13:53.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.355 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:53.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:53.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:13:53.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-15T00:13:53.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853401+0000", 2024-09-15T00:13:53.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:13:53.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853407+0000", 2024-09-15T00:13:53.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-15T00:13:53.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853411+0000", 2024-09-15T00:13:53.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.481e-06 2024-09-15T00:13:53.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886544+0000", 2024-09-15T00:13:53.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033132861999999999 2024-09-15T00:13:53.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886587+0000", 2024-09-15T00:13:53.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2942000000000003e-05 2024-09-15T00:13:53.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886655+0000", 2024-09-15T00:13:53.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7711000000000003e-05 2024-09-15T00:13:53.359 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.359 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.359 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.359 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97494 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:53.359 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:53.359 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 104.47540144, 2024-09-15T00:13:53.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033392390000000001, 2024-09-15T00:13:53.359 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.359 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.359 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.360 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.360 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97494 2024-09-15T00:13:53.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.360 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:53.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:53.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:13:53.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-15T00:13:53.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853419+0000", 2024-09-15T00:13:53.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.917e-06 2024-09-15T00:13:53.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853425+0000", 2024-09-15T00:13:53.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:13:53.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853433+0000", 2024-09-15T00:13:53.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8120000000000008e-06 2024-09-15T00:13:53.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886680+0000", 2024-09-15T00:13:53.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033247207000000001 2024-09-15T00:13:53.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886726+0000", 2024-09-15T00:13:53.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5902000000000002e-05 2024-09-15T00:13:53.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886809+0000", 2024-09-15T00:13:53.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3475999999999998e-05 2024-09-15T00:13:53.363 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.363 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.363 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.364 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97495 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:53.364 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:53.364 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 104.475380487, 2024-09-15T00:13:53.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033640503000000002, 2024-09-15T00:13:53.364 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.364 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.364 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.364 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97495 2024-09-15T00:13:53.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.365 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:53.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:53.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:13:53.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853440+0000", 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.804e-06 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853446+0000", 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5559999999999998e-06 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853450+0000", 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9519999999999996e-06 2024-09-15T00:13:53.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886904+0000", 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033454207999999999 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887007+0000", 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102904 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887078+0000", 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1719999999999998e-05 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.367 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.368 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.368 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97496 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:53.368 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:53.368 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 104.47536338899999, 2024-09-15T00:13:53.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033802431000000001, 2024-09-15T00:13:53.368 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.368 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.368 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.368 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.368 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.368 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.368 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97496 2024-09-15T00:13:53.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.53e-07 2024-09-15T00:13:53.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853462+0000", 2024-09-15T00:13:53.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-15T00:13:53.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853468+0000", 2024-09-15T00:13:53.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3609999999999997e-06 2024-09-15T00:13:53.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853473+0000", 2024-09-15T00:13:53.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-15T00:13:53.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887111+0000", 2024-09-15T00:13:53.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033638376999999997 2024-09-15T00:13:53.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887178+0000", 2024-09-15T00:13:53.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6877000000000004e-05 2024-09-15T00:13:53.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887257+0000", 2024-09-15T00:13:53.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9616999999999993e-05 2024-09-15T00:13:53.372 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.372 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.372 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.372 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97587 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:53.372 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:53.372 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 104.475340155, 2024-09-15T00:13:53.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033973642999999998, 2024-09-15T00:13:53.372 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.372 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.372 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.372 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.372 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97587 2024-09-15T00:13:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:13:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853479+0000", 2024-09-15T00:13:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:13:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853481+0000", 2024-09-15T00:13:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.136e-06 2024-09-15T00:13:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853487+0000", 2024-09-15T00:13:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2439999999999998e-06 2024-09-15T00:13:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853492+0000", 2024-09-15T00:13:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:13:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887284+0000", 2024-09-15T00:13:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033791827000000003 2024-09-15T00:13:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887361+0000", 2024-09-15T00:13:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6983000000000005e-05 2024-09-15T00:13:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887452+0000", 2024-09-15T00:13:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1236999999999994e-05 2024-09-15T00:13:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97588 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:13:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 104.475319998, 2024-09-15T00:13:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034134617999999999, 2024-09-15T00:13:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97588 2024-09-15T00:13:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:13:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853499+0000", 2024-09-15T00:13:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853501+0000", 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.26e-06 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853507+0000", 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2899999999999999e-06 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853512+0000", 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8330000000000001e-06 2024-09-15T00:13:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887495+0000", 2024-09-15T00:13:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033982529999999997 2024-09-15T00:13:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887548+0000", 2024-09-15T00:13:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3445000000000001e-05 2024-09-15T00:13:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887633+0000", 2024-09-15T00:13:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4890999999999997e-05 2024-09-15T00:13:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126866 3.6 3.973c9fa6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.371998052999999, 2024-09-15T00:13:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153301000000003, 2024-09-15T00:13:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126866 2024-09-15T00:13:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956820+0000", 2024-09-15T00:13:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956821+0000", 2024-09-15T00:13:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.91e-07 2024-09-15T00:13:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956822+0000", 2024-09-15T00:13:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.705e-06 2024-09-15T00:13:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956828+0000", 2024-09-15T00:13:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6760000000000004e-06 2024-09-15T00:13:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956833+0000", 2024-09-15T00:13:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5379999999999996e-06 2024-09-15T00:13:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989970+0000", 2024-09-15T00:13:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033137560000000003 2024-09-15T00:13:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.989974+0000", 2024-09-15T00:13:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-15T00:13:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126952 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.371855381000003, 2024-09-15T00:13:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033156066999999997, 2024-09-15T00:13:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126952 2024-09-15T00:13:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:13:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:13:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956965+0000", 2024-09-15T00:13:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3379999999999999e-06 2024-09-15T00:13:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956968+0000", 2024-09-15T00:13:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0419999999999998e-06 2024-09-15T00:13:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956970+0000", 2024-09-15T00:13:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4899999999999999e-06 2024-09-15T00:13:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990113+0000", 2024-09-15T00:13:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033142642 2024-09-15T00:13:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990119+0000", 2024-09-15T00:13:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2979999999999997e-06 2024-09-15T00:13:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126954 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.371809885999994, 2024-09-15T00:13:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033158449, 2024-09-15T00:13:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126954 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:13:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-15T00:13:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957010+0000", 2024-09-15T00:13:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3659999999999999e-06 2024-09-15T00:13:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957013+0000", 2024-09-15T00:13:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1140000000000002e-06 2024-09-15T00:13:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957016+0000", 2024-09-15T00:13:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1209999999999998e-06 2024-09-15T00:13:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990163+0000", 2024-09-15T00:13:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033146451 2024-09-15T00:13:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990167+0000", 2024-09-15T00:13:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0579999999999999e-06 2024-09-15T00:13:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126955 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.371798671999997, 2024-09-15T00:13:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033160617000000003, 2024-09-15T00:13:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126955 2024-09-15T00:13:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:13:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-15T00:13:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957021+0000", 2024-09-15T00:13:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-15T00:13:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957024+0000", 2024-09-15T00:13:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1599999999999998e-06 2024-09-15T00:13:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957027+0000", 2024-09-15T00:13:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000001e-06 2024-09-15T00:13:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990177+0000", 2024-09-15T00:13:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033149741000000003 2024-09-15T00:13:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990180+0000", 2024-09-15T00:13:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4910000000000001e-06 2024-09-15T00:13:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126956 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.371788550999995, 2024-09-15T00:13:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033163633999999997, 2024-09-15T00:13:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126956 2024-09-15T00:13:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:13:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957031+0000", 2024-09-15T00:13:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-06 2024-09-15T00:13:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957034+0000", 2024-09-15T00:13:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9960000000000002e-06 2024-09-15T00:13:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957037+0000", 2024-09-15T00:13:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6419999999999999e-06 2024-09-15T00:13:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990190+0000", 2024-09-15T00:13:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153152999999998 2024-09-15T00:13:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990193+0000", 2024-09-15T00:13:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6069999999999999e-06 2024-09-15T00:13:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126957 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.371778900999999, 2024-09-15T00:13:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033166442999999997, 2024-09-15T00:13:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126957 2024-09-15T00:13:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:13:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957040+0000", 2024-09-15T00:13:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:13:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957041+0000", 2024-09-15T00:13:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0440000000000001e-06 2024-09-15T00:13:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957044+0000", 2024-09-15T00:13:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0170000000000001e-06 2024-09-15T00:13:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957048+0000", 2024-09-15T00:13:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0319999999999997e-06 2024-09-15T00:13:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990202+0000", 2024-09-15T00:13:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033154640999999999 2024-09-15T00:13:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990206+0000", 2024-09-15T00:13:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.507e-06 2024-09-15T00:13:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126958 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.371767671000001, 2024-09-15T00:13:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033167613999999998, 2024-09-15T00:13:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126958 2024-09-15T00:13:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:53.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:13:53.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:13:53.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957053+0000", 2024-09-15T00:13:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.756e-06 2024-09-15T00:13:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957059+0000", 2024-09-15T00:13:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2770000000000002e-06 2024-09-15T00:13:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957063+0000", 2024-09-15T00:13:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2459999999999997e-06 2024-09-15T00:13:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990215+0000", 2024-09-15T00:13:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033151509000000003 2024-09-15T00:13:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990218+0000", 2024-09-15T00:13:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5099999999999999e-06 2024-09-15T00:13:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126959 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.371748667000006, 2024-09-15T00:13:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033161673000000003, 2024-09-15T00:13:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126959 2024-09-15T00:13:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957072+0000", 2024-09-15T00:13:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2239999999999998e-06 2024-09-15T00:13:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957079+0000", 2024-09-15T00:13:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.19e-06 2024-09-15T00:13:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957083+0000", 2024-09-15T00:13:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:13:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990228+0000", 2024-09-15T00:13:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033144818 2024-09-15T00:13:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990231+0000", 2024-09-15T00:13:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:13:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126960 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.371729860000002, 2024-09-15T00:13:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033155166, 2024-09-15T00:13:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:13:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126960 2024-09-15T00:13:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:13:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:13:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957091+0000", 2024-09-15T00:13:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:13:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957097+0000", 2024-09-15T00:13:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6050000000000003e-06 2024-09-15T00:13:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957102+0000", 2024-09-15T00:13:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5539999999999999e-06 2024-09-15T00:13:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990240+0000", 2024-09-15T00:13:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033138289000000001 2024-09-15T00:13:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990244+0000", 2024-09-15T00:13:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4740000000000001e-06 2024-09-15T00:13:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:174575 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138)", 2024-09-15T00:13:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:26.073788+0000", 2024-09-15T00:13:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.255030545, 2024-09-15T00:13:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.95024539600000002, 2024-09-15T00:13:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 174575 2024-09-15T00:13:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073788+0000", 2024-09-15T00:13:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073788+0000", 2024-09-15T00:13:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073789+0000", 2024-09-15T00:13:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3999999999999999e-06 2024-09-15T00:13:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073796+0000", 2024-09-15T00:13:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5529999999999998e-06 2024-09-15T00:13:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073807+0000", 2024-09-15T00:13:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0825000000000001e-05 2024-09-15T00:13:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073812+0000", 2024-09-15T00:13:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6690000000000003e-06 2024-09-15T00:13:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073870+0000", 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7241999999999998e-05 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073878+0000", 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0760000000000003e-06 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:27.023756+0000", 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94987846899999995 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:27.023891+0000", 2024-09-15T00:13:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000134785 2024-09-15T00:13:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:27.024033+0000", 2024-09-15T00:13:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014237700000000001 2024-09-15T00:13:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.426 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:53.428 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:53.428 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:53.426+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:53.528 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:53.527+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:53.572 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:53.572 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:53.572 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:53.581 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:53.629 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:53.627+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:53.686 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:53.686 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:53.686 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:53.694 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:53.730 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:53.729+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:53.831 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:53.829+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:53.840 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:53.840 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:53.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:53.840 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:53.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.840 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:13:53.841 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:53.841 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 125.59503533199999, 2024-09-15T00:13:53.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:13:53.841 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.841 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.841 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:53.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.841 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:13:53.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.841 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:53.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:53.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:13:53.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:13:53.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:13:53.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:13:53.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:13:53.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:13:53.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:13:53.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:53.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:13:53.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:13:53.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:13:53.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:13:53.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:53.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:13:53.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:13:53.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:53.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:13:53.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:13:53.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:53.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:13:53.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:13:53.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:53.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:13:53.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:13:53.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:13:53.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:13:53.846 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.846 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.846 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.847 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58878 3.8 3:12456482:::rbd_data.1139766160da.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:53.847 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:53.847 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 120.548729748, 2024-09-15T00:13:53.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061956199000000003, 2024-09-15T00:13:53.847 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.847 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.847 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:53.847 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.847 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.847 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.847 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58878 2024-09-15T00:13:53.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.848 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:53.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:53.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289151+0000", 2024-09-15T00:13:53.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:13:53.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289473+0000", 2024-09-15T00:13:53.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032183800000000002 2024-09-15T00:13:53.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289487+0000", 2024-09-15T00:13:53.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-05 2024-09-15T00:13:53.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289493+0000", 2024-09-15T00:13:53.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4489999999999996e-06 2024-09-15T00:13:53.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289542+0000", 2024-09-15T00:13:53.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8844999999999997e-05 2024-09-15T00:13:53.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289642+0000", 2024-09-15T00:13:53.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9607999999999999e-05 2024-09-15T00:13:53.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:13:53.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.293908+0000", 2024-09-15T00:13:53.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659350000000002 2024-09-15T00:13:53.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:53.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351022+0000", 2024-09-15T00:13:53.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057114535000000001 2024-09-15T00:13:53.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:53.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351033+0000", 2024-09-15T00:13:53.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0263e-05 2024-09-15T00:13:53.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:53.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351070+0000", 2024-09-15T00:13:53.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7666000000000002e-05 2024-09-15T00:13:53.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351106+0000", 2024-09-15T00:13:53.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999997e-05 2024-09-15T00:13:53.852 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.852 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.852 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.853 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58879 3.7 3:e169894d:::rbd_data.1139766160da.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:53.853 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:53.853 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 120.54637826, 2024-09-15T00:13:53.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043716294000000003, 2024-09-15T00:13:53.853 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.853 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.853 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:53.853 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.853 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.853 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.853 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58879 2024-09-15T00:13:53.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.854 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:53.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:53.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291502+0000", 2024-09-15T00:13:53.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-15T00:13:53.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291664+0000", 2024-09-15T00:13:53.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016216599999999999 2024-09-15T00:13:53.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291674+0000", 2024-09-15T00:13:53.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0276e-05 2024-09-15T00:13:53.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291680+0000", 2024-09-15T00:13:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2580000000000003e-06 2024-09-15T00:13:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291732+0000", 2024-09-15T00:13:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2114999999999998e-05 2024-09-15T00:13:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291866+0000", 2024-09-15T00:13:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013416900000000001 2024-09-15T00:13:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:13:53.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.296426+0000", 2024-09-15T00:13:53.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045595779999999999 2024-09-15T00:13:53.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:53.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335075+0000", 2024-09-15T00:13:53.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038649347000000001 2024-09-15T00:13:53.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:53.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335097+0000", 2024-09-15T00:13:53.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2565000000000002e-05 2024-09-15T00:13:53.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:53.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335161+0000", 2024-09-15T00:13:53.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3928000000000002e-05 2024-09-15T00:13:53.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335217+0000", 2024-09-15T00:13:53.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6042000000000003e-05 2024-09-15T00:13:53.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.858 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.858 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58880 3.8 3:13d91cfd:::rbd_data.1139766160da.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 120.542986153, 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056331463999999998, 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58880 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:53.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:53.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294896+0000", 2024-09-15T00:13:53.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.356e-06 2024-09-15T00:13:53.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295288+0000", 2024-09-15T00:13:53.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039223200000000001 2024-09-15T00:13:53.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295304+0000", 2024-09-15T00:13:53.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5937e-05 2024-09-15T00:13:53.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295312+0000", 2024-09-15T00:13:53.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1489999999999994e-06 2024-09-15T00:13:53.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333162+0000", 2024-09-15T00:13:53.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037849840000000003 2024-09-15T00:13:53.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333269+0000", 2024-09-15T00:13:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010749800000000001 2024-09-15T00:13:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:13:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.336198+0000", 2024-09-15T00:13:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029284739999999999 2024-09-15T00:13:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351137+0000", 2024-09-15T00:13:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014939103 2024-09-15T00:13:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:53.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351150+0000", 2024-09-15T00:13:53.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3542e-05 2024-09-15T00:13:53.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:53.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351191+0000", 2024-09-15T00:13:53.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0791000000000003e-05 2024-09-15T00:13:53.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351225+0000", 2024-09-15T00:13:53.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3541999999999998e-05 2024-09-15T00:13:53.864 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.864 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.865 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.865 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58878 3.8 e56/23 ondisk, result = 0)", 2024-09-15T00:13:53.865 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:53.865 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 120.53645637, 2024-09-15T00:13:53.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049689096000000002, 2024-09-15T00:13:53.865 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.865 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.865 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.865 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:53.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:53.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301425+0000", 2024-09-15T00:13:53.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.782e-06 2024-09-15T00:13:53.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301433+0000", 2024-09-15T00:13:53.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689999999999994e-06 2024-09-15T00:13:53.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301449+0000", 2024-09-15T00:13:53.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5767e-05 2024-09-15T00:13:53.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301458+0000", 2024-09-15T00:13:53.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:13:53.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351009+0000", 2024-09-15T00:13:53.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049551523 2024-09-15T00:13:53.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351017+0000", 2024-09-15T00:13:53.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8359999999999992e-06 2024-09-15T00:13:53.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351112+0000", 2024-09-15T00:13:53.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5173999999999994e-05 2024-09-15T00:13:53.868 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.868 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.868 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.869 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127004 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 6881280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.869 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:53.869 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.869550906000001, 2024-09-15T00:13:53.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735720000000003, 2024-09-15T00:13:53.869 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.869 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.869 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.869 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.869 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.869 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.869 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127004 2024-09-15T00:13:53.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.870 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:53.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:53.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:53.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:13:53.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968331+0000", 2024-09-15T00:13:53.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.849e-06 2024-09-15T00:13:53.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968334+0000", 2024-09-15T00:13:53.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7689999999999998e-06 2024-09-15T00:13:53.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968338+0000", 2024-09-15T00:13:53.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2490000000000001e-06 2024-09-15T00:13:53.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008987+0000", 2024-09-15T00:13:53.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040649117999999998 2024-09-15T00:13:53.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009027+0000", 2024-09-15T00:13:53.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0357e-05 2024-09-15T00:13:53.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009064+0000", 2024-09-15T00:13:53.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7036000000000003e-05 2024-09-15T00:13:53.873 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.873 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.873 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.873 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127005 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7004160~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.873 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:53.874 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.869538353999999, 2024-09-15T00:13:53.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040824358999999998, 2024-09-15T00:13:53.874 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.874 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.874 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.874 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.874 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.874 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.874 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127005 2024-09-15T00:13:53.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.874 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:53.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:53.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:53.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-15T00:13:53.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968343+0000", 2024-09-15T00:13:53.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3430000000000001e-06 2024-09-15T00:13:53.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968349+0000", 2024-09-15T00:13:53.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3080000000000004e-06 2024-09-15T00:13:53.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968355+0000", 2024-09-15T00:13:53.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9499999999999998e-06 2024-09-15T00:13:53.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009091+0000", 2024-09-15T00:13:53.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735645000000001 2024-09-15T00:13:53.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009130+0000", 2024-09-15T00:13:53.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8951e-05 2024-09-15T00:13:53.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009165+0000", 2024-09-15T00:13:53.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5926000000000001e-05 2024-09-15T00:13:53.877 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.878 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.878 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.878 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127006 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7127040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.878 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:53.878 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.869521296000002, 2024-09-15T00:13:53.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040912783000000001, 2024-09-15T00:13:53.878 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.878 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.878 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.878 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.878 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.879 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.879 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127006 2024-09-15T00:13:53.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.879 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:53.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:53.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:53.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:53.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968359+0000", 2024-09-15T00:13:53.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0419999999999999e-06 2024-09-15T00:13:53.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968363+0000", 2024-09-15T00:13:53.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5999999999999998e-06 2024-09-15T00:13:53.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968366+0000", 2024-09-15T00:13:53.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5179999999999999e-06 2024-09-15T00:13:53.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009196+0000", 2024-09-15T00:13:53.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040830313 2024-09-15T00:13:53.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009234+0000", 2024-09-15T00:13:53.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8562000000000001e-05 2024-09-15T00:13:53.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009271+0000", 2024-09-15T00:13:53.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6538000000000002e-05 2024-09-15T00:13:53.882 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.882 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.882 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.882 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127007 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7249920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.882 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:53.883 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.869511097, 2024-09-15T00:13:53.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041019218000000003, 2024-09-15T00:13:53.883 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.883 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.883 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.883 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.883 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.883 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.883 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127007 2024-09-15T00:13:53.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.883 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:53.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:53.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968369+0000", 2024-09-15T00:13:53.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-15T00:13:53.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968370+0000", 2024-09-15T00:13:53.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.012e-06 2024-09-15T00:13:53.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968374+0000", 2024-09-15T00:13:53.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3839999999999999e-06 2024-09-15T00:13:53.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968376+0000", 2024-09-15T00:13:53.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5339999999999998e-06 2024-09-15T00:13:53.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009292+0000", 2024-09-15T00:13:53.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040915092 2024-09-15T00:13:53.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009333+0000", 2024-09-15T00:13:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1105e-05 2024-09-15T00:13:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009388+0000", 2024-09-15T00:13:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4868999999999999e-05 2024-09-15T00:13:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127008 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7364608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.869499976, 2024-09-15T00:13:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041115290999999998, 2024-09-15T00:13:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127008 2024-09-15T00:13:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968380+0000", 2024-09-15T00:13:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3999999999999998e-07 2024-09-15T00:13:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968381+0000", 2024-09-15T00:13:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8700000000000004e-07 2024-09-15T00:13:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968385+0000", 2024-09-15T00:13:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9260000000000002e-06 2024-09-15T00:13:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968389+0000", 2024-09-15T00:13:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.685e-06 2024-09-15T00:13:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009419+0000", 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041029852999999998 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009458+0000", 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8538000000000003e-05 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009495+0000", 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7061999999999999e-05 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127009 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7487488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.869465859000002, 2024-09-15T00:13:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041233063, 2024-09-15T00:13:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127009 2024-09-15T00:13:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7199999999999999e-07 2024-09-15T00:13:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968417+0000", 2024-09-15T00:13:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3810000000000002e-06 2024-09-15T00:13:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968423+0000", 2024-09-15T00:13:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8820000000000003e-06 2024-09-15T00:13:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968433+0000", 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0005e-05 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009527+0000", 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041093750999999998 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009601+0000", 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3979000000000004e-05 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009647+0000", 2024-09-15T00:13:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5493e-05 2024-09-15T00:13:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127010 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7610368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.869436855000004, 2024-09-15T00:13:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041325924999999999, 2024-09-15T00:13:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127010 2024-09-15T00:13:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:13:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968445+0000", 2024-09-15T00:13:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:13:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968452+0000", 2024-09-15T00:13:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-15T00:13:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968456+0000", 2024-09-15T00:13:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5210000000000004e-06 2024-09-15T00:13:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009664+0000", 2024-09-15T00:13:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041207404000000003 2024-09-15T00:13:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009723+0000", 2024-09-15T00:13:53.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9036000000000002e-05 2024-09-15T00:13:53.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009768+0000", 2024-09-15T00:13:53.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5578000000000003e-05 2024-09-15T00:13:53.900 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.900 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.900 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.901 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127011 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7733248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.901 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:53.901 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.869418620000005, 2024-09-15T00:13:53.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041482441000000002, 2024-09-15T00:13:53.901 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.901 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.901 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.901 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.901 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.901 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.901 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127011 2024-09-15T00:13:53.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.902 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:53.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:53.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:53.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-15T00:13:53.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968466+0000", 2024-09-15T00:13:53.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:13:53.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968470+0000", 2024-09-15T00:13:53.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6469999999999997e-06 2024-09-15T00:13:53.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968473+0000", 2024-09-15T00:13:53.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.977e-06 2024-09-15T00:13:53.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009800+0000", 2024-09-15T00:13:53.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041326918999999997 2024-09-15T00:13:53.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009869+0000", 2024-09-15T00:13:53.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8935000000000003e-05 2024-09-15T00:13:53.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009943+0000", 2024-09-15T00:13:53.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4188000000000002e-05 2024-09-15T00:13:53.905 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.905 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.905 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.905 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127012 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7856128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.906 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:53.906 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.869403117999994, 2024-09-15T00:13:53.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041640756000000001, 2024-09-15T00:13:53.906 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.906 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.906 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.906 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127012 2024-09-15T00:13:53.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.906 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:53.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:53.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968477+0000", 2024-09-15T00:13:53.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:53.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968482+0000", 2024-09-15T00:13:53.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6180000000000003e-06 2024-09-15T00:13:53.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968486+0000", 2024-09-15T00:13:53.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.501e-06 2024-09-15T00:13:53.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968489+0000", 2024-09-15T00:13:53.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1530000000000001e-06 2024-09-15T00:13:53.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009983+0000", 2024-09-15T00:13:53.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041493980999999999 2024-09-15T00:13:53.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010051+0000", 2024-09-15T00:13:53.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8039000000000004e-05 2024-09-15T00:13:53.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010117+0000", 2024-09-15T00:13:53.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6254000000000006e-05 2024-09-15T00:13:53.910 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.910 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.910 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.910 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127013 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7979008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.869387298000007, 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041763863999999998, 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127013 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:53.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:53.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969767+0000", 2024-09-15T00:13:53.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001275006 2024-09-15T00:13:53.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969772+0000", 2024-09-15T00:13:53.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4769999999999997e-06 2024-09-15T00:13:53.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969781+0000", 2024-09-15T00:13:53.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1959999999999997e-06 2024-09-15T00:13:53.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969788+0000", 2024-09-15T00:13:53.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6359999999999994e-06 2024-09-15T00:13:53.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010162+0000", 2024-09-15T00:13:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040373291999999998 2024-09-15T00:13:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010212+0000", 2024-09-15T00:13:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0543e-05 2024-09-15T00:13:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010256+0000", 2024-09-15T00:13:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3714000000000002e-05 2024-09-15T00:13:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.915 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127015 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8224768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.915 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:53.915 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.868048361999996, 2024-09-15T00:13:53.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040710186000000002, 2024-09-15T00:13:53.915 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.915 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.915 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.915 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.915 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.915 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.915 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127015 2024-09-15T00:13:53.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.915 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:53.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:53.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969832+0000", 2024-09-15T00:13:53.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:13:53.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969834+0000", 2024-09-15T00:13:53.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2919999999999998e-06 2024-09-15T00:13:53.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969843+0000", 2024-09-15T00:13:53.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0480000000000004e-06 2024-09-15T00:13:53.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969848+0000", 2024-09-15T00:13:53.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-15T00:13:53.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010457+0000", 2024-09-15T00:13:53.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040609054999999998 2024-09-15T00:13:53.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010501+0000", 2024-09-15T00:13:53.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3396000000000003e-05 2024-09-15T00:13:53.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010541+0000", 2024-09-15T00:13:53.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0599000000000002e-05 2024-09-15T00:13:53.919 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.919 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.919 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.919 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127016 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:53.919 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:53.919 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.867997736000007, 2024-09-15T00:13:53.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040766216000000001, 2024-09-15T00:13:53.919 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.920 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.920 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.920 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127016 2024-09-15T00:13:53.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.920 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:53.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:53.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:53.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:13:53.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969886+0000", 2024-09-15T00:13:53.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8449999999999996e-06 2024-09-15T00:13:53.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969897+0000", 2024-09-15T00:13:53.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-15T00:13:53.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969904+0000", 2024-09-15T00:13:53.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1169999999999998e-06 2024-09-15T00:13:53.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010561+0000", 2024-09-15T00:13:53.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040657470000000001 2024-09-15T00:13:53.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010607+0000", 2024-09-15T00:13:53.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6108e-05 2024-09-15T00:13:53.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010648+0000", 2024-09-15T00:13:53.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0479000000000003e-05 2024-09-15T00:13:53.923 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.923 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.923 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.923 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127694 3.9 3:9390b205:::rbd_data.11396481e1ec.00000000000000df:head [set-alloc-hint object_size 1048576 write_size 1048576,write 819200~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T00:13:53.924 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:53.924 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.613278618999999, 2024-09-15T00:13:53.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90376419399999997, 2024-09-15T00:13:53.924 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.924 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.924 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:53.924 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.924 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.924 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.924 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127694 2024-09-15T00:13:53.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.924 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:53.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:53.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224602+0000", 2024-09-15T00:13:53.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5899999999999995e-07 2024-09-15T00:13:53.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224610+0000", 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9130000000000003e-06 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224620+0000", 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0426e-05 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224627+0000", 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4390000000000003e-06 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224675+0000", 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7636000000000003e-05 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224681+0000", 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1460000000000001e-06 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126388+0000", 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90170724999999996 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126526+0000", 2024-09-15T00:13:53.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013730599999999999 2024-09-15T00:13:53.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:53.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126684+0000", 2024-09-15T00:13:53.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000157935 2024-09-15T00:13:53.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:53.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.127932+0000", 2024-09-15T00:13:53.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012479240000000001 2024-09-15T00:13:53.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:53.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128269+0000", 2024-09-15T00:13:53.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033761099999999998 2024-09-15T00:13:53.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:53.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128324+0000", 2024-09-15T00:13:53.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5244999999999997e-05 2024-09-15T00:13:53.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128365+0000", 2024-09-15T00:13:53.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0503999999999997e-05 2024-09-15T00:13:53.929 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.930 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.930 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.930 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:134707 3.15 3:af1fbdb5:::rbd_data.113933a4f79a.000000000000001c:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T00:13:53.930 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:53.930 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.145311504999995, 2024-09-15T00:13:53.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057328626000000001, 2024-09-15T00:13:53.930 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.930 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.930 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:53.930 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:53.930 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:53.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:53.931 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 134707 2024-09-15T00:13:53.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.931 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:53.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:53.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692569+0000", 2024-09-15T00:13:53.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.339e-06 2024-09-15T00:13:53.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692574+0000", 2024-09-15T00:13:53.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-15T00:13:53.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692584+0000", 2024-09-15T00:13:53.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0363000000000001e-05 2024-09-15T00:13:53.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692590+0000", 2024-09-15T00:13:53.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.818e-06 2024-09-15T00:13:53.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692633+0000", 2024-09-15T00:13:53.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.265e-05 2024-09-15T00:13:53.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692718+0000", 2024-09-15T00:13:53.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5279999999999997e-05 2024-09-15T00:13:53.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:53.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.695940+0000", 2024-09-15T00:13:53.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003222052 2024-09-15T00:13:53.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:53.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749714+0000", 2024-09-15T00:13:53.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.053774126999999998 2024-09-15T00:13:53.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:53.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749788+0000", 2024-09-15T00:13:53.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3693000000000001e-05 2024-09-15T00:13:53.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:53.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749847+0000", 2024-09-15T00:13:53.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175999999999997e-05 2024-09-15T00:13:53.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749897+0000", 2024-09-15T00:13:53.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9311999999999998e-05 2024-09-15T00:13:53.936 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.936 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.936 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.936 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:134707 3.15 e86/55 ondisk, result = 0)", 2024-09-15T00:13:53.936 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:53.936 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.136750156000005, 2024-09-15T00:13:53.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048596066, 2024-09-15T00:13:53.936 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:53.936 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:53.937 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:53.937 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:53.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:53.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:53.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:53.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:53.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:53.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701131+0000", 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3549999999999999e-06 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701139+0000", 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1650000000000006e-06 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701156+0000", 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6730000000000001e-05 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701167+0000", 2024-09-15T00:13:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1504e-05 2024-09-15T00:13:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749629+0000", 2024-09-15T00:13:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048461952000000003 2024-09-15T00:13:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749640+0000", 2024-09-15T00:13:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0874e-05 2024-09-15T00:13:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:53.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749725+0000", 2024-09-15T00:13:53.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5486000000000001e-05 2024-09-15T00:13:53.940 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.940 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.940 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.940 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:53.940 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:53.940 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:53.941 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:53.942 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:53.939+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:54.026 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is set 2024-09-15T00:13:54.042 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:54.040+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:54.072 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:54.072 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:54.072 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:54.072 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:54.083 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:54.142 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:54.140+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:54.226 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:54.226 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:54.226 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:54.226 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:54.226 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:54.235 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:54.278 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:54.277+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:54.340 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:54.340 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:54.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:54.340 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:54.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.340 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:54.340 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:54.340 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 148.76510384599999, 2024-09-15T00:13:54.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:13:54.340 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.340 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.340 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:54.340 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.341 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.341 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.341 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:13:54.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.341 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:54.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:54.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:13:54.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:13:54.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:13:54.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:13:54.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:13:54.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:13:54.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:54.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:13:54.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:13:54.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:54.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:13:54.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:13:54.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:54.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:13:54.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:13:54.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:54.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:13:54.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:13:54.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:13:54.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:13:54.345 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.345 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 148.765069154, 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:54.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:54.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:54.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:13:54.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:54.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:13:54.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.724395466, 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.351 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:13:54.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.352 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:54.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:54.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:54.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:13:54.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:13:54.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:13:54.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:13:54.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:13:54.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:13:54.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:13:54.355 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.355 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.355 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.355 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.355 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:54.356 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.724378935, 2024-09-15T00:13:54.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:13:54.356 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.356 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.356 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.356 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.356 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.356 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.356 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:13:54.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.357 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:54.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:54.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:13:54.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:54.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:13:54.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:13:54.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:13:54.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:13:54.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:13:54.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:13:54.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:13:54.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:13:54.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:13:54.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:13:54.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:13:54.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:13:54.360 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.360 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.360 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.360 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.361 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:54.361 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.72436215600001, 2024-09-15T00:13:54.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:13:54.361 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.361 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.361 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.361 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:13:54.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.361 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:54.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:54.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:54.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:13:54.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:13:54.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:13:54.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:13:54.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:13:54.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:13:54.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:13:54.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:13:54.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:13:54.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:13:54.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:13:54.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:13:54.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:13:54.365 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.365 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.365 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.365 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.365 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:54.365 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.72434442599999, 2024-09-15T00:13:54.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:13:54.365 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.365 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.366 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.366 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.366 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.366 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.366 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:13:54.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.366 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:54.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:54.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:54.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:13:54.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:13:54.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:13:54.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:13:54.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:13:54.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:13:54.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:13:54.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:13:54.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:13:54.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:13:54.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:13:54.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:13:54.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:13:54.369 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.369 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.369 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.369 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.369 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:54.370 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.724327262, 2024-09-15T00:13:54.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:13:54.370 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.370 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.370 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.370 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:13:54.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.370 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:13:54.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:13:54.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:13:54.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:13:54.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:13:54.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:13:54.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:13:54.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:13:54.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:13:54.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:13:54.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:13:54.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:13:54.373 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.373 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.373 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.374 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.374 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:54.374 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.72430558600001, 2024-09-15T00:13:54.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:13:54.374 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.374 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.374 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.374 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:13:54.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.375 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:54.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:54.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:54.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:13:54.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:13:54.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:13:54.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:13:54.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:13:54.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:13:54.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:13:54.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:13:54.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:13:54.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:13:54.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:13:54.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:13:54.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:13:54.378 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.378 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.378 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.378 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.378 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:54.378 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.72425946199999, 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:54.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:54.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:13:54.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:13:54.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:13:54.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:13:54.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:13:54.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:13:54.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:13:54.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:13:54.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:13:54.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:13:54.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:13:54.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:13:54.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:13:54.382 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.382 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.382 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.383 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.383 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:54.383 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.724229705, 2024-09-15T00:13:54.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:13:54.383 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.383 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.383 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.383 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:13:54.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.383 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:54.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:54.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:13:54.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:13:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:13:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:13:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:13:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:13:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:13:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:13:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:13:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:13:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:13:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:13:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:13:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.387 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.724203386, 2024-09-15T00:13:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:13:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:13:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:54.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:13:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:13:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:13:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:13:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:13:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:13:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:13:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:13:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:13:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:13:54.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:13:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:13:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.72416752399999, 2024-09-15T00:13:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:13:54.392 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:13:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:13:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:13:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:13:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.724145806, 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:13:54.396 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:13:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:13:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:13:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:13:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:13:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:13:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:13:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:13:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:13:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:13:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:13:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:13:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:13:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:13:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:54.400 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.723949339, 2024-09-15T00:13:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:13:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:13:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:13:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:13:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:13:54.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:13:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:13:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:13:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:13:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:13:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:13:54.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:13:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:13:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:13:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:13:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.72392350000001, 2024-09-15T00:13:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:13:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:13:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:13:54.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:13:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:13:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:13:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:13:54.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:13:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:13:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:13:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:13:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:13:54.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:13:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:13:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:13:54.412 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.72386185400001, 2024-09-15T00:13:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:13:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.413 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:13:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:13:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:13:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:13:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:13:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:13:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:13:54.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:13:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:13:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:13:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:13:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:13:54.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:13:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:13:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:13:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.72382252599999, 2024-09-15T00:13:54.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:13:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:13:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:13:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:13:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:13:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:13:54.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:13:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:13:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:13:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:13:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:13:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:13:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:13:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:13:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:13:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.421 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.72209712599999, 2024-09-15T00:13:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:13:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.422 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:13:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:54.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:13:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:13:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:13:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:13:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:13:54.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:13:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:13:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:13:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:13:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:13:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:13:54.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:13:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:13:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:13:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.426 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.72207496499999, 2024-09-15T00:13:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:13:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.427 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:13:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:54.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:13:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:13:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:13:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:13:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:13:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:13:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:13:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:13:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:13:54.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:13:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:13:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:13:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:13:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.431 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:145034 3.6 3:618673df:::rbd_object_map.1139646a7d4e.000000000000002b:head [writefull 0~48] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T00:13:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.319821360999995, 2024-09-15T00:13:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0081079740000001, 2024-09-15T00:13:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 145034 2024-09-15T00:13:54.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017198+0000", 2024-09-15T00:13:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0100000000000004e-07 2024-09-15T00:13:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017202+0000", 2024-09-15T00:13:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:13:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017212+0000", 2024-09-15T00:13:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7189999999999996e-06 2024-09-15T00:13:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017219+0000", 2024-09-15T00:13:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7839999999999998e-06 2024-09-15T00:13:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017268+0000", 2024-09-15T00:13:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:13:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:13:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017277+0000", 2024-09-15T00:13:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1899999999999995e-06 2024-09-15T00:13:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019139+0000", 2024-09-15T00:13:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0018626129999999 2024-09-15T00:13:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019236+0000", 2024-09-15T00:13:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6513000000000002e-05 2024-09-15T00:13:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019318+0000", 2024-09-15T00:13:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1848000000000006e-05 2024-09-15T00:13:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.023151+0000", 2024-09-15T00:13:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038332169999999999 2024-09-15T00:13:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025217+0000", 2024-09-15T00:13:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020657610000000002 2024-09-15T00:13:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025267+0000", 2024-09-15T00:13:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0198000000000003e-05 2024-09-15T00:13:54.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025305+0000", 2024-09-15T00:13:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8065999999999998e-05 2024-09-15T00:13:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.438 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.438 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:54.440 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:54.441 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:54.439+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:54.540 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:54.539+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:54.583 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:54.583 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:54.583 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:54.583 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:54.591 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:54.641 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:54.640+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:54.694 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:13:54.695 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:54.695 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:54.695 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:54.695 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:54.696 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:54.703 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:54.741 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:54.740+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:54.808 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:54.808 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:54.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:54.808 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:54.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.808 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.808 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:54.808 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 130.19605912599999, 2024-09-15T00:13:54.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:54.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:54.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:13:54.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:13:54.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:13:54.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:13:54.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:13:54.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:13:54.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:13:54.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:13:54.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:13:54.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:13:54.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:13:54.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:13:54.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:13:54.812 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.812 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.812 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.812 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.812 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:54.813 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 130.196037897, 2024-09-15T00:13:54.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:13:54.813 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.813 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.813 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.813 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.813 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.813 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.813 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:13:54.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.813 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:13:54.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:54.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:13:54.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:13:54.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:13:54.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:13:54.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:13:54.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:13:54.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:13:54.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:13:54.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:13:54.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:13:54.816 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.816 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.817 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.820 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.820 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:54.820 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 130.19601235600001, 2024-09-15T00:13:54.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:13:54.820 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.820 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.820 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.820 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.820 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.820 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.821 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:13:54.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.821 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:54.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:54.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:54.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:54.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:13:54.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:13:54.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:13:54.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:13:54.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:13:54.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:13:54.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:13:54.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:13:54.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:13:54.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:13:54.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:13:54.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:13:54.824 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 130.195989881, 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:13:54.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.826 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:54.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:54.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:13:54.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:13:54.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:13:54.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:13:54.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:13:54.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:13:54.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:13:54.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:13:54.828 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 130.19596937099999, 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:13:54.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:13:54.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:13:54.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:13:54.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:13:54.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:13:54.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:13:54.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:13:54.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:13:54.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:13:54.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:13:54.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:13:54.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:13:54.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:13:54.833 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.833 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.833 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.833 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.833 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:54.833 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 130.19594905599999, 2024-09-15T00:13:54.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:13:54.834 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.834 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.834 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.834 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.834 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.834 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.834 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:13:54.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.834 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:54.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:13:54.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:13:54.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:13:54.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:13:54.837 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.837 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.837 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.837 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.837 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:54.837 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 130.19592428000001, 2024-09-15T00:13:54.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:13:54.838 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.838 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.838 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.838 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:13:54.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.838 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:54.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:54.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:54.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:13:54.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:13:54.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:13:54.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:13:54.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:13:54.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:13:54.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:13:54.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:13:54.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:13:54.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:13:54.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:13:54.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:13:54.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:13:54.841 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.841 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.841 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 130.01658289299999, 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:54.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:54.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:13:54.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:13:54.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:13:54.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:13:54.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:13:54.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:13:54.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:13:54.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:13:54.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:13:54.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:13:54.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:13:54.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:13:54.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:54.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:13:54.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:13:54.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:13:54.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:13:54.847 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.847 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.847 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.847 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:13:54.847 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:54.847 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 129.999629441, 2024-09-15T00:13:54.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:13:54.848 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.848 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.848 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.848 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:54.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:54.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:13:54.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:13:54.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:13:54.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:13:54.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:13:54.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:13:54.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:13:54.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:13:54.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:13:54.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:13:54.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:13:54.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:13:54.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:13:54.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:13:54.852 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.852 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.852 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.852 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:13:54.852 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:54.852 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.705198534, 2024-09-15T00:13:54.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:13:54.852 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.852 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.852 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:54.853 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.853 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.853 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.853 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:13:54.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.853 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:54.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:13:54.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:13:54.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:13:54.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:13:54.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:13:54.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:13:54.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:13:54.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:13:54.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:54.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:13:54.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:13:54.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:54.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:13:54.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:13:54.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:54.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:13:54.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:13:54.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:54.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:13:54.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:13:54.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:13:54.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:13:54.857 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.858 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.858 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.858 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:13:54.858 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:54.858 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 116.69274272, 2024-09-15T00:13:54.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:13:54.858 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.858 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.858 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.858 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:13:54.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:13:54.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:13:54.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:13:54.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:13:54.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:13:54.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:13:54.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.861 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:146935 3.16 3:6c79e092:::rbd_data.11394501364.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:54.862 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:54.862 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 70.051963201999996, 2024-09-15T00:13:54.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.058987488999999997, 2024-09-15T00:13:54.862 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.862 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.862 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:54.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.862 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 146935 2024-09-15T00:13:54.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.862 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:54.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:54.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753644+0000", 2024-09-15T00:13:54.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6199999999999997e-07 2024-09-15T00:13:54.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753648+0000", 2024-09-15T00:13:54.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-15T00:13:54.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753659+0000", 2024-09-15T00:13:54.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0703000000000001e-05 2024-09-15T00:13:54.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753664+0000", 2024-09-15T00:13:54.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.04e-06 2024-09-15T00:13:54.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753709+0000", 2024-09-15T00:13:54.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4728000000000003e-05 2024-09-15T00:13:54.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753756+0000", 2024-09-15T00:13:54.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7808e-05 2024-09-15T00:13:54.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:54.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.757051+0000", 2024-09-15T00:13:54.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294249 2024-09-15T00:13:54.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812553+0000", 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055501807 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812564+0000", 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1467999999999999e-05 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812598+0000", 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3553000000000002e-05 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812630+0000", 2024-09-15T00:13:54.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2895000000000003e-05 2024-09-15T00:13:54.867 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.867 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.867 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.867 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:146935 3.16 e98/95 ondisk, result = 0)", 2024-09-15T00:13:54.867 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:54.867 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 70.041160505999997, 2024-09-15T00:13:54.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048111830000000001, 2024-09-15T00:13:54.867 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.867 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.867 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.867 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:54.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:54.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764447+0000", 2024-09-15T00:13:54.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0500000000000002e-07 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764450+0000", 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4400000000000001e-06 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764462+0000", 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1603000000000001e-05 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764469+0000", 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5089999999999996e-06 2024-09-15T00:13:54.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812541+0000", 2024-09-15T00:13:54.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048071585 2024-09-15T00:13:54.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812548+0000", 2024-09-15T00:13:54.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9979999999999997e-06 2024-09-15T00:13:54.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812557+0000", 2024-09-15T00:13:54.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7899999999999994e-06 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:152505 3.17 3:ea3f268c:::rbd_data.113958cb0e55.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.139054544000004, 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.060817959999999997, 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 152505 2024-09-15T00:13:54.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.872 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:54.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:54.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666553+0000", 2024-09-15T00:13:54.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-15T00:13:54.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666562+0000", 2024-09-15T00:13:54.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0909999999999999e-06 2024-09-15T00:13:54.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666572+0000", 2024-09-15T00:13:54.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0214000000000001e-05 2024-09-15T00:13:54.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666577+0000", 2024-09-15T00:13:54.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3800000000000002e-06 2024-09-15T00:13:54.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666617+0000", 2024-09-15T00:13:54.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9304000000000002e-05 2024-09-15T00:13:54.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666672+0000", 2024-09-15T00:13:54.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5206e-05 2024-09-15T00:13:54.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.670821+0000", 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041493169999999996 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727290+0000", 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.056468580999999997 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727302+0000", 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2605e-05 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:54.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727337+0000", 2024-09-15T00:13:54.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4783999999999998e-05 2024-09-15T00:13:54.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727370+0000", 2024-09-15T00:13:54.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2366000000000001e-05 2024-09-15T00:13:54.876 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.876 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.876 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.876 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:152505 3.17 e103/68 ondisk, result = 0)", 2024-09-15T00:13:54.876 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:54.877 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 65.126976107999994, 2024-09-15T00:13:54.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048665938999999998, 2024-09-15T00:13:54.877 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.877 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.877 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.878 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:54.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:54.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678631+0000", 2024-09-15T00:13:54.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5099999999999998e-07 2024-09-15T00:13:54.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678634+0000", 2024-09-15T00:13:54.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.478e-06 2024-09-15T00:13:54.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678647+0000", 2024-09-15T00:13:54.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2085e-05 2024-09-15T00:13:54.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678653+0000", 2024-09-15T00:13:54.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2099999999999998e-06 2024-09-15T00:13:54.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727277+0000", 2024-09-15T00:13:54.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048624405000000002 2024-09-15T00:13:54.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727285+0000", 2024-09-15T00:13:54.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2060000000000007e-06 2024-09-15T00:13:54.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727296+0000", 2024-09-15T00:13:54.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0604e-05 2024-09-15T00:13:54.941 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.941 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.941 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.941 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153959 3.c 3:326b8454:::rbd_data.113958cb0e55.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:54.941 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:54.941 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.715748161999997, 2024-09-15T00:13:54.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.068102041000000002, 2024-09-15T00:13:54.942 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.942 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.942 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:54.942 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.942 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.942 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.942 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153959 2024-09-15T00:13:54.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.942 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:54.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:54.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089859+0000", 2024-09-15T00:13:54.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1800000000000004e-07 2024-09-15T00:13:54.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089864+0000", 2024-09-15T00:13:54.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2079999999999999e-06 2024-09-15T00:13:54.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089880+0000", 2024-09-15T00:13:54.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5695000000000001e-05 2024-09-15T00:13:54.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089889+0000", 2024-09-15T00:13:54.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.935e-06 2024-09-15T00:13:54.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089942+0000", 2024-09-15T00:13:54.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3219e-05 2024-09-15T00:13:54.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.090066+0000", 2024-09-15T00:13:54.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012385099999999999 2024-09-15T00:13:54.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:54.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.095262+0000", 2024-09-15T00:13:54.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051963950000000004 2024-09-15T00:13:54.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:54.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157865+0000", 2024-09-15T00:13:54.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.062602865999999993 2024-09-15T00:13:54.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:54.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157891+0000", 2024-09-15T00:13:54.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5778e-05 2024-09-15T00:13:54.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:54.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157924+0000", 2024-09-15T00:13:54.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3133000000000003e-05 2024-09-15T00:13:54.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157960+0000", 2024-09-15T00:13:54.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6043e-05 2024-09-15T00:13:54.947 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.947 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.947 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.947 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:153959 3.c e103/23 ondisk, result = 0)", 2024-09-15T00:13:54.947 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:54.947 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.703092503999997, 2024-09-15T00:13:54.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055370211000000003, 2024-09-15T00:13:54.947 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.948 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.948 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.948 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:54.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:54.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102515+0000", 2024-09-15T00:13:54.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.206e-06 2024-09-15T00:13:54.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102520+0000", 2024-09-15T00:13:54.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1209999999999998e-06 2024-09-15T00:13:54.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102537+0000", 2024-09-15T00:13:54.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:54.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102546+0000", 2024-09-15T00:13:54.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2329999999999995e-06 2024-09-15T00:13:54.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157847+0000", 2024-09-15T00:13:54.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055300798999999998 2024-09-15T00:13:54.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157857+0000", 2024-09-15T00:13:54.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.046e-05 2024-09-15T00:13:54.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157884+0000", 2024-09-15T00:13:54.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.671e-05 2024-09-15T00:13:54.951 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.951 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.951 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.951 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163346 3.1d 3:baa43c12:::rbd_data.11395d771336.0000000000000065:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:13:54.951 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:54.951 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.034477930999998, 2024-09-15T00:13:54.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047558544000000001, 2024-09-15T00:13:54.951 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.952 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.952 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:54.952 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.952 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.952 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.952 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163346 2024-09-15T00:13:54.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.952 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:54.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:54.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771130+0000", 2024-09-15T00:13:54.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5939999999999999e-06 2024-09-15T00:13:54.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771141+0000", 2024-09-15T00:13:54.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1001e-05 2024-09-15T00:13:54.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771154+0000", 2024-09-15T00:13:54.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2921e-05 2024-09-15T00:13:54.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771160+0000", 2024-09-15T00:13:54.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3849999999999999e-06 2024-09-15T00:13:54.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771207+0000", 2024-09-15T00:13:54.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6888999999999998e-05 2024-09-15T00:13:54.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771306+0000", 2024-09-15T00:13:54.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9259000000000006e-05 2024-09-15T00:13:54.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:54.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.774692+0000", 2024-09-15T00:13:54.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033854850000000001 2024-09-15T00:13:54.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:54.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818600+0000", 2024-09-15T00:13:54.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043908006999999999 2024-09-15T00:13:54.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:54.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818616+0000", 2024-09-15T00:13:54.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6688e-05 2024-09-15T00:13:54.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:54.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818651+0000", 2024-09-15T00:13:54.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4885000000000002e-05 2024-09-15T00:13:54.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818687+0000", 2024-09-15T00:13:54.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5429999999999998e-05 2024-09-15T00:13:54.957 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.957 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.957 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.957 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163578 3.1c 3:3efd71ab:::rbd_data.11395d771336.0000000000000060:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-15T00:13:54.957 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:54.957 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 60.635055182000002, 2024-09-15T00:13:54.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.085154081000000006, 2024-09-15T00:13:54.958 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.958 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.958 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:54.958 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:54.958 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:54.958 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:54.958 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163578 2024-09-15T00:13:54.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.958 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:54.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:54.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170552+0000", 2024-09-15T00:13:54.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8599999999999997e-07 2024-09-15T00:13:54.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170557+0000", 2024-09-15T00:13:54.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-15T00:13:54.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170567+0000", 2024-09-15T00:13:54.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0628999999999999e-05 2024-09-15T00:13:54.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170573+0000", 2024-09-15T00:13:54.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1649999999999997e-06 2024-09-15T00:13:54.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170619+0000", 2024-09-15T00:13:54.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6405e-05 2024-09-15T00:13:54.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170666+0000", 2024-09-15T00:13:54.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7259000000000003e-05 2024-09-15T00:13:54.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:54.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.175060+0000", 2024-09-15T00:13:54.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043940170000000001 2024-09-15T00:13:54.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:54.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255559+0000", 2024-09-15T00:13:54.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080499065999999994 2024-09-15T00:13:54.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:54.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255616+0000", 2024-09-15T00:13:54.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6422000000000003e-05 2024-09-15T00:13:54.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:54.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255655+0000", 2024-09-15T00:13:54.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9632999999999998e-05 2024-09-15T00:13:54.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255705+0000", 2024-09-15T00:13:54.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.973e-05 2024-09-15T00:13:54.963 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.963 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.963 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.963 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:163578 3.1c e107/23 ondisk, result = 0)", 2024-09-15T00:13:54.963 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:54.963 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 60.619833090999997, 2024-09-15T00:13:54.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069939984999999996, 2024-09-15T00:13:54.963 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:54.963 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:54.963 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:54.964 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:54.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:54.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:54.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:54.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:54.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:54.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:54.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185774+0000", 2024-09-15T00:13:54.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.093e-06 2024-09-15T00:13:54.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:54.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185779+0000", 2024-09-15T00:13:54.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9470000000000002e-06 2024-09-15T00:13:54.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:54.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185794+0000", 2024-09-15T00:13:54.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4450000000000001e-05 2024-09-15T00:13:54.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:54.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185801+0000", 2024-09-15T00:13:54.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9189999999999999e-06 2024-09-15T00:13:54.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:54.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255604+0000", 2024-09-15T00:13:54.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069802201999999994 2024-09-15T00:13:54.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:54.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255611+0000", 2024-09-15T00:13:54.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7740000000000004e-06 2024-09-15T00:13:54.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:54.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:54.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:54.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255713+0000", 2024-09-15T00:13:54.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0001016 2024-09-15T00:13:54.967 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.967 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.967 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.967 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:54.967 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:54.968 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:54.968 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:54.969 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:54.969 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:54.970 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:54.970 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:54.971 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:54.971 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:54.971 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:54.971 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:54.972 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:54.972 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:54.972 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:54.972 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:54.973 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:54.973 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:54.973 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:54.974 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:54.974 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:54.967+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:55.070 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:55.067+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:55.097 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:55.097 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:55.107 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:55.191 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:55.188+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:55.254 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:55.255 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:55.263 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:55.305 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:55.302+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:55.372 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:55.370+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:55.407 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:55.407 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:55.416 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:55.473 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:55.471+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:55.518 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:55.518 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:55.527 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:55.610 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:55.608+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:55.670 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:55.670 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:55.675 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:55.672+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:55.679 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:55.776 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:55.774+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:55.822 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:55.822 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:55.830 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:55.877 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:55.874+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:55.941 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:55.941 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:55.950 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:55.978 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:55.976+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:56.045 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 0, 1, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-15T00:13:56.045 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:13:56.045 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [3, 4, 5, 6, 0, 1, 7, 2] 2024-09-15T00:13:56.046 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-15T00:13:56.046 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:13:56.066 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:56.066 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:56.077 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:56.120 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:56.118+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:56.179 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:56.177+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:56.238 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:56.238 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:56.247 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:13:56.290 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:56.289+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:56.353 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:56.353 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:56.361 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:13:56.366 INFO:tasks.ceph.mon.a.smithi171.stderr:2024-09-15T00:13:56.364+0000 7f0425d35640 -1 mon.a@0(leader).osd e168 definitely_dead 0 2024-09-15T00:13:56.381 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:56.380+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:56.469 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:56.469 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:56.478 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:13:56.522 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:56.521+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:56.582 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:56.583 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:56.583 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:56.581+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:56.591 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:13:56.683 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:56.682+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:56.735 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:56.735 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:56.744 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:13:56.785 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:56.782+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:56.885 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:56.884+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:56.888 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:56.888 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:56.897 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:13:56.975 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:13:57.026 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:57.023+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:57.028 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is set 2024-09-15T00:13:57.030 INFO:teuthology.orchestra.run.smithi171.stderr:marked out osd.1. 2024-09-15T00:13:57.057 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:57.057 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:57.066 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:13:57.108 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:57.106+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:57.116 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:57.116 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:57.134 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:57.134 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:57.150 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:57.150 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:57.166 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:57.166 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:57.183 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:57.184 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:57.188 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:57.185+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:57.200 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:57.200 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:57.213 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:57.213 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:57.213 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:13:57.213 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:57.217 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:57.217 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:57.222 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:13:57.234 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:57.234 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:57.289 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:57.287+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:57.337 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:57.337 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:13:57.337 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:13:57.338 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:13:57.338 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:57.346 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:13:57.390 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:57.389+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:57.451 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:13:57.451 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:13:57.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:13:57.451 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:13:57.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.452 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:13:57.452 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:57.452 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 129.20548091200001, 2024-09-15T00:13:57.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:13:57.452 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.452 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.452 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:57.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.453 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:13:57.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.453 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:57.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:13:57.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:13:57.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:13:57.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:13:57.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:13:57.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:13:57.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:13:57.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:13:57.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:57.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:13:57.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:13:57.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:13:57.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:13:57.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:57.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:13:57.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:13:57.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:57.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:13:57.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:13:57.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:57.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:13:57.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:13:57.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:57.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:13:57.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:13:57.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:13:57.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:13:57.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.459 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.459 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58878 3.8 3:12456482:::rbd_data.1139766160da.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:57.459 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:57.459 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.159175328, 2024-09-15T00:13:57.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061956199000000003, 2024-09-15T00:13:57.459 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.459 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.459 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:57.460 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.460 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.460 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.460 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58878 2024-09-15T00:13:57.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.460 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:57.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:13:57.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289151+0000", 2024-09-15T00:13:57.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:13:57.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289473+0000", 2024-09-15T00:13:57.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032183800000000002 2024-09-15T00:13:57.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289487+0000", 2024-09-15T00:13:57.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-05 2024-09-15T00:13:57.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289493+0000", 2024-09-15T00:13:57.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4489999999999996e-06 2024-09-15T00:13:57.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289542+0000", 2024-09-15T00:13:57.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8844999999999997e-05 2024-09-15T00:13:57.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289642+0000", 2024-09-15T00:13:57.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9607999999999999e-05 2024-09-15T00:13:57.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:13:57.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.293908+0000", 2024-09-15T00:13:57.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659350000000002 2024-09-15T00:13:57.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:57.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351022+0000", 2024-09-15T00:13:57.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057114535000000001 2024-09-15T00:13:57.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:57.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351033+0000", 2024-09-15T00:13:57.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0263e-05 2024-09-15T00:13:57.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:57.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351070+0000", 2024-09-15T00:13:57.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7666000000000002e-05 2024-09-15T00:13:57.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351106+0000", 2024-09-15T00:13:57.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999997e-05 2024-09-15T00:13:57.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.465 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.466 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58879 3.7 3:e169894d:::rbd_data.1139766160da.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:57.466 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:57.466 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.15682384, 2024-09-15T00:13:57.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043716294000000003, 2024-09-15T00:13:57.466 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.466 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.466 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:57.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.466 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58879 2024-09-15T00:13:57.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.467 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:57.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:13:57.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291502+0000", 2024-09-15T00:13:57.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-15T00:13:57.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291664+0000", 2024-09-15T00:13:57.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016216599999999999 2024-09-15T00:13:57.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291674+0000", 2024-09-15T00:13:57.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0276e-05 2024-09-15T00:13:57.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291680+0000", 2024-09-15T00:13:57.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2580000000000003e-06 2024-09-15T00:13:57.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291732+0000", 2024-09-15T00:13:57.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2114999999999998e-05 2024-09-15T00:13:57.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291866+0000", 2024-09-15T00:13:57.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013416900000000001 2024-09-15T00:13:57.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:13:57.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.296426+0000", 2024-09-15T00:13:57.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045595779999999999 2024-09-15T00:13:57.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:57.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335075+0000", 2024-09-15T00:13:57.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038649347000000001 2024-09-15T00:13:57.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:57.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335097+0000", 2024-09-15T00:13:57.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2565000000000002e-05 2024-09-15T00:13:57.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:57.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335161+0000", 2024-09-15T00:13:57.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3928000000000002e-05 2024-09-15T00:13:57.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335217+0000", 2024-09-15T00:13:57.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6042000000000003e-05 2024-09-15T00:13:57.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.473 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.473 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58880 3.8 3:13d91cfd:::rbd_data.1139766160da.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:13:57.474 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:57.474 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.15343173300001, 2024-09-15T00:13:57.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056331463999999998, 2024-09-15T00:13:57.474 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.474 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.474 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:57.474 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.474 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.474 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.474 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58880 2024-09-15T00:13:57.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.474 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:57.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:13:57.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294896+0000", 2024-09-15T00:13:57.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.356e-06 2024-09-15T00:13:57.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295288+0000", 2024-09-15T00:13:57.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039223200000000001 2024-09-15T00:13:57.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295304+0000", 2024-09-15T00:13:57.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5937e-05 2024-09-15T00:13:57.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295312+0000", 2024-09-15T00:13:57.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1489999999999994e-06 2024-09-15T00:13:57.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333162+0000", 2024-09-15T00:13:57.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037849840000000003 2024-09-15T00:13:57.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333269+0000", 2024-09-15T00:13:57.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010749800000000001 2024-09-15T00:13:57.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:13:57.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.336198+0000", 2024-09-15T00:13:57.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029284739999999999 2024-09-15T00:13:57.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:57.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351137+0000", 2024-09-15T00:13:57.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014939103 2024-09-15T00:13:57.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:57.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351150+0000", 2024-09-15T00:13:57.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3542e-05 2024-09-15T00:13:57.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:57.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351191+0000", 2024-09-15T00:13:57.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0791000000000003e-05 2024-09-15T00:13:57.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351225+0000", 2024-09-15T00:13:57.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3541999999999998e-05 2024-09-15T00:13:57.481 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.481 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.481 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.481 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58878 3.8 e56/23 ondisk, result = 0)", 2024-09-15T00:13:57.481 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:57.481 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.14690195, 2024-09-15T00:13:57.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049689096000000002, 2024-09-15T00:13:57.481 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.481 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.481 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:57.482 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:57.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:13:57.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301425+0000", 2024-09-15T00:13:57.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.782e-06 2024-09-15T00:13:57.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301433+0000", 2024-09-15T00:13:57.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689999999999994e-06 2024-09-15T00:13:57.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301449+0000", 2024-09-15T00:13:57.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5767e-05 2024-09-15T00:13:57.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301458+0000", 2024-09-15T00:13:57.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:13:57.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351009+0000", 2024-09-15T00:13:57.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049551523 2024-09-15T00:13:57.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351017+0000", 2024-09-15T00:13:57.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8359999999999992e-06 2024-09-15T00:13:57.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351112+0000", 2024-09-15T00:13:57.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5173999999999994e-05 2024-09-15T00:13:57.486 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.486 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.486 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.486 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127004 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 6881280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:57.486 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:57.486 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.479996486000005, 2024-09-15T00:13:57.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735720000000003, 2024-09-15T00:13:57.486 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.486 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.487 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:57.487 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.487 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.487 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.487 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127004 2024-09-15T00:13:57.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.487 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:57.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:57.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:13:57.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:13:57.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968331+0000", 2024-09-15T00:13:57.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.849e-06 2024-09-15T00:13:57.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968334+0000", 2024-09-15T00:13:57.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7689999999999998e-06 2024-09-15T00:13:57.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968338+0000", 2024-09-15T00:13:57.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2490000000000001e-06 2024-09-15T00:13:57.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008987+0000", 2024-09-15T00:13:57.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040649117999999998 2024-09-15T00:13:57.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009027+0000", 2024-09-15T00:13:57.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0357e-05 2024-09-15T00:13:57.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009064+0000", 2024-09-15T00:13:57.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7036000000000003e-05 2024-09-15T00:13:57.491 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.491 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.491 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.491 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127005 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7004160~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:57.491 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:57.491 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.479983934000003, 2024-09-15T00:13:57.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040824358999999998, 2024-09-15T00:13:57.491 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.491 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.492 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:57.492 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.492 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.492 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.492 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127005 2024-09-15T00:13:57.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.492 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:57.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:57.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:13:57.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-15T00:13:57.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968343+0000", 2024-09-15T00:13:57.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3430000000000001e-06 2024-09-15T00:13:57.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968349+0000", 2024-09-15T00:13:57.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3080000000000004e-06 2024-09-15T00:13:57.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968355+0000", 2024-09-15T00:13:57.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9499999999999998e-06 2024-09-15T00:13:57.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009091+0000", 2024-09-15T00:13:57.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735645000000001 2024-09-15T00:13:57.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009130+0000", 2024-09-15T00:13:57.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8951e-05 2024-09-15T00:13:57.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009165+0000", 2024-09-15T00:13:57.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5926000000000001e-05 2024-09-15T00:13:57.496 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.496 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.496 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.497 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127006 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7127040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:57.497 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:57.497 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.479966876000006, 2024-09-15T00:13:57.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040912783000000001, 2024-09-15T00:13:57.497 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.497 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.497 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:57.497 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.497 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.497 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.498 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127006 2024-09-15T00:13:57.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.498 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:57.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:57.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:13:57.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:57.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968359+0000", 2024-09-15T00:13:57.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0419999999999999e-06 2024-09-15T00:13:57.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968363+0000", 2024-09-15T00:13:57.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5999999999999998e-06 2024-09-15T00:13:57.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968366+0000", 2024-09-15T00:13:57.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5179999999999999e-06 2024-09-15T00:13:57.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009196+0000", 2024-09-15T00:13:57.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040830313 2024-09-15T00:13:57.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009234+0000", 2024-09-15T00:13:57.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8562000000000001e-05 2024-09-15T00:13:57.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009271+0000", 2024-09-15T00:13:57.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6538000000000002e-05 2024-09-15T00:13:57.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.501 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.502 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127007 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7249920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:57.502 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:57.502 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.479956677000004, 2024-09-15T00:13:57.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041019218000000003, 2024-09-15T00:13:57.502 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.502 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.502 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:57.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.502 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127007 2024-09-15T00:13:57.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.503 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:57.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:13:57.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968369+0000", 2024-09-15T00:13:57.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-15T00:13:57.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968370+0000", 2024-09-15T00:13:57.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.012e-06 2024-09-15T00:13:57.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968374+0000", 2024-09-15T00:13:57.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3839999999999999e-06 2024-09-15T00:13:57.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968376+0000", 2024-09-15T00:13:57.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5339999999999998e-06 2024-09-15T00:13:57.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009292+0000", 2024-09-15T00:13:57.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040915092 2024-09-15T00:13:57.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009333+0000", 2024-09-15T00:13:57.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1105e-05 2024-09-15T00:13:57.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009388+0000", 2024-09-15T00:13:57.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4868999999999999e-05 2024-09-15T00:13:57.507 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.507 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.507 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.507 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127008 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7364608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:57.508 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:57.508 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.479945556000004, 2024-09-15T00:13:57.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041115290999999998, 2024-09-15T00:13:57.508 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.508 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.508 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:57.508 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.508 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.508 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.508 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127008 2024-09-15T00:13:57.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.508 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:57.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:13:57.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968380+0000", 2024-09-15T00:13:57.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3999999999999998e-07 2024-09-15T00:13:57.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968381+0000", 2024-09-15T00:13:57.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8700000000000004e-07 2024-09-15T00:13:57.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968385+0000", 2024-09-15T00:13:57.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9260000000000002e-06 2024-09-15T00:13:57.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968389+0000", 2024-09-15T00:13:57.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.685e-06 2024-09-15T00:13:57.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009419+0000", 2024-09-15T00:13:57.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041029852999999998 2024-09-15T00:13:57.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009458+0000", 2024-09-15T00:13:57.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8538000000000003e-05 2024-09-15T00:13:57.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009495+0000", 2024-09-15T00:13:57.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7061999999999999e-05 2024-09-15T00:13:57.513 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.513 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.513 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.513 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127009 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7487488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:57.513 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:57.513 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.479911439000006, 2024-09-15T00:13:57.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041233063, 2024-09-15T00:13:57.513 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.514 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.514 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:57.514 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.514 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.514 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.514 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127009 2024-09-15T00:13:57.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.514 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:57.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:57.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:13:57.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7199999999999999e-07 2024-09-15T00:13:57.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968417+0000", 2024-09-15T00:13:57.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3810000000000002e-06 2024-09-15T00:13:57.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968423+0000", 2024-09-15T00:13:57.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8820000000000003e-06 2024-09-15T00:13:57.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968433+0000", 2024-09-15T00:13:57.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0005e-05 2024-09-15T00:13:57.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009527+0000", 2024-09-15T00:13:57.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041093750999999998 2024-09-15T00:13:57.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009601+0000", 2024-09-15T00:13:57.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3979000000000004e-05 2024-09-15T00:13:57.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009647+0000", 2024-09-15T00:13:57.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5493e-05 2024-09-15T00:13:57.518 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.518 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.518 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.519 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127010 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7610368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:57.519 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:57.519 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.479882434999993, 2024-09-15T00:13:57.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041325924999999999, 2024-09-15T00:13:57.519 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.519 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.519 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:57.520 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.520 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.520 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.520 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127010 2024-09-15T00:13:57.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.520 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:57.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:57.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:13:57.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:13:57.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968445+0000", 2024-09-15T00:13:57.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:13:57.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968452+0000", 2024-09-15T00:13:57.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-15T00:13:57.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968456+0000", 2024-09-15T00:13:57.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5210000000000004e-06 2024-09-15T00:13:57.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009664+0000", 2024-09-15T00:13:57.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041207404000000003 2024-09-15T00:13:57.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009723+0000", 2024-09-15T00:13:57.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9036000000000002e-05 2024-09-15T00:13:57.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009768+0000", 2024-09-15T00:13:57.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5578000000000003e-05 2024-09-15T00:13:57.524 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.525 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.525 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.525 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127011 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7733248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:57.525 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:57.525 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.479864199999994, 2024-09-15T00:13:57.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041482441000000002, 2024-09-15T00:13:57.525 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.525 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.525 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:57.526 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.526 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.526 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.526 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127011 2024-09-15T00:13:57.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.526 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:57.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:57.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:13:57.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-15T00:13:57.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968466+0000", 2024-09-15T00:13:57.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:13:57.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968470+0000", 2024-09-15T00:13:57.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6469999999999997e-06 2024-09-15T00:13:57.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968473+0000", 2024-09-15T00:13:57.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.977e-06 2024-09-15T00:13:57.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009800+0000", 2024-09-15T00:13:57.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041326918999999997 2024-09-15T00:13:57.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009869+0000", 2024-09-15T00:13:57.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8935000000000003e-05 2024-09-15T00:13:57.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009943+0000", 2024-09-15T00:13:57.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4188000000000002e-05 2024-09-15T00:13:57.531 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.531 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.531 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.531 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127012 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7856128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:57.531 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:57.531 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.479848697999998, 2024-09-15T00:13:57.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041640756000000001, 2024-09-15T00:13:57.531 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.531 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.532 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:57.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.532 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127012 2024-09-15T00:13:57.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.532 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:57.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:13:57.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968477+0000", 2024-09-15T00:13:57.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:13:57.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968482+0000", 2024-09-15T00:13:57.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6180000000000003e-06 2024-09-15T00:13:57.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968486+0000", 2024-09-15T00:13:57.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.501e-06 2024-09-15T00:13:57.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968489+0000", 2024-09-15T00:13:57.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1530000000000001e-06 2024-09-15T00:13:57.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009983+0000", 2024-09-15T00:13:57.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041493980999999999 2024-09-15T00:13:57.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010051+0000", 2024-09-15T00:13:57.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8039000000000004e-05 2024-09-15T00:13:57.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010117+0000", 2024-09-15T00:13:57.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6254000000000006e-05 2024-09-15T00:13:57.537 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.537 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.537 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.537 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127013 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7979008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:57.537 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:57.537 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.479832877999996, 2024-09-15T00:13:57.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041763863999999998, 2024-09-15T00:13:57.537 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.537 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.537 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:57.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.538 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127013 2024-09-15T00:13:57.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.538 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:57.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:13:57.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969767+0000", 2024-09-15T00:13:57.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001275006 2024-09-15T00:13:57.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969772+0000", 2024-09-15T00:13:57.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4769999999999997e-06 2024-09-15T00:13:57.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969781+0000", 2024-09-15T00:13:57.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1959999999999997e-06 2024-09-15T00:13:57.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969788+0000", 2024-09-15T00:13:57.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6359999999999994e-06 2024-09-15T00:13:57.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010162+0000", 2024-09-15T00:13:57.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040373291999999998 2024-09-15T00:13:57.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010212+0000", 2024-09-15T00:13:57.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0543e-05 2024-09-15T00:13:57.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010256+0000", 2024-09-15T00:13:57.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3714000000000002e-05 2024-09-15T00:13:57.541 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.542 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.542 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.542 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127015 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8224768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:57.542 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:57.542 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.478493942, 2024-09-15T00:13:57.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040710186000000002, 2024-09-15T00:13:57.542 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.542 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.542 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:57.542 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.543 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.543 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.543 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127015 2024-09-15T00:13:57.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.543 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:57.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:13:57.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969832+0000", 2024-09-15T00:13:57.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:13:57.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969834+0000", 2024-09-15T00:13:57.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2919999999999998e-06 2024-09-15T00:13:57.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969843+0000", 2024-09-15T00:13:57.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0480000000000004e-06 2024-09-15T00:13:57.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969848+0000", 2024-09-15T00:13:57.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-15T00:13:57.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010457+0000", 2024-09-15T00:13:57.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040609054999999998 2024-09-15T00:13:57.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010501+0000", 2024-09-15T00:13:57.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3396000000000003e-05 2024-09-15T00:13:57.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010541+0000", 2024-09-15T00:13:57.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0599000000000002e-05 2024-09-15T00:13:57.547 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.547 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.547 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.547 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127016 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:57.547 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:57.548 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.478443315999996, 2024-09-15T00:13:57.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040766216000000001, 2024-09-15T00:13:57.548 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.548 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.548 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:57.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.548 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127016 2024-09-15T00:13:57.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.548 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:57.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:57.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:13:57.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:13:57.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969886+0000", 2024-09-15T00:13:57.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8449999999999996e-06 2024-09-15T00:13:57.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969897+0000", 2024-09-15T00:13:57.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-15T00:13:57.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969904+0000", 2024-09-15T00:13:57.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1169999999999998e-06 2024-09-15T00:13:57.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010561+0000", 2024-09-15T00:13:57.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040657470000000001 2024-09-15T00:13:57.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010607+0000", 2024-09-15T00:13:57.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6108e-05 2024-09-15T00:13:57.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010648+0000", 2024-09-15T00:13:57.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0479000000000003e-05 2024-09-15T00:13:57.552 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.552 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.552 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.553 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127694 3.9 3:9390b205:::rbd_data.11396481e1ec.00000000000000df:head [set-alloc-hint object_size 1048576 write_size 1048576,write 819200~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T00:13:57.553 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:57.553 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.223724199000003, 2024-09-15T00:13:57.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90376419399999997, 2024-09-15T00:13:57.553 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.553 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.553 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:57.553 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.553 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.553 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.553 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127694 2024-09-15T00:13:57.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.554 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:57.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:13:57.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224602+0000", 2024-09-15T00:13:57.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5899999999999995e-07 2024-09-15T00:13:57.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224610+0000", 2024-09-15T00:13:57.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9130000000000003e-06 2024-09-15T00:13:57.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224620+0000", 2024-09-15T00:13:57.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0426e-05 2024-09-15T00:13:57.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224627+0000", 2024-09-15T00:13:57.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4390000000000003e-06 2024-09-15T00:13:57.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224675+0000", 2024-09-15T00:13:57.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7636000000000003e-05 2024-09-15T00:13:57.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:13:57.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224681+0000", 2024-09-15T00:13:57.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1460000000000001e-06 2024-09-15T00:13:57.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126388+0000", 2024-09-15T00:13:57.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90170724999999996 2024-09-15T00:13:57.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126526+0000", 2024-09-15T00:13:57.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013730599999999999 2024-09-15T00:13:57.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:57.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126684+0000", 2024-09-15T00:13:57.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000157935 2024-09-15T00:13:57.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:57.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.127932+0000", 2024-09-15T00:13:57.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012479240000000001 2024-09-15T00:13:57.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:57.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128269+0000", 2024-09-15T00:13:57.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033761099999999998 2024-09-15T00:13:57.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:57.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128324+0000", 2024-09-15T00:13:57.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5244999999999997e-05 2024-09-15T00:13:57.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128365+0000", 2024-09-15T00:13:57.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0503999999999997e-05 2024-09-15T00:13:57.560 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.560 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.560 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.561 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:134707 3.15 3:af1fbdb5:::rbd_data.113933a4f79a.000000000000001c:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T00:13:57.561 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:57.561 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 85.755757084999999, 2024-09-15T00:13:57.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057328626000000001, 2024-09-15T00:13:57.561 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.561 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.561 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:57.561 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:13:57.562 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:13:57.562 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.562 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 134707 2024-09-15T00:13:57.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.562 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:57.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:13:57.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692569+0000", 2024-09-15T00:13:57.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.339e-06 2024-09-15T00:13:57.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692574+0000", 2024-09-15T00:13:57.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-15T00:13:57.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692584+0000", 2024-09-15T00:13:57.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0363000000000001e-05 2024-09-15T00:13:57.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692590+0000", 2024-09-15T00:13:57.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.818e-06 2024-09-15T00:13:57.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692633+0000", 2024-09-15T00:13:57.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.265e-05 2024-09-15T00:13:57.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692718+0000", 2024-09-15T00:13:57.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5279999999999997e-05 2024-09-15T00:13:57.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:57.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.695940+0000", 2024-09-15T00:13:57.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003222052 2024-09-15T00:13:57.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:57.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749714+0000", 2024-09-15T00:13:57.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.053774126999999998 2024-09-15T00:13:57.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:13:57.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749788+0000", 2024-09-15T00:13:57.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3693000000000001e-05 2024-09-15T00:13:57.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:13:57.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749847+0000", 2024-09-15T00:13:57.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175999999999997e-05 2024-09-15T00:13:57.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749897+0000", 2024-09-15T00:13:57.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9311999999999998e-05 2024-09-15T00:13:57.567 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.567 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.567 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.568 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:134707 3.15 e86/55 ondisk, result = 0)", 2024-09-15T00:13:57.568 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:57.568 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 85.747195735999995, 2024-09-15T00:13:57.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048596066, 2024-09-15T00:13:57.568 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:13:57.568 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:13:57.568 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:13:57.568 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:13:57.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:13:57.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:57.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:13:57.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:13:57.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:13:57.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:13:57.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701131+0000", 2024-09-15T00:13:57.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3549999999999999e-06 2024-09-15T00:13:57.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:13:57.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701139+0000", 2024-09-15T00:13:57.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1650000000000006e-06 2024-09-15T00:13:57.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:13:57.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701156+0000", 2024-09-15T00:13:57.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6730000000000001e-05 2024-09-15T00:13:57.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701167+0000", 2024-09-15T00:13:57.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1504e-05 2024-09-15T00:13:57.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:13:57.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749629+0000", 2024-09-15T00:13:57.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048461952000000003 2024-09-15T00:13:57.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:13:57.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749640+0000", 2024-09-15T00:13:57.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0874e-05 2024-09-15T00:13:57.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:13:57.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:13:57.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:13:57.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749725+0000", 2024-09-15T00:13:57.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5486000000000001e-05 2024-09-15T00:13:57.572 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.572 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.572 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.573 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:13:57.573 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:13:57.573 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:13:57.575 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:13:57.575 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:57.572+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:57.675 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:57.672+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:57.688 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:57.688 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:57.688 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:57.688 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:57.697 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:13:57.776 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:57.774+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:57.840 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:57.840 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:57.840 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:57.849 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:13:57.893 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:57.890+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:57.953 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:57.953 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:57.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:57.954 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:57.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.954 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:863 3.9 3:9334bf01:::rbd_data.11397bff333d.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:57.954 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:57.954 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 152.37872094100001, 2024-09-15T00:13:57.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077183384999999993, 2024-09-15T00:13:57.954 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:57.954 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:57.954 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:57.954 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:57.954 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:57.954 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.954 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 863 2024-09-15T00:13:57.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.955 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:57.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:57.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:57.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:57.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571914+0000", 2024-09-15T00:13:57.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:57.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571915+0000", 2024-09-15T00:13:57.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:13:57.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:57.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571917+0000", 2024-09-15T00:13:57.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6639999999999998e-06 2024-09-15T00:13:57.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:57.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571924+0000", 2024-09-15T00:13:57.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8600000000000004e-06 2024-09-15T00:13:57.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571931+0000", 2024-09-15T00:13:57.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9829999999999999e-06 2024-09-15T00:13:57.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:57.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571962+0000", 2024-09-15T00:13:57.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1086e-05 2024-09-15T00:13:57.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:57.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.572002+0000", 2024-09-15T00:13:57.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0036000000000001e-05 2024-09-15T00:13:57.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:57.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.575269+0000", 2024-09-15T00:13:57.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00326662 2024-09-15T00:13:57.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:57.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649026+0000", 2024-09-15T00:13:57.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073757328999999996 2024-09-15T00:13:57.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:57.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649036+0000", 2024-09-15T00:13:57.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2520000000000005e-06 2024-09-15T00:13:57.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:57.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649065+0000", 2024-09-15T00:13:57.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9447000000000001e-05 2024-09-15T00:13:57.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:57.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649098+0000", 2024-09-15T00:13:57.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2616000000000001e-05 2024-09-15T00:13:57.960 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.960 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:57.960 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.960 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:864 3.9 3:90ee4fa6:::rbd_object_map.11397bff333d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-15T00:13:57.960 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:57.960 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 152.378686249, 2024-09-15T00:13:57.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077894726999999997, 2024-09-15T00:13:57.960 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:57.961 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:57.961 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:57.961 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:57.961 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:57.961 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.961 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 864 2024-09-15T00:13:57.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.961 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:57.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:57.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:57.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:57.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:57.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:57.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571949+0000", 2024-09-15T00:13:57.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6899999999999998e-07 2024-09-15T00:13:57.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:57.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571952+0000", 2024-09-15T00:13:57.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:13:57.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:57.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571959+0000", 2024-09-15T00:13:57.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-15T00:13:57.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.571965+0000", 2024-09-15T00:13:57.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2390000000000002e-06 2024-09-15T00:13:57.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:57.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648347+0000", 2024-09-15T00:13:57.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076382962999999998 2024-09-15T00:13:57.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:57.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648421+0000", 2024-09-15T00:13:57.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3332999999999997e-05 2024-09-15T00:13:57.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-15T00:13:57.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.648664+0000", 2024-09-15T00:13:57.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024288200000000001 2024-09-15T00:13:57.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:57.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649347+0000", 2024-09-15T00:13:57.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000683027 2024-09-15T00:13:57.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:57.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649797+0000", 2024-09-15T00:13:57.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044991900000000001 2024-09-15T00:13:57.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:57.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649824+0000", 2024-09-15T00:13:57.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6922000000000001e-05 2024-09-15T00:13:57.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:57.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:25.649844+0000", 2024-09-15T00:13:57.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0182000000000001e-05 2024-09-15T00:13:57.966 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.966 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:57.966 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.966 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48946 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:57.966 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:57.966 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.338012561, 2024-09-15T00:13:57.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390088000000005, 2024-09-15T00:13:57.966 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:57.967 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:57.967 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:57.967 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:57.967 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:57.967 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.967 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48946 2024-09-15T00:13:57.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.967 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:57.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:57.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:57.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:57.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:57.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:57.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612623+0000", 2024-09-15T00:13:57.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:13:57.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:57.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612625+0000", 2024-09-15T00:13:57.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.725e-06 2024-09-15T00:13:57.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:57.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612630+0000", 2024-09-15T00:13:57.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3739999999999997e-06 2024-09-15T00:13:57.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612634+0000", 2024-09-15T00:13:57.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:13:57.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:57.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684386+0000", 2024-09-15T00:13:57.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.071751879000000005 2024-09-15T00:13:57.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:57.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.684465+0000", 2024-09-15T00:13:57.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8596000000000004e-05 2024-09-15T00:13:57.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:57.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685013+0000", 2024-09-15T00:13:57.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054800900000000002 2024-09-15T00:13:57.970 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.971 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:57.971 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.971 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48947 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:57.971 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:57.971 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.33799603, 2024-09-15T00:13:57.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073080015999999998, 2024-09-15T00:13:57.971 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:57.971 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:57.971 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:57.971 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:57.971 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:57.971 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.971 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48947 2024-09-15T00:13:57.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.972 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:57.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:57.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:57.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:57.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612639+0000", 2024-09-15T00:13:57.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:57.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612640+0000", 2024-09-15T00:13:57.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:13:57.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:57.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612641+0000", 2024-09-15T00:13:57.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.457e-06 2024-09-15T00:13:57.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:57.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612647+0000", 2024-09-15T00:13:57.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5779999999999998e-06 2024-09-15T00:13:57.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612651+0000", 2024-09-15T00:13:57.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1509999999999997e-06 2024-09-15T00:13:57.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:57.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685042+0000", 2024-09-15T00:13:57.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.072390825000000006 2024-09-15T00:13:57.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:57.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685191+0000", 2024-09-15T00:13:57.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000149312 2024-09-15T00:13:57.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:57.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685719+0000", 2024-09-15T00:13:57.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052839800000000002 2024-09-15T00:13:57.975 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.975 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:57.975 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.975 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48948 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:57.975 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:57.975 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.33797925100001, 2024-09-15T00:13:57.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073779763999999998, 2024-09-15T00:13:57.976 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:57.976 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:57.976 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:57.976 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:57.976 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:57.976 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.976 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48948 2024-09-15T00:13:57.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.976 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:57.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:57.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:57.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:57.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:57.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:57.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612656+0000", 2024-09-15T00:13:57.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.34e-07 2024-09-15T00:13:57.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:57.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612658+0000", 2024-09-15T00:13:57.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.968e-06 2024-09-15T00:13:57.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:57.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612664+0000", 2024-09-15T00:13:57.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:13:57.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612669+0000", 2024-09-15T00:13:57.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3660000000000002e-06 2024-09-15T00:13:57.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:57.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685742+0000", 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073073414000000003 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.685823+0000", 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.106e-05 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686436+0000", 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00061267899999999998 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.980 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48949 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:57.980 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:57.980 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.33796152100001, 2024-09-15T00:13:57.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074495425000000004, 2024-09-15T00:13:57.980 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:57.980 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:57.980 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:57.980 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:57.980 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:57.980 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.980 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48949 2024-09-15T00:13:57.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.980 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:57.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:57.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:57.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:57.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:57.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:57.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612674+0000", 2024-09-15T00:13:57.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:13:57.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:57.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612676+0000", 2024-09-15T00:13:57.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.669e-06 2024-09-15T00:13:57.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:57.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612682+0000", 2024-09-15T00:13:57.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2009999999999996e-06 2024-09-15T00:13:57.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612686+0000", 2024-09-15T00:13:57.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1500000000000001e-06 2024-09-15T00:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686465+0000", 2024-09-15T00:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.073778701000000002 2024-09-15T00:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.686541+0000", 2024-09-15T00:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6636000000000003e-05 2024-09-15T00:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687169+0000", 2024-09-15T00:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00062776300000000004 2024-09-15T00:13:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48950 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.337944357, 2024-09-15T00:13:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074692316999999994, 2024-09-15T00:13:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:57.984 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.985 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48950 2024-09-15T00:13:57.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.985 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:57.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:57.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:57.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:57.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:57.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612691+0000", 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612693+0000", 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612698+0000", 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612707+0000", 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8370000000000003e-06 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687197+0000", 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074490009999999995 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687265+0000", 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8384000000000001e-05 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687383+0000", 2024-09-15T00:13:57.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117772 2024-09-15T00:13:57.988 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.988 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:57.988 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.988 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48951 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:57.988 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:57.988 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.33792268100001, 2024-09-15T00:13:57.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074923363000000007, 2024-09-15T00:13:57.988 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:57.988 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:57.988 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:57.988 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:57.988 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:57.989 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.989 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48951 2024-09-15T00:13:57.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.989 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:57.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:57.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:57.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:57.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:57.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612713+0000", 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.39e-07 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612715+0000", 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612720+0000", 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:13:57.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612724+0000", 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687410+0000", 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074685318000000001 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687506+0000", 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6873999999999994e-05 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:57.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687636+0000", 2024-09-15T00:13:57.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012946799999999999 2024-09-15T00:13:57.992 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.992 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:57.992 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.992 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48952 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:57.992 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:57.992 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.33787655699999, 2024-09-15T00:13:57.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075130369000000002, 2024-09-15T00:13:57.992 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:57.993 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:57.993 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:57.993 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:57.993 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:57.993 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.993 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48952 2024-09-15T00:13:57.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.993 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:57.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:57.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:57.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:57.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:57.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:57.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612759+0000", 2024-09-15T00:13:57.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:13:57.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:57.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612763+0000", 2024-09-15T00:13:57.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9179999999999996e-06 2024-09-15T00:13:57.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:57.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612771+0000", 2024-09-15T00:13:57.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8800000000000008e-06 2024-09-15T00:13:57.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612778+0000", 2024-09-15T00:13:57.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.472e-06 2024-09-15T00:13:57.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:57.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687664+0000", 2024-09-15T00:13:57.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074885967999999997 2024-09-15T00:13:57.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:57.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687754+0000", 2024-09-15T00:13:57.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0341999999999997e-05 2024-09-15T00:13:57.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:57.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687889+0000", 2024-09-15T00:13:57.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013513300000000001 2024-09-15T00:13:57.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.996 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:57.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:57.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.996 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48953 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:57.997 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:57.997 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.33784679999999, 2024-09-15T00:13:57.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075337262000000002, 2024-09-15T00:13:57.997 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:57.997 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:57.997 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:57.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:57.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:57.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:57.997 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48953 2024-09-15T00:13:57.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.997 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:57.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:57.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:57.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:57.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612788+0000", 2024-09-15T00:13:57.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:57.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:57.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612789+0000", 2024-09-15T00:13:57.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:13:57.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:57.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612791+0000", 2024-09-15T00:13:57.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:13:57.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:57.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612798+0000", 2024-09-15T00:13:57.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:57.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:57.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:57.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:57.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612802+0000", 2024-09-15T00:13:58.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-15T00:13:58.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.687925+0000", 2024-09-15T00:13:58.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075122775000000003 2024-09-15T00:13:58.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688012+0000", 2024-09-15T00:13:58.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7318999999999995e-05 2024-09-15T00:13:58.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688126+0000", 2024-09-15T00:13:58.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113262 2024-09-15T00:13:58.001 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.001 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.001 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.001 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48954 3.1a 3:5ca45140:::rbd_data.11395b9829f3.0000000000000002:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.001 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:58.001 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.33782048099999, 2024-09-15T00:13:58.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075568916, 2024-09-15T00:13:58.001 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.001 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.001 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.002 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.002 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.002 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.002 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48954 2024-09-15T00:13:58.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.002 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:58.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612815+0000", 2024-09-15T00:13:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612830+0000", 2024-09-15T00:13:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5118e-05 2024-09-15T00:13:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612837+0000", 2024-09-15T00:13:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-15T00:13:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612842+0000", 2024-09-15T00:13:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.113e-06 2024-09-15T00:13:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688148+0000", 2024-09-15T00:13:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075305491000000002 2024-09-15T00:13:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688258+0000", 2024-09-15T00:13:58.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110689 2024-09-15T00:13:58.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688384+0000", 2024-09-15T00:13:58.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012529699999999999 2024-09-15T00:13:58.005 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.005 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.005 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.006 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49013 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 1703936~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.006 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:58.006 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.33778461899999, 2024-09-15T00:13:58.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076335985999999995, 2024-09-15T00:13:58.006 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.006 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.006 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.006 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.006 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.006 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.006 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49013 2024-09-15T00:13:58.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.006 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:58.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:58.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612851+0000", 2024-09-15T00:13:58.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1500000000000005e-07 2024-09-15T00:13:58.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612853+0000", 2024-09-15T00:13:58.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0880000000000002e-06 2024-09-15T00:13:58.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612861+0000", 2024-09-15T00:13:58.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3420000000000002e-06 2024-09-15T00:13:58.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612866+0000", 2024-09-15T00:13:58.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-15T00:13:58.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688411+0000", 2024-09-15T00:13:58.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075545883999999994 2024-09-15T00:13:58.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.688494+0000", 2024-09-15T00:13:58.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2490999999999993e-05 2024-09-15T00:13:58.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689187+0000", 2024-09-15T00:13:58.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00069269600000000002 2024-09-15T00:13:58.010 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.010 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.010 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.010 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49014 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.010 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:58.010 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.33776290099999, 2024-09-15T00:13:58.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076522333999999997, 2024-09-15T00:13:58.010 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.010 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.010 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.011 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.011 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.011 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.011 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49014 2024-09-15T00:13:58.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.011 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:58.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612872+0000", 2024-09-15T00:13:58.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612873+0000", 2024-09-15T00:13:58.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.32e-07 2024-09-15T00:13:58.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.612881+0000", 2024-09-15T00:13:58.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9340000000000006e-06 2024-09-15T00:13:58.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613041+0000", 2024-09-15T00:13:58.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016054599999999999 2024-09-15T00:13:58.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613050+0000", 2024-09-15T00:13:58.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4039999999999995e-06 2024-09-15T00:13:58.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689217+0000", 2024-09-15T00:13:58.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076166848999999995 2024-09-15T00:13:58.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689290+0000", 2024-09-15T00:13:58.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3889999999999999e-05 2024-09-15T00:13:58.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689395+0000", 2024-09-15T00:13:58.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104279 2024-09-15T00:13:58.014 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.014 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.014 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.014 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49015 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 5373952~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.015 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:58.015 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.337566434, 2024-09-15T00:13:58.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076966768000000005, 2024-09-15T00:13:58.015 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.015 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.015 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.015 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.015 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.015 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.015 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49015 2024-09-15T00:13:58.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.015 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:58.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:58.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613069+0000", 2024-09-15T00:13:58.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0699999999999997e-07 2024-09-15T00:13:58.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613073+0000", 2024-09-15T00:13:58.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:13:58.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613081+0000", 2024-09-15T00:13:58.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9039999999999992e-06 2024-09-15T00:13:58.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613086+0000", 2024-09-15T00:13:58.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3630000000000002e-06 2024-09-15T00:13:58.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689423+0000", 2024-09-15T00:13:58.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076336599000000005 2024-09-15T00:13:58.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.689498+0000", 2024-09-15T00:13:58.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5641000000000003e-05 2024-09-15T00:13:58.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690036+0000", 2024-09-15T00:13:58.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053731999999999999 2024-09-15T00:13:58.018 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.019 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.019 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.019 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49016 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.019 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:58.019 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.33754059500001, 2024-09-15T00:13:58.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077139160999999998, 2024-09-15T00:13:58.019 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.019 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.019 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.019 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.019 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.019 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.020 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49016 2024-09-15T00:13:58.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.020 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:58.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:58.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613095+0000", 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613097+0000", 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1469999999999999e-06 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613103+0000", 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613108+0000", 2024-09-15T00:13:58.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7169999999999999e-06 2024-09-15T00:13:58.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690065+0000", 2024-09-15T00:13:58.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.076956803000000004 2024-09-15T00:13:58.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690144+0000", 2024-09-15T00:13:58.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9274e-05 2024-09-15T00:13:58.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690234+0000", 2024-09-15T00:13:58.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9909999999999998e-05 2024-09-15T00:13:58.023 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.023 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.023 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.023 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49017 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.023 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:58.023 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.337478949, 2024-09-15T00:13:58.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077307829999999994, 2024-09-15T00:13:58.023 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.023 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.023 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.024 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.024 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.024 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.024 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49017 2024-09-15T00:13:58.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.024 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:58.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613156+0000", 2024-09-15T00:13:58.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613157+0000", 2024-09-15T00:13:58.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4199999999999995e-07 2024-09-15T00:13:58.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613161+0000", 2024-09-15T00:13:58.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:13:58.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613171+0000", 2024-09-15T00:13:58.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5259999999999999e-06 2024-09-15T00:13:58.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613184+0000", 2024-09-15T00:13:58.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:13:58.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690260+0000", 2024-09-15T00:13:58.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077075377 2024-09-15T00:13:58.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690335+0000", 2024-09-15T00:13:58.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5301000000000003e-05 2024-09-15T00:13:58.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690464+0000", 2024-09-15T00:13:58.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000129023 2024-09-15T00:13:58.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.027 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.027 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49018 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.027 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:58.028 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.33743962099999, 2024-09-15T00:13:58.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077473281000000005, 2024-09-15T00:13:58.028 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.028 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.028 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.028 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49018 2024-09-15T00:13:58.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.028 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:58.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.613196+0000", 2024-09-15T00:13:58.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614882+0000", 2024-09-15T00:13:58.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001686208 2024-09-15T00:13:58.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614887+0000", 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614898+0000", 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0662999999999999e-05 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614905+0000", 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0910000000000004e-06 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690499+0000", 2024-09-15T00:13:58.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075593661000000006 2024-09-15T00:13:58.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690573+0000", 2024-09-15T00:13:58.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:58.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690669+0000", 2024-09-15T00:13:58.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6247000000000002e-05 2024-09-15T00:13:58.031 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.031 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.031 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49019 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.33571422099999, 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075892398999999999, 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49019 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:58.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614921+0000", 2024-09-15T00:13:58.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614922+0000", 2024-09-15T00:13:58.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:13:58.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614924+0000", 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7369999999999998e-06 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614933+0000", 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3229999999999998e-06 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614937+0000", 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6179999999999997e-06 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690696+0000", 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075758935999999999 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690757+0000", 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0408000000000002e-05 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690813+0000", 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6817999999999997e-05 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.036 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49020 3.1a 3:5f35a6c3:::rbd_data.11395b9829f3.000000000000000a:e [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.036 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:58.036 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.33569206000001, 2024-09-15T00:13:58.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075986125000000002, 2024-09-15T00:13:58.036 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.036 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.036 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.036 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49020 2024-09-15T00:13:58.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.037 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:58.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:58.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614943+0000", 2024-09-15T00:13:58.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6e-07 2024-09-15T00:13:58.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614945+0000", 2024-09-15T00:13:58.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7570000000000001e-06 2024-09-15T00:13:58.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614949+0000", 2024-09-15T00:13:58.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:13:58.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.614951+0000", 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4990000000000001e-06 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690828+0000", 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.075876498000000001 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690870+0000", 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1962999999999998e-05 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690929+0000", 2024-09-15T00:13:58.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9757999999999999e-05 2024-09-15T00:13:58.040 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.040 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.040 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.040 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:145034 3.6 3:618673df:::rbd_object_map.1139646a7d4e.000000000000002b:head [writefull 0~48] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T00:13:58.040 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:58.040 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 77.933438456000005, 2024-09-15T00:13:58.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0081079740000001, 2024-09-15T00:13:58.040 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.040 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.040 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:58.040 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.040 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.040 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.041 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 145034 2024-09-15T00:13:58.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.041 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:58.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017197+0000", 2024-09-15T00:13:58.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017198+0000", 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0100000000000004e-07 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017202+0000", 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017212+0000", 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7189999999999996e-06 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017219+0000", 2024-09-15T00:13:58.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7839999999999998e-06 2024-09-15T00:13:58.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017268+0000", 2024-09-15T00:13:58.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:13:58.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:13:58.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:40.017277+0000", 2024-09-15T00:13:58.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1899999999999995e-06 2024-09-15T00:13:58.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019139+0000", 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0018626129999999 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019236+0000", 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6513000000000002e-05 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.019318+0000", 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1848000000000006e-05 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:58.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.023151+0000", 2024-09-15T00:13:58.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038332169999999999 2024-09-15T00:13:58.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:58.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025217+0000", 2024-09-15T00:13:58.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020657610000000002 2024-09-15T00:13:58.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:58.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025267+0000", 2024-09-15T00:13:58.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0198000000000003e-05 2024-09-15T00:13:58.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:41.025305+0000", 2024-09-15T00:13:58.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8065999999999998e-05 2024-09-15T00:13:58.046 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.046 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.046 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.046 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.046 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.046 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:58.048 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:13:58.048 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:58.046+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:58.148 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:58.146+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:58.191 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:58.191 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:58.191 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:58.191 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:58.200 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:13:58.284 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:58.282+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:58.344 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:58.344 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:58.344 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:58.350 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:13:58.348+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:13:58.353 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:13:58.451 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:58.449+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:58.499 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:58.499 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:58.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:58.500 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:58.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.500 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.500 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:58.500 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.887254273, 2024-09-15T00:13:58.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:13:58.500 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.500 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.500 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.500 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.500 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:13:58.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.501 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:58.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:58.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:13:58.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:13:58.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:13:58.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:13:58.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:13:58.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:13:58.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:13:58.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:13:58.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:13:58.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:13:58.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:13:58.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:13:58.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:13:58.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:13:58.504 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.504 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.504 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.504 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.504 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:58.504 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.887233044, 2024-09-15T00:13:58.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:13:58.504 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.505 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.505 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.505 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.505 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.505 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.505 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:13:58.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.505 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:58.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:13:58.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:13:58.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:13:58.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:13:58.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:58.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:13:58.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:13:58.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:13:58.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:13:58.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:13:58.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:13:58.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:13:58.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:13:58.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:13:58.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:13:58.508 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.508 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.508 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.509 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.509 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:58.509 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.88720750300001, 2024-09-15T00:13:58.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:13:58.509 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.509 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.509 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.509 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:13:58.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.509 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:13:58.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:13:58.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:13:58.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:13:58.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:13:58.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:13:58.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:13:58.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:13:58.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:13:58.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:13:58.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:13:58.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:13:58.513 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.513 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.513 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.513 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.513 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:58.513 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.887185028, 2024-09-15T00:13:58.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:13:58.513 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.513 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.513 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.513 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.513 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.514 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.514 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:13:58.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.514 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:58.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:13:58.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:13:58.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:13:58.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:13:58.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:13:58.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:13:58.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:13:58.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:13:58.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:13:58.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:13:58.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:13:58.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:13:58.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:13:58.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:13:58.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:13:58.517 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.517 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.517 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.517 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.517 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:58.517 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.88716451799999, 2024-09-15T00:13:58.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:13:58.517 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:13:58.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:13:58.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:13:58.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:13:58.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:13:58.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:13:58.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:13:58.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:13:58.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:13:58.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:13:58.521 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.521 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.521 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.521 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.521 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:58.522 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.88714420299999, 2024-09-15T00:13:58.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:13:58.522 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.522 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.522 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.522 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.522 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.522 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.522 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:13:58.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.522 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:58.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:13:58.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:13:58.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:13:58.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:13:58.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:13:58.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:13:58.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:13:58.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:13:58.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:13:58.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:13:58.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:13:58.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:13:58.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:13:58.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:13:58.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:13:58.525 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.526 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.526 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.526 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.526 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:58.526 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.88711942699999, 2024-09-15T00:13:58.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:13:58.526 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.526 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.526 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.526 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.526 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.527 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.527 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:13:58.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.527 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:58.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:58.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:13:58.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:13:58.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:13:58.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:13:58.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:13:58.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:13:58.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:13:58.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:13:58.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:13:58.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:13:58.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:13:58.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:13:58.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:13:58.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:13:58.530 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.530 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.530 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.531 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:58.531 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:58.531 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.70777803999999, 2024-09-15T00:13:58.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:13:58.531 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.531 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.531 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:58.531 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.531 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.531 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.531 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:13:58.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.531 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:58.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:13:58.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:13:58.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:13:58.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:13:58.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:13:58.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:13:58.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:13:58.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:13:58.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:13:58.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:13:58.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:13:58.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:13:58.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:13:58.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:13:58.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:13:58.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:13:58.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:58.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:13:58.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:13:58.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:58.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:13:58.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:13:58.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:58.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:13:58.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:13:58.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:13:58.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:13:58.536 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.536 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.536 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.536 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:13:58.537 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:58.537 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 133.690824588, 2024-09-15T00:13:58.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:13:58.537 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.537 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.537 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.537 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:58.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:13:58.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:13:58.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:13:58.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:13:58.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:13:58.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:13:58.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:13:58.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:13:58.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:13:58.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:13:58.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:13:58.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:13:58.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:13:58.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:13:58.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:13:58.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.540 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.541 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:13:58.541 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:58.541 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.39639368100001, 2024-09-15T00:13:58.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:13:58.541 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.541 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.541 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:58.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.541 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:13:58.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.541 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:58.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:13:58.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:13:58.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:13:58.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:13:58.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:13:58.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:13:58.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:13:58.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:13:58.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:13:58.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:13:58.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:13:58.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:13:58.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:13:58.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:58.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:13:58.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:13:58.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:58.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:13:58.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:13:58.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:58.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:13:58.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:13:58.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:58.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:13:58.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:13:58.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:13:58.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:13:58.546 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.546 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.546 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.547 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:13:58.547 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:58.547 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 120.383937867, 2024-09-15T00:13:58.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:13:58.547 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.547 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.547 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.547 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:58.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:13:58.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:13:58.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:13:58.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:13:58.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:13:58.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:13:58.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:13:58.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:13:58.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:13:58.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:13:58.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.550 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:146935 3.16 3:6c79e092:::rbd_data.11394501364.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 73.743158348999998, 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.058987488999999997, 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 146935 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:58.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:13:58.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753644+0000", 2024-09-15T00:13:58.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6199999999999997e-07 2024-09-15T00:13:58.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753648+0000", 2024-09-15T00:13:58.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-15T00:13:58.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753659+0000", 2024-09-15T00:13:58.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0703000000000001e-05 2024-09-15T00:13:58.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753664+0000", 2024-09-15T00:13:58.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.04e-06 2024-09-15T00:13:58.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753709+0000", 2024-09-15T00:13:58.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4728000000000003e-05 2024-09-15T00:13:58.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753756+0000", 2024-09-15T00:13:58.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7808e-05 2024-09-15T00:13:58.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:58.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.757051+0000", 2024-09-15T00:13:58.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294249 2024-09-15T00:13:58.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:58.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812553+0000", 2024-09-15T00:13:58.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055501807 2024-09-15T00:13:58.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:58.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812564+0000", 2024-09-15T00:13:58.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1467999999999999e-05 2024-09-15T00:13:58.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:58.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812598+0000", 2024-09-15T00:13:58.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3553000000000002e-05 2024-09-15T00:13:58.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812630+0000", 2024-09-15T00:13:58.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2895000000000003e-05 2024-09-15T00:13:58.556 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.556 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.556 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.556 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:146935 3.16 e98/95 ondisk, result = 0)", 2024-09-15T00:13:58.557 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:58.557 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 73.732355652999999, 2024-09-15T00:13:58.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048111830000000001, 2024-09-15T00:13:58.557 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.557 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.557 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.557 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:58.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:13:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764447+0000", 2024-09-15T00:13:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0500000000000002e-07 2024-09-15T00:13:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764450+0000", 2024-09-15T00:13:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4400000000000001e-06 2024-09-15T00:13:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764462+0000", 2024-09-15T00:13:58.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1603000000000001e-05 2024-09-15T00:13:58.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764469+0000", 2024-09-15T00:13:58.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5089999999999996e-06 2024-09-15T00:13:58.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812541+0000", 2024-09-15T00:13:58.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048071585 2024-09-15T00:13:58.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812548+0000", 2024-09-15T00:13:58.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9979999999999997e-06 2024-09-15T00:13:58.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812557+0000", 2024-09-15T00:13:58.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7899999999999994e-06 2024-09-15T00:13:58.560 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.560 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.560 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.560 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:152505 3.17 3:ea3f268c:::rbd_data.113958cb0e55.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:58.560 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:58.561 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.830249691000006, 2024-09-15T00:13:58.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.060817959999999997, 2024-09-15T00:13:58.561 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.561 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.561 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:58.561 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.561 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.561 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.561 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 152505 2024-09-15T00:13:58.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.561 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:58.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:13:58.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666553+0000", 2024-09-15T00:13:58.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-15T00:13:58.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666562+0000", 2024-09-15T00:13:58.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0909999999999999e-06 2024-09-15T00:13:58.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666572+0000", 2024-09-15T00:13:58.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0214000000000001e-05 2024-09-15T00:13:58.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666577+0000", 2024-09-15T00:13:58.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3800000000000002e-06 2024-09-15T00:13:58.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666617+0000", 2024-09-15T00:13:58.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9304000000000002e-05 2024-09-15T00:13:58.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666672+0000", 2024-09-15T00:13:58.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5206e-05 2024-09-15T00:13:58.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:58.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.670821+0000", 2024-09-15T00:13:58.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041493169999999996 2024-09-15T00:13:58.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:58.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727290+0000", 2024-09-15T00:13:58.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.056468580999999997 2024-09-15T00:13:58.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:58.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727302+0000", 2024-09-15T00:13:58.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2605e-05 2024-09-15T00:13:58.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:58.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727337+0000", 2024-09-15T00:13:58.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4783999999999998e-05 2024-09-15T00:13:58.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727370+0000", 2024-09-15T00:13:58.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2366000000000001e-05 2024-09-15T00:13:58.566 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.566 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.566 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.566 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:152505 3.17 e103/68 ondisk, result = 0)", 2024-09-15T00:13:58.566 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:58.567 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.818171254999996, 2024-09-15T00:13:58.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048665938999999998, 2024-09-15T00:13:58.567 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.567 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.567 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.567 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:58.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:13:58.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678631+0000", 2024-09-15T00:13:58.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5099999999999998e-07 2024-09-15T00:13:58.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678634+0000", 2024-09-15T00:13:58.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.478e-06 2024-09-15T00:13:58.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678647+0000", 2024-09-15T00:13:58.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2085e-05 2024-09-15T00:13:58.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678653+0000", 2024-09-15T00:13:58.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2099999999999998e-06 2024-09-15T00:13:58.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727277+0000", 2024-09-15T00:13:58.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048624405000000002 2024-09-15T00:13:58.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727285+0000", 2024-09-15T00:13:58.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2060000000000007e-06 2024-09-15T00:13:58.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727296+0000", 2024-09-15T00:13:58.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0604e-05 2024-09-15T00:13:58.570 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.570 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.570 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.570 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153959 3.c 3:326b8454:::rbd_data.113958cb0e55.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:58.570 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:58.570 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.406943308999999, 2024-09-15T00:13:58.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.068102041000000002, 2024-09-15T00:13:58.571 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.571 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.571 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:58.571 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.571 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.571 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.571 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153959 2024-09-15T00:13:58.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.571 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:58.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:13:58.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089859+0000", 2024-09-15T00:13:58.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1800000000000004e-07 2024-09-15T00:13:58.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089864+0000", 2024-09-15T00:13:58.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2079999999999999e-06 2024-09-15T00:13:58.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089880+0000", 2024-09-15T00:13:58.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5695000000000001e-05 2024-09-15T00:13:58.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089889+0000", 2024-09-15T00:13:58.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.935e-06 2024-09-15T00:13:58.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089942+0000", 2024-09-15T00:13:58.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3219e-05 2024-09-15T00:13:58.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.090066+0000", 2024-09-15T00:13:58.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012385099999999999 2024-09-15T00:13:58.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:13:58.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.095262+0000", 2024-09-15T00:13:58.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051963950000000004 2024-09-15T00:13:58.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:58.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157865+0000", 2024-09-15T00:13:58.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.062602865999999993 2024-09-15T00:13:58.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:58.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157891+0000", 2024-09-15T00:13:58.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5778e-05 2024-09-15T00:13:58.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:58.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157924+0000", 2024-09-15T00:13:58.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3133000000000003e-05 2024-09-15T00:13:58.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157960+0000", 2024-09-15T00:13:58.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6043e-05 2024-09-15T00:13:58.576 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.576 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.576 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.576 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:153959 3.c e103/23 ondisk, result = 0)", 2024-09-15T00:13:58.576 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:58.576 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.394287650999999, 2024-09-15T00:13:58.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055370211000000003, 2024-09-15T00:13:58.576 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.576 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.577 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.577 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:58.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:13:58.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102515+0000", 2024-09-15T00:13:58.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.206e-06 2024-09-15T00:13:58.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102520+0000", 2024-09-15T00:13:58.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1209999999999998e-06 2024-09-15T00:13:58.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102537+0000", 2024-09-15T00:13:58.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:13:58.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102546+0000", 2024-09-15T00:13:58.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2329999999999995e-06 2024-09-15T00:13:58.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157847+0000", 2024-09-15T00:13:58.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055300798999999998 2024-09-15T00:13:58.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157857+0000", 2024-09-15T00:13:58.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.046e-05 2024-09-15T00:13:58.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157884+0000", 2024-09-15T00:13:58.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.671e-05 2024-09-15T00:13:58.580 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.580 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.580 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.580 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163346 3.1d 3:baa43c12:::rbd_data.11395d771336.0000000000000065:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:13:58.580 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:58.581 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.725673078, 2024-09-15T00:13:58.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047558544000000001, 2024-09-15T00:13:58.581 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.581 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.581 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:58.581 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.581 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.581 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.581 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163346 2024-09-15T00:13:58.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.581 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:58.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:13:58.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771130+0000", 2024-09-15T00:13:58.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5939999999999999e-06 2024-09-15T00:13:58.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771141+0000", 2024-09-15T00:13:58.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1001e-05 2024-09-15T00:13:58.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771154+0000", 2024-09-15T00:13:58.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2921e-05 2024-09-15T00:13:58.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771160+0000", 2024-09-15T00:13:58.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3849999999999999e-06 2024-09-15T00:13:58.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771207+0000", 2024-09-15T00:13:58.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6888999999999998e-05 2024-09-15T00:13:58.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771306+0000", 2024-09-15T00:13:58.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9259000000000006e-05 2024-09-15T00:13:58.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:58.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.774692+0000", 2024-09-15T00:13:58.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033854850000000001 2024-09-15T00:13:58.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:58.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818600+0000", 2024-09-15T00:13:58.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043908006999999999 2024-09-15T00:13:58.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:58.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818616+0000", 2024-09-15T00:13:58.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6688e-05 2024-09-15T00:13:58.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:58.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818651+0000", 2024-09-15T00:13:58.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4885000000000002e-05 2024-09-15T00:13:58.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818687+0000", 2024-09-15T00:13:58.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5429999999999998e-05 2024-09-15T00:13:58.586 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.586 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.586 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.586 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163578 3.1c 3:3efd71ab:::rbd_data.11395d771336.0000000000000060:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-15T00:13:58.586 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:58.587 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.326250329000004, 2024-09-15T00:13:58.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.085154081000000006, 2024-09-15T00:13:58.587 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.587 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.587 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:58.587 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:58.587 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:58.587 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:58.587 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163578 2024-09-15T00:13:58.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.587 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:58.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:13:58.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170552+0000", 2024-09-15T00:13:58.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8599999999999997e-07 2024-09-15T00:13:58.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170557+0000", 2024-09-15T00:13:58.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-15T00:13:58.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170567+0000", 2024-09-15T00:13:58.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0628999999999999e-05 2024-09-15T00:13:58.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170573+0000", 2024-09-15T00:13:58.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1649999999999997e-06 2024-09-15T00:13:58.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170619+0000", 2024-09-15T00:13:58.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6405e-05 2024-09-15T00:13:58.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170666+0000", 2024-09-15T00:13:58.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7259000000000003e-05 2024-09-15T00:13:58.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:58.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.175060+0000", 2024-09-15T00:13:58.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043940170000000001 2024-09-15T00:13:58.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:58.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255559+0000", 2024-09-15T00:13:58.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080499065999999994 2024-09-15T00:13:58.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:58.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255616+0000", 2024-09-15T00:13:58.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6422000000000003e-05 2024-09-15T00:13:58.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:58.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255655+0000", 2024-09-15T00:13:58.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9632999999999998e-05 2024-09-15T00:13:58.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255705+0000", 2024-09-15T00:13:58.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.973e-05 2024-09-15T00:13:58.592 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.592 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.592 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.592 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:163578 3.1c e107/23 ondisk, result = 0)", 2024-09-15T00:13:58.592 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:58.592 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.311028238000006, 2024-09-15T00:13:58.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069939984999999996, 2024-09-15T00:13:58.592 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:58.593 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:58.593 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:58.593 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:58.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:58.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:58.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:58.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:13:58.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:58.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:58.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185774+0000", 2024-09-15T00:13:58.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.093e-06 2024-09-15T00:13:58.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:58.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185779+0000", 2024-09-15T00:13:58.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9470000000000002e-06 2024-09-15T00:13:58.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:58.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185794+0000", 2024-09-15T00:13:58.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4450000000000001e-05 2024-09-15T00:13:58.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:58.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185801+0000", 2024-09-15T00:13:58.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9189999999999999e-06 2024-09-15T00:13:58.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:58.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255604+0000", 2024-09-15T00:13:58.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069802201999999994 2024-09-15T00:13:58.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:58.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255611+0000", 2024-09-15T00:13:58.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7740000000000004e-06 2024-09-15T00:13:58.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:58.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:58.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:58.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255713+0000", 2024-09-15T00:13:58.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0001016 2024-09-15T00:13:58.596 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.596 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.596 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.596 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:58.596 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:58.596 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:58.598 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:13:58.598 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:58.596+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:58.698 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:58.697+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:58.741 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:58.741 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:58.749 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:13:58.799 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:13:58.796+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:13:58.852 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:58.852 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:58.853 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:58.853 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:58.861 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:13:58.902 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:58.899+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:59.002 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:59.000+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:59.004 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:59.004 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:13:59.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:13:59.004 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:13:59.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.004 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:13:59.004 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:59.004 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 134.33423245099999, 2024-09-15T00:13:59.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:13:59.004 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.004 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:13:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:13:59.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:13:59.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:13:59.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:13:59.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:13:59.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:13:59.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:13:59.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:13:59.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:13:59.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:13:59.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:13:59.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:13:59.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:13:59.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:59.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:13:59.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:13:59.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:59.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:13:59.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:13:59.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:59.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:13:59.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:13:59.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:59.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:13:59.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:13:59.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:13:59.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:13:59.010 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.010 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.010 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.010 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125618 3.10 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:59.010 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:59.010 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 97.163093368999995, 2024-09-15T00:13:59.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030860942999999998, 2024-09-15T00:13:59.011 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.011 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.011 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:59.011 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.011 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.011 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.011 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125618 2024-09-15T00:13:59.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.011 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:59.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:13:59.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838499+0000", 2024-09-15T00:13:59.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6189999999999998e-06 2024-09-15T00:13:59.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839599+0000", 2024-09-15T00:13:59.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001099764 2024-09-15T00:13:59.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839620+0000", 2024-09-15T00:13:59.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0764e-05 2024-09-15T00:13:59.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839631+0000", 2024-09-15T00:13:59.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1335000000000001e-05 2024-09-15T00:13:59.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839725+0000", 2024-09-15T00:13:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4217999999999998e-05 2024-09-15T00:13:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839910+0000", 2024-09-15T00:13:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000184298 2024-09-15T00:13:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843171+0000", 2024-09-15T00:13:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032613540000000002 2024-09-15T00:13:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869187+0000", 2024-09-15T00:13:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026015467 2024-09-15T00:13:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869240+0000", 2024-09-15T00:13:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3733000000000002e-05 2024-09-15T00:13:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:59.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869309+0000", 2024-09-15T00:13:59.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8381999999999997e-05 2024-09-15T00:13:59.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869358+0000", 2024-09-15T00:13:59.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-15T00:13:59.016 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.016 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.016 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.016 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125619 3.a 3:507b29bf:::rbd_data.11396481e1ec.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:59.017 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:59.017 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 97.161931240000001, 2024-09-15T00:13:59.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045768870000000003, 2024-09-15T00:13:59.017 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.017 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.017 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:59.017 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.017 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.017 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.017 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125619 2024-09-15T00:13:59.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.017 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:59.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:13:59.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839660+0000", 2024-09-15T00:13:59.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.22e-06 2024-09-15T00:13:59.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840511+0000", 2024-09-15T00:13:59.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000851284 2024-09-15T00:13:59.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840522+0000", 2024-09-15T00:13:59.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0569e-05 2024-09-15T00:13:59.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840527+0000", 2024-09-15T00:13:59.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.412e-06 2024-09-15T00:13:59.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840573+0000", 2024-09-15T00:13:59.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5247000000000002e-05 2024-09-15T00:13:59.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840682+0000", 2024-09-15T00:13:59.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109368 2024-09-15T00:13:59.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:59.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843976+0000", 2024-09-15T00:13:59.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294445 2024-09-15T00:13:59.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:59.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870833+0000", 2024-09-15T00:13:59.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026856944000000001 2024-09-15T00:13:59.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:59.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885344+0000", 2024-09-15T00:13:59.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014510744000000001 2024-09-15T00:13:59.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:59.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885382+0000", 2024-09-15T00:13:59.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7821000000000003e-05 2024-09-15T00:13:59.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885428+0000", 2024-09-15T00:13:59.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5816000000000003e-05 2024-09-15T00:13:59.022 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.022 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.022 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.023 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125621 3.a 3:53594db9:::rbd_data.11396481e1ec.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:13:59.023 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:59.023 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 97.148407860999995, 2024-09-15T00:13:59.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032481797, 2024-09-15T00:13:59.023 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.023 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.023 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:59.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.023 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125621 2024-09-15T00:13:59.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.024 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:59.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:13:59.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853185+0000", 2024-09-15T00:13:59.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.43e-06 2024-09-15T00:13:59.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854125+0000", 2024-09-15T00:13:59.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000940539 2024-09-15T00:13:59.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854141+0000", 2024-09-15T00:13:59.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5568000000000001e-05 2024-09-15T00:13:59.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854148+0000", 2024-09-15T00:13:59.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6529999999999994e-06 2024-09-15T00:13:59.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870872+0000", 2024-09-15T00:13:59.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016723135 2024-09-15T00:13:59.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870975+0000", 2024-09-15T00:13:59.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010314099999999999 2024-09-15T00:13:59.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:59.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.873082+0000", 2024-09-15T00:13:59.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021069859999999999 2024-09-15T00:13:59.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:59.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885439+0000", 2024-09-15T00:13:59.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012357343999999999 2024-09-15T00:13:59.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:59.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885569+0000", 2024-09-15T00:13:59.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012995199999999999 2024-09-15T00:13:59.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:59.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885624+0000", 2024-09-15T00:13:59.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5281999999999997e-05 2024-09-15T00:13:59.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885664+0000", 2024-09-15T00:13:59.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9767000000000001e-05 2024-09-15T00:13:59.028 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.028 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.028 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.028 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:147075 3.0 3:06ab39f2:::rbd_data.11394501364.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:13:59.029 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:59.029 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.052509479999998, 2024-09-15T00:13:59.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046402906000000001, 2024-09-15T00:13:59.029 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.029 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.029 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:59.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.029 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 147075 2024-09-15T00:13:59.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.029 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:59.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:59.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:13:59.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4399999999999999e-07 2024-09-15T00:13:59.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949086+0000", 2024-09-15T00:13:59.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.138e-06 2024-09-15T00:13:59.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949096+0000", 2024-09-15T00:13:59.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0438999999999999e-05 2024-09-15T00:13:59.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949101+0000", 2024-09-15T00:13:59.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8999999999999997e-06 2024-09-15T00:13:59.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949145+0000", 2024-09-15T00:13:59.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4141000000000003e-05 2024-09-15T00:13:59.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949195+0000", 2024-09-15T00:13:59.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9657999999999998e-05 2024-09-15T00:13:59.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:13:59.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.950923+0000", 2024-09-15T00:13:59.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001728753 2024-09-15T00:13:59.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:59.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995390+0000", 2024-09-15T00:13:59.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044466335000000003 2024-09-15T00:13:59.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:59.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995401+0000", 2024-09-15T00:13:59.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1671000000000001e-05 2024-09-15T00:13:59.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:59.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995432+0000", 2024-09-15T00:13:59.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0704999999999998e-05 2024-09-15T00:13:59.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995484+0000", 2024-09-15T00:13:59.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1422000000000003e-05 2024-09-15T00:13:59.034 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.034 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.034 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.034 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:147075 3.0 e98/94 ondisk, result = 0)", 2024-09-15T00:13:59.034 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:59.035 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.044709759, 2024-09-15T00:13:59.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038514459000000001, 2024-09-15T00:13:59.035 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.035 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.035 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:59.035 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:59.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:13:59.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956881+0000", 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0599999999999999e-07 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956885+0000", 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3500000000000001e-06 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956897+0000", 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2303999999999999e-05 2024-09-15T00:13:59.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956902+0000", 2024-09-15T00:13:59.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5169999999999999e-06 2024-09-15T00:13:59.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995377+0000", 2024-09-15T00:13:59.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038474544999999999 2024-09-15T00:13:59.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995385+0000", 2024-09-15T00:13:59.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6240000000000002e-06 2024-09-15T00:13:59.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995395+0000", 2024-09-15T00:13:59.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0213e-05 2024-09-15T00:13:59.038 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.038 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.038 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.038 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153172 3.14 3:2a7843e3:::rbd_data.113958cb0e55.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 983040~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:13:59.038 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:59.038 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.223820695000001, 2024-09-15T00:13:59.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.033232668, 2024-09-15T00:13:59.038 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.038 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.039 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:13:59.039 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.039 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.039 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.039 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153172 2024-09-15T00:13:59.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.039 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:59.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:13:59.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777770+0000", 2024-09-15T00:13:59.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.017e-06 2024-09-15T00:13:59.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788009+0000", 2024-09-15T00:13:59.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010238146 2024-09-15T00:13:59.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788024+0000", 2024-09-15T00:13:59.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5582000000000001e-05 2024-09-15T00:13:59.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788048+0000", 2024-09-15T00:13:59.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4173e-05 2024-09-15T00:13:59.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788097+0000", 2024-09-15T00:13:59.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8498999999999998e-05 2024-09-15T00:13:59.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788201+0000", 2024-09-15T00:13:59.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010398799999999999 2024-09-15T00:13:59.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:13:59.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788393+0000", 2024-09-15T00:13:59.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000192337 2024-09-15T00:13:59.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:13:59.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810637+0000", 2024-09-15T00:13:59.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.022244202000000001 2024-09-15T00:13:59.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:13:59.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810903+0000", 2024-09-15T00:13:59.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00026574600000000003 2024-09-15T00:13:59.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:13:59.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810953+0000", 2024-09-15T00:13:59.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9509000000000003e-05 2024-09-15T00:13:59.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.811002+0000", 2024-09-15T00:13:59.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9468999999999997e-05 2024-09-15T00:13:59.044 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.044 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.044 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.044 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170014 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:59.044 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:59.044 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.154525589000002, 2024-09-15T00:13:59.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030291116999999999, 2024-09-15T00:13:59.044 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.044 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.044 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:59.044 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.045 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170014 2024-09-15T00:13:59.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.045 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:59.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:59.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:13:59.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:13:59.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847073+0000", 2024-09-15T00:13:59.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9589999999999995e-06 2024-09-15T00:13:59.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847080+0000", 2024-09-15T00:13:59.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2930000000000003e-06 2024-09-15T00:13:59.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847086+0000", 2024-09-15T00:13:59.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.631e-06 2024-09-15T00:13:59.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877213+0000", 2024-09-15T00:13:59.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030127017999999998 2024-09-15T00:13:59.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877281+0000", 2024-09-15T00:13:59.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7916999999999994e-05 2024-09-15T00:13:59.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877356+0000", 2024-09-15T00:13:59.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4796999999999996e-05 2024-09-15T00:13:59.048 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.048 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.048 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.049 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170017 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1310720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:59.049 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:59.049 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.154496938000001, 2024-09-15T00:13:59.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030437623, 2024-09-15T00:13:59.049 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.049 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.049 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:59.049 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.049 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.049 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.049 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170017 2024-09-15T00:13:59.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.049 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:59.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:13:59.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847094+0000", 2024-09-15T00:13:59.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:13:59.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847096+0000", 2024-09-15T00:13:59.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.136e-06 2024-09-15T00:13:59.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847104+0000", 2024-09-15T00:13:59.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7190000000000001e-06 2024-09-15T00:13:59.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847109+0000", 2024-09-15T00:13:59.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3279999999999996e-06 2024-09-15T00:13:59.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877384+0000", 2024-09-15T00:13:59.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030275348000000001 2024-09-15T00:13:59.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877451+0000", 2024-09-15T00:13:59.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6509999999999998e-05 2024-09-15T00:13:59.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877531+0000", 2024-09-15T00:13:59.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9993000000000005e-05 2024-09-15T00:13:59.053 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.053 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.053 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.053 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170018 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1351680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:59.053 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:59.054 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.154474593000003, 2024-09-15T00:13:59.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030591884999999999, 2024-09-15T00:13:59.054 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.054 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.054 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:59.054 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.054 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.054 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.054 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170018 2024-09-15T00:13:59.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.054 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:59.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:59.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:13:59.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:13:59.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847118+0000", 2024-09-15T00:13:59.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.03e-06 2024-09-15T00:13:59.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847124+0000", 2024-09-15T00:13:59.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:13:59.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847129+0000", 2024-09-15T00:13:59.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8600000000000001e-06 2024-09-15T00:13:59.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877560+0000", 2024-09-15T00:13:59.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030431145999999999 2024-09-15T00:13:59.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877629+0000", 2024-09-15T00:13:59.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8793000000000004e-05 2024-09-15T00:13:59.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877707+0000", 2024-09-15T00:13:59.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8139999999999994e-05 2024-09-15T00:13:59.058 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.058 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.058 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.058 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170019 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1392640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:59.059 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:59.059 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.154453844999999, 2024-09-15T00:13:59.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030769023, 2024-09-15T00:13:59.059 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.059 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.059 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:59.059 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.059 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.059 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.059 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170019 2024-09-15T00:13:59.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.060 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:59.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:13:59.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847137+0000", 2024-09-15T00:13:59.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:13:59.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847139+0000", 2024-09-15T00:13:59.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.142e-06 2024-09-15T00:13:59.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847146+0000", 2024-09-15T00:13:59.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8120000000000002e-06 2024-09-15T00:13:59.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847157+0000", 2024-09-15T00:13:59.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1093e-05 2024-09-15T00:13:59.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877735+0000", 2024-09-15T00:13:59.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578195999999998 2024-09-15T00:13:59.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877810+0000", 2024-09-15T00:13:59.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5563999999999997e-05 2024-09-15T00:13:59.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877905+0000", 2024-09-15T00:13:59.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4809000000000007e-05 2024-09-15T00:13:59.063 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.063 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.063 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.063 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170020 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:59.063 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:59.063 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.154426604999998, 2024-09-15T00:13:59.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030917150000000001, 2024-09-15T00:13:59.064 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.064 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.064 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:59.064 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.064 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.064 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.064 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170020 2024-09-15T00:13:59.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.064 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:59.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:59.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:13:59.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:13:59.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847166+0000", 2024-09-15T00:13:59.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2790000000000001e-06 2024-09-15T00:13:59.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847173+0000", 2024-09-15T00:13:59.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3969999999999999e-06 2024-09-15T00:13:59.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847177+0000", 2024-09-15T00:13:59.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8620000000000002e-06 2024-09-15T00:13:59.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877934+0000", 2024-09-15T00:13:59.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030756143999999999 2024-09-15T00:13:59.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878002+0000", 2024-09-15T00:13:59.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7951e-05 2024-09-15T00:13:59.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878081+0000", 2024-09-15T00:13:59.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9087999999999999e-05 2024-09-15T00:13:59.067 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.067 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.067 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.068 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170094 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:59.068 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:59.068 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.154169355999997, 2024-09-15T00:13:59.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030836142, 2024-09-15T00:13:59.068 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.068 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.068 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:59.068 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.068 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.068 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.068 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170094 2024-09-15T00:13:59.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.069 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:59.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:59.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:13:59.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3499999999999996e-07 2024-09-15T00:13:59.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847424+0000", 2024-09-15T00:13:59.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2850000000000001e-06 2024-09-15T00:13:59.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847430+0000", 2024-09-15T00:13:59.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3260000000000001e-06 2024-09-15T00:13:59.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847435+0000", 2024-09-15T00:13:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.356e-06 2024-09-15T00:13:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878110+0000", 2024-09-15T00:13:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030675021 2024-09-15T00:13:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878177+0000", 2024-09-15T00:13:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6506000000000003e-05 2024-09-15T00:13:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878257+0000", 2024-09-15T00:13:59.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0012999999999994e-05 2024-09-15T00:13:59.072 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.072 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.072 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.072 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170096 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 458752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:59.072 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:59.072 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.1541274, 2024-09-15T00:13:59.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030977030999999999, 2024-09-15T00:13:59.072 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.072 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.072 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:59.073 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.073 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.073 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.073 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170096 2024-09-15T00:13:59.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.073 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:59.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:59.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:13:59.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:13:59.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847465+0000", 2024-09-15T00:13:59.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0559999999999999e-06 2024-09-15T00:13:59.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847472+0000", 2024-09-15T00:13:59.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3130000000000004e-06 2024-09-15T00:13:59.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847477+0000", 2024-09-15T00:13:59.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9930000000000003e-06 2024-09-15T00:13:59.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878288+0000", 2024-09-15T00:13:59.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030811609 2024-09-15T00:13:59.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878358+0000", 2024-09-15T00:13:59.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0215000000000005e-05 2024-09-15T00:13:59.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878440+0000", 2024-09-15T00:13:59.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1366999999999995e-05 2024-09-15T00:13:59.076 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.076 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.076 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.077 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170097 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:59.077 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:59.077 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.154107314999997, 2024-09-15T00:13:59.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031134757999999998, 2024-09-15T00:13:59.077 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.077 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.077 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:59.077 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.077 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.077 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.077 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170097 2024-09-15T00:13:59.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.077 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:59.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:59.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:13:59.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:13:59.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847485+0000", 2024-09-15T00:13:59.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9190000000000002e-06 2024-09-15T00:13:59.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847492+0000", 2024-09-15T00:13:59.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7109999999999998e-06 2024-09-15T00:13:59.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847497+0000", 2024-09-15T00:13:59.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0259999999999998e-06 2024-09-15T00:13:59.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878469+0000", 2024-09-15T00:13:59.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030972115000000001 2024-09-15T00:13:59.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878538+0000", 2024-09-15T00:13:59.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8669000000000003e-05 2024-09-15T00:13:59.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878618+0000", 2024-09-15T00:13:59.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9874000000000001e-05 2024-09-15T00:13:59.081 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.081 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.081 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.081 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170099 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 532480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:59.081 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:59.081 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.154055753000002, 2024-09-15T00:13:59.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031262183999999998, 2024-09-15T00:13:59.081 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.082 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.082 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:59.082 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.082 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.082 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.082 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170099 2024-09-15T00:13:59.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.082 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:59.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:13:59.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847535+0000", 2024-09-15T00:13:59.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:13:59.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847537+0000", 2024-09-15T00:13:59.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:13:59.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847544+0000", 2024-09-15T00:13:59.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9630000000000001e-06 2024-09-15T00:13:59.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847549+0000", 2024-09-15T00:13:59.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8910000000000003e-06 2024-09-15T00:13:59.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878646+0000", 2024-09-15T00:13:59.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031097400000000001 2024-09-15T00:13:59.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878717+0000", 2024-09-15T00:13:59.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0993000000000003e-05 2024-09-15T00:13:59.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878797+0000", 2024-09-15T00:13:59.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9425000000000006e-05 2024-09-15T00:13:59.085 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.085 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.085 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.086 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170101 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:59.086 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:59.086 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.154014373999999, 2024-09-15T00:13:59.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031674568, 2024-09-15T00:13:59.086 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.086 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.086 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:59.086 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.086 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.086 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.086 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170101 2024-09-15T00:13:59.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.086 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:59.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:59.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:13:59.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:13:59.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847578+0000", 2024-09-15T00:13:59.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0389999999999999e-06 2024-09-15T00:13:59.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847585+0000", 2024-09-15T00:13:59.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.336e-06 2024-09-15T00:13:59.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847596+0000", 2024-09-15T00:13:59.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015999999999999e-05 2024-09-15T00:13:59.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878825+0000", 2024-09-15T00:13:59.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031229848000000001 2024-09-15T00:13:59.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878899+0000", 2024-09-15T00:13:59.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:13:59.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879250+0000", 2024-09-15T00:13:59.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035110000000000002 2024-09-15T00:13:59.089 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170102 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 614400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.153987076999996, 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031577371999999999, 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170102 2024-09-15T00:13:59.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.091 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:59.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:59.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:13:59.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:13:59.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847610+0000", 2024-09-15T00:13:59.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6329999999999999e-06 2024-09-15T00:13:59.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847618+0000", 2024-09-15T00:13:59.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4599999999999997e-06 2024-09-15T00:13:59.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847623+0000", 2024-09-15T00:13:59.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2909999999999998e-06 2024-09-15T00:13:59.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879016+0000", 2024-09-15T00:13:59.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031393227000000003 2024-09-15T00:13:59.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879091+0000", 2024-09-15T00:13:59.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5384999999999997e-05 2024-09-15T00:13:59.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879180+0000", 2024-09-15T00:13:59.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8941000000000001e-05 2024-09-15T00:13:59.094 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.094 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.094 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.094 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170132 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:59.094 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:59.094 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.153939696999998, 2024-09-15T00:13:59.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031722579000000001, 2024-09-15T00:13:59.094 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.094 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.095 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:59.095 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.095 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.095 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.095 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170132 2024-09-15T00:13:59.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.095 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:59.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:13:59.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847651+0000", 2024-09-15T00:13:59.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1900000000000003e-07 2024-09-15T00:13:59.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847653+0000", 2024-09-15T00:13:59.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0329999999999998e-06 2024-09-15T00:13:59.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847659+0000", 2024-09-15T00:13:59.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4169999999999997e-06 2024-09-15T00:13:59.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847670+0000", 2024-09-15T00:13:59.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1031e-05 2024-09-15T00:13:59.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879205+0000", 2024-09-15T00:13:59.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031534982000000003 2024-09-15T00:13:59.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879286+0000", 2024-09-15T00:13:59.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0675999999999998e-05 2024-09-15T00:13:59.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879373+0000", 2024-09-15T00:13:59.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6921000000000003e-05 2024-09-15T00:13:59.098 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.098 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.098 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.098 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170133 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:13:59.099 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:59.099 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 61.153912544000001, 2024-09-15T00:13:59.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031894817999999998, 2024-09-15T00:13:59.099 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:13:59.099 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:13:59.099 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:13:59.099 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:13:59.099 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:13:59.099 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:13:59.099 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170133 2024-09-15T00:13:59.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.099 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:13:59.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:13:59.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:59.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:13:59.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:13:59.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:13:59.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:13:59.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853540+0000", 2024-09-15T00:13:59.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058619409999999999 2024-09-15T00:13:59.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:13:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853545+0000", 2024-09-15T00:13:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3430000000000003e-06 2024-09-15T00:13:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:13:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853554+0000", 2024-09-15T00:13:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4069999999999999e-06 2024-09-15T00:13:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:13:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853561+0000", 2024-09-15T00:13:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5529999999999998e-06 2024-09-15T00:13:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:13:59.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879399+0000", 2024-09-15T00:13:59.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025837941 2024-09-15T00:13:59.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:13:59.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879477+0000", 2024-09-15T00:13:59.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8367000000000004e-05 2024-09-15T00:13:59.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:13:59.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:13:59.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:13:59.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879572+0000", 2024-09-15T00:13:59.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5266000000000006e-05 2024-09-15T00:13:59.102 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.102 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.103 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.103 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:13:59.103 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:13:59.103 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:59.103 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:13:59.105 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:13:59.105 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:59.102+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:59.205 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:13:59.203+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:13:59.246 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:13:59.260 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:59.260 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:59.260 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:13:59.260 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:59.270 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:13:59.306 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:13:59.304+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:13:59.376 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:13:59.376 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:13:59.395 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:13:59.395 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:13:59.406 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:59.404+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:59.412 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:13:59.412 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:13:59.423 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:13:59.423 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:13:59.424 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:13:59.424 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:13:59.424 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:13:59.431 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:13:59.431 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:13:59.433 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:13:59.449 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:13:59.449 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:13:59.467 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:13:59.468 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:13:59.486 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:13:59.487 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:13:59.506 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:13:59.506 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:13:59.507 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:13:59.506+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:13:59.592 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:59.592 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:59.602 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:13:59.645 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:13:59.644+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:13:59.710 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:13:59.707+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:13:59.724 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:59.725 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:59.734 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:13:59.819 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:59.818+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:13:59.889 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:13:59.890 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:13:59.899 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:13:59.939 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:13:59.938+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:14:00.013 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:14:00.012+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:14:00.020 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is unset 2024-09-15T00:14:00.051 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:00.051 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:00.060 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:14:00.114 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:14:00.112+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:14:00.204 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:00.204 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:00.213 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:14:00.215 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:14:00.214+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:14:00.315 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:14:00.314+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:14:00.317 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:00.317 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:00.326 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:14:00.416 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:14:00.415+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:14:00.470 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:00.470 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:00.479 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:14:00.517 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:14:00.515+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:14:00.583 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:00.583 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:00.592 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:14:00.618 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:14:00.616+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:14:00.696 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:00.696 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:00.705 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:14:00.748 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:14:00.747+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:14:00.809 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:00.809 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:00.818 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:14:00.819 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:14:00.817+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:14:00.920 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:14:00.919+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:14:00.962 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:00.962 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:00.971 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:14:01.021 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:14:01.019+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:14:01.121 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:14:01.120+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:14:01.132 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:01.132 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:01.143 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:14:01.222 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:14:01.220+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:14:01.248 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:01.248 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:01.257 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:14:01.323 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:14:01.322+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:14:01.362 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:01.362 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:01.371 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:14:01.424 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:14:01.422+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:14:01.475 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:01.475 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:01.484 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:14:01.519 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:14:01.525 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:14:01.523+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:14:01.625 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:14:01.624+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:14:01.627 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:01.627 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:01.636 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:14:01.641 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:14:01.641 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:14:01.658 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:14:01.658 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:14:01.674 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:14:01.674 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:14:01.690 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:14:01.690 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:14:01.706 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:14:01.706 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:14:01.723 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:14:01.723 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:14:01.726 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:14:01.725+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:14:01.739 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:14:01.740 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:14:01.740 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:14:01.740 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:14:01.740 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:14:01.740 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:14:01.749 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:14:01.756 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:14:01.756 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:14:01.827 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:14:01.826+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:14:01.851 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:14:01.851 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:14:01.851 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:14:01.851 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:14:01.851 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:14:01.859 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:14:01.929 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:14:01.927+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:14:02.004 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:14:02.005 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:14:02.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:14:02.005 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:14:02.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.005 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48973 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 3932160~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:02.005 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:14:02.005 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 137.39259696400001, 2024-09-15T00:14:02.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046641249000000003, 2024-09-15T00:14:02.005 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.005 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.005 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.005 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.005 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.005 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.006 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48973 2024-09-15T00:14:02.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.006 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:14:02.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:14:02.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609547+0000", 2024-09-15T00:14:02.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000003e-07 2024-09-15T00:14:02.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609550+0000", 2024-09-15T00:14:02.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:14:02.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609557+0000", 2024-09-15T00:14:02.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9480000000000002e-06 2024-09-15T00:14:02.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609562+0000", 2024-09-15T00:14:02.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999999e-06 2024-09-15T00:14:02.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655700+0000", 2024-09-15T00:14:02.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046137592999999998 2024-09-15T00:14:02.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.655758+0000", 2024-09-15T00:14:02.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8621e-05 2024-09-15T00:14:02.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656188+0000", 2024-09-15T00:14:02.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00043010200000000002 2024-09-15T00:14:02.009 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.009 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.009 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.009 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48974 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:02.009 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:14:02.009 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 137.39257573500001, 2024-09-15T00:14:02.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046765092000000001, 2024-09-15T00:14:02.010 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.010 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.010 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.010 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48974 2024-09-15T00:14:02.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.010 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:14:02.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609568+0000", 2024-09-15T00:14:02.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609569+0000", 2024-09-15T00:14:02.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:14:02.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609571+0000", 2024-09-15T00:14:02.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:14:02.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609580+0000", 2024-09-15T00:14:02.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7419999999999992e-06 2024-09-15T00:14:02.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609586+0000", 2024-09-15T00:14:02.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:14:02.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656221+0000", 2024-09-15T00:14:02.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046634775000000003 2024-09-15T00:14:02.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656269+0000", 2024-09-15T00:14:02.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7568000000000003e-05 2024-09-15T00:14:02.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656333+0000", 2024-09-15T00:14:02.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4719000000000002e-05 2024-09-15T00:14:02.013 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.013 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.013 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.014 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48975 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:02.014 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:14:02.014 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 137.39255019399999, 2024-09-15T00:14:02.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046920338999999998, 2024-09-15T00:14:02.014 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.014 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.014 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.014 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.014 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.014 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.014 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48975 2024-09-15T00:14:02.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.015 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:14:02.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:14:02.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609594+0000", 2024-09-15T00:14:02.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:14:02.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609597+0000", 2024-09-15T00:14:02.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2689999999999998e-06 2024-09-15T00:14:02.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609604+0000", 2024-09-15T00:14:02.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2860000000000002e-06 2024-09-15T00:14:02.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609610+0000", 2024-09-15T00:14:02.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:14:02.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656371+0000", 2024-09-15T00:14:02.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046761716000000002 2024-09-15T00:14:02.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656453+0000", 2024-09-15T00:14:02.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2064000000000006e-05 2024-09-15T00:14:02.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656514+0000", 2024-09-15T00:14:02.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0838999999999998e-05 2024-09-15T00:14:02.018 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.018 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.018 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.018 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48976 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:02.018 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:14:02.018 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 137.39252771899999, 2024-09-15T00:14:02.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.04711916, 2024-09-15T00:14:02.018 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.018 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.018 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.019 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.019 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.019 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.019 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48976 2024-09-15T00:14:02.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.019 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:14:02.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609616+0000", 2024-09-15T00:14:02.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609617+0000", 2024-09-15T00:14:02.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:14:02.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609619+0000", 2024-09-15T00:14:02.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0849999999999999e-06 2024-09-15T00:14:02.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609625+0000", 2024-09-15T00:14:02.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.669e-06 2024-09-15T00:14:02.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609631+0000", 2024-09-15T00:14:02.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1100000000000002e-06 2024-09-15T00:14:02.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656549+0000", 2024-09-15T00:14:02.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046918814000000003 2024-09-15T00:14:02.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656632+0000", 2024-09-15T00:14:02.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2120000000000007e-05 2024-09-15T00:14:02.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656735+0000", 2024-09-15T00:14:02.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010394 2024-09-15T00:14:02.022 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.022 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.022 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.023 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48977 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:02.023 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:14:02.023 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 137.392507209, 2024-09-15T00:14:02.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047265556, 2024-09-15T00:14:02.023 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.023 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.023 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.023 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48977 2024-09-15T00:14:02.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.024 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:14:02.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:14:02.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609637+0000", 2024-09-15T00:14:02.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.53e-07 2024-09-15T00:14:02.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609639+0000", 2024-09-15T00:14:02.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9889999999999999e-06 2024-09-15T00:14:02.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609646+0000", 2024-09-15T00:14:02.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5830000000000003e-06 2024-09-15T00:14:02.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609651+0000", 2024-09-15T00:14:02.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0860000000000001e-06 2024-09-15T00:14:02.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656767+0000", 2024-09-15T00:14:02.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047116036 2024-09-15T00:14:02.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656819+0000", 2024-09-15T00:14:02.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1731999999999999e-05 2024-09-15T00:14:02.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656902+0000", 2024-09-15T00:14:02.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3776999999999994e-05 2024-09-15T00:14:02.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.027 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.027 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48978 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:02.027 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:14:02.027 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 137.392486894, 2024-09-15T00:14:02.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047387617, 2024-09-15T00:14:02.028 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.028 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.028 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.028 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48978 2024-09-15T00:14:02.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.028 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:14:02.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609657+0000", 2024-09-15T00:14:02.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609658+0000", 2024-09-15T00:14:02.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:14:02.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609664+0000", 2024-09-15T00:14:02.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1589999999999998e-06 2024-09-15T00:14:02.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609671+0000", 2024-09-15T00:14:02.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9099999999999999e-06 2024-09-15T00:14:02.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609675+0000", 2024-09-15T00:14:02.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8060000000000002e-06 2024-09-15T00:14:02.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656936+0000", 2024-09-15T00:14:02.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047260785999999999 2024-09-15T00:14:02.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.656985+0000", 2024-09-15T00:14:02.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8497e-05 2024-09-15T00:14:02.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657045+0000", 2024-09-15T00:14:02.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0059999999999998e-05 2024-09-15T00:14:02.031 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.031 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.032 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.032 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48979 3.17 3:edca63c2:::rbd_data.11395b9829f3.0000000000000005:e [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:02.032 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:14:02.032 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 137.392462118, 2024-09-15T00:14:02.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047491152000000002, 2024-09-15T00:14:02.032 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.032 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.032 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.033 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48979 2024-09-15T00:14:02.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.033 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:14:02.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:14:02.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609682+0000", 2024-09-15T00:14:02.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:14:02.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609684+0000", 2024-09-15T00:14:02.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9580000000000001e-06 2024-09-15T00:14:02.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609691+0000", 2024-09-15T00:14:02.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4590000000000003e-06 2024-09-15T00:14:02.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.609697+0000", 2024-09-15T00:14:02.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8000000000000004e-06 2024-09-15T00:14:02.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657075+0000", 2024-09-15T00:14:02.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047378614999999999 2024-09-15T00:14:02.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657122+0000", 2024-09-15T00:14:02.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6706999999999999e-05 2024-09-15T00:14:02.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.657173+0000", 2024-09-15T00:14:02.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1172999999999999e-05 2024-09-15T00:14:02.036 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.036 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.036 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.036 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49121 3.16 3:690493bc:::rbd_data.113916cf513.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:02.036 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:14:02.037 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 137.213120731, 2024-09-15T00:14:02.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.095753142999999999, 2024-09-15T00:14:02.037 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.037 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.037 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:02.037 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.037 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.037 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.037 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49121 2024-09-15T00:14:02.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.038 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:14:02.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789023+0000", 2024-09-15T00:14:02.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789024+0000", 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9899999999999999e-07 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789029+0000", 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6670000000000004e-06 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789040+0000", 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1229e-05 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789045+0000", 2024-09-15T00:14:02.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2229999999999998e-06 2024-09-15T00:14:02.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789088+0000", 2024-09-15T00:14:02.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799999999999997e-05 2024-09-15T00:14:02.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.789137+0000", 2024-09-15T00:14:02.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8807999999999998e-05 2024-09-15T00:14:02.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-15T00:14:02.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.793267+0000", 2024-09-15T00:14:02.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041302630000000003 2024-09-15T00:14:02.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:02.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884622+0000", 2024-09-15T00:14:02.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.091354867000000006 2024-09-15T00:14:02.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:02.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884689+0000", 2024-09-15T00:14:02.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7080000000000001e-05 2024-09-15T00:14:02.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:02.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884730+0000", 2024-09-15T00:14:02.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0488000000000003e-05 2024-09-15T00:14:02.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884776+0000", 2024-09-15T00:14:02.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6819e-05 2024-09-15T00:14:02.042 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.042 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.042 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.042 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:49121 3.16 e47/23 ondisk, result = 0)", 2024-09-15T00:14:02.042 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:14:02.043 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 137.19616727900001, 2024-09-15T00:14:02.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078807507999999998, 2024-09-15T00:14:02.043 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.043 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.043 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.043 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:14:02.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805977+0000", 2024-09-15T00:14:02.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805978+0000", 2024-09-15T00:14:02.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.001e-06 2024-09-15T00:14:02.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805982+0000", 2024-09-15T00:14:02.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1749999999999998e-06 2024-09-15T00:14:02.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.805994+0000", 2024-09-15T00:14:02.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1691999999999999e-05 2024-09-15T00:14:02.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.806000+0000", 2024-09-15T00:14:02.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2910000000000004e-06 2024-09-15T00:14:02.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884646+0000", 2024-09-15T00:14:02.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078646463 2024-09-15T00:14:02.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884653+0000", 2024-09-15T00:14:02.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5130000000000002e-06 2024-09-15T00:14:02.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.884784+0000", 2024-09-15T00:14:02.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000131373 2024-09-15T00:14:02.046 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.046 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.046 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.046 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:80713 3.1d 3:bf7fc0c9:::rbd_data.113947ee3a89.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:14:02.046 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:14:02.047 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.901736372, 2024-09-15T00:14:02.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.078747142000000006, 2024-09-15T00:14:02.047 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.047 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.047 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:02.047 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.047 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.047 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.047 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 80713 2024-09-15T00:14:02.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.047 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:14:02.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100408+0000", 2024-09-15T00:14:02.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100409+0000", 2024-09-15T00:14:02.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6599999999999994e-07 2024-09-15T00:14:02.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100414+0000", 2024-09-15T00:14:02.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:14:02.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100427+0000", 2024-09-15T00:14:02.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.376e-05 2024-09-15T00:14:02.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100452+0000", 2024-09-15T00:14:02.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4973999999999998e-05 2024-09-15T00:14:02.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100530+0000", 2024-09-15T00:14:02.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7422999999999994e-05 2024-09-15T00:14:02.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.100614+0000", 2024-09-15T00:14:02.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4192999999999998e-05 2024-09-15T00:14:02.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:02.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.104694+0000", 2024-09-15T00:14:02.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040796720000000003 2024-09-15T00:14:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179075+0000", 2024-09-15T00:14:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.074381214000000001 2024-09-15T00:14:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179087+0000", 2024-09-15T00:14:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2238e-05 2024-09-15T00:14:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179120+0000", 2024-09-15T00:14:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3476999999999998e-05 2024-09-15T00:14:02.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179155+0000", 2024-09-15T00:14:02.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.43e-05 2024-09-15T00:14:02.052 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.052 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.052 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.052 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:80713 3.1d e58/23 ondisk, result = 0)", 2024-09-15T00:14:02.052 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:14:02.052 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 123.889280558, 2024-09-15T00:14:02.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0662166, 2024-09-15T00:14:02.053 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.053 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.053 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.053 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:14:02.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112863+0000", 2024-09-15T00:14:02.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112864+0000", 2024-09-15T00:14:02.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.043e-06 2024-09-15T00:14:02.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112869+0000", 2024-09-15T00:14:02.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9080000000000003e-06 2024-09-15T00:14:02.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112887+0000", 2024-09-15T00:14:02.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7261e-05 2024-09-15T00:14:02.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.112893+0000", 2024-09-15T00:14:02.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5980000000000002e-06 2024-09-15T00:14:02.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179062+0000", 2024-09-15T00:14:02.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.066168785999999993 2024-09-15T00:14:02.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179070+0000", 2024-09-15T00:14:02.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0129999999999993e-06 2024-09-15T00:14:02.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:58.179080+0000", 2024-09-15T00:14:02.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-15T00:14:02.056 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.056 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.056 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.056 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:146935 3.16 3:6c79e092:::rbd_data.11394501364.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:14:02.056 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:14:02.057 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 77.248501039999994, 2024-09-15T00:14:02.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.058987488999999997, 2024-09-15T00:14:02.057 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.057 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.057 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:02.057 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.057 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.057 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.057 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 146935 2024-09-15T00:14:02.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.057 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:14:02.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753643+0000", 2024-09-15T00:14:02.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753644+0000", 2024-09-15T00:14:02.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6199999999999997e-07 2024-09-15T00:14:02.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753648+0000", 2024-09-15T00:14:02.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-15T00:14:02.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753659+0000", 2024-09-15T00:14:02.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0703000000000001e-05 2024-09-15T00:14:02.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753664+0000", 2024-09-15T00:14:02.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.04e-06 2024-09-15T00:14:02.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753709+0000", 2024-09-15T00:14:02.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4728000000000003e-05 2024-09-15T00:14:02.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.753756+0000", 2024-09-15T00:14:02.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7808e-05 2024-09-15T00:14:02.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:14:02.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.757051+0000", 2024-09-15T00:14:02.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294249 2024-09-15T00:14:02.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:02.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812553+0000", 2024-09-15T00:14:02.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055501807 2024-09-15T00:14:02.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:02.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812564+0000", 2024-09-15T00:14:02.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1467999999999999e-05 2024-09-15T00:14:02.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:02.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812598+0000", 2024-09-15T00:14:02.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3553000000000002e-05 2024-09-15T00:14:02.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812630+0000", 2024-09-15T00:14:02.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2895000000000003e-05 2024-09-15T00:14:02.062 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.062 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.062 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.062 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:146935 3.16 e98/95 ondisk, result = 0)", 2024-09-15T00:14:02.062 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:14:02.062 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 77.237698343999995, 2024-09-15T00:14:02.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048111830000000001, 2024-09-15T00:14:02.062 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.062 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.062 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.063 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:14:02.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764446+0000", 2024-09-15T00:14:02.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764447+0000", 2024-09-15T00:14:02.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0500000000000002e-07 2024-09-15T00:14:02.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764450+0000", 2024-09-15T00:14:02.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4400000000000001e-06 2024-09-15T00:14:02.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764462+0000", 2024-09-15T00:14:02.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1603000000000001e-05 2024-09-15T00:14:02.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.764469+0000", 2024-09-15T00:14:02.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5089999999999996e-06 2024-09-15T00:14:02.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812541+0000", 2024-09-15T00:14:02.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048071585 2024-09-15T00:14:02.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812548+0000", 2024-09-15T00:14:02.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9979999999999997e-06 2024-09-15T00:14:02.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.812557+0000", 2024-09-15T00:14:02.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7899999999999994e-06 2024-09-15T00:14:02.066 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.066 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.066 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.066 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:152505 3.17 3:ea3f268c:::rbd_data.113958cb0e55.0000000000000014:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:14:02.066 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:14:02.066 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 72.335592382000002, 2024-09-15T00:14:02.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.060817959999999997, 2024-09-15T00:14:02.066 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.066 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.066 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:02.067 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.067 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.067 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.067 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 152505 2024-09-15T00:14:02.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.067 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:14:02.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666552+0000", 2024-09-15T00:14:02.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666553+0000", 2024-09-15T00:14:02.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-15T00:14:02.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666562+0000", 2024-09-15T00:14:02.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0909999999999999e-06 2024-09-15T00:14:02.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666572+0000", 2024-09-15T00:14:02.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0214000000000001e-05 2024-09-15T00:14:02.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666577+0000", 2024-09-15T00:14:02.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3800000000000002e-06 2024-09-15T00:14:02.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666617+0000", 2024-09-15T00:14:02.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9304000000000002e-05 2024-09-15T00:14:02.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.666672+0000", 2024-09-15T00:14:02.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5206e-05 2024-09-15T00:14:02.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:14:02.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.670821+0000", 2024-09-15T00:14:02.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041493169999999996 2024-09-15T00:14:02.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:02.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727290+0000", 2024-09-15T00:14:02.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.056468580999999997 2024-09-15T00:14:02.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:02.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727302+0000", 2024-09-15T00:14:02.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2605e-05 2024-09-15T00:14:02.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:02.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727337+0000", 2024-09-15T00:14:02.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4783999999999998e-05 2024-09-15T00:14:02.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727370+0000", 2024-09-15T00:14:02.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2366000000000001e-05 2024-09-15T00:14:02.072 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.072 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.072 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.072 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:152505 3.17 e103/68 ondisk, result = 0)", 2024-09-15T00:14:02.072 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:14:02.073 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 72.323513946000006, 2024-09-15T00:14:02.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048665938999999998, 2024-09-15T00:14:02.073 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.073 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.073 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.073 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:14:02.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678630+0000", 2024-09-15T00:14:02.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678631+0000", 2024-09-15T00:14:02.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5099999999999998e-07 2024-09-15T00:14:02.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678634+0000", 2024-09-15T00:14:02.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.478e-06 2024-09-15T00:14:02.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678647+0000", 2024-09-15T00:14:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2085e-05 2024-09-15T00:14:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.678653+0000", 2024-09-15T00:14:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2099999999999998e-06 2024-09-15T00:14:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727277+0000", 2024-09-15T00:14:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048624405000000002 2024-09-15T00:14:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727285+0000", 2024-09-15T00:14:02.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2060000000000007e-06 2024-09-15T00:14:02.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.727296+0000", 2024-09-15T00:14:02.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0604e-05 2024-09-15T00:14:02.076 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.076 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.076 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.076 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153959 3.c 3:326b8454:::rbd_data.113958cb0e55.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:14:02.077 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:14:02.077 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 71.912285999999995, 2024-09-15T00:14:02.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.068102041000000002, 2024-09-15T00:14:02.077 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.077 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.077 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:02.077 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.077 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.077 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.077 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153959 2024-09-15T00:14:02.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.077 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:14:02.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089858+0000", 2024-09-15T00:14:02.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089859+0000", 2024-09-15T00:14:02.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1800000000000004e-07 2024-09-15T00:14:02.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089864+0000", 2024-09-15T00:14:02.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2079999999999999e-06 2024-09-15T00:14:02.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089880+0000", 2024-09-15T00:14:02.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5695000000000001e-05 2024-09-15T00:14:02.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089889+0000", 2024-09-15T00:14:02.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.935e-06 2024-09-15T00:14:02.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.089942+0000", 2024-09-15T00:14:02.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3219e-05 2024-09-15T00:14:02.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.090066+0000", 2024-09-15T00:14:02.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012385099999999999 2024-09-15T00:14:02.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:14:02.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.095262+0000", 2024-09-15T00:14:02.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051963950000000004 2024-09-15T00:14:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157865+0000", 2024-09-15T00:14:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.062602865999999993 2024-09-15T00:14:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157891+0000", 2024-09-15T00:14:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5778e-05 2024-09-15T00:14:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157924+0000", 2024-09-15T00:14:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3133000000000003e-05 2024-09-15T00:14:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157960+0000", 2024-09-15T00:14:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6043e-05 2024-09-15T00:14:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:153959 3.c e103/23 ondisk, result = 0)", 2024-09-15T00:14:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:14:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 71.899630341999995, 2024-09-15T00:14:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055370211000000003, 2024-09-15T00:14:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.083 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.083 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.083 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:14:02.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102514+0000", 2024-09-15T00:14:02.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102515+0000", 2024-09-15T00:14:02.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.206e-06 2024-09-15T00:14:02.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102520+0000", 2024-09-15T00:14:02.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1209999999999998e-06 2024-09-15T00:14:02.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102537+0000", 2024-09-15T00:14:02.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:14:02.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.102546+0000", 2024-09-15T00:14:02.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2329999999999995e-06 2024-09-15T00:14:02.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157847+0000", 2024-09-15T00:14:02.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.055300798999999998 2024-09-15T00:14:02.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157857+0000", 2024-09-15T00:14:02.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.046e-05 2024-09-15T00:14:02.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:50.157884+0000", 2024-09-15T00:14:02.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.671e-05 2024-09-15T00:14:02.086 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.086 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.086 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.086 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163346 3.1d 3:baa43c12:::rbd_data.11395d771336.0000000000000065:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-15T00:14:02.086 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:14:02.086 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 68.231015768999995, 2024-09-15T00:14:02.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.047558544000000001, 2024-09-15T00:14:02.086 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.086 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.087 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:02.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.087 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163346 2024-09-15T00:14:02.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.087 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:14:02.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771128+0000", 2024-09-15T00:14:02.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771130+0000", 2024-09-15T00:14:02.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5939999999999999e-06 2024-09-15T00:14:02.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771141+0000", 2024-09-15T00:14:02.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1001e-05 2024-09-15T00:14:02.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771154+0000", 2024-09-15T00:14:02.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2921e-05 2024-09-15T00:14:02.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771160+0000", 2024-09-15T00:14:02.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3849999999999999e-06 2024-09-15T00:14:02.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771207+0000", 2024-09-15T00:14:02.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6888999999999998e-05 2024-09-15T00:14:02.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.771306+0000", 2024-09-15T00:14:02.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9259000000000006e-05 2024-09-15T00:14:02.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:02.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.774692+0000", 2024-09-15T00:14:02.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033854850000000001 2024-09-15T00:14:02.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:02.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818600+0000", 2024-09-15T00:14:02.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043908006999999999 2024-09-15T00:14:02.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:02.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818616+0000", 2024-09-15T00:14:02.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6688e-05 2024-09-15T00:14:02.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:02.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818651+0000", 2024-09-15T00:14:02.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4885000000000002e-05 2024-09-15T00:14:02.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:53.818687+0000", 2024-09-15T00:14:02.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5429999999999998e-05 2024-09-15T00:14:02.092 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.092 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.092 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.092 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:163578 3.1c 3:3efd71ab:::rbd_data.11395d771336.0000000000000060:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-15T00:14:02.092 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:14:02.092 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 67.83159302, 2024-09-15T00:14:02.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.085154081000000006, 2024-09-15T00:14:02.092 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.092 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.092 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:02.093 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.093 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.093 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.093 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 163578 2024-09-15T00:14:02.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.093 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:14:02.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170551+0000", 2024-09-15T00:14:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170552+0000", 2024-09-15T00:14:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8599999999999997e-07 2024-09-15T00:14:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170557+0000", 2024-09-15T00:14:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-15T00:14:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170567+0000", 2024-09-15T00:14:02.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0628999999999999e-05 2024-09-15T00:14:02.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170573+0000", 2024-09-15T00:14:02.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1649999999999997e-06 2024-09-15T00:14:02.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170619+0000", 2024-09-15T00:14:02.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6405e-05 2024-09-15T00:14:02.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.170666+0000", 2024-09-15T00:14:02.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7259000000000003e-05 2024-09-15T00:14:02.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:02.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.175060+0000", 2024-09-15T00:14:02.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043940170000000001 2024-09-15T00:14:02.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:02.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255559+0000", 2024-09-15T00:14:02.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080499065999999994 2024-09-15T00:14:02.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:02.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255616+0000", 2024-09-15T00:14:02.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6422000000000003e-05 2024-09-15T00:14:02.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:02.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255655+0000", 2024-09-15T00:14:02.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9632999999999998e-05 2024-09-15T00:14:02.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255705+0000", 2024-09-15T00:14:02.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.973e-05 2024-09-15T00:14:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:163578 3.1c e107/23 ondisk, result = 0)", 2024-09-15T00:14:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:14:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 67.816370929000001, 2024-09-15T00:14:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069939984999999996, 2024-09-15T00:14:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:14:02.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185773+0000", 2024-09-15T00:14:02.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185774+0000", 2024-09-15T00:14:02.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.093e-06 2024-09-15T00:14:02.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185779+0000", 2024-09-15T00:14:02.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9470000000000002e-06 2024-09-15T00:14:02.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185794+0000", 2024-09-15T00:14:02.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4450000000000001e-05 2024-09-15T00:14:02.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.185801+0000", 2024-09-15T00:14:02.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9189999999999999e-06 2024-09-15T00:14:02.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255604+0000", 2024-09-15T00:14:02.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.069802201999999994 2024-09-15T00:14:02.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255611+0000", 2024-09-15T00:14:02.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7740000000000004e-06 2024-09-15T00:14:02.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:54.255713+0000", 2024-09-15T00:14:02.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0001016 2024-09-15T00:14:02.102 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.102 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.102 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.102 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.102 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.102 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:14:02.102 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:14:02.103 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 0, 7, 2] out_osds: [1] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-15T00:14:02.103 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:14:02.103 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-15T00:14:02.103 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T00:14:02.103 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T00:14:02.104 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-15T00:14:02.105 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:14:02.106 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:14:02.101+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:14:02.204 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:14:02.202+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:14:02.251 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:14:02.251 INFO:teuthology.orchestra.run.smithi184.stdout: "success": "" 2024-09-15T00:14:02.251 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:14:02.252 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:14:02.252 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:14:02.252 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:14:02.252 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:14:02.262 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:14:02.304 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:14:02.303+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:14:02.405 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:14:02.403+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:14:02.406 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:14:02.406 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:14:02.406 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:14:02.406 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:14:02.406 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:14:02.415 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:14:02.505 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:14:02.504+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:14:02.560 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:14:02.560 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:14:02.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:14:02.560 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:14:02.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.560 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49032 3.a 3:576c9cba:::rbd_data.113916cf513.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2228224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:02.560 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:14:02.560 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 137.89014347899999, 2024-09-15T00:14:02.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043779850000000002, 2024-09-15T00:14:02.561 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.561 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.561 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:02.561 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.561 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.561 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.561 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49032 2024-09-15T00:14:02.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.561 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:14:02.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667358+0000", 2024-09-15T00:14:02.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.667360+0000", 2024-09-15T00:14:02.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-15T00:14:02.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674977+0000", 2024-09-15T00:14:02.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076173760000000004 2024-09-15T00:14:02.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.674999+0000", 2024-09-15T00:14:02.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1948999999999999e-05 2024-09-15T00:14:02.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675009+0000", 2024-09-15T00:14:02.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7489999999999993e-06 2024-09-15T00:14:02.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675080+0000", 2024-09-15T00:14:02.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0728000000000005e-05 2024-09-15T00:14:02.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675224+0000", 2024-09-15T00:14:02.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00014454600000000001 2024-09-15T00:14:02.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:02.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.675676+0000", 2024-09-15T00:14:02.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00045150700000000003 2024-09-15T00:14:02.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:02.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.705941+0000", 2024-09-15T00:14:02.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030265351999999999 2024-09-15T00:14:02.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:02.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711016+0000", 2024-09-15T00:14:02.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050750980000000001 2024-09-15T00:14:02.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:02.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711083+0000", 2024-09-15T00:14:02.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7152999999999999e-05 2024-09-15T00:14:02.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.711138+0000", 2024-09-15T00:14:02.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4228000000000003e-05 2024-09-15T00:14:02.567 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.567 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.567 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.567 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125618 3.10 3:0aaeae86:::rbd_data.11396481e1ec.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:02.567 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:14:02.567 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.71900439700001, 2024-09-15T00:14:02.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030860942999999998, 2024-09-15T00:14:02.567 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.567 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.567 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:02.567 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.568 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.568 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.568 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125618 2024-09-15T00:14:02.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.568 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:14:02.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838497+0000", 2024-09-15T00:14:02.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.838499+0000", 2024-09-15T00:14:02.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6189999999999998e-06 2024-09-15T00:14:02.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839599+0000", 2024-09-15T00:14:02.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001099764 2024-09-15T00:14:02.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839620+0000", 2024-09-15T00:14:02.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0764e-05 2024-09-15T00:14:02.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839631+0000", 2024-09-15T00:14:02.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1335000000000001e-05 2024-09-15T00:14:02.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839725+0000", 2024-09-15T00:14:02.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4217999999999998e-05 2024-09-15T00:14:02.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839910+0000", 2024-09-15T00:14:02.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000184298 2024-09-15T00:14:02.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:14:02.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843171+0000", 2024-09-15T00:14:02.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032613540000000002 2024-09-15T00:14:02.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:02.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869187+0000", 2024-09-15T00:14:02.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026015467 2024-09-15T00:14:02.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:02.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869240+0000", 2024-09-15T00:14:02.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3733000000000002e-05 2024-09-15T00:14:02.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:02.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869309+0000", 2024-09-15T00:14:02.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8381999999999997e-05 2024-09-15T00:14:02.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.869358+0000", 2024-09-15T00:14:02.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-15T00:14:02.572 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.573 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.573 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.573 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125619 3.a 3:507b29bf:::rbd_data.11396481e1ec.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:02.573 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:14:02.573 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.717842268, 2024-09-15T00:14:02.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045768870000000003, 2024-09-15T00:14:02.573 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.573 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.573 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:02.573 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.573 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.573 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.574 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125619 2024-09-15T00:14:02.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.574 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:14:02.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839659+0000", 2024-09-15T00:14:02.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.839660+0000", 2024-09-15T00:14:02.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.22e-06 2024-09-15T00:14:02.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840511+0000", 2024-09-15T00:14:02.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000851284 2024-09-15T00:14:02.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840522+0000", 2024-09-15T00:14:02.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0569e-05 2024-09-15T00:14:02.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840527+0000", 2024-09-15T00:14:02.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.412e-06 2024-09-15T00:14:02.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840573+0000", 2024-09-15T00:14:02.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5247000000000002e-05 2024-09-15T00:14:02.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.840682+0000", 2024-09-15T00:14:02.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000109368 2024-09-15T00:14:02.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:02.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.843976+0000", 2024-09-15T00:14:02.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003294445 2024-09-15T00:14:02.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:02.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870833+0000", 2024-09-15T00:14:02.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026856944000000001 2024-09-15T00:14:02.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:02.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885344+0000", 2024-09-15T00:14:02.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014510744000000001 2024-09-15T00:14:02.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:02.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885382+0000", 2024-09-15T00:14:02.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7821000000000003e-05 2024-09-15T00:14:02.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885428+0000", 2024-09-15T00:14:02.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5816000000000003e-05 2024-09-15T00:14:02.578 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.578 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.579 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.579 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:125621 3.a 3:53594db9:::rbd_data.11396481e1ec.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:02.579 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:14:02.579 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 100.70431888900001, 2024-09-15T00:14:02.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.032481797, 2024-09-15T00:14:02.579 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.579 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.579 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:02.579 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.579 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.579 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.579 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 125621 2024-09-15T00:14:02.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.580 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:14:02.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853182+0000", 2024-09-15T00:14:02.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.853185+0000", 2024-09-15T00:14:02.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.43e-06 2024-09-15T00:14:02.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854125+0000", 2024-09-15T00:14:02.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000940539 2024-09-15T00:14:02.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854141+0000", 2024-09-15T00:14:02.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5568000000000001e-05 2024-09-15T00:14:02.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.854148+0000", 2024-09-15T00:14:02.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6529999999999994e-06 2024-09-15T00:14:02.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870872+0000", 2024-09-15T00:14:02.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016723135 2024-09-15T00:14:02.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.870975+0000", 2024-09-15T00:14:02.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010314099999999999 2024-09-15T00:14:02.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:02.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.873082+0000", 2024-09-15T00:14:02.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021069859999999999 2024-09-15T00:14:02.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:02.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885439+0000", 2024-09-15T00:14:02.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012357343999999999 2024-09-15T00:14:02.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:02.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885569+0000", 2024-09-15T00:14:02.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012995199999999999 2024-09-15T00:14:02.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:02.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885624+0000", 2024-09-15T00:14:02.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5281999999999997e-05 2024-09-15T00:14:02.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:21.885664+0000", 2024-09-15T00:14:02.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9767000000000001e-05 2024-09-15T00:14:02.584 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.584 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.584 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.585 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:147075 3.0 3:06ab39f2:::rbd_data.11394501364.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:14:02.585 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:14:02.585 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 77.608420507999995, 2024-09-15T00:14:02.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.046402906000000001, 2024-09-15T00:14:02.585 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.585 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.585 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:02.585 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.585 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.585 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.585 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 147075 2024-09-15T00:14:02.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.586 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:14:02.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:14:02.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949081+0000", 2024-09-15T00:14:02.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4399999999999999e-07 2024-09-15T00:14:02.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949086+0000", 2024-09-15T00:14:02.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.138e-06 2024-09-15T00:14:02.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949096+0000", 2024-09-15T00:14:02.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0438999999999999e-05 2024-09-15T00:14:02.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949101+0000", 2024-09-15T00:14:02.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8999999999999997e-06 2024-09-15T00:14:02.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949145+0000", 2024-09-15T00:14:02.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4141000000000003e-05 2024-09-15T00:14:02.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.949195+0000", 2024-09-15T00:14:02.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9657999999999998e-05 2024-09-15T00:14:02.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-15T00:14:02.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.950923+0000", 2024-09-15T00:14:02.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001728753 2024-09-15T00:14:02.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:02.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995390+0000", 2024-09-15T00:14:02.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044466335000000003 2024-09-15T00:14:02.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:02.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995401+0000", 2024-09-15T00:14:02.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1671000000000001e-05 2024-09-15T00:14:02.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:02.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995432+0000", 2024-09-15T00:14:02.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0704999999999998e-05 2024-09-15T00:14:02.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995484+0000", 2024-09-15T00:14:02.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1422000000000003e-05 2024-09-15T00:14:02.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.591 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.591 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:147075 3.0 e98/94 ondisk, result = 0)", 2024-09-15T00:14:02.591 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:14:02.591 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 77.600620786999997, 2024-09-15T00:14:02.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038514459000000001, 2024-09-15T00:14:02.591 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.592 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.592 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.592 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:14:02.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956880+0000", 2024-09-15T00:14:02.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956881+0000", 2024-09-15T00:14:02.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0599999999999999e-07 2024-09-15T00:14:02.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956885+0000", 2024-09-15T00:14:02.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3500000000000001e-06 2024-09-15T00:14:02.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956897+0000", 2024-09-15T00:14:02.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2303999999999999e-05 2024-09-15T00:14:02.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.956902+0000", 2024-09-15T00:14:02.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5169999999999999e-06 2024-09-15T00:14:02.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995377+0000", 2024-09-15T00:14:02.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038474544999999999 2024-09-15T00:14:02.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995385+0000", 2024-09-15T00:14:02.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6240000000000002e-06 2024-09-15T00:14:02.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:44.995395+0000", 2024-09-15T00:14:02.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0213e-05 2024-09-15T00:14:02.595 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.595 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.595 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.595 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:153172 3.14 3:2a7843e3:::rbd_data.113958cb0e55.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 983040~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:14:02.595 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:14:02.595 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 72.779731722999998, 2024-09-15T00:14:02.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.033232668, 2024-09-15T00:14:02.596 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.596 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.596 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:02.596 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.596 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.596 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.596 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 153172 2024-09-15T00:14:02.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.596 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:14:02.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777769+0000", 2024-09-15T00:14:02.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.777770+0000", 2024-09-15T00:14:02.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.017e-06 2024-09-15T00:14:02.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788009+0000", 2024-09-15T00:14:02.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010238146 2024-09-15T00:14:02.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788024+0000", 2024-09-15T00:14:02.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5582000000000001e-05 2024-09-15T00:14:02.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788048+0000", 2024-09-15T00:14:02.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4173e-05 2024-09-15T00:14:02.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788097+0000", 2024-09-15T00:14:02.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8498999999999998e-05 2024-09-15T00:14:02.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788201+0000", 2024-09-15T00:14:02.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010398799999999999 2024-09-15T00:14:02.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:02.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.788393+0000", 2024-09-15T00:14:02.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000192337 2024-09-15T00:14:02.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:02.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810637+0000", 2024-09-15T00:14:02.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.022244202000000001 2024-09-15T00:14:02.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:02.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810903+0000", 2024-09-15T00:14:02.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00026574600000000003 2024-09-15T00:14:02.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:02.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.810953+0000", 2024-09-15T00:14:02.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9509000000000003e-05 2024-09-15T00:14:02.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:49.811002+0000", 2024-09-15T00:14:02.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9468999999999997e-05 2024-09-15T00:14:02.601 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.601 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.601 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.601 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170014 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:14:02.601 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:14:02.601 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.710436616999999, 2024-09-15T00:14:02.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030291116999999999, 2024-09-15T00:14:02.601 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.602 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.602 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.602 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.602 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.602 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.602 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170014 2024-09-15T00:14:02.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.602 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:14:02.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:14:02.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847065+0000", 2024-09-15T00:14:02.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:14:02.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847073+0000", 2024-09-15T00:14:02.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9589999999999995e-06 2024-09-15T00:14:02.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847080+0000", 2024-09-15T00:14:02.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2930000000000003e-06 2024-09-15T00:14:02.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847086+0000", 2024-09-15T00:14:02.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.631e-06 2024-09-15T00:14:02.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877213+0000", 2024-09-15T00:14:02.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030127017999999998 2024-09-15T00:14:02.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877281+0000", 2024-09-15T00:14:02.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7916999999999994e-05 2024-09-15T00:14:02.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877356+0000", 2024-09-15T00:14:02.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4796999999999996e-05 2024-09-15T00:14:02.605 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.605 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.605 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.606 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170017 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1310720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:14:02.606 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:14:02.606 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.710407966000005, 2024-09-15T00:14:02.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030437623, 2024-09-15T00:14:02.606 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.606 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.606 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.606 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170017 2024-09-15T00:14:02.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.607 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:14:02.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847093+0000", 2024-09-15T00:14:02.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847094+0000", 2024-09-15T00:14:02.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8899999999999999e-07 2024-09-15T00:14:02.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847096+0000", 2024-09-15T00:14:02.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.136e-06 2024-09-15T00:14:02.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847104+0000", 2024-09-15T00:14:02.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7190000000000001e-06 2024-09-15T00:14:02.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847109+0000", 2024-09-15T00:14:02.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3279999999999996e-06 2024-09-15T00:14:02.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877384+0000", 2024-09-15T00:14:02.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030275348000000001 2024-09-15T00:14:02.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877451+0000", 2024-09-15T00:14:02.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6509999999999998e-05 2024-09-15T00:14:02.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877531+0000", 2024-09-15T00:14:02.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9993000000000005e-05 2024-09-15T00:14:02.610 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.610 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.610 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.610 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170018 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1351680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:14:02.610 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:14:02.610 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.710385621, 2024-09-15T00:14:02.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030591884999999999, 2024-09-15T00:14:02.610 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.610 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.611 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.611 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.611 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.611 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.611 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170018 2024-09-15T00:14:02.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.611 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:14:02.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:14:02.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847116+0000", 2024-09-15T00:14:02.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:14:02.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847118+0000", 2024-09-15T00:14:02.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.03e-06 2024-09-15T00:14:02.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847124+0000", 2024-09-15T00:14:02.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:14:02.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847129+0000", 2024-09-15T00:14:02.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8600000000000001e-06 2024-09-15T00:14:02.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877560+0000", 2024-09-15T00:14:02.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030431145999999999 2024-09-15T00:14:02.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877629+0000", 2024-09-15T00:14:02.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8793000000000004e-05 2024-09-15T00:14:02.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877707+0000", 2024-09-15T00:14:02.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8139999999999994e-05 2024-09-15T00:14:02.614 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.614 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.614 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.615 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170019 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1392640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:14:02.615 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:14:02.615 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.710364873000003, 2024-09-15T00:14:02.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030769023, 2024-09-15T00:14:02.615 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.615 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.615 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.615 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.615 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.615 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.615 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170019 2024-09-15T00:14:02.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.615 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:14:02.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847136+0000", 2024-09-15T00:14:02.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847137+0000", 2024-09-15T00:14:02.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:14:02.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847139+0000", 2024-09-15T00:14:02.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.142e-06 2024-09-15T00:14:02.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847146+0000", 2024-09-15T00:14:02.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8120000000000002e-06 2024-09-15T00:14:02.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847157+0000", 2024-09-15T00:14:02.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1093e-05 2024-09-15T00:14:02.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877735+0000", 2024-09-15T00:14:02.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578195999999998 2024-09-15T00:14:02.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877810+0000", 2024-09-15T00:14:02.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5563999999999997e-05 2024-09-15T00:14:02.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877905+0000", 2024-09-15T00:14:02.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4809000000000007e-05 2024-09-15T00:14:02.619 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.619 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.619 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.620 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170020 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:14:02.620 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:14:02.620 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.710337632999995, 2024-09-15T00:14:02.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030917150000000001, 2024-09-15T00:14:02.620 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.620 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.620 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.620 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.620 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.620 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.620 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170020 2024-09-15T00:14:02.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.621 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:14:02.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:14:02.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847164+0000", 2024-09-15T00:14:02.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:14:02.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847166+0000", 2024-09-15T00:14:02.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2790000000000001e-06 2024-09-15T00:14:02.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847173+0000", 2024-09-15T00:14:02.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3969999999999999e-06 2024-09-15T00:14:02.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847177+0000", 2024-09-15T00:14:02.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8620000000000002e-06 2024-09-15T00:14:02.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.877934+0000", 2024-09-15T00:14:02.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030756143999999999 2024-09-15T00:14:02.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878002+0000", 2024-09-15T00:14:02.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7951e-05 2024-09-15T00:14:02.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878081+0000", 2024-09-15T00:14:02.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9087999999999999e-05 2024-09-15T00:14:02.624 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.624 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.624 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.624 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170094 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:14:02.624 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:14:02.624 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.710080383999994, 2024-09-15T00:14:02.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030836142, 2024-09-15T00:14:02.624 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.625 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.625 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.625 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.625 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.625 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.625 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170094 2024-09-15T00:14:02.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.625 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:14:02.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:14:02.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847421+0000", 2024-09-15T00:14:02.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3499999999999996e-07 2024-09-15T00:14:02.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847424+0000", 2024-09-15T00:14:02.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2850000000000001e-06 2024-09-15T00:14:02.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847430+0000", 2024-09-15T00:14:02.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3260000000000001e-06 2024-09-15T00:14:02.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847435+0000", 2024-09-15T00:14:02.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.356e-06 2024-09-15T00:14:02.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878110+0000", 2024-09-15T00:14:02.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030675021 2024-09-15T00:14:02.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878177+0000", 2024-09-15T00:14:02.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6506000000000003e-05 2024-09-15T00:14:02.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878257+0000", 2024-09-15T00:14:02.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0012999999999994e-05 2024-09-15T00:14:02.628 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.628 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.628 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.629 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170096 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 458752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:14:02.629 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:14:02.629 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.710038428000004, 2024-09-15T00:14:02.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030977030999999999, 2024-09-15T00:14:02.629 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.629 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.629 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.629 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.629 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.629 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.629 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170096 2024-09-15T00:14:02.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.629 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:14:02.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:14:02.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847463+0000", 2024-09-15T00:14:02.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:14:02.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847465+0000", 2024-09-15T00:14:02.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0559999999999999e-06 2024-09-15T00:14:02.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847472+0000", 2024-09-15T00:14:02.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3130000000000004e-06 2024-09-15T00:14:02.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847477+0000", 2024-09-15T00:14:02.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9930000000000003e-06 2024-09-15T00:14:02.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878288+0000", 2024-09-15T00:14:02.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030811609 2024-09-15T00:14:02.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878358+0000", 2024-09-15T00:14:02.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0215000000000005e-05 2024-09-15T00:14:02.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878440+0000", 2024-09-15T00:14:02.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1366999999999995e-05 2024-09-15T00:14:02.633 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.633 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.633 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.633 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170097 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:14:02.633 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:14:02.633 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.710018343000002, 2024-09-15T00:14:02.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031134757999999998, 2024-09-15T00:14:02.633 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.633 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170097 2024-09-15T00:14:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:14:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:14:02.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847483+0000", 2024-09-15T00:14:02.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:14:02.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847485+0000", 2024-09-15T00:14:02.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9190000000000002e-06 2024-09-15T00:14:02.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847492+0000", 2024-09-15T00:14:02.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7109999999999998e-06 2024-09-15T00:14:02.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847497+0000", 2024-09-15T00:14:02.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0259999999999998e-06 2024-09-15T00:14:02.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878469+0000", 2024-09-15T00:14:02.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030972115000000001 2024-09-15T00:14:02.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878538+0000", 2024-09-15T00:14:02.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8669000000000003e-05 2024-09-15T00:14:02.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878618+0000", 2024-09-15T00:14:02.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9874000000000001e-05 2024-09-15T00:14:02.637 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.637 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.637 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.638 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170099 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 532480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:14:02.638 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:14:02.638 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.709966781000006, 2024-09-15T00:14:02.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031262183999999998, 2024-09-15T00:14:02.638 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.638 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.638 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.638 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.638 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.638 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.638 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170099 2024-09-15T00:14:02.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.638 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:14:02.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847534+0000", 2024-09-15T00:14:02.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847535+0000", 2024-09-15T00:14:02.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:14:02.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847537+0000", 2024-09-15T00:14:02.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:14:02.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847544+0000", 2024-09-15T00:14:02.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9630000000000001e-06 2024-09-15T00:14:02.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847549+0000", 2024-09-15T00:14:02.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8910000000000003e-06 2024-09-15T00:14:02.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878646+0000", 2024-09-15T00:14:02.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031097400000000001 2024-09-15T00:14:02.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878717+0000", 2024-09-15T00:14:02.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0993000000000003e-05 2024-09-15T00:14:02.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878797+0000", 2024-09-15T00:14:02.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9425000000000006e-05 2024-09-15T00:14:02.642 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.642 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.642 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.642 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170101 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:14:02.642 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:14:02.642 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.709925401999996, 2024-09-15T00:14:02.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031674568, 2024-09-15T00:14:02.642 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.642 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.643 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.643 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.643 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.643 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.643 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170101 2024-09-15T00:14:02.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.643 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:14:02.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:14:02.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847576+0000", 2024-09-15T00:14:02.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:14:02.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847578+0000", 2024-09-15T00:14:02.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0389999999999999e-06 2024-09-15T00:14:02.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847585+0000", 2024-09-15T00:14:02.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.336e-06 2024-09-15T00:14:02.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847596+0000", 2024-09-15T00:14:02.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015999999999999e-05 2024-09-15T00:14:02.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878825+0000", 2024-09-15T00:14:02.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031229848000000001 2024-09-15T00:14:02.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.878899+0000", 2024-09-15T00:14:02.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3831999999999994e-05 2024-09-15T00:14:02.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879250+0000", 2024-09-15T00:14:02.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035110000000000002 2024-09-15T00:14:02.647 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.647 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.647 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.647 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170102 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 614400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:14:02.647 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:14:02.647 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.709898104999994, 2024-09-15T00:14:02.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031577371999999999, 2024-09-15T00:14:02.647 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.648 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.648 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.648 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.648 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.648 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.648 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170102 2024-09-15T00:14:02.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.648 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:14:02.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:14:02.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847603+0000", 2024-09-15T00:14:02.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:14:02.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847610+0000", 2024-09-15T00:14:02.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6329999999999999e-06 2024-09-15T00:14:02.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847618+0000", 2024-09-15T00:14:02.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4599999999999997e-06 2024-09-15T00:14:02.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847623+0000", 2024-09-15T00:14:02.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2909999999999998e-06 2024-09-15T00:14:02.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879016+0000", 2024-09-15T00:14:02.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031393227000000003 2024-09-15T00:14:02.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879091+0000", 2024-09-15T00:14:02.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5384999999999997e-05 2024-09-15T00:14:02.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879180+0000", 2024-09-15T00:14:02.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8941000000000001e-05 2024-09-15T00:14:02.651 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.651 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.651 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.652 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170132 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:14:02.652 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:14:02.652 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.709850724999995, 2024-09-15T00:14:02.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031722579000000001, 2024-09-15T00:14:02.652 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.652 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.652 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.652 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170132 2024-09-15T00:14:02.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.653 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:14:02.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847650+0000", 2024-09-15T00:14:02.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847651+0000", 2024-09-15T00:14:02.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1900000000000003e-07 2024-09-15T00:14:02.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847653+0000", 2024-09-15T00:14:02.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0329999999999998e-06 2024-09-15T00:14:02.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847659+0000", 2024-09-15T00:14:02.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4169999999999997e-06 2024-09-15T00:14:02.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847670+0000", 2024-09-15T00:14:02.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1031e-05 2024-09-15T00:14:02.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879205+0000", 2024-09-15T00:14:02.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031534982000000003 2024-09-15T00:14:02.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879286+0000", 2024-09-15T00:14:02.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0675999999999998e-05 2024-09-15T00:14:02.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879373+0000", 2024-09-15T00:14:02.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6921000000000003e-05 2024-09-15T00:14:02.656 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.656 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.656 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.656 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:170133 3.a 3:5786fe14:::rbd_data.11395d771336.000000000000000a:34 [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-15T00:14:02.656 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:14:02.656 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 64.709823572000005, 2024-09-15T00:14:02.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031894817999999998, 2024-09-15T00:14:02.656 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:02.656 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:02.657 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:02.657 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:02.657 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:02.657 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:02.657 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 170133 2024-09-15T00:14:02.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.657 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:02.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:02.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:14:02.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:02.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.847678+0000", 2024-09-15T00:14:02.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:02.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:02.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853540+0000", 2024-09-15T00:14:02.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058619409999999999 2024-09-15T00:14:02.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:02.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853545+0000", 2024-09-15T00:14:02.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3430000000000003e-06 2024-09-15T00:14:02.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:02.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853554+0000", 2024-09-15T00:14:02.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4069999999999999e-06 2024-09-15T00:14:02.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:02.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.853561+0000", 2024-09-15T00:14:02.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5529999999999998e-06 2024-09-15T00:14:02.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:02.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879399+0000", 2024-09-15T00:14:02.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025837941 2024-09-15T00:14:02.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:02.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879477+0000", 2024-09-15T00:14:02.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8367000000000004e-05 2024-09-15T00:14:02.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:02.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:02.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:02.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:12:57.879572+0000", 2024-09-15T00:14:02.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5266000000000006e-05 2024-09-15T00:14:02.660 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.660 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.660 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.660 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:02.660 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:02.661 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:14:02.662 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:14:02.662 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:14:02.660+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:14:02.762 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:14:02.761+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:14:02.806 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:14:02.806 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:14:02.806 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:14:02.806 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:14:02.815 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:14:02.864 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:14:02.863+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:14:02.918 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:14:02.918 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:14:02.918 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-15T00:14:02.918 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-15T00:14:02.918 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:14:02.926 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:14:02.971 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:14:02.969+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:14:03.031 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:14:03.032 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:14:03.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:14:03.032 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:14:03.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.032 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:14:03.032 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:14:03.032 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 156.95038190099999, 2024-09-15T00:14:03.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:14:03.032 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.032 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.032 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:03.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.033 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.033 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.033 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:14:03.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.033 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:14:03.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:14:03.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:14:03.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:14:03.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:14:03.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:14:03.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:14:03.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:14:03.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:14:03.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:14:03.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:14:03.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:14:03.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:14:03.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:14:03.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:14:03.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:14:03.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:14:03.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:14:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:14:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:14:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:14:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:14:03.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:14:03.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:14:03.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:14:03.039 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.039 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.039 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.040 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:14:03.040 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:14:03.040 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 156.937641359, 2024-09-15T00:14:03.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:14:03.040 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.040 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.040 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:03.040 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:14:03.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:14:03.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:14:03.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:14:03.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:14:03.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:14:03.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:14:03.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:14:03.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:14:03.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:14:03.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:14:03.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:14:03.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:14:03.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:14:03.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:14:03.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:14:03.043 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.043 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.043 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.044 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:14:03.044 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:14:03.044 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 156.80528559999999, 2024-09-15T00:14:03.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:14:03.044 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.044 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.044 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:03.044 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.044 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.044 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.044 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:14:03.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.045 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:14:03.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:14:03.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:14:03.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:14:03.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:14:03.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:14:03.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:14:03.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:14:03.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:14:03.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:14:03.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:14:03.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:14:03.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:14:03.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:14:03.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:03.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:14:03.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:14:03.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:03.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:14:03.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:14:03.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:03.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:14:03.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:14:03.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:03.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:14:03.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:14:03.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:14:03.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:14:03.049 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.049 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.049 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.050 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:14:03.050 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:14:03.050 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 156.79182395500001, 2024-09-15T00:14:03.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:14:03.050 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.050 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.050 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:03.050 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:14:03.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:14:03.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:14:03.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:14:03.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:14:03.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:14:03.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:14:03.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:14:03.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:14:03.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:14:03.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:14:03.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:14:03.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:14:03.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:14:03.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:14:03.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:14:03.054 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.054 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.054 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.054 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:14:03.055 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:14:03.055 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 147.606822472, 2024-09-15T00:14:03.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:14:03.055 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.055 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.055 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:03.055 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.055 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.055 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.055 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:14:03.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.055 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:14:03.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:14:03.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:14:03.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:14:03.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:14:03.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:14:03.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:14:03.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:14:03.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:14:03.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:14:03.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:14:03.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:14:03.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:14:03.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:14:03.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:03.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:14:03.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:14:03.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:03.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:14:03.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:14:03.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:03.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:14:03.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:14:03.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:03.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:14:03.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:14:03.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:14:03.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:14:03.061 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.061 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.061 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.061 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:14:03.061 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:14:03.061 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 147.59394620500001, 2024-09-15T00:14:03.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:14:03.061 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.061 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.062 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:03.062 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:14:03.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:14:03.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:14:03.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:14:03.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:14:03.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:14:03.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:14:03.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:14:03.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:14:03.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:14:03.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:14:03.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:14:03.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:14:03.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:14:03.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:14:03.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:14:03.065 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.065 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.065 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.065 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:14:03.066 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:14:03.066 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 142.13543106099999, 2024-09-15T00:14:03.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:14:03.066 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.066 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.066 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:03.066 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.066 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.066 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.066 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:14:03.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.066 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:14:03.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:14:03.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:14:03.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:14:03.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:14:03.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:14:03.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:14:03.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:14:03.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:14:03.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:14:03.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:14:03.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:14:03.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:14:03.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:14:03.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:14:03.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:14:03.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:14:03.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:03.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:14:03.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:14:03.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:03.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:14:03.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:14:03.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:03.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:14:03.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:14:03.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:14:03.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:14:03.072 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.072 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.072 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.072 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:03.072 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:14:03.072 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 138.417858244, 2024-09-15T00:14:03.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:14:03.072 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.072 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.072 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:03.072 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.073 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.073 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.073 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:14:03.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.073 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:14:03.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:14:03.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:14:03.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:14:03.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:14:03.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:14:03.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:14:03.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:14:03.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:14:03.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:14:03.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:14:03.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:14:03.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:14:03.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:14:03.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:14:03.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:14:03.076 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.076 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.076 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.076 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:03.076 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:14:03.076 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 138.41767567100001, 2024-09-15T00:14:03.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:14:03.077 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.077 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.077 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:03.077 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.077 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.077 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.077 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:14:03.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.077 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:14:03.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:14:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:14:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:14:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:14:03.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:14:03.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:14:03.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:14:03.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:14:03.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:14:03.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:14:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:14:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:14:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:14:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:14:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:14:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.081 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:03.081 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:14:03.081 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 138.41764683, 2024-09-15T00:14:03.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:14:03.081 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.081 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.081 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:03.081 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.081 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.081 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.081 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:14:03.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.081 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:14:03.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:14:03.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:14:03.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:14:03.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:14:03.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:14:03.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:14:03.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:14:03.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:14:03.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:14:03.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:14:03.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:14:03.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:14:03.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:14:03.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:14:03.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:14:03.085 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.085 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.085 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.085 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:03.086 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:14:03.086 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 138.41762461900001, 2024-09-15T00:14:03.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:14:03.086 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.086 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.086 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:03.086 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.086 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.086 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.086 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:14:03.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.086 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:14:03.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:14:03.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:14:03.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:14:03.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:14:03.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:14:03.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:14:03.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:14:03.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:14:03.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:14:03.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:14:03.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:14:03.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:14:03.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:14:03.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:14:03.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:14:03.090 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.090 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.090 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.090 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:03.091 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:14:03.091 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 138.41760346000001, 2024-09-15T00:14:03.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:14:03.091 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.091 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.091 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:03.091 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.091 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.091 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.091 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:14:03.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.091 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:14:03.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:14:03.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:14:03.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:14:03.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:14:03.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:14:03.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:14:03.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:14:03.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:14:03.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:14:03.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:14:03.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:14:03.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:14:03.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:14:03.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:14:03.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:14:03.095 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.095 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.095 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.095 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:03.095 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:14:03.095 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 138.41757351199999, 2024-09-15T00:14:03.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:14:03.095 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.095 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.095 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:03.095 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.096 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.096 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.096 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:14:03.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.096 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:14:03.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:14:03.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:14:03.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:14:03.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:14:03.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:14:03.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:14:03.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:14:03.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:14:03.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:14:03.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:14:03.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:14:03.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:14:03.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:14:03.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:14:03.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:14:03.099 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.099 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.099 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.100 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:03.100 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:14:03.100 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 138.41755154800001, 2024-09-15T00:14:03.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:14:03.100 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.100 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.100 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:03.100 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.100 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.100 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.100 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:14:03.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.100 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:14:03.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:14:03.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:14:03.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:14:03.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:14:03.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:14:03.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:14:03.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:14:03.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:14:03.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:14:03.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:14:03.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:14:03.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:14:03.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:14:03.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:14:03.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:14:03.104 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.104 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.104 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.104 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:03.104 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:14:03.104 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 138.41753412099999, 2024-09-15T00:14:03.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:14:03.104 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.104 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.104 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:03.104 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.104 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.105 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.105 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:14:03.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.105 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:14:03.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:14:03.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:14:03.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:14:03.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:14:03.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:14:03.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:14:03.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:14:03.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:14:03.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:14:03.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:14:03.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:14:03.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:14:03.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:14:03.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:14:03.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:14:03.108 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.108 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.109 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.109 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:03.109 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:14:03.109 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 138.41751232199999, 2024-09-15T00:14:03.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:14:03.109 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.109 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.110 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:03.110 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.110 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.110 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.110 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:14:03.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.110 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:14:03.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:14:03.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:14:03.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:14:03.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:14:03.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:14:03.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:14:03.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:14:03.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:14:03.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:14:03.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:14:03.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:14:03.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:14:03.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:14:03.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:14:03.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:14:03.114 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.114 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.114 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.114 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:03.114 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:14:03.115 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 138.36828396300001, 2024-09-15T00:14:03.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:14:03.115 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.115 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.115 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:03.115 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.115 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.115 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.115 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:14:03.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.115 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:14:03.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:14:03.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:14:03.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:14:03.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:14:03.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:14:03.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:14:03.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:14:03.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:14:03.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:14:03.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:14:03.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:14:03.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:14:03.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:14:03.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:03.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:14:03.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:14:03.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:03.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:14:03.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:14:03.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:03.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:14:03.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:14:03.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:03.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:14:03.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:14:03.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:14:03.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:14:03.120 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.120 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.120 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.120 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:177892 1.0 1:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-15T00:14:03.120 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:13:48.311833+0000", 2024-09-15T00:14:03.120 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.717067965, 2024-09-15T00:14:03.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.90705656700000004, 2024-09-15T00:14:03.120 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.121 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.121 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:14:03.121 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.121 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.121 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.121 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 177892 2024-09-15T00:14:03.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.121 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311833+0000", 2024-09-15T00:14:03.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311833+0000", 2024-09-15T00:14:03.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311836+0000", 2024-09-15T00:14:03.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2400000000000002e-06 2024-09-15T00:14:03.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311842+0000", 2024-09-15T00:14:03.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9540000000000001e-06 2024-09-15T00:14:03.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311858+0000", 2024-09-15T00:14:03.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6424999999999999e-05 2024-09-15T00:14:03.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311866+0000", 2024-09-15T00:14:03.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1289999999999996e-06 2024-09-15T00:14:03.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311944+0000", 2024-09-15T00:14:03.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8158000000000006e-05 2024-09-15T00:14:03.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:14:03.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311953+0000", 2024-09-15T00:14:03.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8759999999999994e-06 2024-09-15T00:14:03.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.218787+0000", 2024-09-15T00:14:03.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.90683381299999999 2024-09-15T00:14:03.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.218890+0000", 2024-09-15T00:14:03.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000102972 2024-09-15T00:14:03.125 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.125 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.125 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.125 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:177895 3.14 3:292bb7d6:::rbd_header.113916cf513:head [watch reconnect cookie 139649723465488 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e161)", 2024-09-15T00:14:03.125 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:13:49.208730+0000", 2024-09-15T00:14:03.125 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.820171124, 2024-09-15T00:14:03.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.82280920000000002, 2024-09-15T00:14:03.125 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.125 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.125 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:03.125 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.126 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.126 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.126 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 177895 2024-09-15T00:14:03.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.126 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208730+0000", 2024-09-15T00:14:03.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208730+0000", 2024-09-15T00:14:03.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208732+0000", 2024-09-15T00:14:03.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5829999999999999e-06 2024-09-15T00:14:03.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208738+0000", 2024-09-15T00:14:03.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3690000000000003e-06 2024-09-15T00:14:03.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208753+0000", 2024-09-15T00:14:03.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5051999999999999e-05 2024-09-15T00:14:03.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208768+0000", 2024-09-15T00:14:03.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4243999999999999e-05 2024-09-15T00:14:03.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208823+0000", 2024-09-15T00:14:03.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5652999999999997e-05 2024-09-15T00:14:03.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-15T00:14:03.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208831+0000", 2024-09-15T00:14:03.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7309999999999993e-06 2024-09-15T00:14:03.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.214458+0000", 2024-09-15T00:14:03.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0056266780000000004 2024-09-15T00:14:03.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:14:03.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.214468+0000", 2024-09-15T00:14:03.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0631e-05 2024-09-15T00:14:03.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.025026+0000", 2024-09-15T00:14:03.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.81055788500000003 2024-09-15T00:14:03.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.025208+0000", 2024-09-15T00:14:03.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000181829 2024-09-15T00:14:03.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.031540+0000", 2024-09-15T00:14:03.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0063315450000000001 2024-09-15T00:14:03.130 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.130 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:177896 3.14 3:292bb7d6:::rbd_header.113916cf513:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e161)", 2024-09-15T00:14:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:13:49.219101+0000", 2024-09-15T00:14:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.809800203, 2024-09-15T00:14:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.81263984899999997, 2024-09-15T00:14:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 177896 2024-09-15T00:14:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.132 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:03.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:03.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219101+0000", 2024-09-15T00:14:03.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:03.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219101+0000", 2024-09-15T00:14:03.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:03.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:03.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219103+0000", 2024-09-15T00:14:03.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.449e-06 2024-09-15T00:14:03.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:03.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219108+0000", 2024-09-15T00:14:03.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5210000000000002e-06 2024-09-15T00:14:03.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:03.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219119+0000", 2024-09-15T00:14:03.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0641e-05 2024-09-15T00:14:03.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219126+0000", 2024-09-15T00:14:03.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7290000000000003e-06 2024-09-15T00:14:03.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219166+0000", 2024-09-15T00:14:03.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0246000000000001e-05 2024-09-15T00:14:03.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:14:03.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219172+0000", 2024-09-15T00:14:03.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2230000000000004e-06 2024-09-15T00:14:03.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:03.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.027379+0000", 2024-09-15T00:14:03.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.80820711700000003 2024-09-15T00:14:03.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:03.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.027474+0000", 2024-09-15T00:14:03.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5111999999999993e-05 2024-09-15T00:14:03.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.027727+0000", 2024-09-15T00:14:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025279600000000001 2024-09-15T00:14:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.031417+0000", 2024-09-15T00:14:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036897980000000002 2024-09-15T00:14:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.031635+0000", 2024-09-15T00:14:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021787799999999999 2024-09-15T00:14:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:03.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.031693+0000", 2024-09-15T00:14:03.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8452999999999998e-05 2024-09-15T00:14:03.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:03.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:03.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:03.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.031741+0000", 2024-09-15T00:14:03.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7886000000000002e-05 2024-09-15T00:14:03.137 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.137 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.137 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.137 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:03.137 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:03.137 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:14:03.139 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is unset 2024-09-15T00:14:03.140 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:14:03.140 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:14:03.137+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:14:03.239 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:14:03.238+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:14:03.285 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:03.285 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:14:03.285 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:14:03.285 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:03.295 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:14:03.380 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:14:03.378+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:14:03.438 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:03.438 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:14:03.439 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:14:03.439 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:14:03.439 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:03.441 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:14:03.439+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:14:03.447 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:14:03.542 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:14:03.541+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:14:03.593 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:03.593 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:14:03.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:14:03.593 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:14:03.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.593 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9, version: 37'47, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9@37'47, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:03.593 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:14:03.593 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.736620475000002, 2024-09-15T00:14:03.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054905559999999997, 2024-09-15T00:14:03.594 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.594 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.594 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.594 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:14:03.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:14:03.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853793+0000", 2024-09-15T00:14:03.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9599999999999999e-07 2024-09-15T00:14:03.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854798+0000", 2024-09-15T00:14:03.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010050770000000001 2024-09-15T00:14:03.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854814+0000", 2024-09-15T00:14:03.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6478999999999999e-05 2024-09-15T00:14:03.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854825+0000", 2024-09-15T00:14:03.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0492999999999999e-05 2024-09-15T00:14:03.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854889+0000", 2024-09-15T00:14:03.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4134999999999995e-05 2024-09-15T00:14:03.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854898+0000", 2024-09-15T00:14:03.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6260000000000001e-06 2024-09-15T00:14:03.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.859283+0000", 2024-09-15T00:14:03.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043852500000000003 2024-09-15T00:14:03.597 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.597 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.597 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.597 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15, version: 59'205, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15@59'205, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:03.598 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:14:03.598 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.61040002, 2024-09-15T00:14:03.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059047939999999997, 2024-09-15T00:14:03.598 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.598 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.598 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.598 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:14:03.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:14:03.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:14:03.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.44e-07 2024-09-15T00:14:03.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980977+0000", 2024-09-15T00:14:03.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00096402300000000005 2024-09-15T00:14:03.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980995+0000", 2024-09-15T00:14:03.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7214000000000002e-05 2024-09-15T00:14:03.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981007+0000", 2024-09-15T00:14:03.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2028000000000001e-05 2024-09-15T00:14:03.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981069+0000", 2024-09-15T00:14:03.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1977000000000006e-05 2024-09-15T00:14:03.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981077+0000", 2024-09-15T00:14:03.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2120000000000003e-06 2024-09-15T00:14:03.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.985918+0000", 2024-09-15T00:14:03.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048406960000000002 2024-09-15T00:14:03.601 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.601 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.601 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.601 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head, version: 62'221, data_included: [0~262144,393216~786432], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head@62'221, size: 1572864, copy_subset: [(0, 1572864)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:03.601 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:14:03.602 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.588131754999999, 2024-09-15T00:14:03.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054170770000000002, 2024-09-15T00:14:03.602 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.602 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.602 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.602 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:14:03.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:14:03.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002282+0000", 2024-09-15T00:14:03.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4499999999999997e-07 2024-09-15T00:14:03.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003196+0000", 2024-09-15T00:14:03.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091476399999999999 2024-09-15T00:14:03.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003213+0000", 2024-09-15T00:14:03.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6554000000000001e-05 2024-09-15T00:14:03.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003230+0000", 2024-09-15T00:14:03.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6617999999999999e-05 2024-09-15T00:14:03.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003292+0000", 2024-09-15T00:14:03.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2193999999999994e-05 2024-09-15T00:14:03.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003300+0000", 2024-09-15T00:14:03.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1429999999999998e-06 2024-09-15T00:14:03.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.007698+0000", 2024-09-15T00:14:03.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043981589999999996 2024-09-15T00:14:03.605 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.605 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.605 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.605 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:03.605 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:14:03.606 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.537401674999998, 2024-09-15T00:14:03.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067633270000000004, 2024-09-15T00:14:03.606 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.606 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.606 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.606 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:14:03.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:14:03.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053012+0000", 2024-09-15T00:14:03.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1659999999999999e-06 2024-09-15T00:14:03.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054086+0000", 2024-09-15T00:14:03.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010732789999999999 2024-09-15T00:14:03.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054100+0000", 2024-09-15T00:14:03.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4123e-05 2024-09-15T00:14:03.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054114+0000", 2024-09-15T00:14:03.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4442e-05 2024-09-15T00:14:03.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054192+0000", 2024-09-15T00:14:03.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8288000000000007e-05 2024-09-15T00:14:03.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054200+0000", 2024-09-15T00:14:03.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-15T00:14:03.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.059774+0000", 2024-09-15T00:14:03.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055741230000000003 2024-09-15T00:14:03.609 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.609 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.609 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.610 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:14:03.610 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:14:03.610 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.529084347000001, 2024-09-15T00:14:03.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049777950000000001, 2024-09-15T00:14:03.610 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.610 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.610 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.610 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:14:03.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:14:03.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061330+0000", 2024-09-15T00:14:03.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.11e-06 2024-09-15T00:14:03.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062350+0000", 2024-09-15T00:14:03.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010207969999999999 2024-09-15T00:14:03.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062365+0000", 2024-09-15T00:14:03.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4157e-05 2024-09-15T00:14:03.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062380+0000", 2024-09-15T00:14:03.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5129e-05 2024-09-15T00:14:03.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062467+0000", 2024-09-15T00:14:03.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7247e-05 2024-09-15T00:14:03.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062475+0000", 2024-09-15T00:14:03.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4300000000000006e-06 2024-09-15T00:14:03.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.066306+0000", 2024-09-15T00:14:03.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038309250000000002 2024-09-15T00:14:03.613 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.616 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.616 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.617 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:14:03.617 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:14:03.617 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.523201389, 2024-09-15T00:14:03.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052454889999999999, 2024-09-15T00:14:03.617 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.617 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.617 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.617 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:14:03.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:14:03.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067212+0000", 2024-09-15T00:14:03.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-07 2024-09-15T00:14:03.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068119+0000", 2024-09-15T00:14:03.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090671199999999995 2024-09-15T00:14:03.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068135+0000", 2024-09-15T00:14:03.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5739e-05 2024-09-15T00:14:03.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068150+0000", 2024-09-15T00:14:03.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5693e-05 2024-09-15T00:14:03.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068236+0000", 2024-09-15T00:14:03.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6100000000000006e-05 2024-09-15T00:14:03.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068245+0000", 2024-09-15T00:14:03.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1960000000000008e-06 2024-09-15T00:14:03.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.072457+0000", 2024-09-15T00:14:03.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042123719999999998 2024-09-15T00:14:03.620 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.621 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.621 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.621 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:14:03.621 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:14:03.621 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.517223039999998, 2024-09-15T00:14:03.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049532020000000003, 2024-09-15T00:14:03.621 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.621 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.621 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.621 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:14:03.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:14:03.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:14:03.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:14:03.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074164+0000", 2024-09-15T00:14:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097354400000000002 2024-09-15T00:14:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074178+0000", 2024-09-15T00:14:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4134999999999999e-05 2024-09-15T00:14:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074192+0000", 2024-09-15T00:14:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.451e-05 2024-09-15T00:14:03.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074272+0000", 2024-09-15T00:14:03.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9989999999999998e-05 2024-09-15T00:14:03.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074280+0000", 2024-09-15T00:14:03.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9990000000000008e-06 2024-09-15T00:14:03.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078143+0000", 2024-09-15T00:14:03.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00386258 2024-09-15T00:14:03.625 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.625 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.625 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.625 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [4194304~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:14:03.625 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:14:03.625 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.511512228000001, 2024-09-15T00:14:03.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049408730000000001, 2024-09-15T00:14:03.625 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.625 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.625 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.625 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:14:03.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:14:03.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:14:03.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2399999999999999e-07 2024-09-15T00:14:03.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079875+0000", 2024-09-15T00:14:03.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097349100000000005 2024-09-15T00:14:03.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079889+0000", 2024-09-15T00:14:03.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4148000000000001e-05 2024-09-15T00:14:03.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079904+0000", 2024-09-15T00:14:03.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5159e-05 2024-09-15T00:14:03.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079964+0000", 2024-09-15T00:14:03.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0204999999999997e-05 2024-09-15T00:14:03.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079971+0000", 2024-09-15T00:14:03.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2490000000000004e-06 2024-09-15T00:14:03.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.083841+0000", 2024-09-15T00:14:03.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038701970000000001 2024-09-15T00:14:03.628 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.629 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.629 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.629 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [5242880~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 6291456, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:14:03.629 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:14:03.629 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.505807247999996, 2024-09-15T00:14:03.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052525740000000003, 2024-09-15T00:14:03.629 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.629 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.629 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.629 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:14:03.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:14:03.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:14:03.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:14:03.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085579+0000", 2024-09-15T00:14:03.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097307600000000002 2024-09-15T00:14:03.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085592+0000", 2024-09-15T00:14:03.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3277e-05 2024-09-15T00:14:03.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085608+0000", 2024-09-15T00:14:03.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5773e-05 2024-09-15T00:14:03.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085687+0000", 2024-09-15T00:14:03.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8956999999999996e-05 2024-09-15T00:14:03.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085694+0000", 2024-09-15T00:14:03.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.956e-06 2024-09-15T00:14:03.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.089858+0000", 2024-09-15T00:14:03.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041641159999999998 2024-09-15T00:14:03.633 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.633 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.633 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.633 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:172552 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-15T00:14:03.633 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:14:03.633 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.401349940000003, 2024-09-15T00:14:03.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.997031471, 2024-09-15T00:14:03.633 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.633 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.633 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.634 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:03.634 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:03.634 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:03.634 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 172552 2024-09-15T00:14:03.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.634 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:14:03.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:14:03.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189064+0000", 2024-09-15T00:14:03.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.198e-06 2024-09-15T00:14:03.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189069+0000", 2024-09-15T00:14:03.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9069999999999998e-06 2024-09-15T00:14:03.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189080+0000", 2024-09-15T00:14:03.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:14:03.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189086+0000", 2024-09-15T00:14:03.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5429999999999999e-06 2024-09-15T00:14:03.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189128+0000", 2024-09-15T00:14:03.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2255000000000001e-05 2024-09-15T00:14:03.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:14:03.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189135+0000", 2024-09-15T00:14:03.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6200000000000001e-06 2024-09-15T00:14:03.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185787+0000", 2024-09-15T00:14:03.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99665157199999999 2024-09-15T00:14:03.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185926+0000", 2024-09-15T00:14:03.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013970399999999999 2024-09-15T00:14:03.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.186094+0000", 2024-09-15T00:14:03.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016813099999999999 2024-09-15T00:14:03.638 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.638 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.638 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.638 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9, version: 37'60, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9@37'60, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:03.638 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:14:03.638 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.342489581000002, 2024-09-15T00:14:03.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049454060000000003, 2024-09-15T00:14:03.638 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.639 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.639 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.639 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:14:03.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:14:03.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247924+0000", 2024-09-15T00:14:03.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:14:03.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248394+0000", 2024-09-15T00:14:03.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047063399999999999 2024-09-15T00:14:03.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248411+0000", 2024-09-15T00:14:03.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6384000000000001e-05 2024-09-15T00:14:03.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248441+0000", 2024-09-15T00:14:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0559999999999999e-05 2024-09-15T00:14:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248499+0000", 2024-09-15T00:14:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7274000000000001e-05 2024-09-15T00:14:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248507+0000", 2024-09-15T00:14:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2390000000000002e-06 2024-09-15T00:14:03.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.252869+0000", 2024-09-15T00:14:03.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043617200000000004 2024-09-15T00:14:03.642 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.642 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.642 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.642 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head, version: 59'393, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head@59'393, size: 5701632, copy_subset: [(0, 5701632)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:03.642 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:14:03.642 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.213041314000002, 2024-09-15T00:14:03.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004807469, 2024-09-15T00:14:03.642 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.643 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.643 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.643 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:14:03.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:14:03.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:14:03.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4799999999999998e-07 2024-09-15T00:14:03.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377830+0000", 2024-09-15T00:14:03.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000457954 2024-09-15T00:14:03.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377847+0000", 2024-09-15T00:14:03.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6741000000000001e-05 2024-09-15T00:14:03.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377858+0000", 2024-09-15T00:14:03.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0951999999999999e-05 2024-09-15T00:14:03.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377929+0000", 2024-09-15T00:14:03.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0697000000000004e-05 2024-09-15T00:14:03.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377941+0000", 2024-09-15T00:14:03.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2529e-05 2024-09-15T00:14:03.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.382179+0000", 2024-09-15T00:14:03.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042379480000000001 2024-09-15T00:14:03.646 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.646 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.646 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.646 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head, version: 88'473, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head@88'473, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:03.646 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:14:03.646 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 56.130921880999999, 2024-09-15T00:14:03.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005823272, 2024-09-15T00:14:03.647 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.647 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.647 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.647 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:14:03.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:14:03.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459492+0000", 2024-09-15T00:14:03.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9800000000000003e-07 2024-09-15T00:14:03.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459940+0000", 2024-09-15T00:14:03.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044854699999999999 2024-09-15T00:14:03.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459957+0000", 2024-09-15T00:14:03.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6586e-05 2024-09-15T00:14:03.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459968+0000", 2024-09-15T00:14:03.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.145e-05 2024-09-15T00:14:03.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460043+0000", 2024-09-15T00:14:03.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4451999999999998e-05 2024-09-15T00:14:03.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460057+0000", 2024-09-15T00:14:03.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.417e-05 2024-09-15T00:14:03.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.465314+0000", 2024-09-15T00:14:03.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052574689999999999 2024-09-15T00:14:03.650 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.650 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.650 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.650 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:03.650 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:14:03.650 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.895745699999999, 2024-09-15T00:14:03.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051811399999999999, 2024-09-15T00:14:03.651 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.651 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.651 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.651 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:14:03.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:14:03.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694668+0000", 2024-09-15T00:14:03.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4600000000000005e-07 2024-09-15T00:14:03.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695136+0000", 2024-09-15T00:14:03.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00046864399999999999 2024-09-15T00:14:03.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695153+0000", 2024-09-15T00:14:03.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6218e-05 2024-09-15T00:14:03.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695189+0000", 2024-09-15T00:14:03.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6393000000000002e-05 2024-09-15T00:14:03.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695263+0000", 2024-09-15T00:14:03.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4175e-05 2024-09-15T00:14:03.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695271+0000", 2024-09-15T00:14:03.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2800000000000003e-06 2024-09-15T00:14:03.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.699848+0000", 2024-09-15T00:14:03.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045768839999999998 2024-09-15T00:14:03.654 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.654 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.654 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.654 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:14:03.654 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:14:03.655 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.887647491999999, 2024-09-15T00:14:03.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048716510000000003, 2024-09-15T00:14:03.655 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.655 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.655 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.655 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:14:03.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:14:03.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702783+0000", 2024-09-15T00:14:03.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7268000000000001e-05 2024-09-15T00:14:03.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703221+0000", 2024-09-15T00:14:03.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00043866899999999998 2024-09-15T00:14:03.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703237+0000", 2024-09-15T00:14:03.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5553000000000002e-05 2024-09-15T00:14:03.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703247+0000", 2024-09-15T00:14:03.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0392e-05 2024-09-15T00:14:03.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703358+0000", 2024-09-15T00:14:03.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110676 2024-09-15T00:14:03.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703369+0000", 2024-09-15T00:14:03.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0716e-05 2024-09-15T00:14:03.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.707637+0000", 2024-09-15T00:14:03.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042683770000000003 2024-09-15T00:14:03.658 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.658 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.658 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.658 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18, version: 62'274, data_included: [0~786432], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18@62'274, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:03.658 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:14:03.659 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.841566409999999, 2024-09-15T00:14:03.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005453014, 2024-09-15T00:14:03.659 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.659 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.659 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.659 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:14:03.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:14:03.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748847+0000", 2024-09-15T00:14:03.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8299999999999997e-07 2024-09-15T00:14:03.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749010+0000", 2024-09-15T00:14:03.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000163144 2024-09-15T00:14:03.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749027+0000", 2024-09-15T00:14:03.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6518999999999999e-05 2024-09-15T00:14:03.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749037+0000", 2024-09-15T00:14:03.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0438000000000001e-05 2024-09-15T00:14:03.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749112+0000", 2024-09-15T00:14:03.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4946999999999999e-05 2024-09-15T00:14:03.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749123+0000", 2024-09-15T00:14:03.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1014e-05 2024-09-15T00:14:03.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.754299+0000", 2024-09-15T00:14:03.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051763690000000001 2024-09-15T00:14:03.662 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.662 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.662 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.662 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20, version: 80'334, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20@80'334, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 20=[]:{20: [20]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:03.663 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:14:03.663 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.786929366000003, 2024-09-15T00:14:03.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048933570000000001, 2024-09-15T00:14:03.663 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.663 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.663 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.663 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:14:03.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:14:03.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803484+0000", 2024-09-15T00:14:03.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:14:03.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803938+0000", 2024-09-15T00:14:03.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045364599999999999 2024-09-15T00:14:03.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803954+0000", 2024-09-15T00:14:03.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6456999999999998e-05 2024-09-15T00:14:03.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803965+0000", 2024-09-15T00:14:03.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.076e-05 2024-09-15T00:14:03.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804023+0000", 2024-09-15T00:14:03.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7787000000000001e-05 2024-09-15T00:14:03.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804031+0000", 2024-09-15T00:14:03.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2160000000000006e-06 2024-09-15T00:14:03.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.808377+0000", 2024-09-15T00:14:03.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043458969999999996 2024-09-15T00:14:03.666 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.666 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.666 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.666 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [1179648~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2228224, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:14:03.666 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:14:03.667 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.742118722999997, 2024-09-15T00:14:03.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053056819999999999, 2024-09-15T00:14:03.667 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.667 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.667 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.667 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:14:03.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:14:03.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848295+0000", 2024-09-15T00:14:03.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4000000000000001e-07 2024-09-15T00:14:03.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848728+0000", 2024-09-15T00:14:03.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000433709 2024-09-15T00:14:03.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848745+0000", 2024-09-15T00:14:03.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6167999999999998e-05 2024-09-15T00:14:03.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848788+0000", 2024-09-15T00:14:03.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3269000000000002e-05 2024-09-15T00:14:03.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848857+0000", 2024-09-15T00:14:03.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9453000000000001e-05 2024-09-15T00:14:03.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848866+0000", 2024-09-15T00:14:03.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2779999999999993e-06 2024-09-15T00:14:03.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.853600+0000", 2024-09-15T00:14:03.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047341650000000002 2024-09-15T00:14:03.670 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.670 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.670 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.670 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head, version: 93'455, data_included: [0~98304,114688~16384,147456~589824,753664~65536,835584~16384,868352~262144], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head@93'455, size: 2342912, copy_subset: [(0, 2342912)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1130496, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:03.670 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:14:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.682984697000002, 2024-09-15T00:14:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063250900000000002, 2024-09-15T00:14:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:14:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:14:03.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907429+0000", 2024-09-15T00:14:03.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.85e-07 2024-09-15T00:14:03.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908008+0000", 2024-09-15T00:14:03.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00057949599999999998 2024-09-15T00:14:03.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908026+0000", 2024-09-15T00:14:03.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7700999999999999e-05 2024-09-15T00:14:03.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908038+0000", 2024-09-15T00:14:03.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2131e-05 2024-09-15T00:14:03.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908101+0000", 2024-09-15T00:14:03.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2596999999999997e-05 2024-09-15T00:14:03.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908110+0000", 2024-09-15T00:14:03.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8540000000000003e-06 2024-09-15T00:14:03.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.913753+0000", 2024-09-15T00:14:03.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056435260000000003 2024-09-15T00:14:03.674 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.674 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.674 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.674 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 165/164 [PushOp(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9, version: 37'60, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9@37'60, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:03.674 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:53.061053+0000", 2024-09-15T00:14:03.674 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.529359412, 2024-09-15T00:14:03.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054243870000000001, 2024-09-15T00:14:03.675 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:03.675 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:03.675 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:03.675 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:03.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:03.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061053+0000", 2024-09-15T00:14:03.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:03.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061053+0000", 2024-09-15T00:14:03.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:03.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:03.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061054+0000", 2024-09-15T00:14:03.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5499999999999997e-07 2024-09-15T00:14:03.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:03.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061854+0000", 2024-09-15T00:14:03.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00080025500000000004 2024-09-15T00:14:03.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:03.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061880+0000", 2024-09-15T00:14:03.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5224000000000001e-05 2024-09-15T00:14:03.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:03.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061898+0000", 2024-09-15T00:14:03.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8484000000000001e-05 2024-09-15T00:14:03.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:03.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061983+0000", 2024-09-15T00:14:03.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4845999999999994e-05 2024-09-15T00:14:03.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:03.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061993+0000", 2024-09-15T00:14:03.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5419999999999994e-06 2024-09-15T00:14:03.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:03.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:03.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:03.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.066478+0000", 2024-09-15T00:14:03.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044852809999999998 2024-09-15T00:14:03.678 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.678 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.678 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.678 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:03.678 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:03.678 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:03.680 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:14:03.678+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:14:03.680 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:14:03.768 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:14:03.812 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:14:03.811+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:14:03.826 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:03.826 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:14:03.826 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:14:03.826 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:03.836 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:14:03.890 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:14:03.891 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:14:03.891 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:14:03.889+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:14:03.908 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:14:03.908 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:14:03.926 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:14:03.926 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:14:03.942 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:14:03.942 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:14:03.960 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:14:03.960 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:14:03.982 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:03.982 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:03.982 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:14:03.981+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:14:03.991 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:14:04.083 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:14:04.081+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:14:04.152 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:04.152 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:04.162 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:14:04.178 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:14:04.178 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:14:04.183 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:14:04.182+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:14:04.198 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:14:04.198 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:14:04.215 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:14:04.215 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:14:04.270 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:04.270 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:04.279 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:14:04.284 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:14:04.282+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:14:04.383 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:04.383 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:04.385 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:14:04.384+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:14:04.392 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:14:04.486 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:14:04.485+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:14:04.495 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:04.495 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:04.504 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:14:04.587 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:14:04.585+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:14:04.608 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:04.608 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:04.617 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:14:04.687 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:14:04.686+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:14:04.761 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:04.761 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:04.770 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:14:04.788 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:14:04.786+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:14:04.872 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:04.873 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:04.881 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:14:04.926 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:14:04.924+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:14:04.988 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:04.988 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:04.989 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:14:04.988+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:14:04.997 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:14:05.090 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:14:05.089+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:14:05.141 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:14:05.147 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:05.147 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:05.156 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:14:05.191 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:14:05.189+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:14:05.259 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:05.260 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:05.268 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:14:05.311 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:14:05.310+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:05.380 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:14:05.421 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:14:05.419+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:14:05.493 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:14:05.491+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:14:05.524 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:05.524 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:05.532 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:14:05.617 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:14:05.615+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:14:05.677 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:05.677 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:05.686 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:14:05.694 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:14:05.693+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:14:05.795 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:14:05.793+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:14:05.829 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:05.829 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:05.838 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:14:05.896 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:14:05.895+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:14:05.981 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:05.981 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:05.990 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:14:06.033 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:14:06.032+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:14:06.053 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is set 2024-09-15T00:14:06.096 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:06.096 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:06.105 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:14:06.115 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:14:06.113+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:14:06.198 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:14:06.195+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:14:06.229 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:14:06.249 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:06.249 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:06.258 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:14:06.298 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:14:06.296+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:14:06.350 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:14:06.350 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:14:06.361 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:06.361 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:06.367 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:14:06.367 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:14:06.370 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:14:06.383 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:14:06.383 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:14:06.399 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:14:06.399 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:14:06.413 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:14:06.412+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:14:06.415 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:14:06.415 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:14:06.432 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:14:06.432 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:14:06.448 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:14:06.449 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:14:06.465 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:14:06.465 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:14:06.477 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:14:06.477 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-15T00:14:06.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-15T00:14:06.477 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-15T00:14:06.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.478 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1257 3.12 3:489584bc:::rbd_data.11397bff333d.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:14:06.478 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:14:06.478 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 160.39562769, 2024-09-15T00:14:06.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.080607395999999998, 2024-09-15T00:14:06.478 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.478 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.478 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:06.478 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.478 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.478 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.478 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1257 2024-09-15T00:14:06.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.478 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:14:06.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078520+0000", 2024-09-15T00:14:06.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078521+0000", 2024-09-15T00:14:06.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.339e-06 2024-09-15T00:14:06.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078528+0000", 2024-09-15T00:14:06.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-15T00:14:06.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078542+0000", 2024-09-15T00:14:06.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.446e-05 2024-09-15T00:14:06.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078548+0000", 2024-09-15T00:14:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2659999999999998e-06 2024-09-15T00:14:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078602+0000", 2024-09-15T00:14:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3149999999999998e-05 2024-09-15T00:14:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.078649+0000", 2024-09-15T00:14:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7648000000000002e-05 2024-09-15T00:14:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:14:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.081683+0000", 2024-09-15T00:14:06.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030338019999999999 2024-09-15T00:14:06.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:06.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159004+0000", 2024-09-15T00:14:06.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.077320625000000004 2024-09-15T00:14:06.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:06.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159019+0000", 2024-09-15T00:14:06.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4871999999999999e-05 2024-09-15T00:14:06.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:06.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159076+0000", 2024-09-15T00:14:06.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7095000000000001e-05 2024-09-15T00:14:06.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159127+0000", 2024-09-15T00:14:06.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1329000000000003e-05 2024-09-15T00:14:06.483 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.483 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.483 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.483 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1257 3.12 e30/28 ondisk, result = 0)", 2024-09-15T00:14:06.484 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:14:06.484 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 160.38288714800001, 2024-09-15T00:14:06.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067749796000000001, 2024-09-15T00:14:06.484 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.484 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.484 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:06.484 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:14:06.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091260+0000", 2024-09-15T00:14:06.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091261+0000", 2024-09-15T00:14:06.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:14:06.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091264+0000", 2024-09-15T00:14:06.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6550000000000002e-06 2024-09-15T00:14:06.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091276+0000", 2024-09-15T00:14:06.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2065000000000001e-05 2024-09-15T00:14:06.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.091300+0000", 2024-09-15T00:14:06.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3345000000000001e-05 2024-09-15T00:14:06.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158990+0000", 2024-09-15T00:14:06.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.067690393000000001 2024-09-15T00:14:06.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.158998+0000", 2024-09-15T00:14:06.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9270000000000005e-06 2024-09-15T00:14:06.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.159010+0000", 2024-09-15T00:14:06.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1718e-05 2024-09-15T00:14:06.487 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.487 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.487 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.487 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:1294 3.13 3:cfd1699b:::rbd_data.11397bff333d.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:14:06.488 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:14:06.488 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 160.250531389, 2024-09-15T00:14:06.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042274394999999999, 2024-09-15T00:14:06.488 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.488 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.488 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:06.488 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.488 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.488 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.488 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1294 2024-09-15T00:14:06.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.488 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:14:06.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223616+0000", 2024-09-15T00:14:06.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223617+0000", 2024-09-15T00:14:06.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0389999999999999e-06 2024-09-15T00:14:06.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223622+0000", 2024-09-15T00:14:06.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4689999999999996e-06 2024-09-15T00:14:06.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223636+0000", 2024-09-15T00:14:06.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3368e-05 2024-09-15T00:14:06.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223643+0000", 2024-09-15T00:14:06.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2289999999999997e-06 2024-09-15T00:14:06.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223690+0000", 2024-09-15T00:14:06.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7512999999999997e-05 2024-09-15T00:14:06.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.223762+0000", 2024-09-15T00:14:06.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1155000000000006e-05 2024-09-15T00:14:06.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:06.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.226898+0000", 2024-09-15T00:14:06.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031364409999999998 2024-09-15T00:14:06.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:06.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265782+0000", 2024-09-15T00:14:06.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.038883521999999997 2024-09-15T00:14:06.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:06.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265795+0000", 2024-09-15T00:14:06.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3879e-05 2024-09-15T00:14:06.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:06.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265830+0000", 2024-09-15T00:14:06.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4452000000000002e-05 2024-09-15T00:14:06.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265890+0000", 2024-09-15T00:14:06.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0328000000000002e-05 2024-09-15T00:14:06.493 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.493 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.493 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.493 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:1294 3.13 e30/23 ondisk, result = 0)", 2024-09-15T00:14:06.493 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:14:06.493 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 160.237069744, 2024-09-15T00:14:06.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028712298000000001, 2024-09-15T00:14:06.493 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.493 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.493 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:06.494 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:14:06.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237077+0000", 2024-09-15T00:14:06.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237078+0000", 2024-09-15T00:14:06.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8100000000000001e-07 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237083+0000", 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.211e-06 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237093+0000", 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0879999999999999e-05 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.237100+0000", 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-15T00:14:06.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265765+0000", 2024-09-15T00:14:06.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028664955999999998 2024-09-15T00:14:06.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265775+0000", 2024-09-15T00:14:06.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8379999999999997e-06 2024-09-15T00:14:06.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:26.265790+0000", 2024-09-15T00:14:06.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4953999999999999e-05 2024-09-15T00:14:06.496 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.497 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.497 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.497 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:18398 3.e 3:7431e516:::rbd_data.11398e1a091a.0000000000000048:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:14:06.497 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:14:06.497 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 151.05206826099999, 2024-09-15T00:14:06.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.061184250000000003, 2024-09-15T00:14:06.497 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.497 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.497 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:06.497 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.497 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.497 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.497 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 18398 2024-09-15T00:14:06.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.498 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:14:06.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422079+0000", 2024-09-15T00:14:06.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422080+0000", 2024-09-15T00:14:06.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.274e-06 2024-09-15T00:14:06.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422086+0000", 2024-09-15T00:14:06.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:14:06.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422096+0000", 2024-09-15T00:14:06.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679999999999999e-05 2024-09-15T00:14:06.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422102+0000", 2024-09-15T00:14:06.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:14:06.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422151+0000", 2024-09-15T00:14:06.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9027999999999999e-05 2024-09-15T00:14:06.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.422276+0000", 2024-09-15T00:14:06.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000124728 2024-09-15T00:14:06.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.425800+0000", 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003523984 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483147+0000", 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.057347326999999997 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483181+0000", 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4309e-05 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483225+0000", 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3149000000000003e-05 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483263+0000", 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4409.0:18398 3.e e38/23 ondisk, result = 0)", 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 151.03919199399999, 2024-09-15T00:14:06.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048198009, 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434955+0000", 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434956+0000", 2024-09-15T00:14:06.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9800000000000002e-07 2024-09-15T00:14:06.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434961+0000", 2024-09-15T00:14:06.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6260000000000003e-06 2024-09-15T00:14:06.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434973+0000", 2024-09-15T00:14:06.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2680000000000001e-05 2024-09-15T00:14:06.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.434980+0000", 2024-09-15T00:14:06.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7150000000000001e-06 2024-09-15T00:14:06.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483118+0000", 2024-09-15T00:14:06.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.048137657 2024-09-15T00:14:06.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483126+0000", 2024-09-15T00:14:06.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0090000000000007e-06 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:35.483153+0000", 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7424e-05 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:20224 3.12 3:4b67f52d:::rbd_data.113917258088.0000000000000032:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 145.58067685, 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.031423583999999997, 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.506 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 20224 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893470+0000", 2024-09-15T00:14:06.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893471+0000", 2024-09-15T00:14:06.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.75e-07 2024-09-15T00:14:06.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893474+0000", 2024-09-15T00:14:06.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.518e-06 2024-09-15T00:14:06.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893483+0000", 2024-09-15T00:14:06.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0469999999999994e-06 2024-09-15T00:14:06.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893487+0000", 2024-09-15T00:14:06.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:14:06.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893544+0000", 2024-09-15T00:14:06.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7187e-05 2024-09-15T00:14:06.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.893662+0000", 2024-09-15T00:14:06.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118089 2024-09-15T00:14:06.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-15T00:14:06.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.895240+0000", 2024-09-15T00:14:06.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015775959999999999 2024-09-15T00:14:06.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:06.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924800+0000", 2024-09-15T00:14:06.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029560047999999998 2024-09-15T00:14:06.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:06.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924812+0000", 2024-09-15T00:14:06.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2763e-05 2024-09-15T00:14:06.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:06.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924852+0000", 2024-09-15T00:14:06.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9447999999999999e-05 2024-09-15T00:14:06.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:40.924894+0000", 2024-09-15T00:14:06.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2109e-05 2024-09-15T00:14:06.511 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.511 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.512 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.512 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:48995 3.d 3:b7ae20a4:::rbd_data.11395b9829f3.0000000000000007:e [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:06.512 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:14:06.512 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 141.86310403300001, 2024-09-15T00:14:06.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028652522999999999, 2024-09-15T00:14:06.512 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.512 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.512 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:06.512 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.512 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.512 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.512 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 48995 2024-09-15T00:14:06.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.513 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:14:06.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611043+0000", 2024-09-15T00:14:06.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611044+0000", 2024-09-15T00:14:06.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4499999999999997e-07 2024-09-15T00:14:06.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611045+0000", 2024-09-15T00:14:06.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8109999999999999e-06 2024-09-15T00:14:06.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611052+0000", 2024-09-15T00:14:06.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:14:06.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611057+0000", 2024-09-15T00:14:06.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5730000000000001e-06 2024-09-15T00:14:06.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639501+0000", 2024-09-15T00:14:06.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028444325999999999 2024-09-15T00:14:06.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639595+0000", 2024-09-15T00:14:06.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3814000000000007e-05 2024-09-15T00:14:06.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639696+0000", 2024-09-15T00:14:06.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100988 2024-09-15T00:14:06.516 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.516 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.516 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.516 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49005 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:06.516 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:14:06.516 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 141.86292146, 2024-09-15T00:14:06.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028886421999999998, 2024-09-15T00:14:06.516 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.516 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.516 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:06.516 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.516 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.517 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.517 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49005 2024-09-15T00:14:06.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.517 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:14:06.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:14:06.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611226+0000", 2024-09-15T00:14:06.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:14:06.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611233+0000", 2024-09-15T00:14:06.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-15T00:14:06.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611241+0000", 2024-09-15T00:14:06.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8990000000000001e-06 2024-09-15T00:14:06.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611247+0000", 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6869999999999999e-06 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639484+0000", 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028237547000000002 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.639579+0000", 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4746000000000004e-05 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640112+0000", 2024-09-15T00:14:06.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000533348 2024-09-15T00:14:06.520 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.520 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.520 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.520 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49006 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:06.520 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:14:06.520 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 141.86289261900001, 2024-09-15T00:14:06.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029513457999999999, 2024-09-15T00:14:06.520 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.520 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.520 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:06.521 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.521 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.521 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.521 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49006 2024-09-15T00:14:06.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.521 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:14:06.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:14:06.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611255+0000", 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611257+0000", 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1919999999999999e-06 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611264+0000", 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9600000000000003e-06 2024-09-15T00:14:06.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611269+0000", 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8049999999999997e-06 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640142+0000", 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028873044 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640225+0000", 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2894999999999998e-05 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640768+0000", 2024-09-15T00:14:06.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054302099999999998 2024-09-15T00:14:06.524 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.524 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.524 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.524 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49007 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 4718592~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:06.524 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:14:06.524 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 141.86287040799999, 2024-09-15T00:14:06.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030150518000000001, 2024-09-15T00:14:06.524 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.524 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.524 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:06.525 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.525 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.525 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.525 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49007 2024-09-15T00:14:06.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.525 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:14:06.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:14:06.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611277+0000", 2024-09-15T00:14:06.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6199999999999999e-07 2024-09-15T00:14:06.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611279+0000", 2024-09-15T00:14:06.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-15T00:14:06.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611286+0000", 2024-09-15T00:14:06.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6950000000000003e-06 2024-09-15T00:14:06.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611292+0000", 2024-09-15T00:14:06.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:14:06.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640801+0000", 2024-09-15T00:14:06.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029509746 2024-09-15T00:14:06.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.640897+0000", 2024-09-15T00:14:06.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5292999999999997e-05 2024-09-15T00:14:06.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641427+0000", 2024-09-15T00:14:06.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00053077899999999999 2024-09-15T00:14:06.528 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.528 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.529 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.529 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49008 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:06.529 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:14:06.529 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 141.86284924899999, 2024-09-15T00:14:06.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030356833, 2024-09-15T00:14:06.529 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.529 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.529 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:06.529 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.529 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.529 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.529 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49008 2024-09-15T00:14:06.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.530 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:14:06.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:14:06.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611298+0000", 2024-09-15T00:14:06.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:14:06.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611300+0000", 2024-09-15T00:14:06.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:14:06.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611307+0000", 2024-09-15T00:14:06.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4740000000000002e-06 2024-09-15T00:14:06.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611320+0000", 2024-09-15T00:14:06.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2687000000000001e-05 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641471+0000", 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030151889000000001 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641556+0000", 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4456999999999995e-05 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641655+0000", 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8858999999999997e-05 2024-09-15T00:14:06.532 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.533 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.533 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.533 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49009 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:06.533 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:14:06.533 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 141.862819301, 2024-09-15T00:14:06.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030470863000000001, 2024-09-15T00:14:06.533 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.533 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.533 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:06.533 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.533 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.533 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.533 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49009 2024-09-15T00:14:06.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.534 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:14:06.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:14:06.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611328+0000", 2024-09-15T00:14:06.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.82e-07 2024-09-15T00:14:06.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611331+0000", 2024-09-15T00:14:06.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:14:06.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611338+0000", 2024-09-15T00:14:06.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-15T00:14:06.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611343+0000", 2024-09-15T00:14:06.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.101e-06 2024-09-15T00:14:06.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641687+0000", 2024-09-15T00:14:06.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030343831000000002 2024-09-15T00:14:06.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641745+0000", 2024-09-15T00:14:06.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7435999999999997e-05 2024-09-15T00:14:06.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641799+0000", 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4191999999999999e-05 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49010 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 141.86279733699999, 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030576144, 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.537 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.538 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.538 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49010 2024-09-15T00:14:06.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.538 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:14:06.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:14:06.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611350+0000", 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.89e-07 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611352+0000", 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611358+0000", 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2830000000000001e-06 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611361+0000", 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5650000000000002e-06 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641824+0000", 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030462332000000002 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641864+0000", 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0756000000000001e-05 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641926+0000", 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1742000000000006e-05 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49011 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 141.86277991, 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030701052, 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.542 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49011 2024-09-15T00:14:06.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.542 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:14:06.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611367+0000", 2024-09-15T00:14:06.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611368+0000", 2024-09-15T00:14:06.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.08e-07 2024-09-15T00:14:06.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611370+0000", 2024-09-15T00:14:06.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0480000000000001e-06 2024-09-15T00:14:06.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611376+0000", 2024-09-15T00:14:06.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5660000000000003e-06 2024-09-15T00:14:06.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611381+0000", 2024-09-15T00:14:06.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7040000000000002e-06 2024-09-15T00:14:06.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.641960+0000", 2024-09-15T00:14:06.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030578630999999998 2024-09-15T00:14:06.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642013+0000", 2024-09-15T00:14:06.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3338999999999999e-05 2024-09-15T00:14:06.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642068+0000", 2024-09-15T00:14:06.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5356000000000003e-05 2024-09-15T00:14:06.545 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.545 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.545 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.545 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49012 3.15 3:ac1f8a6b:::rbd_data.11395b9829f3.0000000000000009:e [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:06.545 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:14:06.545 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 141.86275811100001, 2024-09-15T00:14:06.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030889294000000001, 2024-09-15T00:14:06.545 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.545 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.545 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:06.545 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.546 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.546 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.546 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49012 2024-09-15T00:14:06.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.546 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:14:06.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611389+0000", 2024-09-15T00:14:06.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611390+0000", 2024-09-15T00:14:06.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:14:06.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611392+0000", 2024-09-15T00:14:06.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.951e-06 2024-09-15T00:14:06.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611398+0000", 2024-09-15T00:14:06.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.754e-06 2024-09-15T00:14:06.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.611404+0000", 2024-09-15T00:14:06.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-15T00:14:06.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642095+0000", 2024-09-15T00:14:06.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030691551000000001 2024-09-15T00:14:06.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642180+0000", 2024-09-15T00:14:06.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4577e-05 2024-09-15T00:14:06.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.642278+0000", 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8518999999999996e-05 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:49027 3.f 3:f67a3014:::rbd_data.113916cf513.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 141.81352975199999, 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.037965413000000003, 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.549 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 49027 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660617+0000", 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.660620+0000", 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1270000000000001e-06 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664560+0000", 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039403850000000002 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664576+0000", 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6121999999999999e-05 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664585+0000", 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1540000000000008e-06 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664642+0000", 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6341999999999997e-05 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.664764+0000", 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000122399 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:06.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.665019+0000", 2024-09-15T00:14:06.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000255133 2024-09-15T00:14:06.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:06.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.690463+0000", 2024-09-15T00:14:06.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.025443757000000001 2024-09-15T00:14:06.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:06.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698468+0000", 2024-09-15T00:14:06.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008005402 2024-09-15T00:14:06.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:06.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698532+0000", 2024-09-15T00:14:06.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3794999999999995e-05 2024-09-15T00:14:06.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:11:44.698583+0000", 2024-09-15T00:14:06.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0797000000000002e-05 2024-09-15T00:14:06.554 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.554 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.554 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.554 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:177892 1.0 1:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-15T00:14:06.554 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:13:48.311833+0000", 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.162313753999999, 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.90705656700000004, 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 177892 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311833+0000", 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311833+0000", 2024-09-15T00:14:06.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311836+0000", 2024-09-15T00:14:06.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2400000000000002e-06 2024-09-15T00:14:06.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311842+0000", 2024-09-15T00:14:06.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9540000000000001e-06 2024-09-15T00:14:06.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311858+0000", 2024-09-15T00:14:06.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6424999999999999e-05 2024-09-15T00:14:06.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311866+0000", 2024-09-15T00:14:06.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1289999999999996e-06 2024-09-15T00:14:06.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311944+0000", 2024-09-15T00:14:06.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8158000000000006e-05 2024-09-15T00:14:06.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:14:06.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:48.311953+0000", 2024-09-15T00:14:06.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8759999999999994e-06 2024-09-15T00:14:06.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.218787+0000", 2024-09-15T00:14:06.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.90683381299999999 2024-09-15T00:14:06.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.218890+0000", 2024-09-15T00:14:06.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000102972 2024-09-15T00:14:06.559 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.559 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.559 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.559 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:177895 3.14 3:292bb7d6:::rbd_header.113916cf513:head [watch reconnect cookie 139649723465488 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e161)", 2024-09-15T00:14:06.559 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:13:49.208730+0000", 2024-09-15T00:14:06.559 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.265416912999999, 2024-09-15T00:14:06.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.82280920000000002, 2024-09-15T00:14:06.559 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.559 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.559 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-15T00:14:06.559 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.559 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 177895 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208730+0000", 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208730+0000", 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208732+0000", 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5829999999999999e-06 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208738+0000", 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3690000000000003e-06 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208753+0000", 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5051999999999999e-05 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208768+0000", 2024-09-15T00:14:06.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4243999999999999e-05 2024-09-15T00:14:06.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208823+0000", 2024-09-15T00:14:06.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5652999999999997e-05 2024-09-15T00:14:06.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-15T00:14:06.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.208831+0000", 2024-09-15T00:14:06.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7309999999999993e-06 2024-09-15T00:14:06.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.214458+0000", 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0056266780000000004 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.214468+0000", 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0631e-05 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.025026+0000", 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.81055788500000003 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.025208+0000", 2024-09-15T00:14:06.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000181829 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.031540+0000", 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0063315450000000001 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4409.0:177896 3.14 3:292bb7d6:::rbd_header.113916cf513:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e161)", 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-15T00:13:49.219101+0000", 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.255045991999999, 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.81263984899999997, 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: "continuous": false, 2024-09-15T00:14:06.564 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-15T00:14:06.565 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:06.565 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-15T00:14:06.565 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4409", 2024-09-15T00:14:06.565 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:06.565 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 177896 2024-09-15T00:14:06.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.565 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-15T00:14:06.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-15T00:14:06.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219101+0000", 2024-09-15T00:14:06.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-15T00:14:06.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219101+0000", 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219103+0000", 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.449e-06 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219108+0000", 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5210000000000002e-06 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219119+0000", 2024-09-15T00:14:06.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0641e-05 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219126+0000", 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7290000000000003e-06 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219166+0000", 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0246000000000001e-05 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:49.219172+0000", 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2230000000000004e-06 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-15T00:14:06.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.027379+0000", 2024-09-15T00:14:06.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.80820711700000003 2024-09-15T00:14:06.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-15T00:14:06.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.027474+0000", 2024-09-15T00:14:06.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5111999999999993e-05 2024-09-15T00:14:06.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:06.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.027727+0000", 2024-09-15T00:14:06.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025279600000000001 2024-09-15T00:14:06.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-15T00:14:06.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.031417+0000", 2024-09-15T00:14:06.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036897980000000002 2024-09-15T00:14:06.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:06.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.031635+0000", 2024-09-15T00:14:06.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021787799999999999 2024-09-15T00:14:06.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-15T00:14:06.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.031693+0000", 2024-09-15T00:14:06.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8452999999999998e-05 2024-09-15T00:14:06.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-15T00:14:06.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-15T00:14:06.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-15T00:14:06.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-15T00:13:50.031741+0000", 2024-09-15T00:14:06.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7886000000000002e-05 2024-09-15T00:14:06.570 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.570 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.570 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.570 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-15T00:14:06.570 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-15T00:14:06.570 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:14:06.571 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:14:06.572 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:14:06.569+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:14:06.672 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:14:06.669+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:14:06.676 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:06.677 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:14:06.677 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:14:06.677 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:06.683 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:14:06.772 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:14:06.770+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:14:06.828 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:06.828 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:14:06.828 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:14:06.828 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:14:06.829 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:06.836 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:14:06.878 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:14:06.875+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:14:06.974 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:14:06.971+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:14:06.983 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:14:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:14:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:14:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9, version: 37'47, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ccfe450e:::rbd_data.1139e2e18583.0000000000000003:9@37'47, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:14:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.126498654000002, 2024-09-15T00:14:06.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054905559999999997, 2024-09-15T00:14:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:14:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853792+0000", 2024-09-15T00:14:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:06.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.853793+0000", 2024-09-15T00:14:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9599999999999999e-07 2024-09-15T00:14:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854798+0000", 2024-09-15T00:14:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010050770000000001 2024-09-15T00:14:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854814+0000", 2024-09-15T00:14:06.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6478999999999999e-05 2024-09-15T00:14:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854825+0000", 2024-09-15T00:14:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0492999999999999e-05 2024-09-15T00:14:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854889+0000", 2024-09-15T00:14:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4134999999999995e-05 2024-09-15T00:14:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:06.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.854898+0000", 2024-09-15T00:14:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6260000000000001e-06 2024-09-15T00:14:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.859283+0000", 2024-09-15T00:14:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043852500000000003 2024-09-15T00:14:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15, version: 59'205, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce7d3ed7:::rbd_data.113947ee3a89.0000000000000008:15@59'205, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:14:06.987 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.000278199, 2024-09-15T00:14:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059047939999999997, 2024-09-15T00:14:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:14:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:14:06.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980013+0000", 2024-09-15T00:14:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.44e-07 2024-09-15T00:14:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980977+0000", 2024-09-15T00:14:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00096402300000000005 2024-09-15T00:14:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.980995+0000", 2024-09-15T00:14:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7214000000000002e-05 2024-09-15T00:14:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981007+0000", 2024-09-15T00:14:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2028000000000001e-05 2024-09-15T00:14:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981069+0000", 2024-09-15T00:14:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1977000000000006e-05 2024-09-15T00:14:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.981077+0000", 2024-09-15T00:14:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2120000000000003e-06 2024-09-15T00:14:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:06.985918+0000", 2024-09-15T00:14:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048406960000000002 2024-09-15T00:14:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head, version: 62'221, data_included: [0~262144,393216~786432], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:cc6525bf:::rbd_data.1139e6d65aff.0000000000000004:head@62'221, size: 1572864, copy_subset: [(0, 1572864)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:06.991 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:14:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.978009933999999, 2024-09-15T00:14:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054170770000000002, 2024-09-15T00:14:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:14:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:06.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002281+0000", 2024-09-15T00:14:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.002282+0000", 2024-09-15T00:14:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4499999999999997e-07 2024-09-15T00:14:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003196+0000", 2024-09-15T00:14:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091476399999999999 2024-09-15T00:14:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:06.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003213+0000", 2024-09-15T00:14:06.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6554000000000001e-05 2024-09-15T00:14:06.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003230+0000", 2024-09-15T00:14:06.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6617999999999999e-05 2024-09-15T00:14:06.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:06.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003292+0000", 2024-09-15T00:14:06.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2193999999999994e-05 2024-09-15T00:14:06.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.003300+0000", 2024-09-15T00:14:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1429999999999998e-06 2024-09-15T00:14:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.007698+0000", 2024-09-15T00:14:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043981589999999996 2024-09-15T00:14:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:06.995 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:14:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.927279853999998, 2024-09-15T00:14:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067633270000000004, 2024-09-15T00:14:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:14:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053011+0000", 2024-09-15T00:14:06.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.053012+0000", 2024-09-15T00:14:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1659999999999999e-06 2024-09-15T00:14:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054086+0000", 2024-09-15T00:14:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010732789999999999 2024-09-15T00:14:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054100+0000", 2024-09-15T00:14:06.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4123e-05 2024-09-15T00:14:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054114+0000", 2024-09-15T00:14:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4442e-05 2024-09-15T00:14:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054192+0000", 2024-09-15T00:14:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8288000000000007e-05 2024-09-15T00:14:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.054200+0000", 2024-09-15T00:14:06.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-15T00:14:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.059774+0000", 2024-09-15T00:14:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055741230000000003 2024-09-15T00:14:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:14:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:14:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.918962526000001, 2024-09-15T00:14:07.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049777950000000001, 2024-09-15T00:14:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:14:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061328+0000", 2024-09-15T00:14:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.061330+0000", 2024-09-15T00:14:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.11e-06 2024-09-15T00:14:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062350+0000", 2024-09-15T00:14:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010207969999999999 2024-09-15T00:14:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062365+0000", 2024-09-15T00:14:07.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4157e-05 2024-09-15T00:14:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062380+0000", 2024-09-15T00:14:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5129e-05 2024-09-15T00:14:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062467+0000", 2024-09-15T00:14:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7247e-05 2024-09-15T00:14:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.062475+0000", 2024-09-15T00:14:07.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4300000000000006e-06 2024-09-15T00:14:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.066306+0000", 2024-09-15T00:14:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038309250000000002 2024-09-15T00:14:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:14:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:14:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.913079568000001, 2024-09-15T00:14:07.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052454889999999999, 2024-09-15T00:14:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:14:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067211+0000", 2024-09-15T00:14:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.067212+0000", 2024-09-15T00:14:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-07 2024-09-15T00:14:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068119+0000", 2024-09-15T00:14:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090671199999999995 2024-09-15T00:14:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068135+0000", 2024-09-15T00:14:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5739e-05 2024-09-15T00:14:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068150+0000", 2024-09-15T00:14:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5693e-05 2024-09-15T00:14:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068236+0000", 2024-09-15T00:14:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6100000000000006e-05 2024-09-15T00:14:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.068245+0000", 2024-09-15T00:14:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1960000000000008e-06 2024-09-15T00:14:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.072457+0000", 2024-09-15T00:14:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042123719999999998 2024-09-15T00:14:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:14:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:14:07.008 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.907101218999998, 2024-09-15T00:14:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049532020000000003, 2024-09-15T00:14:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:14:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:14:07.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.073190+0000", 2024-09-15T00:14:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4400000000000001e-07 2024-09-15T00:14:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074164+0000", 2024-09-15T00:14:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097354400000000002 2024-09-15T00:14:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074178+0000", 2024-09-15T00:14:07.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4134999999999999e-05 2024-09-15T00:14:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074192+0000", 2024-09-15T00:14:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.451e-05 2024-09-15T00:14:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074272+0000", 2024-09-15T00:14:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9989999999999998e-05 2024-09-15T00:14:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.074280+0000", 2024-09-15T00:14:07.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9990000000000008e-06 2024-09-15T00:14:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078143+0000", 2024-09-15T00:14:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00386258 2024-09-15T00:14:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [4194304~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:14:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:14:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.901390407000001, 2024-09-15T00:14:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049408730000000001, 2024-09-15T00:14:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.013 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:14:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:14:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.078901+0000", 2024-09-15T00:14:07.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2399999999999999e-07 2024-09-15T00:14:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079875+0000", 2024-09-15T00:14:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097349100000000005 2024-09-15T00:14:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079889+0000", 2024-09-15T00:14:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4148000000000001e-05 2024-09-15T00:14:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079904+0000", 2024-09-15T00:14:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5159e-05 2024-09-15T00:14:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079964+0000", 2024-09-15T00:14:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0204999999999997e-05 2024-09-15T00:14:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.079971+0000", 2024-09-15T00:14:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2490000000000004e-06 2024-09-15T00:14:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.083841+0000", 2024-09-15T00:14:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038701970000000001 2024-09-15T00:14:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.13 118/117 [PushOp(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f, version: 77'332, data_included: [5242880~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ce4ebbe0:::rbd_data.1139d7422821.0000000000000000:1f@77'332, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 6291456, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:14:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:14:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.895685426999997, 2024-09-15T00:14:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052525740000000003, 2024-09-15T00:14:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.017 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:14:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:14:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.084606+0000", 2024-09-15T00:14:07.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085579+0000", 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097307600000000002 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085592+0000", 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3277e-05 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085608+0000", 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5773e-05 2024-09-15T00:14:07.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085687+0000", 2024-09-15T00:14:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8956999999999996e-05 2024-09-15T00:14:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.085694+0000", 2024-09-15T00:14:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.956e-06 2024-09-15T00:14:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.089858+0000", 2024-09-15T00:14:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041641159999999998 2024-09-15T00:14:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:172552 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-15T00:14:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:14:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.791228119000003, 2024-09-15T00:14:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.997031471, 2024-09-15T00:14:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.021 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 172552 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189063+0000", 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189064+0000", 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.198e-06 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189069+0000", 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9069999999999998e-06 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189080+0000", 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0541e-05 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189086+0000", 2024-09-15T00:14:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5429999999999999e-06 2024-09-15T00:14:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189128+0000", 2024-09-15T00:14:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2255000000000001e-05 2024-09-15T00:14:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:14:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.189135+0000", 2024-09-15T00:14:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6200000000000001e-06 2024-09-15T00:14:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185787+0000", 2024-09-15T00:14:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99665157199999999 2024-09-15T00:14:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.185926+0000", 2024-09-15T00:14:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013970399999999999 2024-09-15T00:14:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:08.186094+0000", 2024-09-15T00:14:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016813099999999999 2024-09-15T00:14:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.025 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9, version: 37'60, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9@37'60, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.732367760000002, 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049454060000000003, 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247923+0000", 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.247924+0000", 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248394+0000", 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047063399999999999 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248411+0000", 2024-09-15T00:14:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6384000000000001e-05 2024-09-15T00:14:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248441+0000", 2024-09-15T00:14:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0559999999999999e-05 2024-09-15T00:14:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248499+0000", 2024-09-15T00:14:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7274000000000001e-05 2024-09-15T00:14:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.248507+0000", 2024-09-15T00:14:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2390000000000002e-06 2024-09-15T00:14:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.252869+0000", 2024-09-15T00:14:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043617200000000004 2024-09-15T00:14:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.029 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head, version: 59'393, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:075f84e1:::rbd_data.11396cb1f0e4.0000000000000005:head@59'393, size: 5701632, copy_subset: [(0, 5701632)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:14:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.602919493000002, 2024-09-15T00:14:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004807469, 2024-09-15T00:14:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:14:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:14:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377372+0000", 2024-09-15T00:14:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4799999999999998e-07 2024-09-15T00:14:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377830+0000", 2024-09-15T00:14:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000457954 2024-09-15T00:14:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377847+0000", 2024-09-15T00:14:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6741000000000001e-05 2024-09-15T00:14:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377858+0000", 2024-09-15T00:14:07.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0951999999999999e-05 2024-09-15T00:14:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377929+0000", 2024-09-15T00:14:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0697000000000004e-05 2024-09-15T00:14:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.377941+0000", 2024-09-15T00:14:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2529e-05 2024-09-15T00:14:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.382179+0000", 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042379480000000001 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 119/117 [PushOp(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head, version: 88'473, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:064c1083:::rbd_data.1139f5381d71.0000000000000002:head@88'473, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.520800059999999, 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005823272, 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.034 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459491+0000", 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459492+0000", 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9800000000000003e-07 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459940+0000", 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044854699999999999 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459957+0000", 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6586e-05 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.459968+0000", 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.145e-05 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460043+0000", 2024-09-15T00:14:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4451999999999998e-05 2024-09-15T00:14:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.460057+0000", 2024-09-15T00:14:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.417e-05 2024-09-15T00:14:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.465314+0000", 2024-09-15T00:14:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052574689999999999 2024-09-15T00:14:07.037 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:14:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.285623878999999, 2024-09-15T00:14:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051811399999999999, 2024-09-15T00:14:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.038 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:14:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694667+0000", 2024-09-15T00:14:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.694668+0000", 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4600000000000005e-07 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695136+0000", 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00046864399999999999 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695153+0000", 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6218e-05 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695189+0000", 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6393000000000002e-05 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695263+0000", 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4175e-05 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.695271+0000", 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2800000000000003e-06 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.699848+0000", 2024-09-15T00:14:07.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045768839999999998 2024-09-15T00:14:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10, version: 50'212, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:220ba920:::rbd_data.1139ee4d340b.0000000000000003:10@50'212, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:14:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:14:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.277525670999999, 2024-09-15T00:14:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048716510000000003, 2024-09-15T00:14:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:14:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702765+0000", 2024-09-15T00:14:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.702783+0000", 2024-09-15T00:14:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7268000000000001e-05 2024-09-15T00:14:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703221+0000", 2024-09-15T00:14:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00043866899999999998 2024-09-15T00:14:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703237+0000", 2024-09-15T00:14:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5553000000000002e-05 2024-09-15T00:14:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703247+0000", 2024-09-15T00:14:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0392e-05 2024-09-15T00:14:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703358+0000", 2024-09-15T00:14:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110676 2024-09-15T00:14:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.703369+0000", 2024-09-15T00:14:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0716e-05 2024-09-15T00:14:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.707637+0000", 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042683770000000003 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18, version: 62'274, data_included: [0~786432], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21f1e967:::rbd_data.11392b39946e.000000000000000b:18@62'274, size: 10354688, copy_subset: [(0, 10354688)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 10354688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.231444588999999, 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005453014, 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:14:07.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748846+0000", 2024-09-15T00:14:07.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.748847+0000", 2024-09-15T00:14:07.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8299999999999997e-07 2024-09-15T00:14:07.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749010+0000", 2024-09-15T00:14:07.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000163144 2024-09-15T00:14:07.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749027+0000", 2024-09-15T00:14:07.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6518999999999999e-05 2024-09-15T00:14:07.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749037+0000", 2024-09-15T00:14:07.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0438000000000001e-05 2024-09-15T00:14:07.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749112+0000", 2024-09-15T00:14:07.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4946999999999999e-05 2024-09-15T00:14:07.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.749123+0000", 2024-09-15T00:14:07.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1014e-05 2024-09-15T00:14:07.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.754299+0000", 2024-09-15T00:14:07.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051763690000000001 2024-09-15T00:14:07.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.050 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.050 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20, version: 80'334, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:21619037:::rbd_data.11396481e1ec.0000000000000006:20@80'334, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 20=[]:{20: [20]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:07.050 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:14:07.050 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.176807545000003, 2024-09-15T00:14:07.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048933570000000001, 2024-09-15T00:14:07.050 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.050 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803483+0000", 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803484+0000", 2024-09-15T00:14:07.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803938+0000", 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00045364599999999999 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803954+0000", 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6456999999999998e-05 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.803965+0000", 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.076e-05 2024-09-15T00:14:07.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804023+0000", 2024-09-15T00:14:07.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7787000000000001e-05 2024-09-15T00:14:07.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.804031+0000", 2024-09-15T00:14:07.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2160000000000006e-06 2024-09-15T00:14:07.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.808377+0000", 2024-09-15T00:14:07.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043458969999999996 2024-09-15T00:14:07.054 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.054 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.054 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.054 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head, version: 83'362, data_included: [1179648~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:24cf0de3:::rbd_data.1139312f7f15.0000000000000000:head@83'362, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2228224, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1179648, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T00:14:07.054 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:14:07.054 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.131996901999997, 2024-09-15T00:14:07.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053056819999999999, 2024-09-15T00:14:07.054 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.055 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.055 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.055 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:14:07.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848294+0000", 2024-09-15T00:14:07.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848295+0000", 2024-09-15T00:14:07.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4000000000000001e-07 2024-09-15T00:14:07.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848728+0000", 2024-09-15T00:14:07.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000433709 2024-09-15T00:14:07.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848745+0000", 2024-09-15T00:14:07.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6167999999999998e-05 2024-09-15T00:14:07.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848788+0000", 2024-09-15T00:14:07.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3269000000000002e-05 2024-09-15T00:14:07.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848857+0000", 2024-09-15T00:14:07.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9453000000000001e-05 2024-09-15T00:14:07.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.848866+0000", 2024-09-15T00:14:07.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2779999999999993e-06 2024-09-15T00:14:07.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.853600+0000", 2024-09-15T00:14:07.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047341650000000002 2024-09-15T00:14:07.058 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.058 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.058 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.058 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.4 119/117 [PushOp(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head, version: 93'455, data_included: [0~98304,114688~16384,147456~589824,753664~65536,835584~16384,868352~262144], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:21d388e1:::rbd_data.1139646a7d4e.0000000000000008:head@93'455, size: 2342912, copy_subset: [(0, 2342912)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1130496, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:07.058 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:14:07.058 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.072862876000002, 2024-09-15T00:14:07.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063250900000000002, 2024-09-15T00:14:07.058 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.058 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.058 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.059 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:14:07.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907428+0000", 2024-09-15T00:14:07.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.907429+0000", 2024-09-15T00:14:07.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.85e-07 2024-09-15T00:14:07.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908008+0000", 2024-09-15T00:14:07.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00057949599999999998 2024-09-15T00:14:07.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908026+0000", 2024-09-15T00:14:07.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7700999999999999e-05 2024-09-15T00:14:07.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908038+0000", 2024-09-15T00:14:07.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2131e-05 2024-09-15T00:14:07.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908101+0000", 2024-09-15T00:14:07.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2596999999999997e-05 2024-09-15T00:14:07.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.908110+0000", 2024-09-15T00:14:07.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8540000000000003e-06 2024-09-15T00:14:07.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:07.913753+0000", 2024-09-15T00:14:07.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056435260000000003 2024-09-15T00:14:07.062 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.062 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.062 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.062 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.0 165/164 [PushOp(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9, version: 37'60, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:01a0675c:::rbd_data.1139e2e18583.0000000000000002:9@37'60, size: 1925120, copy_subset: [(0, 1925120)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T00:14:07.062 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:53.061053+0000", 2024-09-15T00:14:07.063 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.919237591, 2024-09-15T00:14:07.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054243870000000001, 2024-09-15T00:14:07.063 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.063 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.063 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.063 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061053+0000", 2024-09-15T00:14:07.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061053+0000", 2024-09-15T00:14:07.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061054+0000", 2024-09-15T00:14:07.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5499999999999997e-07 2024-09-15T00:14:07.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061854+0000", 2024-09-15T00:14:07.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00080025500000000004 2024-09-15T00:14:07.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061880+0000", 2024-09-15T00:14:07.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5224000000000001e-05 2024-09-15T00:14:07.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061898+0000", 2024-09-15T00:14:07.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8484000000000001e-05 2024-09-15T00:14:07.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061983+0000", 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4845999999999994e-05 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.061993+0000", 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5419999999999994e-06 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:53.066478+0000", 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044852809999999998 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.066 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.067 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.067 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:07.068 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:14:07.109 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:14:07.106+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:14:07.176 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-15T00:14:07.174+0000 7fc51738f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33608) UID: 0 2024-09-15T00:14:07.219 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:07.219 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:14:07.219 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:14:07.219 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:07.228 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:14:07.262 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 0, 7, 2] out_osds: [1] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-15T00:14:07.262 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:14:07.262 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [3, 4, 5, 6, 0, 7, 2] 2024-09-15T00:14:07.262 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-15T00:14:07.277 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:14:07.275+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:14:07.334 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:07.334 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:14:07.335 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:14:07.335 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:14:07.335 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:07.344 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:14:07.388 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:14:07.386+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:14:07.454 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:14:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:14:07.454 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:14:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133834 3.1b e85/69 3:da9a78a3:::rbd_data.11395dfcf2b8.0000000000000000:head v 85'356, mlcod=85'354)", 2024-09-15T00:14:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:14:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.887756693, 2024-09-15T00:14:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026079054000000001, 2024-09-15T00:14:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:14:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563336+0000", 2024-09-15T00:14:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.563338+0000", 2024-09-15T00:14:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.418e-06 2024-09-15T00:14:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569385+0000", 2024-09-15T00:14:07.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006047335 2024-09-15T00:14:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569418+0000", 2024-09-15T00:14:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2957999999999999e-05 2024-09-15T00:14:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569432+0000", 2024-09-15T00:14:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3718e-05 2024-09-15T00:14:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569519+0000", 2024-09-15T00:14:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7022999999999997e-05 2024-09-15T00:14:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.569540+0000", 2024-09-15T00:14:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1426000000000001e-05 2024-09-15T00:14:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589362+0000", 2024-09-15T00:14:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019821398 2024-09-15T00:14:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.589415+0000", 2024-09-15T00:14:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3777999999999999e-05 2024-09-15T00:14:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:133836 3.1 e85/67 3:849f9a98:::rbd_data.11395dfcf2b8.0000000000000002:head v 85'304, mlcod=84'301)", 2024-09-15T00:14:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:14:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.880061783000002, 2024-09-15T00:14:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023231600000000002, 2024-09-15T00:14:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.459 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:14:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571031+0000", 2024-09-15T00:14:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.571032+0000", 2024-09-15T00:14:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-15T00:14:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576635+0000", 2024-09-15T00:14:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056028700000000002 2024-09-15T00:14:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576650+0000", 2024-09-15T00:14:07.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.505e-05 2024-09-15T00:14:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576658+0000", 2024-09-15T00:14:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6879999999999999e-06 2024-09-15T00:14:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576717+0000", 2024-09-15T00:14:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175000000000002e-05 2024-09-15T00:14:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.576742+0000", 2024-09-15T00:14:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4734000000000001e-05 2024-09-15T00:14:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594234+0000", 2024-09-15T00:14:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017492527000000001 2024-09-15T00:14:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:30.594263+0000", 2024-09-15T00:14:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8629e-05 2024-09-15T00:14:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146609 3.b e98/69 3:d39b42ba:::rbd_data.11394501364.0000000000000001:head v 98'283, mlcod=93'281)", 2024-09-15T00:14:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:14:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.970343667999998, 2024-09-15T00:14:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020892945, 2024-09-15T00:14:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:14:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480749+0000", 2024-09-15T00:14:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.480751+0000", 2024-09-15T00:14:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6509999999999999e-06 2024-09-15T00:14:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482738+0000", 2024-09-15T00:14:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019867410000000002 2024-09-15T00:14:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482754+0000", 2024-09-15T00:14:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6171000000000002e-05 2024-09-15T00:14:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482769+0000", 2024-09-15T00:14:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4688000000000001e-05 2024-09-15T00:14:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482835+0000", 2024-09-15T00:14:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6526000000000006e-05 2024-09-15T00:14:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.482858+0000", 2024-09-15T00:14:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2959000000000002e-05 2024-09-15T00:14:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501587+0000", 2024-09-15T00:14:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018728471 2024-09-15T00:14:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.501642+0000", 2024-09-15T00:14:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5738000000000001e-05 2024-09-15T00:14:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:146835 3.12 e98/68 3:4eee39d6:::rbd_data.11394501364.0000000000000000:head v 98'474, mlcod=93'472)", 2024-09-15T00:14:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:14:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.861716397999999, 2024-09-15T00:14:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020080434000000001, 2024-09-15T00:14:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.468 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:14:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589377+0000", 2024-09-15T00:14:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.589378+0000", 2024-09-15T00:14:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7770000000000001e-06 2024-09-15T00:14:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590678+0000", 2024-09-15T00:14:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012991750000000001 2024-09-15T00:14:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590699+0000", 2024-09-15T00:14:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1353e-05 2024-09-15T00:14:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590710+0000", 2024-09-15T00:14:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1245e-05 2024-09-15T00:14:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590800+0000", 2024-09-15T00:14:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9963000000000006e-05 2024-09-15T00:14:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.590823+0000", 2024-09-15T00:14:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3289e-05 2024-09-15T00:14:07.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609368+0000", 2024-09-15T00:14:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018545078 2024-09-15T00:14:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.609457+0000", 2024-09-15T00:14:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8554000000000006e-05 2024-09-15T00:14:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146837 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7201319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:14:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:14:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.850936934999993, 2024-09-15T00:14:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024751038, 2024-09-15T00:14:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146837 2024-09-15T00:14:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:14:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600156+0000", 2024-09-15T00:14:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.600157+0000", 2024-09-15T00:14:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:14:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601965+0000", 2024-09-15T00:14:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001807613 2024-09-15T00:14:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601985+0000", 2024-09-15T00:14:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0679e-05 2024-09-15T00:14:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.601998+0000", 2024-09-15T00:14:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2955999999999999e-05 2024-09-15T00:14:07.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602070+0000", 2024-09-15T00:14:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1958000000000005e-05 2024-09-15T00:14:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602237+0000", 2024-09-15T00:14:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016621299999999999 2024-09-15T00:14:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.602799+0000", 2024-09-15T00:14:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000562723 2024-09-15T00:14:07.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:14:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.620811+0000", 2024-09-15T00:14:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018011207000000001 2024-09-15T00:14:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624793+0000", 2024-09-15T00:14:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0039828770000000001 2024-09-15T00:14:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624855+0000", 2024-09-15T00:14:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.143e-05 2024-09-15T00:14:07.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.624907+0000", 2024-09-15T00:14:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2262000000000002e-05 2024-09-15T00:14:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:146936 3.6 3:67feb552:::rbd_data.11394501364.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-15T00:14:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:14:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.697423615999995, 2024-09-15T00:14:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056847126999999997, 2024-09-15T00:14:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.478 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 146936 2024-09-15T00:14:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:14:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753669+0000", 2024-09-15T00:14:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753671+0000", 2024-09-15T00:14:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2839999999999999e-06 2024-09-15T00:14:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753677+0000", 2024-09-15T00:14:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0260000000000004e-06 2024-09-15T00:14:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753691+0000", 2024-09-15T00:14:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3891e-05 2024-09-15T00:14:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753698+0000", 2024-09-15T00:14:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8480000000000001e-06 2024-09-15T00:14:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753750+0000", 2024-09-15T00:14:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1338000000000002e-05 2024-09-15T00:14:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.753814+0000", 2024-09-15T00:14:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4642999999999998e-05 2024-09-15T00:14:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.756359+0000", 2024-09-15T00:14:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025450619999999998 2024-09-15T00:14:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810435+0000", 2024-09-15T00:14:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.054075397999999997 2024-09-15T00:14:07.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:14:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810446+0000", 2024-09-15T00:14:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1005000000000001e-05 2024-09-15T00:14:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810482+0000", 2024-09-15T00:14:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6494e-05 2024-09-15T00:14:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810517+0000", 2024-09-15T00:14:07.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4137999999999997e-05 2024-09-15T00:14:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:146936 3.6 e98/97 ondisk, result = 0)", 2024-09-15T00:14:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:14:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.687640561999999, 2024-09-15T00:14:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046987205999999997, 2024-09-15T00:14:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:14:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763452+0000", 2024-09-15T00:14:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763453+0000", 2024-09-15T00:14:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5199999999999996e-07 2024-09-15T00:14:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763457+0000", 2024-09-15T00:14:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8569999999999997e-06 2024-09-15T00:14:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763469+0000", 2024-09-15T00:14:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1977999999999999e-05 2024-09-15T00:14:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.763476+0000", 2024-09-15T00:14:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:14:07.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810423+0000", 2024-09-15T00:14:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046946993999999999 2024-09-15T00:14:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810431+0000", 2024-09-15T00:14:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0339999999999997e-06 2024-09-15T00:14:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:44.810440+0000", 2024-09-15T00:14:07.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9989999999999997e-06 2024-09-15T00:14:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:151036 3.1f e103/68 3:f84a1a09:::rbd_data.113958cb0e55.000000000000000c:head v 103'452, mlcod=100'450)", 2024-09-15T00:14:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:14:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.114294333999993, 2024-09-15T00:14:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020618753, 2024-09-15T00:14:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:14:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:14:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.336799+0000", 2024-09-15T00:14:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:14:07.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340692+0000", 2024-09-15T00:14:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038928890000000001 2024-09-15T00:14:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340704+0000", 2024-09-15T00:14:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1497e-05 2024-09-15T00:14:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340709+0000", 2024-09-15T00:14:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1529999999999996e-06 2024-09-15T00:14:07.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340789+0000", 2024-09-15T00:14:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9907999999999995e-05 2024-09-15T00:14:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.340810+0000", 2024-09-15T00:14:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1837000000000001e-05 2024-09-15T00:14:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357368+0000", 2024-09-15T00:14:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016557199000000002 2024-09-15T00:14:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.357417+0000", 2024-09-15T00:14:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9783000000000001e-05 2024-09-15T00:14:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:151042 3.6 3:6419cde4:::rbd_data.113958cb0e55.0000000000000031:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T00:14:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:14:07.492 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.085216216000006, 2024-09-15T00:14:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025627375000000001, 2024-09-15T00:14:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 151042 2024-09-15T00:14:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:14:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:14:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365877+0000", 2024-09-15T00:14:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7899999999999998e-07 2024-09-15T00:14:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365881+0000", 2024-09-15T00:14:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9770000000000002e-06 2024-09-15T00:14:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365890+0000", 2024-09-15T00:14:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5920000000000001e-06 2024-09-15T00:14:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365894+0000", 2024-09-15T00:14:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4569999999999998e-06 2024-09-15T00:14:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365927+0000", 2024-09-15T00:14:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2036000000000003e-05 2024-09-15T00:14:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.365970+0000", 2024-09-15T00:14:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3197e-05 2024-09-15T00:14:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.367432+0000", 2024-09-15T00:14:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014621720000000001 2024-09-15T00:14:07.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:14:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391358+0000", 2024-09-15T00:14:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023925921999999999 2024-09-15T00:14:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391415+0000", 2024-09-15T00:14:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7253999999999998e-05 2024-09-15T00:14:07.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391456+0000", 2024-09-15T00:14:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0497999999999997e-05 2024-09-15T00:14:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391504+0000", 2024-09-15T00:14:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8591000000000003e-05 2024-09-15T00:14:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:151042 3.6 e103/97 ondisk, result = 0)", 2024-09-15T00:14:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:14:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 78.079473203000006, 2024-09-15T00:14:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019892730000000001, 2024-09-15T00:14:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:14:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:14:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371620+0000", 2024-09-15T00:14:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8100000000000003e-07 2024-09-15T00:14:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371623+0000", 2024-09-15T00:14:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6290000000000002e-06 2024-09-15T00:14:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371632+0000", 2024-09-15T00:14:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6379999999999993e-06 2024-09-15T00:14:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.371636+0000", 2024-09-15T00:14:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:14:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391403+0000", 2024-09-15T00:14:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019766982999999998 2024-09-15T00:14:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391411+0000", 2024-09-15T00:14:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7309999999999993e-06 2024-09-15T00:14:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.391513+0000", 2024-09-15T00:14:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101648 2024-09-15T00:14:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.505 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152025 3.3 e103/68 3:c0279197:::rbd_data.113958cb0e55.0000000000000007:head v 103'464, mlcod=102'461)", 2024-09-15T00:14:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:14:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.934985607000002, 2024-09-15T00:14:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021264476000000001, 2024-09-15T00:14:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:14:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516107+0000", 2024-09-15T00:14:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.516109+0000", 2024-09-15T00:14:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.666e-06 2024-09-15T00:14:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.519997+0000", 2024-09-15T00:14:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003888449 2024-09-15T00:14:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520016+0000", 2024-09-15T00:14:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8059000000000001e-05 2024-09-15T00:14:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520026+0000", 2024-09-15T00:14:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9599999999999995e-06 2024-09-15T00:14:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520089+0000", 2024-09-15T00:14:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3074e-05 2024-09-15T00:14:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.520108+0000", 2024-09-15T00:14:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8981999999999999e-05 2024-09-15T00:14:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537326+0000", 2024-09-15T00:14:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017218102999999998 2024-09-15T00:14:07.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.537372+0000", 2024-09-15T00:14:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6183000000000002e-05 2024-09-15T00:14:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152284 3.f e103/68 3:f4272b5d:::rbd_data.113958cb0e55.0000000000000006:head v 103'370, mlcod=103'368)", 2024-09-15T00:14:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:14:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.886509075999996, 2024-09-15T00:14:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020145026999999999, 2024-09-15T00:14:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.510 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:14:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564584+0000", 2024-09-15T00:14:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.564585+0000", 2024-09-15T00:14:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7000000000000003e-07 2024-09-15T00:14:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568352+0000", 2024-09-15T00:14:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037675909999999998 2024-09-15T00:14:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568370+0000", 2024-09-15T00:14:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7158000000000001e-05 2024-09-15T00:14:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568378+0000", 2024-09-15T00:14:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6449999999999994e-06 2024-09-15T00:14:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568454+0000", 2024-09-15T00:14:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5285999999999997e-05 2024-09-15T00:14:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.568472+0000", 2024-09-15T00:14:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8447000000000001e-05 2024-09-15T00:14:07.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584693+0000", 2024-09-15T00:14:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016221467999999999 2024-09-15T00:14:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.584729+0000", 2024-09-15T00:14:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5462e-05 2024-09-15T00:14:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:152500 3.17 e103/68 3:edff5adf:::rbd_data.113958cb0e55.000000000000000d:head v 103'461, mlcod=103'459)", 2024-09-15T00:14:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:14:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.809522490999996, 2024-09-15T00:14:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020217809, 2024-09-15T00:14:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:14:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641570+0000", 2024-09-15T00:14:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.641572+0000", 2024-09-15T00:14:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-15T00:14:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644961+0000", 2024-09-15T00:14:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033882610000000001 2024-09-15T00:14:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644979+0000", 2024-09-15T00:14:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8179e-05 2024-09-15T00:14:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.644987+0000", 2024-09-15T00:14:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8860000000000004e-06 2024-09-15T00:14:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645044+0000", 2024-09-15T00:14:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7141000000000001e-05 2024-09-15T00:14:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.645062+0000", 2024-09-15T00:14:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8258e-05 2024-09-15T00:14:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661749+0000", 2024-09-15T00:14:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016686553999999999 2024-09-15T00:14:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.661788+0000", 2024-09-15T00:14:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9690000000000001e-05 2024-09-15T00:14:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153171 3.18 e103/69 3:1db186e8:::rbd_data.113958cb0e55.0000000000000002:head v 103'520, mlcod=103'518)", 2024-09-15T00:14:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:14:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.671037584999993, 2024-09-15T00:14:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.022266838000000001, 2024-09-15T00:14:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.519 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:14:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780055+0000", 2024-09-15T00:14:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.780057+0000", 2024-09-15T00:14:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5880000000000001e-06 2024-09-15T00:14:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781273+0000", 2024-09-15T00:14:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001216087 2024-09-15T00:14:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781311+0000", 2024-09-15T00:14:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7656e-05 2024-09-15T00:14:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781325+0000", 2024-09-15T00:14:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4508999999999999e-05 2024-09-15T00:14:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781403+0000", 2024-09-15T00:14:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7927000000000001e-05 2024-09-15T00:14:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.781422+0000", 2024-09-15T00:14:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8575000000000001e-05 2024-09-15T00:14:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802249+0000", 2024-09-15T00:14:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.020827577 2024-09-15T00:14:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.802322+0000", 2024-09-15T00:14:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2918999999999998e-05 2024-09-15T00:14:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.523 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:153772 3.12 e103/68 3:4911a4f6:::rbd_data.113958cb0e55.0000000000000001:head v 103'490, mlcod=103'488)", 2024-09-15T00:14:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:14:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 77.568166958000006, 2024-09-15T00:14:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.021181004, 2024-09-15T00:14:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:14:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882926+0000", 2024-09-15T00:14:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.882927+0000", 2024-09-15T00:14:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.426e-06 2024-09-15T00:14:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883966+0000", 2024-09-15T00:14:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010384610000000001 2024-09-15T00:14:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883985+0000", 2024-09-15T00:14:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9276e-05 2024-09-15T00:14:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.883994+0000", 2024-09-15T00:14:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9490000000000002e-06 2024-09-15T00:14:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884054+0000", 2024-09-15T00:14:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9419999999999997e-05 2024-09-15T00:14:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.884073+0000", 2024-09-15T00:14:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9137e-05 2024-09-15T00:14:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904038+0000", 2024-09-15T00:14:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019965092 2024-09-15T00:14:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:49.904107+0000", 2024-09-15T00:14:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9243000000000001e-05 2024-09-15T00:14:07.527 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163112 3.3 e106/68 3:c668f3d5:::rbd_data.11395d771336.0000000000000001:head v 106'489, mlcod=105'487)", 2024-09-15T00:14:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:14:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.986269688999997, 2024-09-15T00:14:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.019317183000000002, 2024-09-15T00:14:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:14:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464823+0000", 2024-09-15T00:14:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.464824+0000", 2024-09-15T00:14:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1100000000000005e-07 2024-09-15T00:14:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466874+0000", 2024-09-15T00:14:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002049497 2024-09-15T00:14:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466884+0000", 2024-09-15T00:14:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0424999999999999e-05 2024-09-15T00:14:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466890+0000", 2024-09-15T00:14:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-15T00:14:07.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466944+0000", 2024-09-15T00:14:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4592999999999997e-05 2024-09-15T00:14:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.466958+0000", 2024-09-15T00:14:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3993e-05 2024-09-15T00:14:07.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484068+0000", 2024-09-15T00:14:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017109994 2024-09-15T00:14:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.484140+0000", 2024-09-15T00:14:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2224000000000005e-05 2024-09-15T00:14:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4409.0:163119 3.0 e106/94 3:06118613:::rbd_data.11395d771336.0000000000000004:head v 106'549, mlcod=105'547)", 2024-09-15T00:14:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:14:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.974654486000006, 2024-09-15T00:14:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.017175867000000001, 2024-09-15T00:14:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:14:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:14:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.476439+0000", 2024-09-15T00:14:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6700000000000002e-07 2024-09-15T00:14:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477786+0000", 2024-09-15T00:14:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013468149999999999 2024-09-15T00:14:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477797+0000", 2024-09-15T00:14:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0635000000000001e-05 2024-09-15T00:14:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477805+0000", 2024-09-15T00:14:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2989999999999997e-06 2024-09-15T00:14:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477859+0000", 2024-09-15T00:14:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3699000000000002e-05 2024-09-15T00:14:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.477878+0000", 2024-09-15T00:14:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9335e-05 2024-09-15T00:14:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493560+0000", 2024-09-15T00:14:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015681748999999998 2024-09-15T00:14:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:53.493614+0000", 2024-09-15T00:14:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4367999999999999e-05 2024-09-15T00:14:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.5.0:5005 3.17 e110/68 3:eb0359e8:::rbd_data.113964deed84.0000000000000251:35 v 110'537, mlcod=109'534)", 2024-09-15T00:14:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:14:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.314804113999998, 2024-09-15T00:14:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015446896999999999, 2024-09-15T00:14:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.537 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:14:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136289+0000", 2024-09-15T00:14:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136290+0000", 2024-09-15T00:14:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7400000000000003e-07 2024-09-15T00:14:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136294+0000", 2024-09-15T00:14:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4989999999999996e-06 2024-09-15T00:14:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136304+0000", 2024-09-15T00:14:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0319999999999999e-05 2024-09-15T00:14:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136310+0000", 2024-09-15T00:14:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.981e-06 2024-09-15T00:14:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136350+0000", 2024-09-15T00:14:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0088e-05 2024-09-15T00:14:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.136364+0000", 2024-09-15T00:14:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3954999999999999e-05 2024-09-15T00:14:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151681+0000", 2024-09-15T00:14:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015316969999999999 2024-09-15T00:14:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.151736+0000", 2024-09-15T00:14:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4410000000000003e-05 2024-09-15T00:14:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.541 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1758 3.10 e110/68 3:08149147:::rbd_data.113964deed84.0000000000000055:35 v 110'594, mlcod=109'591)", 2024-09-15T00:14:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:14:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.310505887999994, 2024-09-15T00:14:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016278652000000001, 2024-09-15T00:14:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:14:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:14:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140587+0000", 2024-09-15T00:14:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9499999999999998e-07 2024-09-15T00:14:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140599+0000", 2024-09-15T00:14:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1109e-05 2024-09-15T00:14:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140605+0000", 2024-09-15T00:14:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9890000000000003e-06 2024-09-15T00:14:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140612+0000", 2024-09-15T00:14:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6089999999999997e-06 2024-09-15T00:14:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140656+0000", 2024-09-15T00:14:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3754999999999997e-05 2024-09-15T00:14:07.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.140674+0000", 2024-09-15T00:14:07.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7640999999999999e-05 2024-09-15T00:14:07.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156823+0000", 2024-09-15T00:14:07.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016149047999999999 2024-09-15T00:14:07.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.156866+0000", 2024-09-15T00:14:07.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3205999999999999e-05 2024-09-15T00:14:07.546 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.546 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.546 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.546 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.4.0:5722 3.e e110/95 3:74a58daf:::rbd_data.113964deed84.00000000000000c1:35 v 110'1287, mlcod=109'1284)", 2024-09-15T00:14:07.546 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:14:07.546 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.301431531999995, 2024-09-15T00:14:07.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01830555, 2024-09-15T00:14:07.546 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.546 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.546 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.546 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:14:07.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149661+0000", 2024-09-15T00:14:07.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149662+0000", 2024-09-15T00:14:07.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:14:07.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149666+0000", 2024-09-15T00:14:07.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2220000000000004e-06 2024-09-15T00:14:07.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149672+0000", 2024-09-15T00:14:07.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3410000000000002e-06 2024-09-15T00:14:07.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149676+0000", 2024-09-15T00:14:07.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.566e-06 2024-09-15T00:14:07.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149709+0000", 2024-09-15T00:14:07.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2403000000000001e-05 2024-09-15T00:14:07.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.149719+0000", 2024-09-15T00:14:07.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0608000000000001e-05 2024-09-15T00:14:07.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167944+0000", 2024-09-15T00:14:07.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.018225183999999998 2024-09-15T00:14:07.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:58.167967+0000", 2024-09-15T00:14:07.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2705e-05 2024-09-15T00:14:07.550 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.550 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.550 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.550 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.550 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.550 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:07.552 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:14:07.552 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:14:07.549+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:14:07.568 INFO:tasks.ceph.mon.a.smithi171.stderr:2024-09-15T00:14:07.565+0000 7f0425d35640 -1 mon.a@0(leader).osd e179 definitely_dead 0 2024-09-15T00:14:07.652 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:14:07.650+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:14:07.671 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:07.671 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:14:07.671 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:14:07.671 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:07.680 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:14:07.753 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:14:07.752+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:14:07.783 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:07.783 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:14:07.784 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:14:07.784 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:14:07.784 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:07.793 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:14:07.855 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:14:07.853+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:14:07.897 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:07.897 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:14:07.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:14:07.898 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:14:07.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.898 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:1287 3.6 3:60e5612d:::rbd_data.11397bff333d.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-15T00:14:07.898 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:14:07.898 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 161.71526197899999, 2024-09-15T00:14:07.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036128928999999997, 2024-09-15T00:14:07.898 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.898 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.898 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.898 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.898 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.898 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.899 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1287 2024-09-15T00:14:07.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.899 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:14:07.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179825+0000", 2024-09-15T00:14:07.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179826+0000", 2024-09-15T00:14:07.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:14:07.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179832+0000", 2024-09-15T00:14:07.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3559999999999998e-06 2024-09-15T00:14:07.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179846+0000", 2024-09-15T00:14:07.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3889e-05 2024-09-15T00:14:07.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179854+0000", 2024-09-15T00:14:07.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9049999999999997e-06 2024-09-15T00:14:07.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179914+0000", 2024-09-15T00:14:07.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9564000000000001e-05 2024-09-15T00:14:07.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.179991+0000", 2024-09-15T00:14:07.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7483000000000003e-05 2024-09-15T00:14:07.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:07.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.182965+0000", 2024-09-15T00:14:07.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002973879 2024-09-15T00:14:07.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:14:07.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215827+0000", 2024-09-15T00:14:07.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.032861856000000002 2024-09-15T00:14:07.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:07.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215863+0000", 2024-09-15T00:14:07.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6389e-05 2024-09-15T00:14:07.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215899+0000", 2024-09-15T00:14:07.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5998999999999999e-05 2024-09-15T00:14:07.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:26.215954+0000", 2024-09-15T00:14:07.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4358999999999999e-05 2024-09-15T00:14:07.903 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.903 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.903 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.904 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:18724 3.0 3:0710fae0:::rbd_data.11398e1a091a.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:14:07.904 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:14:07.904 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 151.98542079999999, 2024-09-15T00:14:07.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046042147999999998, 2024-09-15T00:14:07.904 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.904 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.904 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.904 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.904 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.904 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.904 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 18724 2024-09-15T00:14:07.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.905 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:14:07.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909666+0000", 2024-09-15T00:14:07.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909667+0000", 2024-09-15T00:14:07.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-15T00:14:07.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909671+0000", 2024-09-15T00:14:07.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5719999999999996e-06 2024-09-15T00:14:07.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909681+0000", 2024-09-15T00:14:07.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2220000000000008e-06 2024-09-15T00:14:07.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909687+0000", 2024-09-15T00:14:07.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5690000000000001e-06 2024-09-15T00:14:07.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909726+0000", 2024-09-15T00:14:07.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8634000000000003e-05 2024-09-15T00:14:07.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.909774+0000", 2024-09-15T00:14:07.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7833000000000001e-05 2024-09-15T00:14:07.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-15T00:14:07.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.912105+0000", 2024-09-15T00:14:07.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023309170000000001 2024-09-15T00:14:07.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:07.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955573+0000", 2024-09-15T00:14:07.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043468859999999998 2024-09-15T00:14:07.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:14:07.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955591+0000", 2024-09-15T00:14:07.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7943000000000001e-05 2024-09-15T00:14:07.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955643+0000", 2024-09-15T00:14:07.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1481999999999999e-05 2024-09-15T00:14:07.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955708+0000", 2024-09-15T00:14:07.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5235999999999997e-05 2024-09-15T00:14:07.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.909 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.914 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:18724 3.0 e38/23 ondisk, result = 0)", 2024-09-15T00:14:07.914 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:14:07.914 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 151.97846243699999, 2024-09-15T00:14:07.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038957644, 2024-09-15T00:14:07.914 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.914 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.914 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.914 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:14:07.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916624+0000", 2024-09-15T00:14:07.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916625+0000", 2024-09-15T00:14:07.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-15T00:14:07.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916629+0000", 2024-09-15T00:14:07.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.014e-06 2024-09-15T00:14:07.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916642+0000", 2024-09-15T00:14:07.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2469e-05 2024-09-15T00:14:07.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.916649+0000", 2024-09-15T00:14:07.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1559999999999998e-06 2024-09-15T00:14:07.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955554+0000", 2024-09-15T00:14:07.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905585999999999 2024-09-15T00:14:07.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955566+0000", 2024-09-15T00:14:07.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1311999999999999e-05 2024-09-15T00:14:07.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:35.955582+0000", 2024-09-15T00:14:07.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339000000000001e-05 2024-09-15T00:14:07.917 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.917 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.918 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.918 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:49029 3.4 3:2729eaee:::rbd_data.113916cf513.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14942208~2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T00:14:07.918 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:14:07.918 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 143.23127360000001, 2024-09-15T00:14:07.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034989105, 2024-09-15T00:14:07.918 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.918 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.918 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:07.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.919 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 49029 2024-09-15T00:14:07.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.919 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:14:07.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663813+0000", 2024-09-15T00:14:07.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.663815+0000", 2024-09-15T00:14:07.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.612e-06 2024-09-15T00:14:07.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667565+0000", 2024-09-15T00:14:07.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037503250000000001 2024-09-15T00:14:07.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667587+0000", 2024-09-15T00:14:07.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-05 2024-09-15T00:14:07.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667595+0000", 2024-09-15T00:14:07.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6249999999999996e-06 2024-09-15T00:14:07.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667650+0000", 2024-09-15T00:14:07.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5204000000000002e-05 2024-09-15T00:14:07.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.667758+0000", 2024-09-15T00:14:07.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107932 2024-09-15T00:14:07.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:14:07.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.668043+0000", 2024-09-15T00:14:07.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284787 2024-09-15T00:14:07.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:14:07.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.684876+0000", 2024-09-15T00:14:07.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016832514 2024-09-15T00:14:07.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:07.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698669+0000", 2024-09-15T00:14:07.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013793177 2024-09-15T00:14:07.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:07.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698723+0000", 2024-09-15T00:14:07.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4539000000000001e-05 2024-09-15T00:14:07.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:44.698802+0000", 2024-09-15T00:14:07.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8850000000000006e-05 2024-09-15T00:14:07.923 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.924 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.924 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.924 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97493 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:14:07.924 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:14:07.924 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 119.04169088, 2024-09-15T00:14:07.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033259141999999998, 2024-09-15T00:14:07.924 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.924 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.924 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.924 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.924 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.924 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.925 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97493 2024-09-15T00:14:07.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.925 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:14:07.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:14:07.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853396+0000", 2024-09-15T00:14:07.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-15T00:14:07.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853401+0000", 2024-09-15T00:14:07.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:14:07.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853407+0000", 2024-09-15T00:14:07.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.84e-06 2024-09-15T00:14:07.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853411+0000", 2024-09-15T00:14:07.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.481e-06 2024-09-15T00:14:07.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886544+0000", 2024-09-15T00:14:07.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033132861999999999 2024-09-15T00:14:07.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886587+0000", 2024-09-15T00:14:07.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2942000000000003e-05 2024-09-15T00:14:07.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886655+0000", 2024-09-15T00:14:07.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7711000000000003e-05 2024-09-15T00:14:07.928 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.928 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.928 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.928 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97494 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:14:07.928 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:14:07.928 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 119.04166975699999, 2024-09-15T00:14:07.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033392390000000001, 2024-09-15T00:14:07.929 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.929 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.929 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.929 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.929 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.929 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.929 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97494 2024-09-15T00:14:07.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.929 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:14:07.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:14:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853417+0000", 2024-09-15T00:14:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-15T00:14:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853419+0000", 2024-09-15T00:14:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.917e-06 2024-09-15T00:14:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853425+0000", 2024-09-15T00:14:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.682e-06 2024-09-15T00:14:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853433+0000", 2024-09-15T00:14:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8120000000000008e-06 2024-09-15T00:14:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886680+0000", 2024-09-15T00:14:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033247207000000001 2024-09-15T00:14:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886726+0000", 2024-09-15T00:14:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5902000000000002e-05 2024-09-15T00:14:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886809+0000", 2024-09-15T00:14:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3475999999999998e-05 2024-09-15T00:14:07.932 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97495 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:14:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:14:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 119.041648804, 2024-09-15T00:14:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033640503000000002, 2024-09-15T00:14:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.933 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97495 2024-09-15T00:14:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:14:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:14:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853438+0000", 2024-09-15T00:14:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:14:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853440+0000", 2024-09-15T00:14:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.804e-06 2024-09-15T00:14:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853446+0000", 2024-09-15T00:14:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5559999999999998e-06 2024-09-15T00:14:07.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853450+0000", 2024-09-15T00:14:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9519999999999996e-06 2024-09-15T00:14:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.886904+0000", 2024-09-15T00:14:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033454207999999999 2024-09-15T00:14:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887007+0000", 2024-09-15T00:14:07.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102904 2024-09-15T00:14:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887078+0000", 2024-09-15T00:14:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1719999999999998e-05 2024-09-15T00:14:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97496 3.6 3:64aa4710:::rbd_data.113918df0a50.0000000000000008:1a [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:14:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:14:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 119.041631706, 2024-09-15T00:14:07.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033802431000000001, 2024-09-15T00:14:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97496 2024-09-15T00:14:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:14:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:14:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853455+0000", 2024-09-15T00:14:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.53e-07 2024-09-15T00:14:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853462+0000", 2024-09-15T00:14:07.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-15T00:14:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853468+0000", 2024-09-15T00:14:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3609999999999997e-06 2024-09-15T00:14:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853473+0000", 2024-09-15T00:14:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6809999999999997e-06 2024-09-15T00:14:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887111+0000", 2024-09-15T00:14:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033638376999999997 2024-09-15T00:14:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887178+0000", 2024-09-15T00:14:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6877000000000004e-05 2024-09-15T00:14:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887257+0000", 2024-09-15T00:14:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9616999999999993e-05 2024-09-15T00:14:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97587 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:14:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:14:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 119.04160847199999, 2024-09-15T00:14:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033973642999999998, 2024-09-15T00:14:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97587 2024-09-15T00:14:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.942 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:14:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853478+0000", 2024-09-15T00:14:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853479+0000", 2024-09-15T00:14:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-15T00:14:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853481+0000", 2024-09-15T00:14:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.136e-06 2024-09-15T00:14:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853487+0000", 2024-09-15T00:14:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2439999999999998e-06 2024-09-15T00:14:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853492+0000", 2024-09-15T00:14:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:14:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887284+0000", 2024-09-15T00:14:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033791827000000003 2024-09-15T00:14:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887361+0000", 2024-09-15T00:14:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6983000000000005e-05 2024-09-15T00:14:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887452+0000", 2024-09-15T00:14:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1236999999999994e-05 2024-09-15T00:14:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:97588 3.6 3:630a42a7:::rbd_data.113918df0a50.0000000000000000:1a [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:14:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:14:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 119.041588315, 2024-09-15T00:14:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.034134617999999999, 2024-09-15T00:14:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.946 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 97588 2024-09-15T00:14:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:14:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853498+0000", 2024-09-15T00:14:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853499+0000", 2024-09-15T00:14:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:14:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853501+0000", 2024-09-15T00:14:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.26e-06 2024-09-15T00:14:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853507+0000", 2024-09-15T00:14:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2899999999999999e-06 2024-09-15T00:14:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.853512+0000", 2024-09-15T00:14:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8330000000000001e-06 2024-09-15T00:14:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887495+0000", 2024-09-15T00:14:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033982529999999997 2024-09-15T00:14:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887548+0000", 2024-09-15T00:14:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3445000000000001e-05 2024-09-15T00:14:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:08.887633+0000", 2024-09-15T00:14:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4890999999999997e-05 2024-09-15T00:14:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126952 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:07.950 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:14:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 105.938123698, 2024-09-15T00:14:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033156066999999997, 2024-09-15T00:14:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:14:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126952 2024-09-15T00:14:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:14:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:14:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956963+0000", 2024-09-15T00:14:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5699999999999999e-07 2024-09-15T00:14:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956965+0000", 2024-09-15T00:14:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3379999999999999e-06 2024-09-15T00:14:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956968+0000", 2024-09-15T00:14:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0419999999999998e-06 2024-09-15T00:14:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.956970+0000", 2024-09-15T00:14:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4899999999999999e-06 2024-09-15T00:14:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990113+0000", 2024-09-15T00:14:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033142642 2024-09-15T00:14:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990119+0000", 2024-09-15T00:14:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2979999999999997e-06 2024-09-15T00:14:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.954 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126954 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:14:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 105.938078203, 2024-09-15T00:14:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033158449, 2024-09-15T00:14:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:14:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126954 2024-09-15T00:14:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:14:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:14:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957009+0000", 2024-09-15T00:14:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.39e-07 2024-09-15T00:14:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957010+0000", 2024-09-15T00:14:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3659999999999999e-06 2024-09-15T00:14:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957013+0000", 2024-09-15T00:14:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1140000000000002e-06 2024-09-15T00:14:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957016+0000", 2024-09-15T00:14:07.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1209999999999998e-06 2024-09-15T00:14:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990163+0000", 2024-09-15T00:14:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033146451 2024-09-15T00:14:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990167+0000", 2024-09-15T00:14:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0579999999999999e-06 2024-09-15T00:14:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126955 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:14:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 105.93806698900001, 2024-09-15T00:14:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033160617000000003, 2024-09-15T00:14:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:14:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126955 2024-09-15T00:14:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.959 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:14:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:14:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957020+0000", 2024-09-15T00:14:07.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.11e-07 2024-09-15T00:14:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957021+0000", 2024-09-15T00:14:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-15T00:14:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957024+0000", 2024-09-15T00:14:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1599999999999998e-06 2024-09-15T00:14:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957027+0000", 2024-09-15T00:14:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000001e-06 2024-09-15T00:14:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990177+0000", 2024-09-15T00:14:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033149741000000003 2024-09-15T00:14:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990180+0000", 2024-09-15T00:14:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4910000000000001e-06 2024-09-15T00:14:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.962 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126956 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:14:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 105.938056868, 2024-09-15T00:14:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033163633999999997, 2024-09-15T00:14:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:14:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126956 2024-09-15T00:14:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.963 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:14:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:14:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957030+0000", 2024-09-15T00:14:07.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:14:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957031+0000", 2024-09-15T00:14:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-06 2024-09-15T00:14:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957034+0000", 2024-09-15T00:14:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9960000000000002e-06 2024-09-15T00:14:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957037+0000", 2024-09-15T00:14:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6419999999999999e-06 2024-09-15T00:14:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990190+0000", 2024-09-15T00:14:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033153152999999998 2024-09-15T00:14:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990193+0000", 2024-09-15T00:14:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6069999999999999e-06 2024-09-15T00:14:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.966 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126957 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:14:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 105.93804721799999, 2024-09-15T00:14:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033166442999999997, 2024-09-15T00:14:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:14:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126957 2024-09-15T00:14:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.967 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:14:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957039+0000", 2024-09-15T00:14:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957040+0000", 2024-09-15T00:14:07.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-15T00:14:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957041+0000", 2024-09-15T00:14:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0440000000000001e-06 2024-09-15T00:14:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957044+0000", 2024-09-15T00:14:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0170000000000001e-06 2024-09-15T00:14:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957048+0000", 2024-09-15T00:14:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0319999999999997e-06 2024-09-15T00:14:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990202+0000", 2024-09-15T00:14:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033154640999999999 2024-09-15T00:14:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990206+0000", 2024-09-15T00:14:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.507e-06 2024-09-15T00:14:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126958 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:14:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 105.938035988, 2024-09-15T00:14:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033167613999999998, 2024-09-15T00:14:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:14:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126958 2024-09-15T00:14:07.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:14:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:14:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957051+0000", 2024-09-15T00:14:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1600000000000002e-07 2024-09-15T00:14:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957053+0000", 2024-09-15T00:14:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.756e-06 2024-09-15T00:14:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957059+0000", 2024-09-15T00:14:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2770000000000002e-06 2024-09-15T00:14:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957063+0000", 2024-09-15T00:14:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2459999999999997e-06 2024-09-15T00:14:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990215+0000", 2024-09-15T00:14:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033151509000000003 2024-09-15T00:14:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990218+0000", 2024-09-15T00:14:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5099999999999999e-06 2024-09-15T00:14:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.974 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126959 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:14:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 105.938016984, 2024-09-15T00:14:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033161673000000003, 2024-09-15T00:14:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:14:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126959 2024-09-15T00:14:07.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:14:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:14:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957070+0000", 2024-09-15T00:14:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:14:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957072+0000", 2024-09-15T00:14:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2239999999999998e-06 2024-09-15T00:14:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957079+0000", 2024-09-15T00:14:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.19e-06 2024-09-15T00:14:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957083+0000", 2024-09-15T00:14:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:14:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990228+0000", 2024-09-15T00:14:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033144818 2024-09-15T00:14:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990231+0000", 2024-09-15T00:14:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:14:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.978 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:126960 3.6 3.973c9fa6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:14:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 105.937998177, 2024-09-15T00:14:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033155166, 2024-09-15T00:14:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-15T00:14:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.979 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 126960 2024-09-15T00:14:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:14:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:14:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957089+0000", 2024-09-15T00:14:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.03e-07 2024-09-15T00:14:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957091+0000", 2024-09-15T00:14:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9410000000000001e-06 2024-09-15T00:14:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957097+0000", 2024-09-15T00:14:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6050000000000003e-06 2024-09-15T00:14:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.957102+0000", 2024-09-15T00:14:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5539999999999999e-06 2024-09-15T00:14:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990240+0000", 2024-09-15T00:14:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.033138289000000001 2024-09-15T00:14:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.990244+0000", 2024-09-15T00:14:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4740000000000001e-06 2024-09-15T00:14:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.982 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:174575 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138)", 2024-09-15T00:14:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:26.073788+0000", 2024-09-15T00:14:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.821298861999999, 2024-09-15T00:14:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.95024539600000002, 2024-09-15T00:14:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.983 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 174575 2024-09-15T00:14:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073788+0000", 2024-09-15T00:14:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073788+0000", 2024-09-15T00:14:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073789+0000", 2024-09-15T00:14:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3999999999999999e-06 2024-09-15T00:14:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073796+0000", 2024-09-15T00:14:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5529999999999998e-06 2024-09-15T00:14:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073807+0000", 2024-09-15T00:14:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0825000000000001e-05 2024-09-15T00:14:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073812+0000", 2024-09-15T00:14:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6690000000000003e-06 2024-09-15T00:14:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073870+0000", 2024-09-15T00:14:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7241999999999998e-05 2024-09-15T00:14:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:14:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:26.073878+0000", 2024-09-15T00:14:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0760000000000003e-06 2024-09-15T00:14:07.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:27.023756+0000", 2024-09-15T00:14:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.94987846899999995 2024-09-15T00:14:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:27.023891+0000", 2024-09-15T00:14:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000134785 2024-09-15T00:14:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:27.024033+0000", 2024-09-15T00:14:07.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014237700000000001 2024-09-15T00:14:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:178794 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e170)", 2024-09-15T00:14:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:13:58.023421+0000", 2024-09-15T00:14:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.8716657320000003, 2024-09-15T00:14:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.025621619, 2024-09-15T00:14:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:07.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 178794 2024-09-15T00:14:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:58.023421+0000", 2024-09-15T00:14:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:58.023421+0000", 2024-09-15T00:14:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:07.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:58.023423+0000", 2024-09-15T00:14:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.578e-06 2024-09-15T00:14:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:58.023436+0000", 2024-09-15T00:14:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3930999999999999e-05 2024-09-15T00:14:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:58.023450+0000", 2024-09-15T00:14:07.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3237e-05 2024-09-15T00:14:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:58.023458+0000", 2024-09-15T00:14:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4270000000000008e-06 2024-09-15T00:14:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:58.023525+0000", 2024-09-15T00:14:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6372999999999996e-05 2024-09-15T00:14:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-15T00:14:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:58.023532+0000", 2024-09-15T00:14:07.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9549999999999992e-06 2024-09-15T00:14:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:58.031067+0000", 2024-09-15T00:14:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075341999999999996 2024-09-15T00:14:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:14:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:58.031077+0000", 2024-09-15T00:14:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0402e-05 2024-09-15T00:14:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:59.048733+0000", 2024-09-15T00:14:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.01765591 2024-09-15T00:14:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:59.048904+0000", 2024-09-15T00:14:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000171473 2024-09-15T00:14:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:13:59.049043+0000", 2024-09-15T00:14:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000138133 2024-09-15T00:14:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.993 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.994 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:07.994 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:07.994 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:07.995 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:14:07.996 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:14:07.993+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:14:08.074 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:14:08.076 INFO:teuthology.orchestra.run.smithi171.stderr:marked out osd.0. 2024-09-15T00:14:08.096 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:14:08.094+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:14:08.109 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:08.109 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:14:08.109 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-15T00:14:08.109 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:08.121 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:14:08.204 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:14:08.202+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:14:08.267 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:08.267 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-15T00:14:08.268 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-15T00:14:08.268 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-15T00:14:08.268 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:08.277 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:14:08.317 INFO:tasks.ceph.osd.3.smithi171.stderr:2024-09-15T00:14:08.315+0000 7f0f1bb5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33249) UID: 0 2024-09-15T00:14:08.399 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:14:08.396+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:14:08.423 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-15T00:14:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-15T00:14:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-15T00:14:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:56206 3.7 3:e59b5ae2:::rbd_data.1139167e758e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-15T00:14:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:14:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 140.177837587, 2024-09-15T00:14:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040677761, 2024-09-15T00:14:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 56206 2024-09-15T00:14:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:14:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242844+0000", 2024-09-15T00:14:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.242845+0000", 2024-09-15T00:14:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:14:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245109+0000", 2024-09-15T00:14:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022639510000000002 2024-09-15T00:14:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245127+0000", 2024-09-15T00:14:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8229000000000001e-05 2024-09-15T00:14:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.245144+0000", 2024-09-15T00:14:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6682000000000001e-05 2024-09-15T00:14:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246724+0000", 2024-09-15T00:14:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015801400000000001 2024-09-15T00:14:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.246854+0000", 2024-09-15T00:14:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013030800000000001 2024-09-15T00:14:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-15T00:14:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.247949+0000", 2024-09-15T00:14:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010952100000000001 2024-09-15T00:14:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:14:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.274053+0000", 2024-09-15T00:14:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026103937000000001 2024-09-15T00:14:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283467+0000", 2024-09-15T00:14:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094142280000000002 2024-09-15T00:14:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283497+0000", 2024-09-15T00:14:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.968e-05 2024-09-15T00:14:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:48.283522+0000", 2024-09-15T00:14:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4725000000000001e-05 2024-09-15T00:14:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58878 3.8 3:12456482:::rbd_data.1139766160da.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:14:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:14:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 135.13153200299999, 2024-09-15T00:14:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061956199000000003, 2024-09-15T00:14:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58878 2024-09-15T00:14:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:14:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289150+0000", 2024-09-15T00:14:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289151+0000", 2024-09-15T00:14:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-15T00:14:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289473+0000", 2024-09-15T00:14:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032183800000000002 2024-09-15T00:14:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289487+0000", 2024-09-15T00:14:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-05 2024-09-15T00:14:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289493+0000", 2024-09-15T00:14:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4489999999999996e-06 2024-09-15T00:14:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289542+0000", 2024-09-15T00:14:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8844999999999997e-05 2024-09-15T00:14:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.289642+0000", 2024-09-15T00:14:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9607999999999999e-05 2024-09-15T00:14:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:14:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.293908+0000", 2024-09-15T00:14:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659350000000002 2024-09-15T00:14:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351022+0000", 2024-09-15T00:14:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057114535000000001 2024-09-15T00:14:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:14:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351033+0000", 2024-09-15T00:14:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0263e-05 2024-09-15T00:14:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351070+0000", 2024-09-15T00:14:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7666000000000002e-05 2024-09-15T00:14:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351106+0000", 2024-09-15T00:14:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999997e-05 2024-09-15T00:14:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58879 3.7 3:e169894d:::rbd_data.1139766160da.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:14:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:14:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 135.129180515, 2024-09-15T00:14:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.043716294000000003, 2024-09-15T00:14:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58879 2024-09-15T00:14:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:14:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291501+0000", 2024-09-15T00:14:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291502+0000", 2024-09-15T00:14:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5000000000000001e-07 2024-09-15T00:14:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291664+0000", 2024-09-15T00:14:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016216599999999999 2024-09-15T00:14:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291674+0000", 2024-09-15T00:14:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0276e-05 2024-09-15T00:14:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291680+0000", 2024-09-15T00:14:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2580000000000003e-06 2024-09-15T00:14:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291732+0000", 2024-09-15T00:14:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2114999999999998e-05 2024-09-15T00:14:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.291866+0000", 2024-09-15T00:14:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013416900000000001 2024-09-15T00:14:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-15T00:14:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.296426+0000", 2024-09-15T00:14:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045595779999999999 2024-09-15T00:14:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335075+0000", 2024-09-15T00:14:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038649347000000001 2024-09-15T00:14:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:14:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335097+0000", 2024-09-15T00:14:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2565000000000002e-05 2024-09-15T00:14:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335161+0000", 2024-09-15T00:14:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3928000000000002e-05 2024-09-15T00:14:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.335217+0000", 2024-09-15T00:14:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6042000000000003e-05 2024-09-15T00:14:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:58880 3.8 3:13d91cfd:::rbd_data.1139766160da.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T00:14:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:14:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 135.12578840800001, 2024-09-15T00:14:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.056331463999999998, 2024-09-15T00:14:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 58880 2024-09-15T00:14:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:14:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294893+0000", 2024-09-15T00:14:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.294896+0000", 2024-09-15T00:14:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.356e-06 2024-09-15T00:14:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295288+0000", 2024-09-15T00:14:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039223200000000001 2024-09-15T00:14:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295304+0000", 2024-09-15T00:14:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5937e-05 2024-09-15T00:14:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.295312+0000", 2024-09-15T00:14:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1489999999999994e-06 2024-09-15T00:14:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333162+0000", 2024-09-15T00:14:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037849840000000003 2024-09-15T00:14:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.333269+0000", 2024-09-15T00:14:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010749800000000001 2024-09-15T00:14:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-15T00:14:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.336198+0000", 2024-09-15T00:14:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029284739999999999 2024-09-15T00:14:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351137+0000", 2024-09-15T00:14:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014939103 2024-09-15T00:14:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:14:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351150+0000", 2024-09-15T00:14:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3542e-05 2024-09-15T00:14:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351191+0000", 2024-09-15T00:14:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0791000000000003e-05 2024-09-15T00:14:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351225+0000", 2024-09-15T00:14:08.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3541999999999998e-05 2024-09-15T00:14:08.447 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.447 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.447 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.447 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:58878 3.8 e56/23 ondisk, result = 0)", 2024-09-15T00:14:08.447 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:14:08.447 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 135.11925862499999, 2024-09-15T00:14:08.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049689096000000002, 2024-09-15T00:14:08.447 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.447 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.447 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:08.447 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:14:08.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301423+0000", 2024-09-15T00:14:08.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301425+0000", 2024-09-15T00:14:08.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.782e-06 2024-09-15T00:14:08.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301433+0000", 2024-09-15T00:14:08.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9689999999999994e-06 2024-09-15T00:14:08.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301449+0000", 2024-09-15T00:14:08.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5767e-05 2024-09-15T00:14:08.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.301458+0000", 2024-09-15T00:14:08.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0450000000000006e-06 2024-09-15T00:14:08.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351009+0000", 2024-09-15T00:14:08.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049551523 2024-09-15T00:14:08.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351017+0000", 2024-09-15T00:14:08.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8359999999999992e-06 2024-09-15T00:14:08.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:11:53.351112+0000", 2024-09-15T00:14:08.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5173999999999994e-05 2024-09-15T00:14:08.451 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.451 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.451 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.451 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127004 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 6881280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:08.451 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:14:08.451 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.452353161, 2024-09-15T00:14:08.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735720000000003, 2024-09-15T00:14:08.451 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.451 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.451 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:08.451 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.452 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127004 2024-09-15T00:14:08.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.452 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:14:08.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:14:08.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968329+0000", 2024-09-15T00:14:08.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:14:08.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968331+0000", 2024-09-15T00:14:08.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.849e-06 2024-09-15T00:14:08.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968334+0000", 2024-09-15T00:14:08.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7689999999999998e-06 2024-09-15T00:14:08.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968338+0000", 2024-09-15T00:14:08.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2490000000000001e-06 2024-09-15T00:14:08.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.008987+0000", 2024-09-15T00:14:08.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040649117999999998 2024-09-15T00:14:08.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009027+0000", 2024-09-15T00:14:08.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0357e-05 2024-09-15T00:14:08.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009064+0000", 2024-09-15T00:14:08.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7036000000000003e-05 2024-09-15T00:14:08.455 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.455 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.455 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.455 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127005 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7004160~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:08.456 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:14:08.456 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.452340609, 2024-09-15T00:14:08.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040824358999999998, 2024-09-15T00:14:08.456 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.456 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.456 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:08.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.456 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127005 2024-09-15T00:14:08.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.456 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:14:08.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:14:08.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968341+0000", 2024-09-15T00:14:08.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-15T00:14:08.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968343+0000", 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3430000000000001e-06 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968349+0000", 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3080000000000004e-06 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968355+0000", 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9499999999999998e-06 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009091+0000", 2024-09-15T00:14:08.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040735645000000001 2024-09-15T00:14:08.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009130+0000", 2024-09-15T00:14:08.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8951e-05 2024-09-15T00:14:08.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009165+0000", 2024-09-15T00:14:08.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5926000000000001e-05 2024-09-15T00:14:08.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.459 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.460 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127006 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7127040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:08.460 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:14:08.460 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.45232355100001, 2024-09-15T00:14:08.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040912783000000001, 2024-09-15T00:14:08.460 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.460 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.460 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:08.460 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.460 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.460 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.460 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127006 2024-09-15T00:14:08.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.460 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:14:08.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:14:08.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968358+0000", 2024-09-15T00:14:08.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:14:08.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968359+0000", 2024-09-15T00:14:08.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0419999999999999e-06 2024-09-15T00:14:08.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968363+0000", 2024-09-15T00:14:08.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5999999999999998e-06 2024-09-15T00:14:08.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968366+0000", 2024-09-15T00:14:08.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5179999999999999e-06 2024-09-15T00:14:08.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009196+0000", 2024-09-15T00:14:08.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040830313 2024-09-15T00:14:08.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009234+0000", 2024-09-15T00:14:08.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8562000000000001e-05 2024-09-15T00:14:08.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009271+0000", 2024-09-15T00:14:08.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6538000000000002e-05 2024-09-15T00:14:08.464 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.464 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.464 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.464 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127007 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7249920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:08.464 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:14:08.464 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.452313352, 2024-09-15T00:14:08.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041019218000000003, 2024-09-15T00:14:08.464 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.464 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.465 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:08.465 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.465 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.465 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.465 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127007 2024-09-15T00:14:08.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.465 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:14:08.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968368+0000", 2024-09-15T00:14:08.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968369+0000", 2024-09-15T00:14:08.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-15T00:14:08.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968370+0000", 2024-09-15T00:14:08.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.012e-06 2024-09-15T00:14:08.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968374+0000", 2024-09-15T00:14:08.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3839999999999999e-06 2024-09-15T00:14:08.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968376+0000", 2024-09-15T00:14:08.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5339999999999998e-06 2024-09-15T00:14:08.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009292+0000", 2024-09-15T00:14:08.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040915092 2024-09-15T00:14:08.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009333+0000", 2024-09-15T00:14:08.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1105e-05 2024-09-15T00:14:08.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009388+0000", 2024-09-15T00:14:08.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4868999999999999e-05 2024-09-15T00:14:08.468 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.468 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.468 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.469 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127008 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7364608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:08.469 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:14:08.469 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.452302231, 2024-09-15T00:14:08.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041115290999999998, 2024-09-15T00:14:08.469 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.469 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.469 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:08.469 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.469 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.469 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.469 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127008 2024-09-15T00:14:08.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.469 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968379+0000", 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968380+0000", 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3999999999999998e-07 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968381+0000", 2024-09-15T00:14:08.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8700000000000004e-07 2024-09-15T00:14:08.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968385+0000", 2024-09-15T00:14:08.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9260000000000002e-06 2024-09-15T00:14:08.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968389+0000", 2024-09-15T00:14:08.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.685e-06 2024-09-15T00:14:08.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009419+0000", 2024-09-15T00:14:08.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041029852999999998 2024-09-15T00:14:08.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009458+0000", 2024-09-15T00:14:08.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8538000000000003e-05 2024-09-15T00:14:08.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009495+0000", 2024-09-15T00:14:08.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7061999999999999e-05 2024-09-15T00:14:08.472 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.472 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.473 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127009 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7487488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:08.473 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:14:08.473 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.45226811400001, 2024-09-15T00:14:08.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041233063, 2024-09-15T00:14:08.473 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.473 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.473 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:08.473 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.473 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.473 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.474 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127009 2024-09-15T00:14:08.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.474 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:14:08.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:14:08.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968414+0000", 2024-09-15T00:14:08.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7199999999999999e-07 2024-09-15T00:14:08.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968417+0000", 2024-09-15T00:14:08.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3810000000000002e-06 2024-09-15T00:14:08.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968423+0000", 2024-09-15T00:14:08.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8820000000000003e-06 2024-09-15T00:14:08.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968433+0000", 2024-09-15T00:14:08.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0005e-05 2024-09-15T00:14:08.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009527+0000", 2024-09-15T00:14:08.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041093750999999998 2024-09-15T00:14:08.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009601+0000", 2024-09-15T00:14:08.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3979000000000004e-05 2024-09-15T00:14:08.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009647+0000", 2024-09-15T00:14:08.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5493e-05 2024-09-15T00:14:08.477 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.477 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.477 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.477 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127010 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7610368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:08.477 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:14:08.477 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.45223910999999, 2024-09-15T00:14:08.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041325924999999999, 2024-09-15T00:14:08.478 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.478 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.478 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:08.478 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.478 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.478 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.478 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127010 2024-09-15T00:14:08.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.478 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:14:08.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:14:08.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968443+0000", 2024-09-15T00:14:08.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:14:08.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968445+0000", 2024-09-15T00:14:08.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:14:08.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968452+0000", 2024-09-15T00:14:08.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6250000000000001e-06 2024-09-15T00:14:08.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968456+0000", 2024-09-15T00:14:08.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5210000000000004e-06 2024-09-15T00:14:08.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009664+0000", 2024-09-15T00:14:08.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041207404000000003 2024-09-15T00:14:08.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009723+0000", 2024-09-15T00:14:08.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9036000000000002e-05 2024-09-15T00:14:08.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009768+0000", 2024-09-15T00:14:08.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5578000000000003e-05 2024-09-15T00:14:08.481 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.481 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.481 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.482 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127011 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7733248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:08.482 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:14:08.482 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.45222087499999, 2024-09-15T00:14:08.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041482441000000002, 2024-09-15T00:14:08.482 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.482 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.482 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:08.482 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.482 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.482 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.482 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127011 2024-09-15T00:14:08.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.483 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:14:08.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:14:08.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968461+0000", 2024-09-15T00:14:08.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-15T00:14:08.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968466+0000", 2024-09-15T00:14:08.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4939999999999997e-06 2024-09-15T00:14:08.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968470+0000", 2024-09-15T00:14:08.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6469999999999997e-06 2024-09-15T00:14:08.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968473+0000", 2024-09-15T00:14:08.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.977e-06 2024-09-15T00:14:08.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009800+0000", 2024-09-15T00:14:08.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041326918999999997 2024-09-15T00:14:08.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009869+0000", 2024-09-15T00:14:08.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8935000000000003e-05 2024-09-15T00:14:08.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009943+0000", 2024-09-15T00:14:08.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4188000000000002e-05 2024-09-15T00:14:08.486 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.486 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.486 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.486 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127012 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7856128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:08.486 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:14:08.486 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.452205373, 2024-09-15T00:14:08.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041640756000000001, 2024-09-15T00:14:08.486 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.486 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.487 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:08.487 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.487 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.487 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.487 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127012 2024-09-15T00:14:08.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.487 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:14:08.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968476+0000", 2024-09-15T00:14:08.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968477+0000", 2024-09-15T00:14:08.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-15T00:14:08.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968482+0000", 2024-09-15T00:14:08.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6180000000000003e-06 2024-09-15T00:14:08.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968486+0000", 2024-09-15T00:14:08.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.501e-06 2024-09-15T00:14:08.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968489+0000", 2024-09-15T00:14:08.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1530000000000001e-06 2024-09-15T00:14:08.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.009983+0000", 2024-09-15T00:14:08.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041493980999999999 2024-09-15T00:14:08.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010051+0000", 2024-09-15T00:14:08.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8039000000000004e-05 2024-09-15T00:14:08.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010117+0000", 2024-09-15T00:14:08.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6254000000000006e-05 2024-09-15T00:14:08.490 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.490 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.490 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.491 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127013 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 7979008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:08.491 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:14:08.491 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.452189553, 2024-09-15T00:14:08.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.041763863999999998, 2024-09-15T00:14:08.491 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.491 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.491 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:08.491 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.491 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.491 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.491 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127013 2024-09-15T00:14:08.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.491 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:14:08.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.968492+0000", 2024-09-15T00:14:08.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969767+0000", 2024-09-15T00:14:08.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001275006 2024-09-15T00:14:08.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969772+0000", 2024-09-15T00:14:08.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4769999999999997e-06 2024-09-15T00:14:08.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969781+0000", 2024-09-15T00:14:08.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1959999999999997e-06 2024-09-15T00:14:08.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969788+0000", 2024-09-15T00:14:08.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6359999999999994e-06 2024-09-15T00:14:08.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010162+0000", 2024-09-15T00:14:08.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040373291999999998 2024-09-15T00:14:08.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010212+0000", 2024-09-15T00:14:08.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0543e-05 2024-09-15T00:14:08.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010256+0000", 2024-09-15T00:14:08.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3714000000000002e-05 2024-09-15T00:14:08.495 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.495 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.495 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.495 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127015 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8224768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:08.495 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:14:08.495 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.450850617, 2024-09-15T00:14:08.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040710186000000002, 2024-09-15T00:14:08.495 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.495 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.495 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:08.495 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.496 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.496 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.496 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127015 2024-09-15T00:14:08.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.496 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:14:08.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969831+0000", 2024-09-15T00:14:08.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969832+0000", 2024-09-15T00:14:08.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:14:08.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969834+0000", 2024-09-15T00:14:08.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2919999999999998e-06 2024-09-15T00:14:08.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969843+0000", 2024-09-15T00:14:08.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0480000000000004e-06 2024-09-15T00:14:08.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969848+0000", 2024-09-15T00:14:08.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-15T00:14:08.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010457+0000", 2024-09-15T00:14:08.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040609054999999998 2024-09-15T00:14:08.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010501+0000", 2024-09-15T00:14:08.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3396000000000003e-05 2024-09-15T00:14:08.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010541+0000", 2024-09-15T00:14:08.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0599000000000002e-05 2024-09-15T00:14:08.499 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.499 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.499 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.499 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127016 3.b 3:d1e449a4:::rbd_data.1139d7422821.0000000000000001:1f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:14:08.499 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:14:08.500 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.450799991, 2024-09-15T00:14:08.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040766216000000001, 2024-09-15T00:14:08.500 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.500 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.500 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:08.500 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.500 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.500 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.500 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127016 2024-09-15T00:14:08.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.500 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:14:08.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:14:08.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969882+0000", 2024-09-15T00:14:08.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-15T00:14:08.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969886+0000", 2024-09-15T00:14:08.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8449999999999996e-06 2024-09-15T00:14:08.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969897+0000", 2024-09-15T00:14:08.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0526000000000001e-05 2024-09-15T00:14:08.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:21.969904+0000", 2024-09-15T00:14:08.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1169999999999998e-06 2024-09-15T00:14:08.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010561+0000", 2024-09-15T00:14:08.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040657470000000001 2024-09-15T00:14:08.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010607+0000", 2024-09-15T00:14:08.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6108e-05 2024-09-15T00:14:08.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.010648+0000", 2024-09-15T00:14:08.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0479000000000003e-05 2024-09-15T00:14:08.504 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.504 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.504 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.504 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:127694 3.9 3:9390b205:::rbd_data.11396481e1ec.00000000000000df:head [set-alloc-hint object_size 1048576 write_size 1048576,write 819200~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T00:14:08.504 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:14:08.504 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.196080874, 2024-09-15T00:14:08.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90376419399999997, 2024-09-15T00:14:08.504 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.504 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.505 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:08.505 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.505 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.505 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.505 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 127694 2024-09-15T00:14:08.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.505 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:14:08.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224601+0000", 2024-09-15T00:14:08.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224602+0000", 2024-09-15T00:14:08.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5899999999999995e-07 2024-09-15T00:14:08.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224610+0000", 2024-09-15T00:14:08.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9130000000000003e-06 2024-09-15T00:14:08.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224620+0000", 2024-09-15T00:14:08.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0426e-05 2024-09-15T00:14:08.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224627+0000", 2024-09-15T00:14:08.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4390000000000003e-06 2024-09-15T00:14:08.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224675+0000", 2024-09-15T00:14:08.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7636000000000003e-05 2024-09-15T00:14:08.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-15T00:14:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:22.224681+0000", 2024-09-15T00:14:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1460000000000001e-06 2024-09-15T00:14:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126388+0000", 2024-09-15T00:14:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90170724999999996 2024-09-15T00:14:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126526+0000", 2024-09-15T00:14:08.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013730599999999999 2024-09-15T00:14:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-15T00:14:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.126684+0000", 2024-09-15T00:14:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000157935 2024-09-15T00:14:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:14:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.127932+0000", 2024-09-15T00:14:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012479240000000001 2024-09-15T00:14:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:08.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128269+0000", 2024-09-15T00:14:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00033761099999999998 2024-09-15T00:14:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128324+0000", 2024-09-15T00:14:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5244999999999997e-05 2024-09-15T00:14:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:23.128365+0000", 2024-09-15T00:14:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0503999999999997e-05 2024-09-15T00:14:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.510 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4409.0:134707 3.15 3:af1fbdb5:::rbd_data.113933a4f79a.000000000000001c:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T00:14:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:14:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.728113759999999, 2024-09-15T00:14:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.057328626000000001, 2024-09-15T00:14:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:14:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-15T00:14:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4409", 2024-09-15T00:14:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/3814908215", 2024-09-15T00:14:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 134707 2024-09-15T00:14:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.511 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:14:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692568+0000", 2024-09-15T00:14:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692569+0000", 2024-09-15T00:14:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.339e-06 2024-09-15T00:14:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692574+0000", 2024-09-15T00:14:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-15T00:14:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692584+0000", 2024-09-15T00:14:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0363000000000001e-05 2024-09-15T00:14:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692590+0000", 2024-09-15T00:14:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.818e-06 2024-09-15T00:14:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692633+0000", 2024-09-15T00:14:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.265e-05 2024-09-15T00:14:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.692718+0000", 2024-09-15T00:14:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5279999999999997e-05 2024-09-15T00:14:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-15T00:14:08.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.695940+0000", 2024-09-15T00:14:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003222052 2024-09-15T00:14:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:14:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749714+0000", 2024-09-15T00:14:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.053774126999999998 2024-09-15T00:14:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-15T00:14:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749788+0000", 2024-09-15T00:14:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3693000000000001e-05 2024-09-15T00:14:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-15T00:14:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749847+0000", 2024-09-15T00:14:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9175999999999997e-05 2024-09-15T00:14:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749897+0000", 2024-09-15T00:14:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9311999999999998e-05 2024-09-15T00:14:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop_reply(client.4409.0:134707 3.15 e86/55 ondisk, result = 0)", 2024-09-15T00:14:08.516 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:14:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 96.719552410999995, 2024-09-15T00:14:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048596066, 2024-09-15T00:14:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-15T00:14:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-15T00:14:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-15T00:14:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-15T00:14:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-15T00:14:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:14:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-15T00:14:08.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701129+0000", 2024-09-15T00:14:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-15T00:14:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-15T00:14:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701131+0000", 2024-09-15T00:14:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3549999999999999e-06 2024-09-15T00:14:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-15T00:14:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701139+0000", 2024-09-15T00:14:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1650000000000006e-06 2024-09-15T00:14:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-15T00:14:08.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701156+0000", 2024-09-15T00:14:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6730000000000001e-05 2024-09-15T00:14:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-15T00:14:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.701167+0000", 2024-09-15T00:14:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1504e-05 2024-09-15T00:14:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-15T00:14:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749629+0000", 2024-09-15T00:14:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048461952000000003 2024-09-15T00:14:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-15T00:14:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749640+0000", 2024-09-15T00:14:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0874e-05 2024-09-15T00:14:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-15T00:14:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-15T00:14:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-15T00:14:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-15T00:12:31.749725+0000", 2024-09-15T00:14:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5486000000000001e-05 2024-09-15T00:14:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-15T00:14:08.520 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-15T00:14:08.520 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:08.521 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:14:08.522 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:14:08.523 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:14:08.520+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:14:08.623 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:14:08.621+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:14:08.636 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-15T00:14:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-15T00:14:08.636 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-15T00:14:08.636 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-15T00:14:08.645 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:14:08.683 INFO:teuthology.orchestra.run.smithi171.stdout:osd.0: {} 2024-09-15T00:14:08.683 INFO:teuthology.orchestra.run.smithi171.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:14:08.700 INFO:teuthology.orchestra.run.smithi171.stdout:osd.1: {} 2024-09-15T00:14:08.700 INFO:teuthology.orchestra.run.smithi171.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:14:08.716 INFO:teuthology.orchestra.run.smithi171.stdout:osd.2: {} 2024-09-15T00:14:08.716 INFO:teuthology.orchestra.run.smithi171.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:14:08.724 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:14:08.722+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:14:08.732 INFO:teuthology.orchestra.run.smithi171.stdout:osd.3: {} 2024-09-15T00:14:08.732 INFO:teuthology.orchestra.run.smithi171.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:14:08.750 INFO:teuthology.orchestra.run.smithi171.stdout:osd.4: {} 2024-09-15T00:14:08.750 INFO:teuthology.orchestra.run.smithi171.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:14:08.750 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:08.750 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:08.759 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:14:08.767 INFO:teuthology.orchestra.run.smithi171.stdout:osd.5: {} 2024-09-15T00:14:08.767 INFO:teuthology.orchestra.run.smithi171.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:14:08.783 INFO:teuthology.orchestra.run.smithi171.stdout:osd.6: {} 2024-09-15T00:14:08.783 INFO:teuthology.orchestra.run.smithi171.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T00:14:08.800 INFO:teuthology.orchestra.run.smithi171.stdout:osd.7: {} 2024-09-15T00:14:08.800 INFO:teuthology.orchestra.run.smithi171.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T00:14:08.825 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:14:08.823+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:08.871 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:14:08.956 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:14:08.954+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:14:09.016 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:09.016 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:09.025 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:14:09.026 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:14:09.024+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:14:09.083 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is set 2024-09-15T00:14:09.127 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:14:09.125+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:14:09.171 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:09.171 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:09.180 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:14:09.266 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:14:09.264+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:14:09.323 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:09.324 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:09.328 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-15T00:14:09.326+0000 7f23dcbee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33610) UID: 0 2024-09-15T00:14:09.332 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T00:14:09.429 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:14:09.427+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:14:09.477 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:09.477 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:09.486 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T00:14:09.570 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:14:09.568+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:14:09.629 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:09.629 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:09.630 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-15T00:14:09.629+0000 7f9ef8c46640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33603) UID: 0 2024-09-15T00:14:09.638 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T00:14:09.731 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:14:09.730+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:14:09.781 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:09.781 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:09.790 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T00:14:09.832 INFO:tasks.ceph.osd.0.smithi171.stderr:2024-09-15T00:14:09.830+0000 7fcf0f8c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33229) UID: 0 2024-09-15T00:14:09.933 INFO:tasks.ceph.osd.1.smithi171.stderr:2024-09-15T00:14:09.931+0000 7fef6cf5f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33241) UID: 0 2024-09-15T00:14:09.933 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:09.934 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:09.942 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T00:14:10.039 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-15T00:14:10.036+0000 7f801ac60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33605) UID: 0 2024-09-15T00:14:10.063 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:10.063 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:10.077 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T00:14:10.135 INFO:tasks.ceph.osd.2.smithi171.stderr:2024-09-15T00:14:10.132+0000 7feb5d860640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 49422) UID: 0 2024-09-15T00:14:10.170 INFO:teuthology.orchestra.run.smithi171.stdout:All operations completed A-OK! 2024-09-15T00:14:10.183 DEBUG:teuthology.parallel:result is None 2024-09-15T00:14:10.183 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-15T00:14:10.192 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-15T00:14:10.210 INFO:tasks.thrashosds:joining thrashosds 2024-09-15T00:14:10.253 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:10.253 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:10.264 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:14:10.395 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:10.395 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:10.406 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:14:10.525 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:10.525 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:10.535 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:14:10.655 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-15T00:14:10.655 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:14:11.102 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:14:12.026 INFO:teuthology.orchestra.run.smithi171.stderr:noscrub is unset 2024-09-15T00:14:12.047 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:14:13.038 INFO:teuthology.orchestra.run.smithi171.stderr:nodeep-scrub is unset 2024-09-15T00:14:13.098 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-15T00:14:13.099 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-15T00:14:13.099 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-15T00:14:13.390 INFO:tasks.ceph.mon.a.smithi171.stderr:2024-09-15T00:14:13.387+0000 7f0425d35640 -1 mon.a@0(leader).osd e185 definitely_dead 0 2024-09-15T00:14:14.045 INFO:teuthology.orchestra.run.smithi171.stderr:marked in osd.0. 2024-09-15T00:14:14.065 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-15T00:14:14.065 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-15T00:14:14.065 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-15T00:14:14.065 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-15T00:14:14.362 INFO:tasks.ceph.mon.a.smithi171.stderr:2024-09-15T00:14:14.359+0000 7f0425d35640 -1 mon.a@0(leader).osd e186 definitely_dead 0 2024-09-15T00:14:15.024 INFO:teuthology.orchestra.run.smithi171.stderr:marked in osd.1. 2024-09-15T00:14:15.044 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-15T00:14:15.045 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-15T00:14:15.158 INFO:teuthology.orchestra.run.smithi171.stdout:mon.a: {} 2024-09-15T00:14:15.159 INFO:teuthology.orchestra.run.smithi171.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T00:14:15.168 INFO:teuthology.orchestra.run.smithi171.stdout:mon.b: {} 2024-09-15T00:14:15.168 INFO:teuthology.orchestra.run.smithi171.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T00:14:15.176 INFO:teuthology.orchestra.run.smithi171.stdout:mon.c: {} 2024-09-15T00:14:15.176 INFO:teuthology.orchestra.run.smithi171.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T00:14:15.187 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T00:14:16.030 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T00:14:16.071 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-15T00:14:17.032 INFO:teuthology.orchestra.run.smithi171.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-15T00:14:17.055 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T00:14:18.041 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T00:14:18.058 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-15T00:14:19.037 INFO:teuthology.orchestra.run.smithi171.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-15T00:14:19.055 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T00:14:20.033 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T00:14:20.049 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-15T00:14:21.031 INFO:teuthology.orchestra.run.smithi171.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-15T00:14:21.047 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-15T00:14:22.034 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.4 to 1 (10000) 2024-09-15T00:14:22.051 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-15T00:14:23.095 INFO:teuthology.orchestra.run.smithi171.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-15T00:14:23.109 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-15T00:14:24.089 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.5 to 1 (10000) 2024-09-15T00:14:24.109 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-15T00:14:25.027 INFO:teuthology.orchestra.run.smithi171.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-15T00:14:25.044 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-15T00:14:26.104 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.6 to 1 (10000) 2024-09-15T00:14:26.123 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-15T00:14:27.115 INFO:teuthology.orchestra.run.smithi171.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-15T00:14:27.132 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-15T00:14:28.117 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.7 to 1 (10000) 2024-09-15T00:14:28.135 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-15T00:14:29.127 INFO:teuthology.orchestra.run.smithi171.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-15T00:14:29.143 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T00:14:30.133 INFO:teuthology.orchestra.run.smithi171.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T00:14:30.151 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-15T00:14:31.138 INFO:teuthology.orchestra.run.smithi171.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-15T00:14:31.156 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-15T00:14:31.156 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-15T00:14:31.156 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-15T00:14:31.157 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-15T00:14:31.157 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T00:14:31.157 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T00:14:31.441 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:14:31.441 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":203,"fsid":"bad2533c-9c09-4af4-ab93-147afbacb705","created":"2024-09-15T00:10:50.203030+0000","modified":"2024-09-15T00:14:31.135695+0000","last_up_change":"2024-09-15T00:12:55.092989+0000","last_in_change":"2024-09-15T00:14:15.021696+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T00:11:04.326356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T00:11:08.005967+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-15T00:11:17.676102+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":53,"snap_epoch":180,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7337111f-0e02-490d-976b-3814a03371f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6825","nonce":2906515165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6827","nonce":2906515165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6831","nonce":2906515165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6829","nonce":2906515165}]},"public_addr":"172.21.15.171:6825/2906515165","cluster_addr":"172.21.15.171:6827/2906515165","heartbeat_back_addr":"172.21.15.171:6831/2906515165","heartbeat_front_addr":"172.21.15.171:6829/2906515165","state":["exists","up"]},{"osd":1,"uuid":"64412633-4dd4-4b8d-b3f1-ed3c664ddf48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6817","nonce":3039831960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6819","nonce":3039831960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6823","nonce":3039831960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6821","nonce":3039831960}]},"public_addr":"172.21.15.171:6817/3039831960","cluster_addr":"172.21.15.171:6819/3039831960","heartbeat_back_addr":"172.21.15.171:6823/3039831960","heartbeat_front_addr":"172.21.15.171:6821/3039831960","state":["exists","up"]},{"osd":2,"uuid":"0300adee-5ce3-43ce-997b-2c0e970edbc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":79,"up_from":108,"up_thru":190,"down_at":81,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":213053077},{"type":"v1","addr":"172.21.15.171:6809","nonce":213053077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":213053077},{"type":"v1","addr":"172.21.15.171:6811","nonce":213053077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":213053077},{"type":"v1","addr":"172.21.15.171:6815","nonce":213053077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":213053077},{"type":"v1","addr":"172.21.15.171:6813","nonce":213053077}]},"public_addr":"172.21.15.171:6809/213053077","cluster_addr":"172.21.15.171:6811/213053077","heartbeat_back_addr":"172.21.15.171:6815/213053077","heartbeat_front_addr":"172.21.15.171:6813/213053077","state":["exists","up"]},{"osd":3,"uuid":"ec668801-9b8f-401a-829c-0b1f8d8f08fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6801","nonce":1819098683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6803","nonce":1819098683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6807","nonce":1819098683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6805","nonce":1819098683}]},"public_addr":"172.21.15.171:6801/1819098683","cluster_addr":"172.21.15.171:6803/1819098683","heartbeat_back_addr":"172.21.15.171:6807/1819098683","heartbeat_front_addr":"172.21.15.171:6805/1819098683","state":["exists","up"]},{"osd":4,"uuid":"9cc48936-5644-44c6-84bb-06510e933e8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6801","nonce":2801626173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6803","nonce":2801626173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6807","nonce":2801626173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6805","nonce":2801626173}]},"public_addr":"172.21.15.184:6801/2801626173","cluster_addr":"172.21.15.184:6803/2801626173","heartbeat_back_addr":"172.21.15.184:6807/2801626173","heartbeat_front_addr":"172.21.15.184:6805/2801626173","state":["exists","up"]},{"osd":5,"uuid":"f6c53297-9b00-476f-9be8-e76ff762b044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6809","nonce":3271066302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6811","nonce":3271066302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6815","nonce":3271066302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6813","nonce":3271066302}]},"public_addr":"172.21.15.184:6809/3271066302","cluster_addr":"172.21.15.184:6811/3271066302","heartbeat_back_addr":"172.21.15.184:6815/3271066302","heartbeat_front_addr":"172.21.15.184:6813/3271066302","state":["exists","up"]},{"osd":6,"uuid":"737f08a6-2223-46b3-b9fe-6ae18b067ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6817","nonce":1903994256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6819","nonce":1903994256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6823","nonce":1903994256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6821","nonce":1903994256}]},"public_addr":"172.21.15.184:6817/1903994256","cluster_addr":"172.21.15.184:6819/1903994256","heartbeat_back_addr":"172.21.15.184:6823/1903994256","heartbeat_front_addr":"172.21.15.184:6821/1903994256","state":["exists","up"]},{"osd":7,"uuid":"5a5fba64-9249-426d-814b-fd251e8aa910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6825","nonce":978565004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6827","nonce":978565004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6831","nonce":978565004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6829","nonce":978565004}]},"public_addr":"172.21.15.184:6825/978565004","cluster_addr":"172.21.15.184:6827/978565004","heartbeat_back_addr":"172.21.15.184:6831/978565004","heartbeat_front_addr":"172.21.15.184:6829/978565004","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:59.070337+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.884683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.998882+0000","dead_epoch":81},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.656630+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.802744+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.567865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.739964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:59.565310+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.9","osds":[3,0]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:14:31.456 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T00:14:31.457 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T00:14:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:14:31.733 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":203,"fsid":"bad2533c-9c09-4af4-ab93-147afbacb705","created":"2024-09-15T00:10:50.203030+0000","modified":"2024-09-15T00:14:31.135695+0000","last_up_change":"2024-09-15T00:12:55.092989+0000","last_in_change":"2024-09-15T00:14:15.021696+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T00:11:04.326356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T00:11:08.005967+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-15T00:11:17.676102+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":53,"snap_epoch":180,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7337111f-0e02-490d-976b-3814a03371f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6825","nonce":2906515165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6827","nonce":2906515165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6831","nonce":2906515165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6829","nonce":2906515165}]},"public_addr":"172.21.15.171:6825/2906515165","cluster_addr":"172.21.15.171:6827/2906515165","heartbeat_back_addr":"172.21.15.171:6831/2906515165","heartbeat_front_addr":"172.21.15.171:6829/2906515165","state":["exists","up"]},{"osd":1,"uuid":"64412633-4dd4-4b8d-b3f1-ed3c664ddf48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6817","nonce":3039831960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6819","nonce":3039831960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6823","nonce":3039831960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6821","nonce":3039831960}]},"public_addr":"172.21.15.171:6817/3039831960","cluster_addr":"172.21.15.171:6819/3039831960","heartbeat_back_addr":"172.21.15.171:6823/3039831960","heartbeat_front_addr":"172.21.15.171:6821/3039831960","state":["exists","up"]},{"osd":2,"uuid":"0300adee-5ce3-43ce-997b-2c0e970edbc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":79,"up_from":108,"up_thru":190,"down_at":81,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":213053077},{"type":"v1","addr":"172.21.15.171:6809","nonce":213053077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":213053077},{"type":"v1","addr":"172.21.15.171:6811","nonce":213053077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":213053077},{"type":"v1","addr":"172.21.15.171:6815","nonce":213053077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":213053077},{"type":"v1","addr":"172.21.15.171:6813","nonce":213053077}]},"public_addr":"172.21.15.171:6809/213053077","cluster_addr":"172.21.15.171:6811/213053077","heartbeat_back_addr":"172.21.15.171:6815/213053077","heartbeat_front_addr":"172.21.15.171:6813/213053077","state":["exists","up"]},{"osd":3,"uuid":"ec668801-9b8f-401a-829c-0b1f8d8f08fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6801","nonce":1819098683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6803","nonce":1819098683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6807","nonce":1819098683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6805","nonce":1819098683}]},"public_addr":"172.21.15.171:6801/1819098683","cluster_addr":"172.21.15.171:6803/1819098683","heartbeat_back_addr":"172.21.15.171:6807/1819098683","heartbeat_front_addr":"172.21.15.171:6805/1819098683","state":["exists","up"]},{"osd":4,"uuid":"9cc48936-5644-44c6-84bb-06510e933e8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6801","nonce":2801626173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6803","nonce":2801626173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6807","nonce":2801626173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6805","nonce":2801626173}]},"public_addr":"172.21.15.184:6801/2801626173","cluster_addr":"172.21.15.184:6803/2801626173","heartbeat_back_addr":"172.21.15.184:6807/2801626173","heartbeat_front_addr":"172.21.15.184:6805/2801626173","state":["exists","up"]},{"osd":5,"uuid":"f6c53297-9b00-476f-9be8-e76ff762b044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6809","nonce":3271066302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6811","nonce":3271066302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6815","nonce":3271066302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6813","nonce":3271066302}]},"public_addr":"172.21.15.184:6809/3271066302","cluster_addr":"172.21.15.184:6811/3271066302","heartbeat_back_addr":"172.21.15.184:6815/3271066302","heartbeat_front_addr":"172.21.15.184:6813/3271066302","state":["exists","up"]},{"osd":6,"uuid":"737f08a6-2223-46b3-b9fe-6ae18b067ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6817","nonce":1903994256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6819","nonce":1903994256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6823","nonce":1903994256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6821","nonce":1903994256}]},"public_addr":"172.21.15.184:6817/1903994256","cluster_addr":"172.21.15.184:6819/1903994256","heartbeat_back_addr":"172.21.15.184:6823/1903994256","heartbeat_front_addr":"172.21.15.184:6821/1903994256","state":["exists","up"]},{"osd":7,"uuid":"5a5fba64-9249-426d-814b-fd251e8aa910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6825","nonce":978565004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6827","nonce":978565004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6831","nonce":978565004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6829","nonce":978565004}]},"public_addr":"172.21.15.184:6825/978565004","cluster_addr":"172.21.15.184:6827/978565004","heartbeat_back_addr":"172.21.15.184:6831/978565004","heartbeat_front_addr":"172.21.15.184:6829/978565004","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:59.070337+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.884683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.998882+0000","dead_epoch":81},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.656630+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.802744+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.567865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.739964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:59.565310+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.9","osds":[3,0]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:14:31.746 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T00:14:31.747 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T00:14:31.747 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T00:14:31.748 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T00:14:31.748 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T00:14:31.749 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T00:14:31.749 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-15T00:14:31.750 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-15T00:14:31.989 INFO:teuthology.orchestra.run.smithi171.stdout:60129542191 2024-09-15T00:14:31.989 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T00:14:31.994 INFO:teuthology.orchestra.run.smithi171.stdout:60129542190 2024-09-15T00:14:31.995 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T00:14:31.996 INFO:teuthology.orchestra.run.smithi171.stdout:463856467990 2024-09-15T00:14:31.996 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T00:14:31.997 INFO:teuthology.orchestra.run.smithi171.stdout:60129542193 2024-09-15T00:14:31.997 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T00:14:31.998 INFO:teuthology.orchestra.run.smithi171.stdout:60129542190 2024-09-15T00:14:31.998 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T00:14:31.999 INFO:teuthology.orchestra.run.smithi171.stdout:60129542188 2024-09-15T00:14:31.999 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T00:14:32.000 INFO:teuthology.orchestra.run.smithi171.stdout:60129542191 2024-09-15T00:14:32.001 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T00:14:32.002 INFO:teuthology.orchestra.run.smithi171.stdout:60129542189 2024-09-15T00:14:32.002 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T00:14:32.551 INFO:teuthology.orchestra.run.smithi171.stdout:60129542190 2024-09-15T00:14:32.560 INFO:teuthology.orchestra.run.smithi171.stdout:60129542189 2024-09-15T00:14:32.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542191 got 60129542190 for osd.6 2024-09-15T00:14:32.580 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542189 for osd.7 2024-09-15T00:14:32.597 INFO:teuthology.orchestra.run.smithi171.stdout:60129542191 2024-09-15T00:14:32.605 INFO:teuthology.orchestra.run.smithi171.stdout:60129542188 2024-09-15T00:14:32.611 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542193 got 60129542191 for osd.4 2024-09-15T00:14:32.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542189 got 60129542188 for osd.1 2024-09-15T00:14:32.619 INFO:teuthology.orchestra.run.smithi171.stdout:60129542187 2024-09-15T00:14:32.629 INFO:teuthology.orchestra.run.smithi171.stdout:60129542189 2024-09-15T00:14:32.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542188 got 60129542187 for osd.5 2024-09-15T00:14:32.639 INFO:teuthology.orchestra.run.smithi171.stdout:463856467989 2024-09-15T00:14:32.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542189 for osd.0 2024-09-15T00:14:32.648 INFO:teuthology.orchestra.run.smithi171.stdout:60129542190 2024-09-15T00:14:32.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 463856467990 got 463856467989 for osd.2 2024-09-15T00:14:32.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542191 got 60129542190 for osd.3 2024-09-15T00:14:33.570 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T00:14:33.581 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T00:14:33.612 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T00:14:33.620 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T00:14:33.634 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T00:14:33.643 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T00:14:33.653 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T00:14:33.662 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T00:14:34.112 INFO:teuthology.orchestra.run.smithi171.stdout:60129542191 2024-09-15T00:14:34.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542191 got 60129542191 for osd.6 2024-09-15T00:14:34.133 DEBUG:teuthology.parallel:result is None 2024-09-15T00:14:34.167 INFO:teuthology.orchestra.run.smithi171.stdout:60129542190 2024-09-15T00:14:34.188 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542190 for osd.7 2024-09-15T00:14:34.189 DEBUG:teuthology.parallel:result is None 2024-09-15T00:14:34.193 INFO:teuthology.orchestra.run.smithi171.stdout:60129542190 2024-09-15T00:14:34.193 INFO:teuthology.orchestra.run.smithi171.stdout:60129542193 2024-09-15T00:14:34.202 INFO:teuthology.orchestra.run.smithi171.stdout:60129542188 2024-09-15T00:14:34.209 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542190 for osd.0 2024-09-15T00:14:34.209 DEBUG:teuthology.parallel:result is None 2024-09-15T00:14:34.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542193 got 60129542193 for osd.4 2024-09-15T00:14:34.215 DEBUG:teuthology.parallel:result is None 2024-09-15T00:14:34.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542188 got 60129542188 for osd.5 2024-09-15T00:14:34.219 DEBUG:teuthology.parallel:result is None 2024-09-15T00:14:34.222 INFO:teuthology.orchestra.run.smithi171.stdout:463856467990 2024-09-15T00:14:34.237 INFO:tasks.ceph.ceph_manager.ceph:need seq 463856467990 got 463856467990 for osd.2 2024-09-15T00:14:34.237 DEBUG:teuthology.parallel:result is None 2024-09-15T00:14:34.253 INFO:teuthology.orchestra.run.smithi171.stdout:60129542189 2024-09-15T00:14:34.267 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542189 got 60129542189 for osd.1 2024-09-15T00:14:34.267 DEBUG:teuthology.parallel:result is None 2024-09-15T00:14:34.270 INFO:teuthology.orchestra.run.smithi171.stdout:60129542191 2024-09-15T00:14:34.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542191 got 60129542191 for osd.3 2024-09-15T00:14:34.284 DEBUG:teuthology.parallel:result is None 2024-09-15T00:14:34.284 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T00:14:34.284 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:14:34.603 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:14:34.604 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:14:34.615 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-15T00:14:34.327513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":525,"num_object_clones":0,"num_object_copies":1050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":525,"num_whiteouts":517,"num_read":240395,"num_read_kb":1281361,"num_write":28811,"num_write_kb":587641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10669,"num_bytes_recovered":392695172,"num_keys_recovered":726,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25327,"ondisk_log_size":25327,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":497084,"kb_used_data":19172,"kb_used_omap":17,"kb_used_meta":477678,"kb_avail":754477636,"statfs":{"total":773094113280,"available":772585099264,"internally_reserved":0,"allocated":19632128,"data_stored":10703506,"data_compressed":12736,"data_compressed_allocated":73728,"data_compressed_original":1179648,"omap_allocated":18117,"internal_metadata":489142587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001051"},"pg_stats":[{"pgid":"3.1f","version":"182'643","reported_seq":6819,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142321+0000","last_change":"2024-09-15T00:14:18.045712+0000","last_active":"2024-09-15T00:14:31.142321+0000","last_peered":"2024-09-15T00:14:31.142321+0000","last_clean":"2024-09-15T00:14:31.142321+0000","last_became_active":"2024-09-15T00:14:18.042449+0000","last_became_peered":"2024-09-15T00:14:18.042449+0000","last_unstale":"2024-09-15T00:14:31.142321+0000","last_undegraded":"2024-09-15T00:14:31.142321+0000","last_fullsized":"2024-09-15T00:14:31.142321+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:38:13.616525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011752869999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6942,"num_read_kb":26340,"num_write":697,"num_write_kb":18376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":5475412,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'632","reported_seq":7633,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763549+0000","last_change":"2024-09-15T00:14:08.080286+0000","last_active":"2024-09-15T00:14:31.763549+0000","last_peered":"2024-09-15T00:14:31.763549+0000","last_clean":"2024-09-15T00:14:31.763549+0000","last_became_active":"2024-09-15T00:11:19.598640+0000","last_became_peered":"2024-09-15T00:11:19.598640+0000","last_unstale":"2024-09-15T00:14:31.763549+0000","last_undegraded":"2024-09-15T00:14:31.763549+0000","last_fullsized":"2024-09-15T00:14:31.763549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:16:36.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016674789999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7565,"num_read_kb":39401,"num_write":737,"num_write_kb":10312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'698","reported_seq":4933,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762732+0000","last_change":"2024-09-15T00:14:08.080849+0000","last_active":"2024-09-15T00:14:31.762732+0000","last_peered":"2024-09-15T00:14:31.762732+0000","last_clean":"2024-09-15T00:14:31.762732+0000","last_became_active":"2024-09-15T00:11:19.595954+0000","last_became_peered":"2024-09-15T00:11:19.595954+0000","last_unstale":"2024-09-15T00:14:31.762732+0000","last_undegraded":"2024-09-15T00:14:31.762732+0000","last_fullsized":"2024-09-15T00:14:31.762732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:28.144497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020206820000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5583,"num_read_kb":45090,"num_write":773,"num_write_kb":31484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"182'674","reported_seq":5880,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762701+0000","last_change":"2024-09-15T00:14:08.080354+0000","last_active":"2024-09-15T00:14:31.762701+0000","last_peered":"2024-09-15T00:14:31.762701+0000","last_clean":"2024-09-15T00:14:31.762701+0000","last_became_active":"2024-09-15T00:11:19.602583+0000","last_became_peered":"2024-09-15T00:11:19.602583+0000","last_unstale":"2024-09-15T00:14:31.762701+0000","last_undegraded":"2024-09-15T00:14:31.762701+0000","last_fullsized":"2024-09-15T00:14:31.762701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:33:11.479982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00168294,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":5888,"num_read_kb":35219,"num_write":807,"num_write_kb":10221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'844","reported_seq":6971,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142108+0000","last_change":"2024-09-15T00:14:20.035839+0000","last_active":"2024-09-15T00:14:31.142108+0000","last_peered":"2024-09-15T00:14:31.142108+0000","last_clean":"2024-09-15T00:14:31.142108+0000","last_became_active":"2024-09-15T00:14:20.032662+0000","last_became_peered":"2024-09-15T00:14:20.032662+0000","last_unstale":"2024-09-15T00:14:31.142108+0000","last_undegraded":"2024-09-15T00:14:31.142108+0000","last_fullsized":"2024-09-15T00:14:31.142108+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:24:20.329228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001922899,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7801,"num_read_kb":34537,"num_write":909,"num_write_kb":14651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":10954706,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"179'783","reported_seq":7558,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.141955+0000","last_change":"2024-09-15T00:14:24.098641+0000","last_active":"2024-09-15T00:14:31.141955+0000","last_peered":"2024-09-15T00:14:31.141955+0000","last_clean":"2024-09-15T00:14:31.141955+0000","last_became_active":"2024-09-15T00:14:24.097321+0000","last_became_peered":"2024-09-15T00:14:24.097321+0000","last_unstale":"2024-09-15T00:14:31.141955+0000","last_undegraded":"2024-09-15T00:14:31.141955+0000","last_fullsized":"2024-09-15T00:14:31.141955+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:40:15.897865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012814980000000001,"stat_sum":{"num_bytes":19,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":8900,"num_read_kb":62621,"num_write":837,"num_write_kb":33377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":66497974,"num_keys_recovered":149,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"177'678","reported_seq":5626,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142375+0000","last_change":"2024-09-15T00:14:22.038154+0000","last_active":"2024-09-15T00:14:31.142375+0000","last_peered":"2024-09-15T00:14:31.142375+0000","last_clean":"2024-09-15T00:14:31.142375+0000","last_became_active":"2024-09-15T00:14:22.036152+0000","last_became_peered":"2024-09-15T00:14:22.036152+0000","last_unstale":"2024-09-15T00:14:31.142375+0000","last_undegraded":"2024-09-15T00:14:31.142375+0000","last_fullsized":"2024-09-15T00:14:31.142375+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:30:00.495418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005256073,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4711,"num_read_kb":42462,"num_write":705,"num_write_kb":9831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":558,"num_bytes_recovered":6445817,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"179'706","reported_seq":8764,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.764017+0000","last_change":"2024-09-15T00:14:18.044374+0000","last_active":"2024-09-15T00:14:31.764017+0000","last_peered":"2024-09-15T00:14:31.764017+0000","last_clean":"2024-09-15T00:14:31.764017+0000","last_became_active":"2024-09-15T00:14:18.042710+0000","last_became_peered":"2024-09-15T00:14:18.042710+0000","last_unstale":"2024-09-15T00:14:31.764017+0000","last_undegraded":"2024-09-15T00:14:31.764017+0000","last_fullsized":"2024-09-15T00:14:31.764017+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:52:32.313688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014749780000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8102,"num_read_kb":40115,"num_write":776,"num_write_kb":15981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":4986328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'679","reported_seq":5549,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142315+0000","last_change":"2024-09-15T00:14:20.037042+0000","last_active":"2024-09-15T00:14:31.142315+0000","last_peered":"2024-09-15T00:14:31.142315+0000","last_clean":"2024-09-15T00:14:31.142315+0000","last_became_active":"2024-09-15T00:14:20.034657+0000","last_became_peered":"2024-09-15T00:14:20.034657+0000","last_unstale":"2024-09-15T00:14:31.142315+0000","last_undegraded":"2024-09-15T00:14:31.142315+0000","last_fullsized":"2024-09-15T00:14:31.142315+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:52:39.383067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002708182,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5548,"num_read_kb":45663,"num_write":634,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":12460957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"176'1032","reported_seq":8430,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762889+0000","last_change":"2024-09-15T00:14:08.079912+0000","last_active":"2024-09-15T00:14:31.762889+0000","last_peered":"2024-09-15T00:14:31.762889+0000","last_clean":"2024-09-15T00:14:31.762889+0000","last_became_active":"2024-09-15T00:13:50.031768+0000","last_became_peered":"2024-09-15T00:13:50.031768+0000","last_unstale":"2024-09-15T00:14:31.762889+0000","last_undegraded":"2024-09-15T00:14:31.762889+0000","last_fullsized":"2024-09-15T00:14:31.762889+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1032,"log_dups_size":0,"ondisk_log_size":1032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:21.539927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001158467,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8890,"num_read_kb":39257,"num_write":1264,"num_write_kb":20613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":23803494,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"179'667","reported_seq":6954,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502804+0000","last_change":"2024-09-15T00:14:08.088641+0000","last_active":"2024-09-15T00:14:31.502804+0000","last_peered":"2024-09-15T00:14:31.502804+0000","last_clean":"2024-09-15T00:14:31.502804+0000","last_became_active":"2024-09-15T00:13:50.025342+0000","last_became_peered":"2024-09-15T00:13:50.025342+0000","last_unstale":"2024-09-15T00:14:31.502804+0000","last_undegraded":"2024-09-15T00:14:31.502804+0000","last_fullsized":"2024-09-15T00:14:31.502804+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:50:45.634693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071033100000000003,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7596,"num_read_kb":45634,"num_write":737,"num_write_kb":14836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":19127384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'1079","reported_seq":7470,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142064+0000","last_change":"2024-09-15T00:14:24.095829+0000","last_active":"2024-09-15T00:14:31.142064+0000","last_peered":"2024-09-15T00:14:31.142064+0000","last_clean":"2024-09-15T00:14:31.142064+0000","last_became_active":"2024-09-15T00:14:24.093937+0000","last_became_peered":"2024-09-15T00:14:24.093937+0000","last_unstale":"2024-09-15T00:14:31.142064+0000","last_undegraded":"2024-09-15T00:14:31.142064+0000","last_fullsized":"2024-09-15T00:14:31.142064+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1079,"log_dups_size":0,"ondisk_log_size":1079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:10:03.817976+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.016604621,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":9874,"num_read_kb":58461,"num_write":1482,"num_write_kb":25008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":6083388,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":403,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470173+0000","last_change":"2024-09-15T00:13:46.030592+0000","last_active":"2024-09-15T00:14:31.470173+0000","last_peered":"2024-09-15T00:14:31.470173+0000","last_clean":"2024-09-15T00:14:31.470173+0000","last_became_active":"2024-09-15T00:13:46.030230+0000","last_became_peered":"2024-09-15T00:13:46.030230+0000","last_unstale":"2024-09-15T00:14:31.470173+0000","last_undegraded":"2024-09-15T00:14:31.470173+0000","last_fullsized":"2024-09-15T00:14:31.470173+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:26:48.293426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'715","reported_seq":7972,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142190+0000","last_change":"2024-09-15T00:14:18.046157+0000","last_active":"2024-09-15T00:14:31.142190+0000","last_peered":"2024-09-15T00:14:31.142190+0000","last_clean":"2024-09-15T00:14:31.142190+0000","last_became_active":"2024-09-15T00:14:18.043887+0000","last_became_peered":"2024-09-15T00:14:18.043887+0000","last_unstale":"2024-09-15T00:14:31.142190+0000","last_undegraded":"2024-09-15T00:14:31.142190+0000","last_fullsized":"2024-09-15T00:14:31.142190+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:22:24.520772+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00086129200000000005,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6781,"num_read_kb":42484,"num_write":652,"num_write_kb":25730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":26193256,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142568+0000","last_change":"2024-09-15T00:14:16.040721+0000","last_active":"2024-09-15T00:14:31.142568+0000","last_peered":"2024-09-15T00:14:31.142568+0000","last_clean":"2024-09-15T00:14:31.142568+0000","last_became_active":"2024-09-15T00:14:16.040306+0000","last_became_peered":"2024-09-15T00:14:16.040306+0000","last_unstale":"2024-09-15T00:14:31.142568+0000","last_undegraded":"2024-09-15T00:14:31.142568+0000","last_fullsized":"2024-09-15T00:14:31.142568+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:54:01.189526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'731","reported_seq":9397,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763223+0000","last_change":"2024-09-15T00:14:08.081190+0000","last_active":"2024-09-15T00:14:31.763223+0000","last_peered":"2024-09-15T00:14:31.763223+0000","last_clean":"2024-09-15T00:14:31.763223+0000","last_became_active":"2024-09-15T00:13:49.218857+0000","last_became_peered":"2024-09-15T00:13:49.218857+0000","last_unstale":"2024-09-15T00:14:31.763223+0000","last_undegraded":"2024-09-15T00:14:31.763223+0000","last_fullsized":"2024-09-15T00:14:31.763223+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:04:29.276892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020637310000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":10066,"num_read_kb":57181,"num_write":720,"num_write_kb":17747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":15784649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":110,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502690+0000","last_change":"2024-09-15T00:14:15.038333+0000","last_active":"2024-09-15T00:14:31.502690+0000","last_peered":"2024-09-15T00:14:31.502690+0000","last_clean":"2024-09-15T00:14:31.502690+0000","last_became_active":"2024-09-15T00:14:15.037987+0000","last_became_peered":"2024-09-15T00:14:15.037987+0000","last_unstale":"2024-09-15T00:14:31.502690+0000","last_undegraded":"2024-09-15T00:14:31.502690+0000","last_fullsized":"2024-09-15T00:14:31.502690+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:27.770396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"177'774","reported_seq":12041,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142529+0000","last_change":"2024-09-15T00:14:20.036384+0000","last_active":"2024-09-15T00:14:31.142529+0000","last_peered":"2024-09-15T00:14:31.142529+0000","last_clean":"2024-09-15T00:14:31.142529+0000","last_became_active":"2024-09-15T00:14:20.033228+0000","last_became_peered":"2024-09-15T00:14:20.033228+0000","last_unstale":"2024-09-15T00:14:31.142529+0000","last_undegraded":"2024-09-15T00:14:31.142529+0000","last_fullsized":"2024-09-15T00:14:31.142529+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:36:37.663215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017650229999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12816,"num_read_kb":60614,"num_write":771,"num_write_kb":25261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1083,"num_bytes_recovered":41768720,"num_keys_recovered":205,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142653+0000","last_change":"2024-09-15T00:14:16.034807+0000","last_active":"2024-09-15T00:14:31.142653+0000","last_peered":"2024-09-15T00:14:31.142653+0000","last_clean":"2024-09-15T00:14:31.142653+0000","last_became_active":"2024-09-15T00:14:16.034489+0000","last_became_peered":"2024-09-15T00:14:16.034489+0000","last_unstale":"2024-09-15T00:14:31.142653+0000","last_undegraded":"2024-09-15T00:14:31.142653+0000","last_fullsized":"2024-09-15T00:14:31.142653+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:38:39.946046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"179'552","reported_seq":3247,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762929+0000","last_change":"2024-09-15T00:14:08.081714+0000","last_active":"2024-09-15T00:14:31.762929+0000","last_peered":"2024-09-15T00:14:31.762929+0000","last_clean":"2024-09-15T00:14:31.762929+0000","last_became_active":"2024-09-15T00:11:19.595633+0000","last_became_peered":"2024-09-15T00:11:19.595633+0000","last_unstale":"2024-09-15T00:14:31.762929+0000","last_undegraded":"2024-09-15T00:14:31.762929+0000","last_fullsized":"2024-09-15T00:14:31.762929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:24:30.596801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026087110000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3741,"num_read_kb":17273,"num_write":546,"num_write_kb":8984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"182'819","reported_seq":8159,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142046+0000","last_change":"2024-09-15T00:14:24.096290+0000","last_active":"2024-09-15T00:14:31.142046+0000","last_peered":"2024-09-15T00:14:31.142046+0000","last_clean":"2024-09-15T00:14:31.142046+0000","last_became_active":"2024-09-15T00:14:24.094819+0000","last_became_peered":"2024-09-15T00:14:24.094819+0000","last_unstale":"2024-09-15T00:14:31.142046+0000","last_undegraded":"2024-09-15T00:14:31.142046+0000","last_fullsized":"2024-09-15T00:14:31.142046+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:32:14.972007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090289999999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":7230,"num_read_kb":43219,"num_write":761,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":9273484,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":420,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762759+0000","last_change":"2024-09-15T00:14:16.050758+0000","last_active":"2024-09-15T00:14:31.762759+0000","last_peered":"2024-09-15T00:14:31.762759+0000","last_clean":"2024-09-15T00:14:31.762759+0000","last_became_active":"2024-09-15T00:14:16.046906+0000","last_became_peered":"2024-09-15T00:14:16.046906+0000","last_unstale":"2024-09-15T00:14:31.762759+0000","last_undegraded":"2024-09-15T00:14:31.762759+0000","last_fullsized":"2024-09-15T00:14:31.762759+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:49:39.277929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"179'696","reported_seq":7804,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142817+0000","last_change":"2024-09-15T00:14:19.050801+0000","last_active":"2024-09-15T00:14:31.142817+0000","last_peered":"2024-09-15T00:14:31.142817+0000","last_clean":"2024-09-15T00:14:31.142817+0000","last_became_active":"2024-09-15T00:14:19.048768+0000","last_became_peered":"2024-09-15T00:14:19.048768+0000","last_unstale":"2024-09-15T00:14:31.142817+0000","last_undegraded":"2024-09-15T00:14:31.142817+0000","last_fullsized":"2024-09-15T00:14:31.142817+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:42:10.166497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092066200000000004,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7368,"num_read_kb":33386,"num_write":726,"num_write_kb":14440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":31813799,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762886+0000","last_change":"2024-09-15T00:13:52.026234+0000","last_active":"2024-09-15T00:14:31.762886+0000","last_peered":"2024-09-15T00:14:31.762886+0000","last_clean":"2024-09-15T00:14:31.762886+0000","last_became_active":"2024-09-15T00:13:52.025864+0000","last_became_peered":"2024-09-15T00:13:52.025864+0000","last_unstale":"2024-09-15T00:14:31.762886+0000","last_undegraded":"2024-09-15T00:14:31.762886+0000","last_fullsized":"2024-09-15T00:14:31.762886+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:23:52.397810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"182'708","reported_seq":6729,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142024+0000","last_change":"2024-09-15T00:14:21.036152+0000","last_active":"2024-09-15T00:14:31.142024+0000","last_peered":"2024-09-15T00:14:31.142024+0000","last_clean":"2024-09-15T00:14:31.142024+0000","last_became_active":"2024-09-15T00:14:21.034477+0000","last_became_peered":"2024-09-15T00:14:21.034477+0000","last_unstale":"2024-09-15T00:14:31.142024+0000","last_undegraded":"2024-09-15T00:14:31.142024+0000","last_fullsized":"2024-09-15T00:14:31.142024+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:50.622032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075419699999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6724,"num_read_kb":49050,"num_write":759,"num_write_kb":22019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":6058433,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":114,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502620+0000","last_change":"2024-09-15T00:14:16.041987+0000","last_active":"2024-09-15T00:14:31.502620+0000","last_peered":"2024-09-15T00:14:31.502620+0000","last_clean":"2024-09-15T00:14:31.502620+0000","last_became_active":"2024-09-15T00:14:16.041672+0000","last_became_peered":"2024-09-15T00:14:16.041672+0000","last_unstale":"2024-09-15T00:14:31.502620+0000","last_undegraded":"2024-09-15T00:14:31.502620+0000","last_fullsized":"2024-09-15T00:14:31.502620+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:04:25.406297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"179'619","reported_seq":5394,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763692+0000","last_change":"2024-09-15T00:14:08.080584+0000","last_active":"2024-09-15T00:14:31.763692+0000","last_peered":"2024-09-15T00:14:31.763692+0000","last_clean":"2024-09-15T00:14:31.763692+0000","last_became_active":"2024-09-15T00:11:19.594528+0000","last_became_peered":"2024-09-15T00:11:19.594528+0000","last_unstale":"2024-09-15T00:14:31.763692+0000","last_undegraded":"2024-09-15T00:14:31.763692+0000","last_fullsized":"2024-09-15T00:14:31.763692+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:56:44.230264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001727704,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5349,"num_read_kb":42137,"num_write":591,"num_write_kb":17763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":437,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762777+0000","last_change":"2024-09-15T00:13:52.028546+0000","last_active":"2024-09-15T00:14:31.762777+0000","last_peered":"2024-09-15T00:14:31.762777+0000","last_clean":"2024-09-15T00:14:31.762777+0000","last_became_active":"2024-09-15T00:13:52.026457+0000","last_became_peered":"2024-09-15T00:13:52.026457+0000","last_unstale":"2024-09-15T00:14:31.762777+0000","last_undegraded":"2024-09-15T00:14:31.762777+0000","last_fullsized":"2024-09-15T00:14:31.762777+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:58:54.983326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":348,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502648+0000","last_change":"2024-09-15T00:14:18.046986+0000","last_active":"2024-09-15T00:14:31.502648+0000","last_peered":"2024-09-15T00:14:31.502648+0000","last_clean":"2024-09-15T00:14:31.502648+0000","last_became_active":"2024-09-15T00:14:18.046593+0000","last_became_peered":"2024-09-15T00:14:18.046593+0000","last_unstale":"2024-09-15T00:14:31.502648+0000","last_undegraded":"2024-09-15T00:14:31.502648+0000","last_fullsized":"2024-09-15T00:14:31.502648+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_clean_scrub_stamp":"2024-09-15T00:11:04.453672+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:59:10.044950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"179'844","reported_seq":8284,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.764021+0000","last_change":"2024-09-15T00:14:22.040149+0000","last_active":"2024-09-15T00:14:31.764021+0000","last_peered":"2024-09-15T00:14:31.764021+0000","last_clean":"2024-09-15T00:14:31.764021+0000","last_became_active":"2024-09-15T00:14:22.038746+0000","last_became_peered":"2024-09-15T00:14:22.038746+0000","last_unstale":"2024-09-15T00:14:31.764021+0000","last_undegraded":"2024-09-15T00:14:31.764021+0000","last_fullsized":"2024-09-15T00:14:31.764021+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:36:14.331079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012708680000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8015,"num_read_kb":57970,"num_write":928,"num_write_kb":24350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":14074270,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"179'1526","reported_seq":9116,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763808+0000","last_change":"2024-09-15T00:14:19.051081+0000","last_active":"2024-09-15T00:14:31.763808+0000","last_peered":"2024-09-15T00:14:31.763808+0000","last_clean":"2024-09-15T00:14:31.763808+0000","last_became_active":"2024-09-15T00:14:19.048942+0000","last_became_peered":"2024-09-15T00:14:19.048942+0000","last_unstale":"2024-09-15T00:14:31.763808+0000","last_undegraded":"2024-09-15T00:14:31.763808+0000","last_fullsized":"2024-09-15T00:14:31.763808+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1526,"log_dups_size":0,"ondisk_log_size":1526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:54:02.659520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061724499999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":14886,"num_read_kb":35520,"num_write":2067,"num_write_kb":12930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":409,"num_bytes_recovered":10512832,"num_keys_recovered":83,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"179'733","reported_seq":6627,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470085+0000","last_change":"2024-09-15T00:14:08.082753+0000","last_active":"2024-09-15T00:14:31.470085+0000","last_peered":"2024-09-15T00:14:31.470085+0000","last_clean":"2024-09-15T00:14:31.470085+0000","last_became_active":"2024-09-15T00:11:19.602625+0000","last_became_peered":"2024-09-15T00:11:19.602625+0000","last_unstale":"2024-09-15T00:14:31.470085+0000","last_undegraded":"2024-09-15T00:14:31.470085+0000","last_fullsized":"2024-09-15T00:14:31.470085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:24:41.939171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012377130000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6965,"num_read_kb":22637,"num_write":697,"num_write_kb":11622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"177'426","reported_seq":5524,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763453+0000","last_change":"2024-09-15T00:14:20.037949+0000","last_active":"2024-09-15T00:14:31.763453+0000","last_peered":"2024-09-15T00:14:31.763453+0000","last_clean":"2024-09-15T00:14:31.763453+0000","last_became_active":"2024-09-15T00:14:20.036368+0000","last_became_peered":"2024-09-15T00:14:20.036368+0000","last_unstale":"2024-09-15T00:14:31.763453+0000","last_undegraded":"2024-09-15T00:14:31.763453+0000","last_fullsized":"2024-09-15T00:14:31.763453+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:10:24.358861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087100100000000005,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3886,"num_read_kb":33280,"num_write":476,"num_write_kb":22691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":10092575,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'919","reported_seq":4934,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762892+0000","last_change":"2024-09-15T00:14:08.081241+0000","last_active":"2024-09-15T00:14:31.762892+0000","last_peered":"2024-09-15T00:14:31.762892+0000","last_clean":"2024-09-15T00:14:31.762892+0000","last_became_active":"2024-09-15T00:11:19.595088+0000","last_became_peered":"2024-09-15T00:11:19.595088+0000","last_unstale":"2024-09-15T00:14:31.762892+0000","last_undegraded":"2024-09-15T00:14:31.762892+0000","last_fullsized":"2024-09-15T00:14:31.762892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:47:04.219653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024006000000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7681,"num_read_kb":43201,"num_write":1093,"num_write_kb":15373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"177'985","reported_seq":6046,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502672+0000","last_change":"2024-09-15T00:14:08.088640+0000","last_active":"2024-09-15T00:14:31.502672+0000","last_peered":"2024-09-15T00:14:31.502672+0000","last_clean":"2024-09-15T00:14:31.502672+0000","last_became_active":"2024-09-15T00:13:51.028205+0000","last_became_peered":"2024-09-15T00:13:51.028205+0000","last_unstale":"2024-09-15T00:14:31.502672+0000","last_undegraded":"2024-09-15T00:14:31.502672+0000","last_fullsized":"2024-09-15T00:14:31.502672+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:13:15.922674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012164389999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7667,"num_read_kb":20584,"num_write":1269,"num_write_kb":5933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":1861870,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"168'1421","reported_seq":6068,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502758+0000","last_change":"2024-09-15T00:14:08.088029+0000","last_active":"2024-09-15T00:14:31.502758+0000","last_peered":"2024-09-15T00:14:31.502758+0000","last_clean":"2024-09-15T00:14:31.502758+0000","last_became_active":"2024-09-15T00:13:50.023348+0000","last_became_peered":"2024-09-15T00:13:50.023348+0000","last_unstale":"2024-09-15T00:14:31.502758+0000","last_undegraded":"2024-09-15T00:14:31.502758+0000","last_fullsized":"2024-09-15T00:14:31.502758+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1421,"log_dups_size":0,"ondisk_log_size":1421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:46:55.906750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001473739,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8919,"num_read_kb":45201,"num_write":2133,"num_write_kb":24448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":18482007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"176'516","reported_seq":6414,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502850+0000","last_change":"2024-09-15T00:14:08.086840+0000","last_active":"2024-09-15T00:14:31.502850+0000","last_peered":"2024-09-15T00:14:31.502850+0000","last_clean":"2024-09-15T00:14:31.502850+0000","last_became_active":"2024-09-15T00:13:49.218512+0000","last_became_peered":"2024-09-15T00:13:49.218512+0000","last_unstale":"2024-09-15T00:14:31.502850+0000","last_undegraded":"2024-09-15T00:14:31.502850+0000","last_fullsized":"2024-09-15T00:14:31.502850+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:27:37.063625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014419529999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5794,"num_read_kb":31109,"num_write":499,"num_write_kb":15000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5158884,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"182'731","reported_seq":7089,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470085+0000","last_change":"2024-09-15T00:14:19.055694+0000","last_active":"2024-09-15T00:14:31.470085+0000","last_peered":"2024-09-15T00:14:31.470085+0000","last_clean":"2024-09-15T00:14:31.470085+0000","last_became_active":"2024-09-15T00:14:19.054103+0000","last_became_peered":"2024-09-15T00:14:19.054103+0000","last_unstale":"2024-09-15T00:14:31.470085+0000","last_undegraded":"2024-09-15T00:14:31.470085+0000","last_fullsized":"2024-09-15T00:14:31.470085+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:41:29.549456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086791500000000003,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7073,"num_read_kb":31603,"num_write":836,"num_write_kb":14934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":8339863,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"176'596","reported_seq":4626,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502629+0000","last_change":"2024-09-15T00:14:08.086089+0000","last_active":"2024-09-15T00:14:31.502629+0000","last_peered":"2024-09-15T00:14:31.502629+0000","last_clean":"2024-09-15T00:14:31.502629+0000","last_became_active":"2024-09-15T00:13:49.218581+0000","last_became_peered":"2024-09-15T00:13:49.218581+0000","last_unstale":"2024-09-15T00:14:31.502629+0000","last_undegraded":"2024-09-15T00:14:31.502629+0000","last_fullsized":"2024-09-15T00:14:31.502629+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:17.388753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010536860000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4205,"num_read_kb":15404,"num_write":601,"num_write_kb":11097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":1517056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"173'674","reported_seq":7804,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.141963+0000","last_change":"2024-09-15T00:14:21.034205+0000","last_active":"2024-09-15T00:14:31.141963+0000","last_peered":"2024-09-15T00:14:31.141963+0000","last_clean":"2024-09-15T00:14:31.141963+0000","last_became_active":"2024-09-15T00:14:21.032233+0000","last_became_peered":"2024-09-15T00:14:21.032233+0000","last_unstale":"2024-09-15T00:14:31.141963+0000","last_undegraded":"2024-09-15T00:14:31.141963+0000","last_fullsized":"2024-09-15T00:14:31.141963+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:46:53.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089427500000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6790,"num_read_kb":50206,"num_write":784,"num_write_kb":25712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":5797342,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'1107","reported_seq":8225,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502722+0000","last_change":"2024-09-15T00:14:08.087888+0000","last_active":"2024-09-15T00:14:31.502722+0000","last_peered":"2024-09-15T00:14:31.502722+0000","last_clean":"2024-09-15T00:14:31.502722+0000","last_became_active":"2024-09-15T00:13:51.025771+0000","last_became_peered":"2024-09-15T00:13:51.025771+0000","last_unstale":"2024-09-15T00:14:31.502722+0000","last_undegraded":"2024-09-15T00:14:31.502722+0000","last_fullsized":"2024-09-15T00:14:31.502722+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:28:00.228817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001944035,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":10923,"num_read_kb":34402,"num_write":1383,"num_write_kb":19662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":27769124,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":521,"num_object_clones":0,"num_object_copies":1042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":521,"num_whiteouts":517,"num_read":240279,"num_read_kb":1281261,"num_write":28650,"num_write_kb":585223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10651,"num_bytes_recovered":390333624,"num_keys_recovered":726,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5367,"internal_metadata":0},"log_size":25207,"ondisk_log_size":25207,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":2361472,"data_compressed":25472,"data_compressed_allocated":147456,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87736,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284104,"statfs":{"total":96636764160,"available":96546922496,"internally_reserved":0,"allocated":2453504,"data_stored":1194007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51620,"kb_used_data":2380,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320220,"statfs":{"total":96636764160,"available":96583905280,"internally_reserved":0,"allocated":2437120,"data_stored":1188146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55836,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316004,"statfs":{"total":96636764160,"available":96579588096,"internally_reserved":0,"allocated":2490368,"data_stored":1784356,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542191,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77996,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293844,"statfs":{"total":96636764160,"available":96556896256,"internally_reserved":0,"allocated":2449408,"data_stored":1193988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47476,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324364,"statfs":{"total":96636764160,"available":96588148736,"internally_reserved":0,"allocated":2453504,"data_stored":1194007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":108,"seq":463856467990,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42588,"kb_used_data":2368,"kb_used_omap":6,"kb_used_meta":40185,"kb_avail":94329252,"statfs":{"total":96636764160,"available":96593154048,"internally_reserved":0,"allocated":2424832,"data_stored":1182304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6962,"internal_metadata":41149646},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55784,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316056,"statfs":{"total":96636764160,"available":96579641344,"internally_reserved":0,"allocated":2433024,"data_stored":1182342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78048,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293792,"statfs":{"total":96636764160,"available":96556843008,"internally_reserved":0,"allocated":2490368,"data_stored":1784356,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5367,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:14:34.619 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:14:34.892 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:14:34.893 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:14:34.905 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-15T00:14:34.327513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":525,"num_object_clones":0,"num_object_copies":1050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":525,"num_whiteouts":517,"num_read":240395,"num_read_kb":1281361,"num_write":28811,"num_write_kb":587641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10669,"num_bytes_recovered":392695172,"num_keys_recovered":726,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25327,"ondisk_log_size":25327,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":497084,"kb_used_data":19172,"kb_used_omap":17,"kb_used_meta":477678,"kb_avail":754477636,"statfs":{"total":773094113280,"available":772585099264,"internally_reserved":0,"allocated":19632128,"data_stored":10703506,"data_compressed":12736,"data_compressed_allocated":73728,"data_compressed_original":1179648,"omap_allocated":18117,"internal_metadata":489142587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001051"},"pg_stats":[{"pgid":"3.1f","version":"182'643","reported_seq":6819,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142321+0000","last_change":"2024-09-15T00:14:18.045712+0000","last_active":"2024-09-15T00:14:31.142321+0000","last_peered":"2024-09-15T00:14:31.142321+0000","last_clean":"2024-09-15T00:14:31.142321+0000","last_became_active":"2024-09-15T00:14:18.042449+0000","last_became_peered":"2024-09-15T00:14:18.042449+0000","last_unstale":"2024-09-15T00:14:31.142321+0000","last_undegraded":"2024-09-15T00:14:31.142321+0000","last_fullsized":"2024-09-15T00:14:31.142321+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:38:13.616525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011752869999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6942,"num_read_kb":26340,"num_write":697,"num_write_kb":18376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":5475412,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'632","reported_seq":7633,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763549+0000","last_change":"2024-09-15T00:14:08.080286+0000","last_active":"2024-09-15T00:14:31.763549+0000","last_peered":"2024-09-15T00:14:31.763549+0000","last_clean":"2024-09-15T00:14:31.763549+0000","last_became_active":"2024-09-15T00:11:19.598640+0000","last_became_peered":"2024-09-15T00:11:19.598640+0000","last_unstale":"2024-09-15T00:14:31.763549+0000","last_undegraded":"2024-09-15T00:14:31.763549+0000","last_fullsized":"2024-09-15T00:14:31.763549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:16:36.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016674789999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7565,"num_read_kb":39401,"num_write":737,"num_write_kb":10312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'698","reported_seq":4933,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762732+0000","last_change":"2024-09-15T00:14:08.080849+0000","last_active":"2024-09-15T00:14:31.762732+0000","last_peered":"2024-09-15T00:14:31.762732+0000","last_clean":"2024-09-15T00:14:31.762732+0000","last_became_active":"2024-09-15T00:11:19.595954+0000","last_became_peered":"2024-09-15T00:11:19.595954+0000","last_unstale":"2024-09-15T00:14:31.762732+0000","last_undegraded":"2024-09-15T00:14:31.762732+0000","last_fullsized":"2024-09-15T00:14:31.762732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:28.144497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020206820000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5583,"num_read_kb":45090,"num_write":773,"num_write_kb":31484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"182'674","reported_seq":5880,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762701+0000","last_change":"2024-09-15T00:14:08.080354+0000","last_active":"2024-09-15T00:14:31.762701+0000","last_peered":"2024-09-15T00:14:31.762701+0000","last_clean":"2024-09-15T00:14:31.762701+0000","last_became_active":"2024-09-15T00:11:19.602583+0000","last_became_peered":"2024-09-15T00:11:19.602583+0000","last_unstale":"2024-09-15T00:14:31.762701+0000","last_undegraded":"2024-09-15T00:14:31.762701+0000","last_fullsized":"2024-09-15T00:14:31.762701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:33:11.479982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00168294,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":5888,"num_read_kb":35219,"num_write":807,"num_write_kb":10221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'844","reported_seq":6971,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142108+0000","last_change":"2024-09-15T00:14:20.035839+0000","last_active":"2024-09-15T00:14:31.142108+0000","last_peered":"2024-09-15T00:14:31.142108+0000","last_clean":"2024-09-15T00:14:31.142108+0000","last_became_active":"2024-09-15T00:14:20.032662+0000","last_became_peered":"2024-09-15T00:14:20.032662+0000","last_unstale":"2024-09-15T00:14:31.142108+0000","last_undegraded":"2024-09-15T00:14:31.142108+0000","last_fullsized":"2024-09-15T00:14:31.142108+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:24:20.329228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001922899,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7801,"num_read_kb":34537,"num_write":909,"num_write_kb":14651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":10954706,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"179'783","reported_seq":7558,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.141955+0000","last_change":"2024-09-15T00:14:24.098641+0000","last_active":"2024-09-15T00:14:31.141955+0000","last_peered":"2024-09-15T00:14:31.141955+0000","last_clean":"2024-09-15T00:14:31.141955+0000","last_became_active":"2024-09-15T00:14:24.097321+0000","last_became_peered":"2024-09-15T00:14:24.097321+0000","last_unstale":"2024-09-15T00:14:31.141955+0000","last_undegraded":"2024-09-15T00:14:31.141955+0000","last_fullsized":"2024-09-15T00:14:31.141955+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:40:15.897865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012814980000000001,"stat_sum":{"num_bytes":19,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":8900,"num_read_kb":62621,"num_write":837,"num_write_kb":33377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":66497974,"num_keys_recovered":149,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"177'678","reported_seq":5626,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142375+0000","last_change":"2024-09-15T00:14:22.038154+0000","last_active":"2024-09-15T00:14:31.142375+0000","last_peered":"2024-09-15T00:14:31.142375+0000","last_clean":"2024-09-15T00:14:31.142375+0000","last_became_active":"2024-09-15T00:14:22.036152+0000","last_became_peered":"2024-09-15T00:14:22.036152+0000","last_unstale":"2024-09-15T00:14:31.142375+0000","last_undegraded":"2024-09-15T00:14:31.142375+0000","last_fullsized":"2024-09-15T00:14:31.142375+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:30:00.495418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005256073,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4711,"num_read_kb":42462,"num_write":705,"num_write_kb":9831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":558,"num_bytes_recovered":6445817,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"179'706","reported_seq":8764,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.764017+0000","last_change":"2024-09-15T00:14:18.044374+0000","last_active":"2024-09-15T00:14:31.764017+0000","last_peered":"2024-09-15T00:14:31.764017+0000","last_clean":"2024-09-15T00:14:31.764017+0000","last_became_active":"2024-09-15T00:14:18.042710+0000","last_became_peered":"2024-09-15T00:14:18.042710+0000","last_unstale":"2024-09-15T00:14:31.764017+0000","last_undegraded":"2024-09-15T00:14:31.764017+0000","last_fullsized":"2024-09-15T00:14:31.764017+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:52:32.313688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014749780000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8102,"num_read_kb":40115,"num_write":776,"num_write_kb":15981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":4986328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'679","reported_seq":5549,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142315+0000","last_change":"2024-09-15T00:14:20.037042+0000","last_active":"2024-09-15T00:14:31.142315+0000","last_peered":"2024-09-15T00:14:31.142315+0000","last_clean":"2024-09-15T00:14:31.142315+0000","last_became_active":"2024-09-15T00:14:20.034657+0000","last_became_peered":"2024-09-15T00:14:20.034657+0000","last_unstale":"2024-09-15T00:14:31.142315+0000","last_undegraded":"2024-09-15T00:14:31.142315+0000","last_fullsized":"2024-09-15T00:14:31.142315+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:52:39.383067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002708182,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5548,"num_read_kb":45663,"num_write":634,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":12460957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"176'1032","reported_seq":8430,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762889+0000","last_change":"2024-09-15T00:14:08.079912+0000","last_active":"2024-09-15T00:14:31.762889+0000","last_peered":"2024-09-15T00:14:31.762889+0000","last_clean":"2024-09-15T00:14:31.762889+0000","last_became_active":"2024-09-15T00:13:50.031768+0000","last_became_peered":"2024-09-15T00:13:50.031768+0000","last_unstale":"2024-09-15T00:14:31.762889+0000","last_undegraded":"2024-09-15T00:14:31.762889+0000","last_fullsized":"2024-09-15T00:14:31.762889+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1032,"log_dups_size":0,"ondisk_log_size":1032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:21.539927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001158467,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8890,"num_read_kb":39257,"num_write":1264,"num_write_kb":20613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":23803494,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"179'667","reported_seq":6954,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502804+0000","last_change":"2024-09-15T00:14:08.088641+0000","last_active":"2024-09-15T00:14:31.502804+0000","last_peered":"2024-09-15T00:14:31.502804+0000","last_clean":"2024-09-15T00:14:31.502804+0000","last_became_active":"2024-09-15T00:13:50.025342+0000","last_became_peered":"2024-09-15T00:13:50.025342+0000","last_unstale":"2024-09-15T00:14:31.502804+0000","last_undegraded":"2024-09-15T00:14:31.502804+0000","last_fullsized":"2024-09-15T00:14:31.502804+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:50:45.634693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071033100000000003,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7596,"num_read_kb":45634,"num_write":737,"num_write_kb":14836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":19127384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'1079","reported_seq":7470,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142064+0000","last_change":"2024-09-15T00:14:24.095829+0000","last_active":"2024-09-15T00:14:31.142064+0000","last_peered":"2024-09-15T00:14:31.142064+0000","last_clean":"2024-09-15T00:14:31.142064+0000","last_became_active":"2024-09-15T00:14:24.093937+0000","last_became_peered":"2024-09-15T00:14:24.093937+0000","last_unstale":"2024-09-15T00:14:31.142064+0000","last_undegraded":"2024-09-15T00:14:31.142064+0000","last_fullsized":"2024-09-15T00:14:31.142064+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1079,"log_dups_size":0,"ondisk_log_size":1079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:10:03.817976+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.016604621,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":9874,"num_read_kb":58461,"num_write":1482,"num_write_kb":25008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":6083388,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":403,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470173+0000","last_change":"2024-09-15T00:13:46.030592+0000","last_active":"2024-09-15T00:14:31.470173+0000","last_peered":"2024-09-15T00:14:31.470173+0000","last_clean":"2024-09-15T00:14:31.470173+0000","last_became_active":"2024-09-15T00:13:46.030230+0000","last_became_peered":"2024-09-15T00:13:46.030230+0000","last_unstale":"2024-09-15T00:14:31.470173+0000","last_undegraded":"2024-09-15T00:14:31.470173+0000","last_fullsized":"2024-09-15T00:14:31.470173+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:26:48.293426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'715","reported_seq":7972,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142190+0000","last_change":"2024-09-15T00:14:18.046157+0000","last_active":"2024-09-15T00:14:31.142190+0000","last_peered":"2024-09-15T00:14:31.142190+0000","last_clean":"2024-09-15T00:14:31.142190+0000","last_became_active":"2024-09-15T00:14:18.043887+0000","last_became_peered":"2024-09-15T00:14:18.043887+0000","last_unstale":"2024-09-15T00:14:31.142190+0000","last_undegraded":"2024-09-15T00:14:31.142190+0000","last_fullsized":"2024-09-15T00:14:31.142190+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:22:24.520772+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00086129200000000005,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6781,"num_read_kb":42484,"num_write":652,"num_write_kb":25730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":26193256,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142568+0000","last_change":"2024-09-15T00:14:16.040721+0000","last_active":"2024-09-15T00:14:31.142568+0000","last_peered":"2024-09-15T00:14:31.142568+0000","last_clean":"2024-09-15T00:14:31.142568+0000","last_became_active":"2024-09-15T00:14:16.040306+0000","last_became_peered":"2024-09-15T00:14:16.040306+0000","last_unstale":"2024-09-15T00:14:31.142568+0000","last_undegraded":"2024-09-15T00:14:31.142568+0000","last_fullsized":"2024-09-15T00:14:31.142568+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:54:01.189526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'731","reported_seq":9397,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763223+0000","last_change":"2024-09-15T00:14:08.081190+0000","last_active":"2024-09-15T00:14:31.763223+0000","last_peered":"2024-09-15T00:14:31.763223+0000","last_clean":"2024-09-15T00:14:31.763223+0000","last_became_active":"2024-09-15T00:13:49.218857+0000","last_became_peered":"2024-09-15T00:13:49.218857+0000","last_unstale":"2024-09-15T00:14:31.763223+0000","last_undegraded":"2024-09-15T00:14:31.763223+0000","last_fullsized":"2024-09-15T00:14:31.763223+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:04:29.276892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020637310000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":10066,"num_read_kb":57181,"num_write":720,"num_write_kb":17747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":15784649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":110,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502690+0000","last_change":"2024-09-15T00:14:15.038333+0000","last_active":"2024-09-15T00:14:31.502690+0000","last_peered":"2024-09-15T00:14:31.502690+0000","last_clean":"2024-09-15T00:14:31.502690+0000","last_became_active":"2024-09-15T00:14:15.037987+0000","last_became_peered":"2024-09-15T00:14:15.037987+0000","last_unstale":"2024-09-15T00:14:31.502690+0000","last_undegraded":"2024-09-15T00:14:31.502690+0000","last_fullsized":"2024-09-15T00:14:31.502690+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:27.770396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"177'774","reported_seq":12041,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142529+0000","last_change":"2024-09-15T00:14:20.036384+0000","last_active":"2024-09-15T00:14:31.142529+0000","last_peered":"2024-09-15T00:14:31.142529+0000","last_clean":"2024-09-15T00:14:31.142529+0000","last_became_active":"2024-09-15T00:14:20.033228+0000","last_became_peered":"2024-09-15T00:14:20.033228+0000","last_unstale":"2024-09-15T00:14:31.142529+0000","last_undegraded":"2024-09-15T00:14:31.142529+0000","last_fullsized":"2024-09-15T00:14:31.142529+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:36:37.663215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017650229999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12816,"num_read_kb":60614,"num_write":771,"num_write_kb":25261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1083,"num_bytes_recovered":41768720,"num_keys_recovered":205,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142653+0000","last_change":"2024-09-15T00:14:16.034807+0000","last_active":"2024-09-15T00:14:31.142653+0000","last_peered":"2024-09-15T00:14:31.142653+0000","last_clean":"2024-09-15T00:14:31.142653+0000","last_became_active":"2024-09-15T00:14:16.034489+0000","last_became_peered":"2024-09-15T00:14:16.034489+0000","last_unstale":"2024-09-15T00:14:31.142653+0000","last_undegraded":"2024-09-15T00:14:31.142653+0000","last_fullsized":"2024-09-15T00:14:31.142653+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:38:39.946046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"179'552","reported_seq":3247,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762929+0000","last_change":"2024-09-15T00:14:08.081714+0000","last_active":"2024-09-15T00:14:31.762929+0000","last_peered":"2024-09-15T00:14:31.762929+0000","last_clean":"2024-09-15T00:14:31.762929+0000","last_became_active":"2024-09-15T00:11:19.595633+0000","last_became_peered":"2024-09-15T00:11:19.595633+0000","last_unstale":"2024-09-15T00:14:31.762929+0000","last_undegraded":"2024-09-15T00:14:31.762929+0000","last_fullsized":"2024-09-15T00:14:31.762929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:24:30.596801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026087110000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3741,"num_read_kb":17273,"num_write":546,"num_write_kb":8984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"182'819","reported_seq":8159,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142046+0000","last_change":"2024-09-15T00:14:24.096290+0000","last_active":"2024-09-15T00:14:31.142046+0000","last_peered":"2024-09-15T00:14:31.142046+0000","last_clean":"2024-09-15T00:14:31.142046+0000","last_became_active":"2024-09-15T00:14:24.094819+0000","last_became_peered":"2024-09-15T00:14:24.094819+0000","last_unstale":"2024-09-15T00:14:31.142046+0000","last_undegraded":"2024-09-15T00:14:31.142046+0000","last_fullsized":"2024-09-15T00:14:31.142046+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:32:14.972007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090289999999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":7230,"num_read_kb":43219,"num_write":761,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":9273484,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":420,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762759+0000","last_change":"2024-09-15T00:14:16.050758+0000","last_active":"2024-09-15T00:14:31.762759+0000","last_peered":"2024-09-15T00:14:31.762759+0000","last_clean":"2024-09-15T00:14:31.762759+0000","last_became_active":"2024-09-15T00:14:16.046906+0000","last_became_peered":"2024-09-15T00:14:16.046906+0000","last_unstale":"2024-09-15T00:14:31.762759+0000","last_undegraded":"2024-09-15T00:14:31.762759+0000","last_fullsized":"2024-09-15T00:14:31.762759+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:49:39.277929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"179'696","reported_seq":7804,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142817+0000","last_change":"2024-09-15T00:14:19.050801+0000","last_active":"2024-09-15T00:14:31.142817+0000","last_peered":"2024-09-15T00:14:31.142817+0000","last_clean":"2024-09-15T00:14:31.142817+0000","last_became_active":"2024-09-15T00:14:19.048768+0000","last_became_peered":"2024-09-15T00:14:19.048768+0000","last_unstale":"2024-09-15T00:14:31.142817+0000","last_undegraded":"2024-09-15T00:14:31.142817+0000","last_fullsized":"2024-09-15T00:14:31.142817+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:42:10.166497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092066200000000004,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7368,"num_read_kb":33386,"num_write":726,"num_write_kb":14440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":31813799,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762886+0000","last_change":"2024-09-15T00:13:52.026234+0000","last_active":"2024-09-15T00:14:31.762886+0000","last_peered":"2024-09-15T00:14:31.762886+0000","last_clean":"2024-09-15T00:14:31.762886+0000","last_became_active":"2024-09-15T00:13:52.025864+0000","last_became_peered":"2024-09-15T00:13:52.025864+0000","last_unstale":"2024-09-15T00:14:31.762886+0000","last_undegraded":"2024-09-15T00:14:31.762886+0000","last_fullsized":"2024-09-15T00:14:31.762886+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:23:52.397810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"182'708","reported_seq":6729,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142024+0000","last_change":"2024-09-15T00:14:21.036152+0000","last_active":"2024-09-15T00:14:31.142024+0000","last_peered":"2024-09-15T00:14:31.142024+0000","last_clean":"2024-09-15T00:14:31.142024+0000","last_became_active":"2024-09-15T00:14:21.034477+0000","last_became_peered":"2024-09-15T00:14:21.034477+0000","last_unstale":"2024-09-15T00:14:31.142024+0000","last_undegraded":"2024-09-15T00:14:31.142024+0000","last_fullsized":"2024-09-15T00:14:31.142024+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:50.622032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075419699999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6724,"num_read_kb":49050,"num_write":759,"num_write_kb":22019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":6058433,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":114,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502620+0000","last_change":"2024-09-15T00:14:16.041987+0000","last_active":"2024-09-15T00:14:31.502620+0000","last_peered":"2024-09-15T00:14:31.502620+0000","last_clean":"2024-09-15T00:14:31.502620+0000","last_became_active":"2024-09-15T00:14:16.041672+0000","last_became_peered":"2024-09-15T00:14:16.041672+0000","last_unstale":"2024-09-15T00:14:31.502620+0000","last_undegraded":"2024-09-15T00:14:31.502620+0000","last_fullsized":"2024-09-15T00:14:31.502620+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:04:25.406297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"179'619","reported_seq":5394,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763692+0000","last_change":"2024-09-15T00:14:08.080584+0000","last_active":"2024-09-15T00:14:31.763692+0000","last_peered":"2024-09-15T00:14:31.763692+0000","last_clean":"2024-09-15T00:14:31.763692+0000","last_became_active":"2024-09-15T00:11:19.594528+0000","last_became_peered":"2024-09-15T00:11:19.594528+0000","last_unstale":"2024-09-15T00:14:31.763692+0000","last_undegraded":"2024-09-15T00:14:31.763692+0000","last_fullsized":"2024-09-15T00:14:31.763692+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:56:44.230264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001727704,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5349,"num_read_kb":42137,"num_write":591,"num_write_kb":17763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":437,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762777+0000","last_change":"2024-09-15T00:13:52.028546+0000","last_active":"2024-09-15T00:14:31.762777+0000","last_peered":"2024-09-15T00:14:31.762777+0000","last_clean":"2024-09-15T00:14:31.762777+0000","last_became_active":"2024-09-15T00:13:52.026457+0000","last_became_peered":"2024-09-15T00:13:52.026457+0000","last_unstale":"2024-09-15T00:14:31.762777+0000","last_undegraded":"2024-09-15T00:14:31.762777+0000","last_fullsized":"2024-09-15T00:14:31.762777+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:58:54.983326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":348,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502648+0000","last_change":"2024-09-15T00:14:18.046986+0000","last_active":"2024-09-15T00:14:31.502648+0000","last_peered":"2024-09-15T00:14:31.502648+0000","last_clean":"2024-09-15T00:14:31.502648+0000","last_became_active":"2024-09-15T00:14:18.046593+0000","last_became_peered":"2024-09-15T00:14:18.046593+0000","last_unstale":"2024-09-15T00:14:31.502648+0000","last_undegraded":"2024-09-15T00:14:31.502648+0000","last_fullsized":"2024-09-15T00:14:31.502648+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_clean_scrub_stamp":"2024-09-15T00:11:04.453672+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:59:10.044950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"179'844","reported_seq":8284,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.764021+0000","last_change":"2024-09-15T00:14:22.040149+0000","last_active":"2024-09-15T00:14:31.764021+0000","last_peered":"2024-09-15T00:14:31.764021+0000","last_clean":"2024-09-15T00:14:31.764021+0000","last_became_active":"2024-09-15T00:14:22.038746+0000","last_became_peered":"2024-09-15T00:14:22.038746+0000","last_unstale":"2024-09-15T00:14:31.764021+0000","last_undegraded":"2024-09-15T00:14:31.764021+0000","last_fullsized":"2024-09-15T00:14:31.764021+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:36:14.331079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012708680000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8015,"num_read_kb":57970,"num_write":928,"num_write_kb":24350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":14074270,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"179'1526","reported_seq":9116,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763808+0000","last_change":"2024-09-15T00:14:19.051081+0000","last_active":"2024-09-15T00:14:31.763808+0000","last_peered":"2024-09-15T00:14:31.763808+0000","last_clean":"2024-09-15T00:14:31.763808+0000","last_became_active":"2024-09-15T00:14:19.048942+0000","last_became_peered":"2024-09-15T00:14:19.048942+0000","last_unstale":"2024-09-15T00:14:31.763808+0000","last_undegraded":"2024-09-15T00:14:31.763808+0000","last_fullsized":"2024-09-15T00:14:31.763808+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1526,"log_dups_size":0,"ondisk_log_size":1526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:54:02.659520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061724499999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":14886,"num_read_kb":35520,"num_write":2067,"num_write_kb":12930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":409,"num_bytes_recovered":10512832,"num_keys_recovered":83,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"179'733","reported_seq":6627,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470085+0000","last_change":"2024-09-15T00:14:08.082753+0000","last_active":"2024-09-15T00:14:31.470085+0000","last_peered":"2024-09-15T00:14:31.470085+0000","last_clean":"2024-09-15T00:14:31.470085+0000","last_became_active":"2024-09-15T00:11:19.602625+0000","last_became_peered":"2024-09-15T00:11:19.602625+0000","last_unstale":"2024-09-15T00:14:31.470085+0000","last_undegraded":"2024-09-15T00:14:31.470085+0000","last_fullsized":"2024-09-15T00:14:31.470085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:24:41.939171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012377130000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6965,"num_read_kb":22637,"num_write":697,"num_write_kb":11622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"177'426","reported_seq":5524,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763453+0000","last_change":"2024-09-15T00:14:20.037949+0000","last_active":"2024-09-15T00:14:31.763453+0000","last_peered":"2024-09-15T00:14:31.763453+0000","last_clean":"2024-09-15T00:14:31.763453+0000","last_became_active":"2024-09-15T00:14:20.036368+0000","last_became_peered":"2024-09-15T00:14:20.036368+0000","last_unstale":"2024-09-15T00:14:31.763453+0000","last_undegraded":"2024-09-15T00:14:31.763453+0000","last_fullsized":"2024-09-15T00:14:31.763453+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:10:24.358861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087100100000000005,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3886,"num_read_kb":33280,"num_write":476,"num_write_kb":22691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":10092575,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'919","reported_seq":4934,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762892+0000","last_change":"2024-09-15T00:14:08.081241+0000","last_active":"2024-09-15T00:14:31.762892+0000","last_peered":"2024-09-15T00:14:31.762892+0000","last_clean":"2024-09-15T00:14:31.762892+0000","last_became_active":"2024-09-15T00:11:19.595088+0000","last_became_peered":"2024-09-15T00:11:19.595088+0000","last_unstale":"2024-09-15T00:14:31.762892+0000","last_undegraded":"2024-09-15T00:14:31.762892+0000","last_fullsized":"2024-09-15T00:14:31.762892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:47:04.219653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024006000000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7681,"num_read_kb":43201,"num_write":1093,"num_write_kb":15373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"177'985","reported_seq":6046,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502672+0000","last_change":"2024-09-15T00:14:08.088640+0000","last_active":"2024-09-15T00:14:31.502672+0000","last_peered":"2024-09-15T00:14:31.502672+0000","last_clean":"2024-09-15T00:14:31.502672+0000","last_became_active":"2024-09-15T00:13:51.028205+0000","last_became_peered":"2024-09-15T00:13:51.028205+0000","last_unstale":"2024-09-15T00:14:31.502672+0000","last_undegraded":"2024-09-15T00:14:31.502672+0000","last_fullsized":"2024-09-15T00:14:31.502672+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:13:15.922674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012164389999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7667,"num_read_kb":20584,"num_write":1269,"num_write_kb":5933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":1861870,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"168'1421","reported_seq":6068,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502758+0000","last_change":"2024-09-15T00:14:08.088029+0000","last_active":"2024-09-15T00:14:31.502758+0000","last_peered":"2024-09-15T00:14:31.502758+0000","last_clean":"2024-09-15T00:14:31.502758+0000","last_became_active":"2024-09-15T00:13:50.023348+0000","last_became_peered":"2024-09-15T00:13:50.023348+0000","last_unstale":"2024-09-15T00:14:31.502758+0000","last_undegraded":"2024-09-15T00:14:31.502758+0000","last_fullsized":"2024-09-15T00:14:31.502758+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1421,"log_dups_size":0,"ondisk_log_size":1421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:46:55.906750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001473739,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8919,"num_read_kb":45201,"num_write":2133,"num_write_kb":24448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":18482007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"176'516","reported_seq":6414,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502850+0000","last_change":"2024-09-15T00:14:08.086840+0000","last_active":"2024-09-15T00:14:31.502850+0000","last_peered":"2024-09-15T00:14:31.502850+0000","last_clean":"2024-09-15T00:14:31.502850+0000","last_became_active":"2024-09-15T00:13:49.218512+0000","last_became_peered":"2024-09-15T00:13:49.218512+0000","last_unstale":"2024-09-15T00:14:31.502850+0000","last_undegraded":"2024-09-15T00:14:31.502850+0000","last_fullsized":"2024-09-15T00:14:31.502850+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:27:37.063625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014419529999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5794,"num_read_kb":31109,"num_write":499,"num_write_kb":15000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5158884,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"182'731","reported_seq":7089,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470085+0000","last_change":"2024-09-15T00:14:19.055694+0000","last_active":"2024-09-15T00:14:31.470085+0000","last_peered":"2024-09-15T00:14:31.470085+0000","last_clean":"2024-09-15T00:14:31.470085+0000","last_became_active":"2024-09-15T00:14:19.054103+0000","last_became_peered":"2024-09-15T00:14:19.054103+0000","last_unstale":"2024-09-15T00:14:31.470085+0000","last_undegraded":"2024-09-15T00:14:31.470085+0000","last_fullsized":"2024-09-15T00:14:31.470085+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:41:29.549456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086791500000000003,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7073,"num_read_kb":31603,"num_write":836,"num_write_kb":14934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":8339863,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"176'596","reported_seq":4626,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502629+0000","last_change":"2024-09-15T00:14:08.086089+0000","last_active":"2024-09-15T00:14:31.502629+0000","last_peered":"2024-09-15T00:14:31.502629+0000","last_clean":"2024-09-15T00:14:31.502629+0000","last_became_active":"2024-09-15T00:13:49.218581+0000","last_became_peered":"2024-09-15T00:13:49.218581+0000","last_unstale":"2024-09-15T00:14:31.502629+0000","last_undegraded":"2024-09-15T00:14:31.502629+0000","last_fullsized":"2024-09-15T00:14:31.502629+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:17.388753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010536860000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4205,"num_read_kb":15404,"num_write":601,"num_write_kb":11097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":1517056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"173'674","reported_seq":7804,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.141963+0000","last_change":"2024-09-15T00:14:21.034205+0000","last_active":"2024-09-15T00:14:31.141963+0000","last_peered":"2024-09-15T00:14:31.141963+0000","last_clean":"2024-09-15T00:14:31.141963+0000","last_became_active":"2024-09-15T00:14:21.032233+0000","last_became_peered":"2024-09-15T00:14:21.032233+0000","last_unstale":"2024-09-15T00:14:31.141963+0000","last_undegraded":"2024-09-15T00:14:31.141963+0000","last_fullsized":"2024-09-15T00:14:31.141963+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:46:53.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089427500000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6790,"num_read_kb":50206,"num_write":784,"num_write_kb":25712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":5797342,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'1107","reported_seq":8225,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502722+0000","last_change":"2024-09-15T00:14:08.087888+0000","last_active":"2024-09-15T00:14:31.502722+0000","last_peered":"2024-09-15T00:14:31.502722+0000","last_clean":"2024-09-15T00:14:31.502722+0000","last_became_active":"2024-09-15T00:13:51.025771+0000","last_became_peered":"2024-09-15T00:13:51.025771+0000","last_unstale":"2024-09-15T00:14:31.502722+0000","last_undegraded":"2024-09-15T00:14:31.502722+0000","last_fullsized":"2024-09-15T00:14:31.502722+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:28:00.228817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001944035,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":10923,"num_read_kb":34402,"num_write":1383,"num_write_kb":19662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":27769124,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":521,"num_object_clones":0,"num_object_copies":1042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":521,"num_whiteouts":517,"num_read":240279,"num_read_kb":1281261,"num_write":28650,"num_write_kb":585223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10651,"num_bytes_recovered":390333624,"num_keys_recovered":726,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5367,"internal_metadata":0},"log_size":25207,"ondisk_log_size":25207,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":2361472,"data_compressed":25472,"data_compressed_allocated":147456,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87736,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284104,"statfs":{"total":96636764160,"available":96546922496,"internally_reserved":0,"allocated":2453504,"data_stored":1194007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51620,"kb_used_data":2380,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320220,"statfs":{"total":96636764160,"available":96583905280,"internally_reserved":0,"allocated":2437120,"data_stored":1188146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55836,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316004,"statfs":{"total":96636764160,"available":96579588096,"internally_reserved":0,"allocated":2490368,"data_stored":1784356,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542191,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77996,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293844,"statfs":{"total":96636764160,"available":96556896256,"internally_reserved":0,"allocated":2449408,"data_stored":1193988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47476,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324364,"statfs":{"total":96636764160,"available":96588148736,"internally_reserved":0,"allocated":2453504,"data_stored":1194007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":108,"seq":463856467990,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42588,"kb_used_data":2368,"kb_used_omap":6,"kb_used_meta":40185,"kb_avail":94329252,"statfs":{"total":96636764160,"available":96593154048,"internally_reserved":0,"allocated":2424832,"data_stored":1182304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6962,"internal_metadata":41149646},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55784,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316056,"statfs":{"total":96636764160,"available":96579641344,"internally_reserved":0,"allocated":2433024,"data_stored":1182342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78048,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293792,"statfs":{"total":96636764160,"available":96556843008,"internally_reserved":0,"allocated":2490368,"data_stored":1784356,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5367,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:14:34.908 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T00:14:34.908 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-15T00:14:35.206 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T00:14:35.215 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T00:14:35.216 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:14:35.491 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:14:35.492 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:14:35.503 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-15T00:14:34.327513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":525,"num_object_clones":0,"num_object_copies":1050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":525,"num_whiteouts":517,"num_read":240395,"num_read_kb":1281361,"num_write":28811,"num_write_kb":587641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10669,"num_bytes_recovered":392695172,"num_keys_recovered":726,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25327,"ondisk_log_size":25327,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":497084,"kb_used_data":19172,"kb_used_omap":17,"kb_used_meta":477678,"kb_avail":754477636,"statfs":{"total":773094113280,"available":772585099264,"internally_reserved":0,"allocated":19632128,"data_stored":10703506,"data_compressed":12736,"data_compressed_allocated":73728,"data_compressed_original":1179648,"omap_allocated":18117,"internal_metadata":489142587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001051"},"pg_stats":[{"pgid":"3.1f","version":"182'643","reported_seq":6819,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142321+0000","last_change":"2024-09-15T00:14:18.045712+0000","last_active":"2024-09-15T00:14:31.142321+0000","last_peered":"2024-09-15T00:14:31.142321+0000","last_clean":"2024-09-15T00:14:31.142321+0000","last_became_active":"2024-09-15T00:14:18.042449+0000","last_became_peered":"2024-09-15T00:14:18.042449+0000","last_unstale":"2024-09-15T00:14:31.142321+0000","last_undegraded":"2024-09-15T00:14:31.142321+0000","last_fullsized":"2024-09-15T00:14:31.142321+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:38:13.616525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011752869999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6942,"num_read_kb":26340,"num_write":697,"num_write_kb":18376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":5475412,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'632","reported_seq":7633,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763549+0000","last_change":"2024-09-15T00:14:08.080286+0000","last_active":"2024-09-15T00:14:31.763549+0000","last_peered":"2024-09-15T00:14:31.763549+0000","last_clean":"2024-09-15T00:14:31.763549+0000","last_became_active":"2024-09-15T00:11:19.598640+0000","last_became_peered":"2024-09-15T00:11:19.598640+0000","last_unstale":"2024-09-15T00:14:31.763549+0000","last_undegraded":"2024-09-15T00:14:31.763549+0000","last_fullsized":"2024-09-15T00:14:31.763549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:16:36.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016674789999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7565,"num_read_kb":39401,"num_write":737,"num_write_kb":10312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'698","reported_seq":4933,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762732+0000","last_change":"2024-09-15T00:14:08.080849+0000","last_active":"2024-09-15T00:14:31.762732+0000","last_peered":"2024-09-15T00:14:31.762732+0000","last_clean":"2024-09-15T00:14:31.762732+0000","last_became_active":"2024-09-15T00:11:19.595954+0000","last_became_peered":"2024-09-15T00:11:19.595954+0000","last_unstale":"2024-09-15T00:14:31.762732+0000","last_undegraded":"2024-09-15T00:14:31.762732+0000","last_fullsized":"2024-09-15T00:14:31.762732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:28.144497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020206820000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5583,"num_read_kb":45090,"num_write":773,"num_write_kb":31484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"182'674","reported_seq":5880,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762701+0000","last_change":"2024-09-15T00:14:08.080354+0000","last_active":"2024-09-15T00:14:31.762701+0000","last_peered":"2024-09-15T00:14:31.762701+0000","last_clean":"2024-09-15T00:14:31.762701+0000","last_became_active":"2024-09-15T00:11:19.602583+0000","last_became_peered":"2024-09-15T00:11:19.602583+0000","last_unstale":"2024-09-15T00:14:31.762701+0000","last_undegraded":"2024-09-15T00:14:31.762701+0000","last_fullsized":"2024-09-15T00:14:31.762701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:33:11.479982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00168294,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":5888,"num_read_kb":35219,"num_write":807,"num_write_kb":10221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'844","reported_seq":6971,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142108+0000","last_change":"2024-09-15T00:14:20.035839+0000","last_active":"2024-09-15T00:14:31.142108+0000","last_peered":"2024-09-15T00:14:31.142108+0000","last_clean":"2024-09-15T00:14:31.142108+0000","last_became_active":"2024-09-15T00:14:20.032662+0000","last_became_peered":"2024-09-15T00:14:20.032662+0000","last_unstale":"2024-09-15T00:14:31.142108+0000","last_undegraded":"2024-09-15T00:14:31.142108+0000","last_fullsized":"2024-09-15T00:14:31.142108+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:24:20.329228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001922899,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7801,"num_read_kb":34537,"num_write":909,"num_write_kb":14651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":10954706,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"179'783","reported_seq":7558,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.141955+0000","last_change":"2024-09-15T00:14:24.098641+0000","last_active":"2024-09-15T00:14:31.141955+0000","last_peered":"2024-09-15T00:14:31.141955+0000","last_clean":"2024-09-15T00:14:31.141955+0000","last_became_active":"2024-09-15T00:14:24.097321+0000","last_became_peered":"2024-09-15T00:14:24.097321+0000","last_unstale":"2024-09-15T00:14:31.141955+0000","last_undegraded":"2024-09-15T00:14:31.141955+0000","last_fullsized":"2024-09-15T00:14:31.141955+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:40:15.897865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012814980000000001,"stat_sum":{"num_bytes":19,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":8900,"num_read_kb":62621,"num_write":837,"num_write_kb":33377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":66497974,"num_keys_recovered":149,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"177'678","reported_seq":5626,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142375+0000","last_change":"2024-09-15T00:14:22.038154+0000","last_active":"2024-09-15T00:14:31.142375+0000","last_peered":"2024-09-15T00:14:31.142375+0000","last_clean":"2024-09-15T00:14:31.142375+0000","last_became_active":"2024-09-15T00:14:22.036152+0000","last_became_peered":"2024-09-15T00:14:22.036152+0000","last_unstale":"2024-09-15T00:14:31.142375+0000","last_undegraded":"2024-09-15T00:14:31.142375+0000","last_fullsized":"2024-09-15T00:14:31.142375+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:30:00.495418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005256073,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4711,"num_read_kb":42462,"num_write":705,"num_write_kb":9831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":558,"num_bytes_recovered":6445817,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"179'706","reported_seq":8764,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.764017+0000","last_change":"2024-09-15T00:14:18.044374+0000","last_active":"2024-09-15T00:14:31.764017+0000","last_peered":"2024-09-15T00:14:31.764017+0000","last_clean":"2024-09-15T00:14:31.764017+0000","last_became_active":"2024-09-15T00:14:18.042710+0000","last_became_peered":"2024-09-15T00:14:18.042710+0000","last_unstale":"2024-09-15T00:14:31.764017+0000","last_undegraded":"2024-09-15T00:14:31.764017+0000","last_fullsized":"2024-09-15T00:14:31.764017+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:52:32.313688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014749780000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8102,"num_read_kb":40115,"num_write":776,"num_write_kb":15981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":4986328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'679","reported_seq":5549,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142315+0000","last_change":"2024-09-15T00:14:20.037042+0000","last_active":"2024-09-15T00:14:31.142315+0000","last_peered":"2024-09-15T00:14:31.142315+0000","last_clean":"2024-09-15T00:14:31.142315+0000","last_became_active":"2024-09-15T00:14:20.034657+0000","last_became_peered":"2024-09-15T00:14:20.034657+0000","last_unstale":"2024-09-15T00:14:31.142315+0000","last_undegraded":"2024-09-15T00:14:31.142315+0000","last_fullsized":"2024-09-15T00:14:31.142315+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:52:39.383067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002708182,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5548,"num_read_kb":45663,"num_write":634,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":12460957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"176'1032","reported_seq":8430,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762889+0000","last_change":"2024-09-15T00:14:08.079912+0000","last_active":"2024-09-15T00:14:31.762889+0000","last_peered":"2024-09-15T00:14:31.762889+0000","last_clean":"2024-09-15T00:14:31.762889+0000","last_became_active":"2024-09-15T00:13:50.031768+0000","last_became_peered":"2024-09-15T00:13:50.031768+0000","last_unstale":"2024-09-15T00:14:31.762889+0000","last_undegraded":"2024-09-15T00:14:31.762889+0000","last_fullsized":"2024-09-15T00:14:31.762889+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1032,"log_dups_size":0,"ondisk_log_size":1032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:21.539927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001158467,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8890,"num_read_kb":39257,"num_write":1264,"num_write_kb":20613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":23803494,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"179'667","reported_seq":6954,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502804+0000","last_change":"2024-09-15T00:14:08.088641+0000","last_active":"2024-09-15T00:14:31.502804+0000","last_peered":"2024-09-15T00:14:31.502804+0000","last_clean":"2024-09-15T00:14:31.502804+0000","last_became_active":"2024-09-15T00:13:50.025342+0000","last_became_peered":"2024-09-15T00:13:50.025342+0000","last_unstale":"2024-09-15T00:14:31.502804+0000","last_undegraded":"2024-09-15T00:14:31.502804+0000","last_fullsized":"2024-09-15T00:14:31.502804+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:50:45.634693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071033100000000003,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7596,"num_read_kb":45634,"num_write":737,"num_write_kb":14836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":19127384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'1079","reported_seq":7470,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142064+0000","last_change":"2024-09-15T00:14:24.095829+0000","last_active":"2024-09-15T00:14:31.142064+0000","last_peered":"2024-09-15T00:14:31.142064+0000","last_clean":"2024-09-15T00:14:31.142064+0000","last_became_active":"2024-09-15T00:14:24.093937+0000","last_became_peered":"2024-09-15T00:14:24.093937+0000","last_unstale":"2024-09-15T00:14:31.142064+0000","last_undegraded":"2024-09-15T00:14:31.142064+0000","last_fullsized":"2024-09-15T00:14:31.142064+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1079,"log_dups_size":0,"ondisk_log_size":1079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:10:03.817976+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.016604621,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":9874,"num_read_kb":58461,"num_write":1482,"num_write_kb":25008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":6083388,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":403,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470173+0000","last_change":"2024-09-15T00:13:46.030592+0000","last_active":"2024-09-15T00:14:31.470173+0000","last_peered":"2024-09-15T00:14:31.470173+0000","last_clean":"2024-09-15T00:14:31.470173+0000","last_became_active":"2024-09-15T00:13:46.030230+0000","last_became_peered":"2024-09-15T00:13:46.030230+0000","last_unstale":"2024-09-15T00:14:31.470173+0000","last_undegraded":"2024-09-15T00:14:31.470173+0000","last_fullsized":"2024-09-15T00:14:31.470173+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:26:48.293426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'715","reported_seq":7972,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142190+0000","last_change":"2024-09-15T00:14:18.046157+0000","last_active":"2024-09-15T00:14:31.142190+0000","last_peered":"2024-09-15T00:14:31.142190+0000","last_clean":"2024-09-15T00:14:31.142190+0000","last_became_active":"2024-09-15T00:14:18.043887+0000","last_became_peered":"2024-09-15T00:14:18.043887+0000","last_unstale":"2024-09-15T00:14:31.142190+0000","last_undegraded":"2024-09-15T00:14:31.142190+0000","last_fullsized":"2024-09-15T00:14:31.142190+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:22:24.520772+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00086129200000000005,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6781,"num_read_kb":42484,"num_write":652,"num_write_kb":25730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":26193256,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142568+0000","last_change":"2024-09-15T00:14:16.040721+0000","last_active":"2024-09-15T00:14:31.142568+0000","last_peered":"2024-09-15T00:14:31.142568+0000","last_clean":"2024-09-15T00:14:31.142568+0000","last_became_active":"2024-09-15T00:14:16.040306+0000","last_became_peered":"2024-09-15T00:14:16.040306+0000","last_unstale":"2024-09-15T00:14:31.142568+0000","last_undegraded":"2024-09-15T00:14:31.142568+0000","last_fullsized":"2024-09-15T00:14:31.142568+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:54:01.189526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'731","reported_seq":9397,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763223+0000","last_change":"2024-09-15T00:14:08.081190+0000","last_active":"2024-09-15T00:14:31.763223+0000","last_peered":"2024-09-15T00:14:31.763223+0000","last_clean":"2024-09-15T00:14:31.763223+0000","last_became_active":"2024-09-15T00:13:49.218857+0000","last_became_peered":"2024-09-15T00:13:49.218857+0000","last_unstale":"2024-09-15T00:14:31.763223+0000","last_undegraded":"2024-09-15T00:14:31.763223+0000","last_fullsized":"2024-09-15T00:14:31.763223+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:04:29.276892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020637310000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":10066,"num_read_kb":57181,"num_write":720,"num_write_kb":17747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":15784649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":110,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502690+0000","last_change":"2024-09-15T00:14:15.038333+0000","last_active":"2024-09-15T00:14:31.502690+0000","last_peered":"2024-09-15T00:14:31.502690+0000","last_clean":"2024-09-15T00:14:31.502690+0000","last_became_active":"2024-09-15T00:14:15.037987+0000","last_became_peered":"2024-09-15T00:14:15.037987+0000","last_unstale":"2024-09-15T00:14:31.502690+0000","last_undegraded":"2024-09-15T00:14:31.502690+0000","last_fullsized":"2024-09-15T00:14:31.502690+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:27.770396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"177'774","reported_seq":12041,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142529+0000","last_change":"2024-09-15T00:14:20.036384+0000","last_active":"2024-09-15T00:14:31.142529+0000","last_peered":"2024-09-15T00:14:31.142529+0000","last_clean":"2024-09-15T00:14:31.142529+0000","last_became_active":"2024-09-15T00:14:20.033228+0000","last_became_peered":"2024-09-15T00:14:20.033228+0000","last_unstale":"2024-09-15T00:14:31.142529+0000","last_undegraded":"2024-09-15T00:14:31.142529+0000","last_fullsized":"2024-09-15T00:14:31.142529+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:36:37.663215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017650229999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12816,"num_read_kb":60614,"num_write":771,"num_write_kb":25261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1083,"num_bytes_recovered":41768720,"num_keys_recovered":205,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142653+0000","last_change":"2024-09-15T00:14:16.034807+0000","last_active":"2024-09-15T00:14:31.142653+0000","last_peered":"2024-09-15T00:14:31.142653+0000","last_clean":"2024-09-15T00:14:31.142653+0000","last_became_active":"2024-09-15T00:14:16.034489+0000","last_became_peered":"2024-09-15T00:14:16.034489+0000","last_unstale":"2024-09-15T00:14:31.142653+0000","last_undegraded":"2024-09-15T00:14:31.142653+0000","last_fullsized":"2024-09-15T00:14:31.142653+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:38:39.946046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"179'552","reported_seq":3247,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762929+0000","last_change":"2024-09-15T00:14:08.081714+0000","last_active":"2024-09-15T00:14:31.762929+0000","last_peered":"2024-09-15T00:14:31.762929+0000","last_clean":"2024-09-15T00:14:31.762929+0000","last_became_active":"2024-09-15T00:11:19.595633+0000","last_became_peered":"2024-09-15T00:11:19.595633+0000","last_unstale":"2024-09-15T00:14:31.762929+0000","last_undegraded":"2024-09-15T00:14:31.762929+0000","last_fullsized":"2024-09-15T00:14:31.762929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:24:30.596801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026087110000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3741,"num_read_kb":17273,"num_write":546,"num_write_kb":8984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"182'819","reported_seq":8159,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142046+0000","last_change":"2024-09-15T00:14:24.096290+0000","last_active":"2024-09-15T00:14:31.142046+0000","last_peered":"2024-09-15T00:14:31.142046+0000","last_clean":"2024-09-15T00:14:31.142046+0000","last_became_active":"2024-09-15T00:14:24.094819+0000","last_became_peered":"2024-09-15T00:14:24.094819+0000","last_unstale":"2024-09-15T00:14:31.142046+0000","last_undegraded":"2024-09-15T00:14:31.142046+0000","last_fullsized":"2024-09-15T00:14:31.142046+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:32:14.972007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090289999999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":7230,"num_read_kb":43219,"num_write":761,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":9273484,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":420,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762759+0000","last_change":"2024-09-15T00:14:16.050758+0000","last_active":"2024-09-15T00:14:31.762759+0000","last_peered":"2024-09-15T00:14:31.762759+0000","last_clean":"2024-09-15T00:14:31.762759+0000","last_became_active":"2024-09-15T00:14:16.046906+0000","last_became_peered":"2024-09-15T00:14:16.046906+0000","last_unstale":"2024-09-15T00:14:31.762759+0000","last_undegraded":"2024-09-15T00:14:31.762759+0000","last_fullsized":"2024-09-15T00:14:31.762759+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:49:39.277929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"179'696","reported_seq":7804,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142817+0000","last_change":"2024-09-15T00:14:19.050801+0000","last_active":"2024-09-15T00:14:31.142817+0000","last_peered":"2024-09-15T00:14:31.142817+0000","last_clean":"2024-09-15T00:14:31.142817+0000","last_became_active":"2024-09-15T00:14:19.048768+0000","last_became_peered":"2024-09-15T00:14:19.048768+0000","last_unstale":"2024-09-15T00:14:31.142817+0000","last_undegraded":"2024-09-15T00:14:31.142817+0000","last_fullsized":"2024-09-15T00:14:31.142817+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:42:10.166497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092066200000000004,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7368,"num_read_kb":33386,"num_write":726,"num_write_kb":14440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":31813799,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762886+0000","last_change":"2024-09-15T00:13:52.026234+0000","last_active":"2024-09-15T00:14:31.762886+0000","last_peered":"2024-09-15T00:14:31.762886+0000","last_clean":"2024-09-15T00:14:31.762886+0000","last_became_active":"2024-09-15T00:13:52.025864+0000","last_became_peered":"2024-09-15T00:13:52.025864+0000","last_unstale":"2024-09-15T00:14:31.762886+0000","last_undegraded":"2024-09-15T00:14:31.762886+0000","last_fullsized":"2024-09-15T00:14:31.762886+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:23:52.397810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"182'708","reported_seq":6729,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142024+0000","last_change":"2024-09-15T00:14:21.036152+0000","last_active":"2024-09-15T00:14:31.142024+0000","last_peered":"2024-09-15T00:14:31.142024+0000","last_clean":"2024-09-15T00:14:31.142024+0000","last_became_active":"2024-09-15T00:14:21.034477+0000","last_became_peered":"2024-09-15T00:14:21.034477+0000","last_unstale":"2024-09-15T00:14:31.142024+0000","last_undegraded":"2024-09-15T00:14:31.142024+0000","last_fullsized":"2024-09-15T00:14:31.142024+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:50.622032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075419699999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6724,"num_read_kb":49050,"num_write":759,"num_write_kb":22019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":6058433,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":114,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502620+0000","last_change":"2024-09-15T00:14:16.041987+0000","last_active":"2024-09-15T00:14:31.502620+0000","last_peered":"2024-09-15T00:14:31.502620+0000","last_clean":"2024-09-15T00:14:31.502620+0000","last_became_active":"2024-09-15T00:14:16.041672+0000","last_became_peered":"2024-09-15T00:14:16.041672+0000","last_unstale":"2024-09-15T00:14:31.502620+0000","last_undegraded":"2024-09-15T00:14:31.502620+0000","last_fullsized":"2024-09-15T00:14:31.502620+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:04:25.406297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"179'619","reported_seq":5394,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763692+0000","last_change":"2024-09-15T00:14:08.080584+0000","last_active":"2024-09-15T00:14:31.763692+0000","last_peered":"2024-09-15T00:14:31.763692+0000","last_clean":"2024-09-15T00:14:31.763692+0000","last_became_active":"2024-09-15T00:11:19.594528+0000","last_became_peered":"2024-09-15T00:11:19.594528+0000","last_unstale":"2024-09-15T00:14:31.763692+0000","last_undegraded":"2024-09-15T00:14:31.763692+0000","last_fullsized":"2024-09-15T00:14:31.763692+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:56:44.230264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001727704,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5349,"num_read_kb":42137,"num_write":591,"num_write_kb":17763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":437,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762777+0000","last_change":"2024-09-15T00:13:52.028546+0000","last_active":"2024-09-15T00:14:31.762777+0000","last_peered":"2024-09-15T00:14:31.762777+0000","last_clean":"2024-09-15T00:14:31.762777+0000","last_became_active":"2024-09-15T00:13:52.026457+0000","last_became_peered":"2024-09-15T00:13:52.026457+0000","last_unstale":"2024-09-15T00:14:31.762777+0000","last_undegraded":"2024-09-15T00:14:31.762777+0000","last_fullsized":"2024-09-15T00:14:31.762777+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:58:54.983326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":348,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502648+0000","last_change":"2024-09-15T00:14:18.046986+0000","last_active":"2024-09-15T00:14:31.502648+0000","last_peered":"2024-09-15T00:14:31.502648+0000","last_clean":"2024-09-15T00:14:31.502648+0000","last_became_active":"2024-09-15T00:14:18.046593+0000","last_became_peered":"2024-09-15T00:14:18.046593+0000","last_unstale":"2024-09-15T00:14:31.502648+0000","last_undegraded":"2024-09-15T00:14:31.502648+0000","last_fullsized":"2024-09-15T00:14:31.502648+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_clean_scrub_stamp":"2024-09-15T00:11:04.453672+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:59:10.044950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"179'844","reported_seq":8284,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.764021+0000","last_change":"2024-09-15T00:14:22.040149+0000","last_active":"2024-09-15T00:14:31.764021+0000","last_peered":"2024-09-15T00:14:31.764021+0000","last_clean":"2024-09-15T00:14:31.764021+0000","last_became_active":"2024-09-15T00:14:22.038746+0000","last_became_peered":"2024-09-15T00:14:22.038746+0000","last_unstale":"2024-09-15T00:14:31.764021+0000","last_undegraded":"2024-09-15T00:14:31.764021+0000","last_fullsized":"2024-09-15T00:14:31.764021+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:36:14.331079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012708680000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8015,"num_read_kb":57970,"num_write":928,"num_write_kb":24350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":14074270,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"179'1526","reported_seq":9116,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763808+0000","last_change":"2024-09-15T00:14:19.051081+0000","last_active":"2024-09-15T00:14:31.763808+0000","last_peered":"2024-09-15T00:14:31.763808+0000","last_clean":"2024-09-15T00:14:31.763808+0000","last_became_active":"2024-09-15T00:14:19.048942+0000","last_became_peered":"2024-09-15T00:14:19.048942+0000","last_unstale":"2024-09-15T00:14:31.763808+0000","last_undegraded":"2024-09-15T00:14:31.763808+0000","last_fullsized":"2024-09-15T00:14:31.763808+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1526,"log_dups_size":0,"ondisk_log_size":1526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:54:02.659520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061724499999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":14886,"num_read_kb":35520,"num_write":2067,"num_write_kb":12930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":409,"num_bytes_recovered":10512832,"num_keys_recovered":83,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"179'733","reported_seq":6627,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470085+0000","last_change":"2024-09-15T00:14:08.082753+0000","last_active":"2024-09-15T00:14:31.470085+0000","last_peered":"2024-09-15T00:14:31.470085+0000","last_clean":"2024-09-15T00:14:31.470085+0000","last_became_active":"2024-09-15T00:11:19.602625+0000","last_became_peered":"2024-09-15T00:11:19.602625+0000","last_unstale":"2024-09-15T00:14:31.470085+0000","last_undegraded":"2024-09-15T00:14:31.470085+0000","last_fullsized":"2024-09-15T00:14:31.470085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:24:41.939171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012377130000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6965,"num_read_kb":22637,"num_write":697,"num_write_kb":11622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"177'426","reported_seq":5524,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763453+0000","last_change":"2024-09-15T00:14:20.037949+0000","last_active":"2024-09-15T00:14:31.763453+0000","last_peered":"2024-09-15T00:14:31.763453+0000","last_clean":"2024-09-15T00:14:31.763453+0000","last_became_active":"2024-09-15T00:14:20.036368+0000","last_became_peered":"2024-09-15T00:14:20.036368+0000","last_unstale":"2024-09-15T00:14:31.763453+0000","last_undegraded":"2024-09-15T00:14:31.763453+0000","last_fullsized":"2024-09-15T00:14:31.763453+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:10:24.358861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087100100000000005,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3886,"num_read_kb":33280,"num_write":476,"num_write_kb":22691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":10092575,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'919","reported_seq":4934,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762892+0000","last_change":"2024-09-15T00:14:08.081241+0000","last_active":"2024-09-15T00:14:31.762892+0000","last_peered":"2024-09-15T00:14:31.762892+0000","last_clean":"2024-09-15T00:14:31.762892+0000","last_became_active":"2024-09-15T00:11:19.595088+0000","last_became_peered":"2024-09-15T00:11:19.595088+0000","last_unstale":"2024-09-15T00:14:31.762892+0000","last_undegraded":"2024-09-15T00:14:31.762892+0000","last_fullsized":"2024-09-15T00:14:31.762892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:47:04.219653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024006000000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7681,"num_read_kb":43201,"num_write":1093,"num_write_kb":15373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"177'985","reported_seq":6046,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502672+0000","last_change":"2024-09-15T00:14:08.088640+0000","last_active":"2024-09-15T00:14:31.502672+0000","last_peered":"2024-09-15T00:14:31.502672+0000","last_clean":"2024-09-15T00:14:31.502672+0000","last_became_active":"2024-09-15T00:13:51.028205+0000","last_became_peered":"2024-09-15T00:13:51.028205+0000","last_unstale":"2024-09-15T00:14:31.502672+0000","last_undegraded":"2024-09-15T00:14:31.502672+0000","last_fullsized":"2024-09-15T00:14:31.502672+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:13:15.922674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012164389999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7667,"num_read_kb":20584,"num_write":1269,"num_write_kb":5933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":1861870,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"168'1421","reported_seq":6068,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502758+0000","last_change":"2024-09-15T00:14:08.088029+0000","last_active":"2024-09-15T00:14:31.502758+0000","last_peered":"2024-09-15T00:14:31.502758+0000","last_clean":"2024-09-15T00:14:31.502758+0000","last_became_active":"2024-09-15T00:13:50.023348+0000","last_became_peered":"2024-09-15T00:13:50.023348+0000","last_unstale":"2024-09-15T00:14:31.502758+0000","last_undegraded":"2024-09-15T00:14:31.502758+0000","last_fullsized":"2024-09-15T00:14:31.502758+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1421,"log_dups_size":0,"ondisk_log_size":1421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:46:55.906750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001473739,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8919,"num_read_kb":45201,"num_write":2133,"num_write_kb":24448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":18482007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"176'516","reported_seq":6414,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502850+0000","last_change":"2024-09-15T00:14:08.086840+0000","last_active":"2024-09-15T00:14:31.502850+0000","last_peered":"2024-09-15T00:14:31.502850+0000","last_clean":"2024-09-15T00:14:31.502850+0000","last_became_active":"2024-09-15T00:13:49.218512+0000","last_became_peered":"2024-09-15T00:13:49.218512+0000","last_unstale":"2024-09-15T00:14:31.502850+0000","last_undegraded":"2024-09-15T00:14:31.502850+0000","last_fullsized":"2024-09-15T00:14:31.502850+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:27:37.063625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014419529999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5794,"num_read_kb":31109,"num_write":499,"num_write_kb":15000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5158884,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"182'731","reported_seq":7089,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470085+0000","last_change":"2024-09-15T00:14:19.055694+0000","last_active":"2024-09-15T00:14:31.470085+0000","last_peered":"2024-09-15T00:14:31.470085+0000","last_clean":"2024-09-15T00:14:31.470085+0000","last_became_active":"2024-09-15T00:14:19.054103+0000","last_became_peered":"2024-09-15T00:14:19.054103+0000","last_unstale":"2024-09-15T00:14:31.470085+0000","last_undegraded":"2024-09-15T00:14:31.470085+0000","last_fullsized":"2024-09-15T00:14:31.470085+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:41:29.549456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086791500000000003,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7073,"num_read_kb":31603,"num_write":836,"num_write_kb":14934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":8339863,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"176'596","reported_seq":4626,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502629+0000","last_change":"2024-09-15T00:14:08.086089+0000","last_active":"2024-09-15T00:14:31.502629+0000","last_peered":"2024-09-15T00:14:31.502629+0000","last_clean":"2024-09-15T00:14:31.502629+0000","last_became_active":"2024-09-15T00:13:49.218581+0000","last_became_peered":"2024-09-15T00:13:49.218581+0000","last_unstale":"2024-09-15T00:14:31.502629+0000","last_undegraded":"2024-09-15T00:14:31.502629+0000","last_fullsized":"2024-09-15T00:14:31.502629+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:17.388753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010536860000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4205,"num_read_kb":15404,"num_write":601,"num_write_kb":11097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":1517056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"173'674","reported_seq":7804,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.141963+0000","last_change":"2024-09-15T00:14:21.034205+0000","last_active":"2024-09-15T00:14:31.141963+0000","last_peered":"2024-09-15T00:14:31.141963+0000","last_clean":"2024-09-15T00:14:31.141963+0000","last_became_active":"2024-09-15T00:14:21.032233+0000","last_became_peered":"2024-09-15T00:14:21.032233+0000","last_unstale":"2024-09-15T00:14:31.141963+0000","last_undegraded":"2024-09-15T00:14:31.141963+0000","last_fullsized":"2024-09-15T00:14:31.141963+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:46:53.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089427500000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6790,"num_read_kb":50206,"num_write":784,"num_write_kb":25712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":5797342,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'1107","reported_seq":8225,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502722+0000","last_change":"2024-09-15T00:14:08.087888+0000","last_active":"2024-09-15T00:14:31.502722+0000","last_peered":"2024-09-15T00:14:31.502722+0000","last_clean":"2024-09-15T00:14:31.502722+0000","last_became_active":"2024-09-15T00:13:51.025771+0000","last_became_peered":"2024-09-15T00:13:51.025771+0000","last_unstale":"2024-09-15T00:14:31.502722+0000","last_undegraded":"2024-09-15T00:14:31.502722+0000","last_fullsized":"2024-09-15T00:14:31.502722+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:28:00.228817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001944035,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":10923,"num_read_kb":34402,"num_write":1383,"num_write_kb":19662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":27769124,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":521,"num_object_clones":0,"num_object_copies":1042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":521,"num_whiteouts":517,"num_read":240279,"num_read_kb":1281261,"num_write":28650,"num_write_kb":585223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10651,"num_bytes_recovered":390333624,"num_keys_recovered":726,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5367,"internal_metadata":0},"log_size":25207,"ondisk_log_size":25207,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":2361472,"data_compressed":25472,"data_compressed_allocated":147456,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87736,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284104,"statfs":{"total":96636764160,"available":96546922496,"internally_reserved":0,"allocated":2453504,"data_stored":1194007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51620,"kb_used_data":2380,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320220,"statfs":{"total":96636764160,"available":96583905280,"internally_reserved":0,"allocated":2437120,"data_stored":1188146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55836,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316004,"statfs":{"total":96636764160,"available":96579588096,"internally_reserved":0,"allocated":2490368,"data_stored":1784356,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542191,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77996,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293844,"statfs":{"total":96636764160,"available":96556896256,"internally_reserved":0,"allocated":2449408,"data_stored":1193988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47476,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324364,"statfs":{"total":96636764160,"available":96588148736,"internally_reserved":0,"allocated":2453504,"data_stored":1194007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":108,"seq":463856467990,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42588,"kb_used_data":2368,"kb_used_omap":6,"kb_used_meta":40185,"kb_avail":94329252,"statfs":{"total":96636764160,"available":96593154048,"internally_reserved":0,"allocated":2424832,"data_stored":1182304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6962,"internal_metadata":41149646},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55784,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316056,"statfs":{"total":96636764160,"available":96579641344,"internally_reserved":0,"allocated":2433024,"data_stored":1182342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78048,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293792,"statfs":{"total":96636764160,"available":96556843008,"internally_reserved":0,"allocated":2490368,"data_stored":1784356,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5367,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:14:35.505 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:14:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:14:35.781 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:14:35.792 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-15T00:14:34.327513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":525,"num_object_clones":0,"num_object_copies":1050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":525,"num_whiteouts":517,"num_read":240395,"num_read_kb":1281361,"num_write":28811,"num_write_kb":587641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10669,"num_bytes_recovered":392695172,"num_keys_recovered":726,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25327,"ondisk_log_size":25327,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":497084,"kb_used_data":19172,"kb_used_omap":17,"kb_used_meta":477678,"kb_avail":754477636,"statfs":{"total":773094113280,"available":772585099264,"internally_reserved":0,"allocated":19632128,"data_stored":10703506,"data_compressed":12736,"data_compressed_allocated":73728,"data_compressed_original":1179648,"omap_allocated":18117,"internal_metadata":489142587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001051"},"pg_stats":[{"pgid":"3.1f","version":"182'643","reported_seq":6819,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142321+0000","last_change":"2024-09-15T00:14:18.045712+0000","last_active":"2024-09-15T00:14:31.142321+0000","last_peered":"2024-09-15T00:14:31.142321+0000","last_clean":"2024-09-15T00:14:31.142321+0000","last_became_active":"2024-09-15T00:14:18.042449+0000","last_became_peered":"2024-09-15T00:14:18.042449+0000","last_unstale":"2024-09-15T00:14:31.142321+0000","last_undegraded":"2024-09-15T00:14:31.142321+0000","last_fullsized":"2024-09-15T00:14:31.142321+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:38:13.616525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011752869999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6942,"num_read_kb":26340,"num_write":697,"num_write_kb":18376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":5475412,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'632","reported_seq":7633,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763549+0000","last_change":"2024-09-15T00:14:08.080286+0000","last_active":"2024-09-15T00:14:31.763549+0000","last_peered":"2024-09-15T00:14:31.763549+0000","last_clean":"2024-09-15T00:14:31.763549+0000","last_became_active":"2024-09-15T00:11:19.598640+0000","last_became_peered":"2024-09-15T00:11:19.598640+0000","last_unstale":"2024-09-15T00:14:31.763549+0000","last_undegraded":"2024-09-15T00:14:31.763549+0000","last_fullsized":"2024-09-15T00:14:31.763549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:16:36.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016674789999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7565,"num_read_kb":39401,"num_write":737,"num_write_kb":10312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'698","reported_seq":4933,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762732+0000","last_change":"2024-09-15T00:14:08.080849+0000","last_active":"2024-09-15T00:14:31.762732+0000","last_peered":"2024-09-15T00:14:31.762732+0000","last_clean":"2024-09-15T00:14:31.762732+0000","last_became_active":"2024-09-15T00:11:19.595954+0000","last_became_peered":"2024-09-15T00:11:19.595954+0000","last_unstale":"2024-09-15T00:14:31.762732+0000","last_undegraded":"2024-09-15T00:14:31.762732+0000","last_fullsized":"2024-09-15T00:14:31.762732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:28.144497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020206820000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5583,"num_read_kb":45090,"num_write":773,"num_write_kb":31484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"182'674","reported_seq":5880,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762701+0000","last_change":"2024-09-15T00:14:08.080354+0000","last_active":"2024-09-15T00:14:31.762701+0000","last_peered":"2024-09-15T00:14:31.762701+0000","last_clean":"2024-09-15T00:14:31.762701+0000","last_became_active":"2024-09-15T00:11:19.602583+0000","last_became_peered":"2024-09-15T00:11:19.602583+0000","last_unstale":"2024-09-15T00:14:31.762701+0000","last_undegraded":"2024-09-15T00:14:31.762701+0000","last_fullsized":"2024-09-15T00:14:31.762701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:33:11.479982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00168294,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":5888,"num_read_kb":35219,"num_write":807,"num_write_kb":10221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'844","reported_seq":6971,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142108+0000","last_change":"2024-09-15T00:14:20.035839+0000","last_active":"2024-09-15T00:14:31.142108+0000","last_peered":"2024-09-15T00:14:31.142108+0000","last_clean":"2024-09-15T00:14:31.142108+0000","last_became_active":"2024-09-15T00:14:20.032662+0000","last_became_peered":"2024-09-15T00:14:20.032662+0000","last_unstale":"2024-09-15T00:14:31.142108+0000","last_undegraded":"2024-09-15T00:14:31.142108+0000","last_fullsized":"2024-09-15T00:14:31.142108+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:24:20.329228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001922899,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7801,"num_read_kb":34537,"num_write":909,"num_write_kb":14651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":10954706,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"179'783","reported_seq":7558,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.141955+0000","last_change":"2024-09-15T00:14:24.098641+0000","last_active":"2024-09-15T00:14:31.141955+0000","last_peered":"2024-09-15T00:14:31.141955+0000","last_clean":"2024-09-15T00:14:31.141955+0000","last_became_active":"2024-09-15T00:14:24.097321+0000","last_became_peered":"2024-09-15T00:14:24.097321+0000","last_unstale":"2024-09-15T00:14:31.141955+0000","last_undegraded":"2024-09-15T00:14:31.141955+0000","last_fullsized":"2024-09-15T00:14:31.141955+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:40:15.897865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012814980000000001,"stat_sum":{"num_bytes":19,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":8900,"num_read_kb":62621,"num_write":837,"num_write_kb":33377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":66497974,"num_keys_recovered":149,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"177'678","reported_seq":5626,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142375+0000","last_change":"2024-09-15T00:14:22.038154+0000","last_active":"2024-09-15T00:14:31.142375+0000","last_peered":"2024-09-15T00:14:31.142375+0000","last_clean":"2024-09-15T00:14:31.142375+0000","last_became_active":"2024-09-15T00:14:22.036152+0000","last_became_peered":"2024-09-15T00:14:22.036152+0000","last_unstale":"2024-09-15T00:14:31.142375+0000","last_undegraded":"2024-09-15T00:14:31.142375+0000","last_fullsized":"2024-09-15T00:14:31.142375+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:30:00.495418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005256073,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4711,"num_read_kb":42462,"num_write":705,"num_write_kb":9831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":558,"num_bytes_recovered":6445817,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"179'706","reported_seq":8764,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.764017+0000","last_change":"2024-09-15T00:14:18.044374+0000","last_active":"2024-09-15T00:14:31.764017+0000","last_peered":"2024-09-15T00:14:31.764017+0000","last_clean":"2024-09-15T00:14:31.764017+0000","last_became_active":"2024-09-15T00:14:18.042710+0000","last_became_peered":"2024-09-15T00:14:18.042710+0000","last_unstale":"2024-09-15T00:14:31.764017+0000","last_undegraded":"2024-09-15T00:14:31.764017+0000","last_fullsized":"2024-09-15T00:14:31.764017+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:52:32.313688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014749780000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8102,"num_read_kb":40115,"num_write":776,"num_write_kb":15981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":4986328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'679","reported_seq":5549,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142315+0000","last_change":"2024-09-15T00:14:20.037042+0000","last_active":"2024-09-15T00:14:31.142315+0000","last_peered":"2024-09-15T00:14:31.142315+0000","last_clean":"2024-09-15T00:14:31.142315+0000","last_became_active":"2024-09-15T00:14:20.034657+0000","last_became_peered":"2024-09-15T00:14:20.034657+0000","last_unstale":"2024-09-15T00:14:31.142315+0000","last_undegraded":"2024-09-15T00:14:31.142315+0000","last_fullsized":"2024-09-15T00:14:31.142315+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:52:39.383067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002708182,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5548,"num_read_kb":45663,"num_write":634,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":12460957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"176'1032","reported_seq":8430,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762889+0000","last_change":"2024-09-15T00:14:08.079912+0000","last_active":"2024-09-15T00:14:31.762889+0000","last_peered":"2024-09-15T00:14:31.762889+0000","last_clean":"2024-09-15T00:14:31.762889+0000","last_became_active":"2024-09-15T00:13:50.031768+0000","last_became_peered":"2024-09-15T00:13:50.031768+0000","last_unstale":"2024-09-15T00:14:31.762889+0000","last_undegraded":"2024-09-15T00:14:31.762889+0000","last_fullsized":"2024-09-15T00:14:31.762889+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1032,"log_dups_size":0,"ondisk_log_size":1032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:21.539927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001158467,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8890,"num_read_kb":39257,"num_write":1264,"num_write_kb":20613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":23803494,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"179'667","reported_seq":6954,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502804+0000","last_change":"2024-09-15T00:14:08.088641+0000","last_active":"2024-09-15T00:14:31.502804+0000","last_peered":"2024-09-15T00:14:31.502804+0000","last_clean":"2024-09-15T00:14:31.502804+0000","last_became_active":"2024-09-15T00:13:50.025342+0000","last_became_peered":"2024-09-15T00:13:50.025342+0000","last_unstale":"2024-09-15T00:14:31.502804+0000","last_undegraded":"2024-09-15T00:14:31.502804+0000","last_fullsized":"2024-09-15T00:14:31.502804+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:50:45.634693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071033100000000003,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7596,"num_read_kb":45634,"num_write":737,"num_write_kb":14836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":19127384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'1079","reported_seq":7470,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142064+0000","last_change":"2024-09-15T00:14:24.095829+0000","last_active":"2024-09-15T00:14:31.142064+0000","last_peered":"2024-09-15T00:14:31.142064+0000","last_clean":"2024-09-15T00:14:31.142064+0000","last_became_active":"2024-09-15T00:14:24.093937+0000","last_became_peered":"2024-09-15T00:14:24.093937+0000","last_unstale":"2024-09-15T00:14:31.142064+0000","last_undegraded":"2024-09-15T00:14:31.142064+0000","last_fullsized":"2024-09-15T00:14:31.142064+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1079,"log_dups_size":0,"ondisk_log_size":1079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:10:03.817976+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.016604621,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":9874,"num_read_kb":58461,"num_write":1482,"num_write_kb":25008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":6083388,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":403,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470173+0000","last_change":"2024-09-15T00:13:46.030592+0000","last_active":"2024-09-15T00:14:31.470173+0000","last_peered":"2024-09-15T00:14:31.470173+0000","last_clean":"2024-09-15T00:14:31.470173+0000","last_became_active":"2024-09-15T00:13:46.030230+0000","last_became_peered":"2024-09-15T00:13:46.030230+0000","last_unstale":"2024-09-15T00:14:31.470173+0000","last_undegraded":"2024-09-15T00:14:31.470173+0000","last_fullsized":"2024-09-15T00:14:31.470173+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:26:48.293426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'715","reported_seq":7972,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142190+0000","last_change":"2024-09-15T00:14:18.046157+0000","last_active":"2024-09-15T00:14:31.142190+0000","last_peered":"2024-09-15T00:14:31.142190+0000","last_clean":"2024-09-15T00:14:31.142190+0000","last_became_active":"2024-09-15T00:14:18.043887+0000","last_became_peered":"2024-09-15T00:14:18.043887+0000","last_unstale":"2024-09-15T00:14:31.142190+0000","last_undegraded":"2024-09-15T00:14:31.142190+0000","last_fullsized":"2024-09-15T00:14:31.142190+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:22:24.520772+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00086129200000000005,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6781,"num_read_kb":42484,"num_write":652,"num_write_kb":25730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":26193256,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142568+0000","last_change":"2024-09-15T00:14:16.040721+0000","last_active":"2024-09-15T00:14:31.142568+0000","last_peered":"2024-09-15T00:14:31.142568+0000","last_clean":"2024-09-15T00:14:31.142568+0000","last_became_active":"2024-09-15T00:14:16.040306+0000","last_became_peered":"2024-09-15T00:14:16.040306+0000","last_unstale":"2024-09-15T00:14:31.142568+0000","last_undegraded":"2024-09-15T00:14:31.142568+0000","last_fullsized":"2024-09-15T00:14:31.142568+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:54:01.189526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'731","reported_seq":9397,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763223+0000","last_change":"2024-09-15T00:14:08.081190+0000","last_active":"2024-09-15T00:14:31.763223+0000","last_peered":"2024-09-15T00:14:31.763223+0000","last_clean":"2024-09-15T00:14:31.763223+0000","last_became_active":"2024-09-15T00:13:49.218857+0000","last_became_peered":"2024-09-15T00:13:49.218857+0000","last_unstale":"2024-09-15T00:14:31.763223+0000","last_undegraded":"2024-09-15T00:14:31.763223+0000","last_fullsized":"2024-09-15T00:14:31.763223+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:04:29.276892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020637310000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":10066,"num_read_kb":57181,"num_write":720,"num_write_kb":17747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":15784649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":110,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502690+0000","last_change":"2024-09-15T00:14:15.038333+0000","last_active":"2024-09-15T00:14:31.502690+0000","last_peered":"2024-09-15T00:14:31.502690+0000","last_clean":"2024-09-15T00:14:31.502690+0000","last_became_active":"2024-09-15T00:14:15.037987+0000","last_became_peered":"2024-09-15T00:14:15.037987+0000","last_unstale":"2024-09-15T00:14:31.502690+0000","last_undegraded":"2024-09-15T00:14:31.502690+0000","last_fullsized":"2024-09-15T00:14:31.502690+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:27.770396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"177'774","reported_seq":12041,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142529+0000","last_change":"2024-09-15T00:14:20.036384+0000","last_active":"2024-09-15T00:14:31.142529+0000","last_peered":"2024-09-15T00:14:31.142529+0000","last_clean":"2024-09-15T00:14:31.142529+0000","last_became_active":"2024-09-15T00:14:20.033228+0000","last_became_peered":"2024-09-15T00:14:20.033228+0000","last_unstale":"2024-09-15T00:14:31.142529+0000","last_undegraded":"2024-09-15T00:14:31.142529+0000","last_fullsized":"2024-09-15T00:14:31.142529+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:36:37.663215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017650229999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12816,"num_read_kb":60614,"num_write":771,"num_write_kb":25261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1083,"num_bytes_recovered":41768720,"num_keys_recovered":205,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142653+0000","last_change":"2024-09-15T00:14:16.034807+0000","last_active":"2024-09-15T00:14:31.142653+0000","last_peered":"2024-09-15T00:14:31.142653+0000","last_clean":"2024-09-15T00:14:31.142653+0000","last_became_active":"2024-09-15T00:14:16.034489+0000","last_became_peered":"2024-09-15T00:14:16.034489+0000","last_unstale":"2024-09-15T00:14:31.142653+0000","last_undegraded":"2024-09-15T00:14:31.142653+0000","last_fullsized":"2024-09-15T00:14:31.142653+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:38:39.946046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"179'552","reported_seq":3247,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762929+0000","last_change":"2024-09-15T00:14:08.081714+0000","last_active":"2024-09-15T00:14:31.762929+0000","last_peered":"2024-09-15T00:14:31.762929+0000","last_clean":"2024-09-15T00:14:31.762929+0000","last_became_active":"2024-09-15T00:11:19.595633+0000","last_became_peered":"2024-09-15T00:11:19.595633+0000","last_unstale":"2024-09-15T00:14:31.762929+0000","last_undegraded":"2024-09-15T00:14:31.762929+0000","last_fullsized":"2024-09-15T00:14:31.762929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:24:30.596801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026087110000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3741,"num_read_kb":17273,"num_write":546,"num_write_kb":8984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"182'819","reported_seq":8159,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142046+0000","last_change":"2024-09-15T00:14:24.096290+0000","last_active":"2024-09-15T00:14:31.142046+0000","last_peered":"2024-09-15T00:14:31.142046+0000","last_clean":"2024-09-15T00:14:31.142046+0000","last_became_active":"2024-09-15T00:14:24.094819+0000","last_became_peered":"2024-09-15T00:14:24.094819+0000","last_unstale":"2024-09-15T00:14:31.142046+0000","last_undegraded":"2024-09-15T00:14:31.142046+0000","last_fullsized":"2024-09-15T00:14:31.142046+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:32:14.972007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090289999999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":7230,"num_read_kb":43219,"num_write":761,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":9273484,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":420,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762759+0000","last_change":"2024-09-15T00:14:16.050758+0000","last_active":"2024-09-15T00:14:31.762759+0000","last_peered":"2024-09-15T00:14:31.762759+0000","last_clean":"2024-09-15T00:14:31.762759+0000","last_became_active":"2024-09-15T00:14:16.046906+0000","last_became_peered":"2024-09-15T00:14:16.046906+0000","last_unstale":"2024-09-15T00:14:31.762759+0000","last_undegraded":"2024-09-15T00:14:31.762759+0000","last_fullsized":"2024-09-15T00:14:31.762759+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:49:39.277929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"179'696","reported_seq":7804,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142817+0000","last_change":"2024-09-15T00:14:19.050801+0000","last_active":"2024-09-15T00:14:31.142817+0000","last_peered":"2024-09-15T00:14:31.142817+0000","last_clean":"2024-09-15T00:14:31.142817+0000","last_became_active":"2024-09-15T00:14:19.048768+0000","last_became_peered":"2024-09-15T00:14:19.048768+0000","last_unstale":"2024-09-15T00:14:31.142817+0000","last_undegraded":"2024-09-15T00:14:31.142817+0000","last_fullsized":"2024-09-15T00:14:31.142817+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:42:10.166497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092066200000000004,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7368,"num_read_kb":33386,"num_write":726,"num_write_kb":14440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":31813799,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762886+0000","last_change":"2024-09-15T00:13:52.026234+0000","last_active":"2024-09-15T00:14:31.762886+0000","last_peered":"2024-09-15T00:14:31.762886+0000","last_clean":"2024-09-15T00:14:31.762886+0000","last_became_active":"2024-09-15T00:13:52.025864+0000","last_became_peered":"2024-09-15T00:13:52.025864+0000","last_unstale":"2024-09-15T00:14:31.762886+0000","last_undegraded":"2024-09-15T00:14:31.762886+0000","last_fullsized":"2024-09-15T00:14:31.762886+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:23:52.397810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"182'708","reported_seq":6729,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142024+0000","last_change":"2024-09-15T00:14:21.036152+0000","last_active":"2024-09-15T00:14:31.142024+0000","last_peered":"2024-09-15T00:14:31.142024+0000","last_clean":"2024-09-15T00:14:31.142024+0000","last_became_active":"2024-09-15T00:14:21.034477+0000","last_became_peered":"2024-09-15T00:14:21.034477+0000","last_unstale":"2024-09-15T00:14:31.142024+0000","last_undegraded":"2024-09-15T00:14:31.142024+0000","last_fullsized":"2024-09-15T00:14:31.142024+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:50.622032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075419699999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6724,"num_read_kb":49050,"num_write":759,"num_write_kb":22019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":6058433,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":114,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502620+0000","last_change":"2024-09-15T00:14:16.041987+0000","last_active":"2024-09-15T00:14:31.502620+0000","last_peered":"2024-09-15T00:14:31.502620+0000","last_clean":"2024-09-15T00:14:31.502620+0000","last_became_active":"2024-09-15T00:14:16.041672+0000","last_became_peered":"2024-09-15T00:14:16.041672+0000","last_unstale":"2024-09-15T00:14:31.502620+0000","last_undegraded":"2024-09-15T00:14:31.502620+0000","last_fullsized":"2024-09-15T00:14:31.502620+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:04:25.406297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"179'619","reported_seq":5394,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763692+0000","last_change":"2024-09-15T00:14:08.080584+0000","last_active":"2024-09-15T00:14:31.763692+0000","last_peered":"2024-09-15T00:14:31.763692+0000","last_clean":"2024-09-15T00:14:31.763692+0000","last_became_active":"2024-09-15T00:11:19.594528+0000","last_became_peered":"2024-09-15T00:11:19.594528+0000","last_unstale":"2024-09-15T00:14:31.763692+0000","last_undegraded":"2024-09-15T00:14:31.763692+0000","last_fullsized":"2024-09-15T00:14:31.763692+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:56:44.230264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001727704,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5349,"num_read_kb":42137,"num_write":591,"num_write_kb":17763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":437,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762777+0000","last_change":"2024-09-15T00:13:52.028546+0000","last_active":"2024-09-15T00:14:31.762777+0000","last_peered":"2024-09-15T00:14:31.762777+0000","last_clean":"2024-09-15T00:14:31.762777+0000","last_became_active":"2024-09-15T00:13:52.026457+0000","last_became_peered":"2024-09-15T00:13:52.026457+0000","last_unstale":"2024-09-15T00:14:31.762777+0000","last_undegraded":"2024-09-15T00:14:31.762777+0000","last_fullsized":"2024-09-15T00:14:31.762777+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:58:54.983326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":348,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502648+0000","last_change":"2024-09-15T00:14:18.046986+0000","last_active":"2024-09-15T00:14:31.502648+0000","last_peered":"2024-09-15T00:14:31.502648+0000","last_clean":"2024-09-15T00:14:31.502648+0000","last_became_active":"2024-09-15T00:14:18.046593+0000","last_became_peered":"2024-09-15T00:14:18.046593+0000","last_unstale":"2024-09-15T00:14:31.502648+0000","last_undegraded":"2024-09-15T00:14:31.502648+0000","last_fullsized":"2024-09-15T00:14:31.502648+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_clean_scrub_stamp":"2024-09-15T00:11:04.453672+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:59:10.044950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"179'844","reported_seq":8284,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.764021+0000","last_change":"2024-09-15T00:14:22.040149+0000","last_active":"2024-09-15T00:14:31.764021+0000","last_peered":"2024-09-15T00:14:31.764021+0000","last_clean":"2024-09-15T00:14:31.764021+0000","last_became_active":"2024-09-15T00:14:22.038746+0000","last_became_peered":"2024-09-15T00:14:22.038746+0000","last_unstale":"2024-09-15T00:14:31.764021+0000","last_undegraded":"2024-09-15T00:14:31.764021+0000","last_fullsized":"2024-09-15T00:14:31.764021+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:36:14.331079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012708680000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8015,"num_read_kb":57970,"num_write":928,"num_write_kb":24350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":14074270,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"179'1526","reported_seq":9116,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763808+0000","last_change":"2024-09-15T00:14:19.051081+0000","last_active":"2024-09-15T00:14:31.763808+0000","last_peered":"2024-09-15T00:14:31.763808+0000","last_clean":"2024-09-15T00:14:31.763808+0000","last_became_active":"2024-09-15T00:14:19.048942+0000","last_became_peered":"2024-09-15T00:14:19.048942+0000","last_unstale":"2024-09-15T00:14:31.763808+0000","last_undegraded":"2024-09-15T00:14:31.763808+0000","last_fullsized":"2024-09-15T00:14:31.763808+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1526,"log_dups_size":0,"ondisk_log_size":1526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:54:02.659520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061724499999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":14886,"num_read_kb":35520,"num_write":2067,"num_write_kb":12930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":409,"num_bytes_recovered":10512832,"num_keys_recovered":83,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"179'733","reported_seq":6627,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470085+0000","last_change":"2024-09-15T00:14:08.082753+0000","last_active":"2024-09-15T00:14:31.470085+0000","last_peered":"2024-09-15T00:14:31.470085+0000","last_clean":"2024-09-15T00:14:31.470085+0000","last_became_active":"2024-09-15T00:11:19.602625+0000","last_became_peered":"2024-09-15T00:11:19.602625+0000","last_unstale":"2024-09-15T00:14:31.470085+0000","last_undegraded":"2024-09-15T00:14:31.470085+0000","last_fullsized":"2024-09-15T00:14:31.470085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:24:41.939171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012377130000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6965,"num_read_kb":22637,"num_write":697,"num_write_kb":11622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"177'426","reported_seq":5524,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763453+0000","last_change":"2024-09-15T00:14:20.037949+0000","last_active":"2024-09-15T00:14:31.763453+0000","last_peered":"2024-09-15T00:14:31.763453+0000","last_clean":"2024-09-15T00:14:31.763453+0000","last_became_active":"2024-09-15T00:14:20.036368+0000","last_became_peered":"2024-09-15T00:14:20.036368+0000","last_unstale":"2024-09-15T00:14:31.763453+0000","last_undegraded":"2024-09-15T00:14:31.763453+0000","last_fullsized":"2024-09-15T00:14:31.763453+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:10:24.358861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087100100000000005,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3886,"num_read_kb":33280,"num_write":476,"num_write_kb":22691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":10092575,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'919","reported_seq":4934,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762892+0000","last_change":"2024-09-15T00:14:08.081241+0000","last_active":"2024-09-15T00:14:31.762892+0000","last_peered":"2024-09-15T00:14:31.762892+0000","last_clean":"2024-09-15T00:14:31.762892+0000","last_became_active":"2024-09-15T00:11:19.595088+0000","last_became_peered":"2024-09-15T00:11:19.595088+0000","last_unstale":"2024-09-15T00:14:31.762892+0000","last_undegraded":"2024-09-15T00:14:31.762892+0000","last_fullsized":"2024-09-15T00:14:31.762892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:47:04.219653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024006000000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7681,"num_read_kb":43201,"num_write":1093,"num_write_kb":15373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"177'985","reported_seq":6046,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502672+0000","last_change":"2024-09-15T00:14:08.088640+0000","last_active":"2024-09-15T00:14:31.502672+0000","last_peered":"2024-09-15T00:14:31.502672+0000","last_clean":"2024-09-15T00:14:31.502672+0000","last_became_active":"2024-09-15T00:13:51.028205+0000","last_became_peered":"2024-09-15T00:13:51.028205+0000","last_unstale":"2024-09-15T00:14:31.502672+0000","last_undegraded":"2024-09-15T00:14:31.502672+0000","last_fullsized":"2024-09-15T00:14:31.502672+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:13:15.922674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012164389999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7667,"num_read_kb":20584,"num_write":1269,"num_write_kb":5933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":1861870,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"168'1421","reported_seq":6068,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502758+0000","last_change":"2024-09-15T00:14:08.088029+0000","last_active":"2024-09-15T00:14:31.502758+0000","last_peered":"2024-09-15T00:14:31.502758+0000","last_clean":"2024-09-15T00:14:31.502758+0000","last_became_active":"2024-09-15T00:13:50.023348+0000","last_became_peered":"2024-09-15T00:13:50.023348+0000","last_unstale":"2024-09-15T00:14:31.502758+0000","last_undegraded":"2024-09-15T00:14:31.502758+0000","last_fullsized":"2024-09-15T00:14:31.502758+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1421,"log_dups_size":0,"ondisk_log_size":1421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:46:55.906750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001473739,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8919,"num_read_kb":45201,"num_write":2133,"num_write_kb":24448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":18482007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"176'516","reported_seq":6414,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502850+0000","last_change":"2024-09-15T00:14:08.086840+0000","last_active":"2024-09-15T00:14:31.502850+0000","last_peered":"2024-09-15T00:14:31.502850+0000","last_clean":"2024-09-15T00:14:31.502850+0000","last_became_active":"2024-09-15T00:13:49.218512+0000","last_became_peered":"2024-09-15T00:13:49.218512+0000","last_unstale":"2024-09-15T00:14:31.502850+0000","last_undegraded":"2024-09-15T00:14:31.502850+0000","last_fullsized":"2024-09-15T00:14:31.502850+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:27:37.063625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014419529999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5794,"num_read_kb":31109,"num_write":499,"num_write_kb":15000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5158884,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"182'731","reported_seq":7089,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470085+0000","last_change":"2024-09-15T00:14:19.055694+0000","last_active":"2024-09-15T00:14:31.470085+0000","last_peered":"2024-09-15T00:14:31.470085+0000","last_clean":"2024-09-15T00:14:31.470085+0000","last_became_active":"2024-09-15T00:14:19.054103+0000","last_became_peered":"2024-09-15T00:14:19.054103+0000","last_unstale":"2024-09-15T00:14:31.470085+0000","last_undegraded":"2024-09-15T00:14:31.470085+0000","last_fullsized":"2024-09-15T00:14:31.470085+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:41:29.549456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086791500000000003,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7073,"num_read_kb":31603,"num_write":836,"num_write_kb":14934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":8339863,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"176'596","reported_seq":4626,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502629+0000","last_change":"2024-09-15T00:14:08.086089+0000","last_active":"2024-09-15T00:14:31.502629+0000","last_peered":"2024-09-15T00:14:31.502629+0000","last_clean":"2024-09-15T00:14:31.502629+0000","last_became_active":"2024-09-15T00:13:49.218581+0000","last_became_peered":"2024-09-15T00:13:49.218581+0000","last_unstale":"2024-09-15T00:14:31.502629+0000","last_undegraded":"2024-09-15T00:14:31.502629+0000","last_fullsized":"2024-09-15T00:14:31.502629+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:17.388753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010536860000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4205,"num_read_kb":15404,"num_write":601,"num_write_kb":11097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":1517056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"173'674","reported_seq":7804,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.141963+0000","last_change":"2024-09-15T00:14:21.034205+0000","last_active":"2024-09-15T00:14:31.141963+0000","last_peered":"2024-09-15T00:14:31.141963+0000","last_clean":"2024-09-15T00:14:31.141963+0000","last_became_active":"2024-09-15T00:14:21.032233+0000","last_became_peered":"2024-09-15T00:14:21.032233+0000","last_unstale":"2024-09-15T00:14:31.141963+0000","last_undegraded":"2024-09-15T00:14:31.141963+0000","last_fullsized":"2024-09-15T00:14:31.141963+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:46:53.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089427500000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6790,"num_read_kb":50206,"num_write":784,"num_write_kb":25712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":5797342,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'1107","reported_seq":8225,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502722+0000","last_change":"2024-09-15T00:14:08.087888+0000","last_active":"2024-09-15T00:14:31.502722+0000","last_peered":"2024-09-15T00:14:31.502722+0000","last_clean":"2024-09-15T00:14:31.502722+0000","last_became_active":"2024-09-15T00:13:51.025771+0000","last_became_peered":"2024-09-15T00:13:51.025771+0000","last_unstale":"2024-09-15T00:14:31.502722+0000","last_undegraded":"2024-09-15T00:14:31.502722+0000","last_fullsized":"2024-09-15T00:14:31.502722+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:28:00.228817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001944035,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":10923,"num_read_kb":34402,"num_write":1383,"num_write_kb":19662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":27769124,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":521,"num_object_clones":0,"num_object_copies":1042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":521,"num_whiteouts":517,"num_read":240279,"num_read_kb":1281261,"num_write":28650,"num_write_kb":585223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10651,"num_bytes_recovered":390333624,"num_keys_recovered":726,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5367,"internal_metadata":0},"log_size":25207,"ondisk_log_size":25207,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":2361472,"data_compressed":25472,"data_compressed_allocated":147456,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87736,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284104,"statfs":{"total":96636764160,"available":96546922496,"internally_reserved":0,"allocated":2453504,"data_stored":1194007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51620,"kb_used_data":2380,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320220,"statfs":{"total":96636764160,"available":96583905280,"internally_reserved":0,"allocated":2437120,"data_stored":1188146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55836,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316004,"statfs":{"total":96636764160,"available":96579588096,"internally_reserved":0,"allocated":2490368,"data_stored":1784356,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542191,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77996,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293844,"statfs":{"total":96636764160,"available":96556896256,"internally_reserved":0,"allocated":2449408,"data_stored":1193988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47476,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324364,"statfs":{"total":96636764160,"available":96588148736,"internally_reserved":0,"allocated":2453504,"data_stored":1194007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":108,"seq":463856467990,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42588,"kb_used_data":2368,"kb_used_omap":6,"kb_used_meta":40185,"kb_avail":94329252,"statfs":{"total":96636764160,"available":96593154048,"internally_reserved":0,"allocated":2424832,"data_stored":1182304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6962,"internal_metadata":41149646},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55784,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316056,"statfs":{"total":96636764160,"available":96579641344,"internally_reserved":0,"allocated":2433024,"data_stored":1182342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78048,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293792,"statfs":{"total":96636764160,"available":96556843008,"internally_reserved":0,"allocated":2490368,"data_stored":1784356,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5367,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:14:35.794 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T00:14:35.794 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:14:36.069 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:14:36.070 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:14:36.081 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-15T00:14:34.327513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":525,"num_object_clones":0,"num_object_copies":1050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":525,"num_whiteouts":517,"num_read":240395,"num_read_kb":1281361,"num_write":28811,"num_write_kb":587641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10669,"num_bytes_recovered":392695172,"num_keys_recovered":726,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25327,"ondisk_log_size":25327,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":497084,"kb_used_data":19172,"kb_used_omap":17,"kb_used_meta":477678,"kb_avail":754477636,"statfs":{"total":773094113280,"available":772585099264,"internally_reserved":0,"allocated":19632128,"data_stored":10703506,"data_compressed":12736,"data_compressed_allocated":73728,"data_compressed_original":1179648,"omap_allocated":18117,"internal_metadata":489142587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001051"},"pg_stats":[{"pgid":"3.1f","version":"182'643","reported_seq":6819,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142321+0000","last_change":"2024-09-15T00:14:18.045712+0000","last_active":"2024-09-15T00:14:31.142321+0000","last_peered":"2024-09-15T00:14:31.142321+0000","last_clean":"2024-09-15T00:14:31.142321+0000","last_became_active":"2024-09-15T00:14:18.042449+0000","last_became_peered":"2024-09-15T00:14:18.042449+0000","last_unstale":"2024-09-15T00:14:31.142321+0000","last_undegraded":"2024-09-15T00:14:31.142321+0000","last_fullsized":"2024-09-15T00:14:31.142321+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:38:13.616525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011752869999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6942,"num_read_kb":26340,"num_write":697,"num_write_kb":18376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":5475412,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'632","reported_seq":7633,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763549+0000","last_change":"2024-09-15T00:14:08.080286+0000","last_active":"2024-09-15T00:14:31.763549+0000","last_peered":"2024-09-15T00:14:31.763549+0000","last_clean":"2024-09-15T00:14:31.763549+0000","last_became_active":"2024-09-15T00:11:19.598640+0000","last_became_peered":"2024-09-15T00:11:19.598640+0000","last_unstale":"2024-09-15T00:14:31.763549+0000","last_undegraded":"2024-09-15T00:14:31.763549+0000","last_fullsized":"2024-09-15T00:14:31.763549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:16:36.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016674789999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7565,"num_read_kb":39401,"num_write":737,"num_write_kb":10312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'698","reported_seq":4933,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762732+0000","last_change":"2024-09-15T00:14:08.080849+0000","last_active":"2024-09-15T00:14:31.762732+0000","last_peered":"2024-09-15T00:14:31.762732+0000","last_clean":"2024-09-15T00:14:31.762732+0000","last_became_active":"2024-09-15T00:11:19.595954+0000","last_became_peered":"2024-09-15T00:11:19.595954+0000","last_unstale":"2024-09-15T00:14:31.762732+0000","last_undegraded":"2024-09-15T00:14:31.762732+0000","last_fullsized":"2024-09-15T00:14:31.762732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:28.144497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020206820000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5583,"num_read_kb":45090,"num_write":773,"num_write_kb":31484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"182'674","reported_seq":5880,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762701+0000","last_change":"2024-09-15T00:14:08.080354+0000","last_active":"2024-09-15T00:14:31.762701+0000","last_peered":"2024-09-15T00:14:31.762701+0000","last_clean":"2024-09-15T00:14:31.762701+0000","last_became_active":"2024-09-15T00:11:19.602583+0000","last_became_peered":"2024-09-15T00:11:19.602583+0000","last_unstale":"2024-09-15T00:14:31.762701+0000","last_undegraded":"2024-09-15T00:14:31.762701+0000","last_fullsized":"2024-09-15T00:14:31.762701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:33:11.479982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00168294,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":5888,"num_read_kb":35219,"num_write":807,"num_write_kb":10221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'844","reported_seq":6971,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142108+0000","last_change":"2024-09-15T00:14:20.035839+0000","last_active":"2024-09-15T00:14:31.142108+0000","last_peered":"2024-09-15T00:14:31.142108+0000","last_clean":"2024-09-15T00:14:31.142108+0000","last_became_active":"2024-09-15T00:14:20.032662+0000","last_became_peered":"2024-09-15T00:14:20.032662+0000","last_unstale":"2024-09-15T00:14:31.142108+0000","last_undegraded":"2024-09-15T00:14:31.142108+0000","last_fullsized":"2024-09-15T00:14:31.142108+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:24:20.329228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001922899,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7801,"num_read_kb":34537,"num_write":909,"num_write_kb":14651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":10954706,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"179'783","reported_seq":7558,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.141955+0000","last_change":"2024-09-15T00:14:24.098641+0000","last_active":"2024-09-15T00:14:31.141955+0000","last_peered":"2024-09-15T00:14:31.141955+0000","last_clean":"2024-09-15T00:14:31.141955+0000","last_became_active":"2024-09-15T00:14:24.097321+0000","last_became_peered":"2024-09-15T00:14:24.097321+0000","last_unstale":"2024-09-15T00:14:31.141955+0000","last_undegraded":"2024-09-15T00:14:31.141955+0000","last_fullsized":"2024-09-15T00:14:31.141955+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:40:15.897865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012814980000000001,"stat_sum":{"num_bytes":19,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":8900,"num_read_kb":62621,"num_write":837,"num_write_kb":33377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":66497974,"num_keys_recovered":149,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"177'678","reported_seq":5626,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142375+0000","last_change":"2024-09-15T00:14:22.038154+0000","last_active":"2024-09-15T00:14:31.142375+0000","last_peered":"2024-09-15T00:14:31.142375+0000","last_clean":"2024-09-15T00:14:31.142375+0000","last_became_active":"2024-09-15T00:14:22.036152+0000","last_became_peered":"2024-09-15T00:14:22.036152+0000","last_unstale":"2024-09-15T00:14:31.142375+0000","last_undegraded":"2024-09-15T00:14:31.142375+0000","last_fullsized":"2024-09-15T00:14:31.142375+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:30:00.495418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005256073,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4711,"num_read_kb":42462,"num_write":705,"num_write_kb":9831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":558,"num_bytes_recovered":6445817,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"179'706","reported_seq":8764,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.764017+0000","last_change":"2024-09-15T00:14:18.044374+0000","last_active":"2024-09-15T00:14:31.764017+0000","last_peered":"2024-09-15T00:14:31.764017+0000","last_clean":"2024-09-15T00:14:31.764017+0000","last_became_active":"2024-09-15T00:14:18.042710+0000","last_became_peered":"2024-09-15T00:14:18.042710+0000","last_unstale":"2024-09-15T00:14:31.764017+0000","last_undegraded":"2024-09-15T00:14:31.764017+0000","last_fullsized":"2024-09-15T00:14:31.764017+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:52:32.313688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014749780000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8102,"num_read_kb":40115,"num_write":776,"num_write_kb":15981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":4986328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'679","reported_seq":5549,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142315+0000","last_change":"2024-09-15T00:14:20.037042+0000","last_active":"2024-09-15T00:14:31.142315+0000","last_peered":"2024-09-15T00:14:31.142315+0000","last_clean":"2024-09-15T00:14:31.142315+0000","last_became_active":"2024-09-15T00:14:20.034657+0000","last_became_peered":"2024-09-15T00:14:20.034657+0000","last_unstale":"2024-09-15T00:14:31.142315+0000","last_undegraded":"2024-09-15T00:14:31.142315+0000","last_fullsized":"2024-09-15T00:14:31.142315+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:52:39.383067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002708182,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5548,"num_read_kb":45663,"num_write":634,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":12460957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"176'1032","reported_seq":8430,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762889+0000","last_change":"2024-09-15T00:14:08.079912+0000","last_active":"2024-09-15T00:14:31.762889+0000","last_peered":"2024-09-15T00:14:31.762889+0000","last_clean":"2024-09-15T00:14:31.762889+0000","last_became_active":"2024-09-15T00:13:50.031768+0000","last_became_peered":"2024-09-15T00:13:50.031768+0000","last_unstale":"2024-09-15T00:14:31.762889+0000","last_undegraded":"2024-09-15T00:14:31.762889+0000","last_fullsized":"2024-09-15T00:14:31.762889+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1032,"log_dups_size":0,"ondisk_log_size":1032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:21.539927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001158467,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8890,"num_read_kb":39257,"num_write":1264,"num_write_kb":20613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":23803494,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"179'667","reported_seq":6954,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502804+0000","last_change":"2024-09-15T00:14:08.088641+0000","last_active":"2024-09-15T00:14:31.502804+0000","last_peered":"2024-09-15T00:14:31.502804+0000","last_clean":"2024-09-15T00:14:31.502804+0000","last_became_active":"2024-09-15T00:13:50.025342+0000","last_became_peered":"2024-09-15T00:13:50.025342+0000","last_unstale":"2024-09-15T00:14:31.502804+0000","last_undegraded":"2024-09-15T00:14:31.502804+0000","last_fullsized":"2024-09-15T00:14:31.502804+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:50:45.634693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071033100000000003,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7596,"num_read_kb":45634,"num_write":737,"num_write_kb":14836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":19127384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'1079","reported_seq":7470,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142064+0000","last_change":"2024-09-15T00:14:24.095829+0000","last_active":"2024-09-15T00:14:31.142064+0000","last_peered":"2024-09-15T00:14:31.142064+0000","last_clean":"2024-09-15T00:14:31.142064+0000","last_became_active":"2024-09-15T00:14:24.093937+0000","last_became_peered":"2024-09-15T00:14:24.093937+0000","last_unstale":"2024-09-15T00:14:31.142064+0000","last_undegraded":"2024-09-15T00:14:31.142064+0000","last_fullsized":"2024-09-15T00:14:31.142064+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1079,"log_dups_size":0,"ondisk_log_size":1079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:10:03.817976+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.016604621,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":9874,"num_read_kb":58461,"num_write":1482,"num_write_kb":25008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":6083388,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":403,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470173+0000","last_change":"2024-09-15T00:13:46.030592+0000","last_active":"2024-09-15T00:14:31.470173+0000","last_peered":"2024-09-15T00:14:31.470173+0000","last_clean":"2024-09-15T00:14:31.470173+0000","last_became_active":"2024-09-15T00:13:46.030230+0000","last_became_peered":"2024-09-15T00:13:46.030230+0000","last_unstale":"2024-09-15T00:14:31.470173+0000","last_undegraded":"2024-09-15T00:14:31.470173+0000","last_fullsized":"2024-09-15T00:14:31.470173+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:26:48.293426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'715","reported_seq":7972,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142190+0000","last_change":"2024-09-15T00:14:18.046157+0000","last_active":"2024-09-15T00:14:31.142190+0000","last_peered":"2024-09-15T00:14:31.142190+0000","last_clean":"2024-09-15T00:14:31.142190+0000","last_became_active":"2024-09-15T00:14:18.043887+0000","last_became_peered":"2024-09-15T00:14:18.043887+0000","last_unstale":"2024-09-15T00:14:31.142190+0000","last_undegraded":"2024-09-15T00:14:31.142190+0000","last_fullsized":"2024-09-15T00:14:31.142190+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:22:24.520772+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00086129200000000005,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6781,"num_read_kb":42484,"num_write":652,"num_write_kb":25730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":26193256,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142568+0000","last_change":"2024-09-15T00:14:16.040721+0000","last_active":"2024-09-15T00:14:31.142568+0000","last_peered":"2024-09-15T00:14:31.142568+0000","last_clean":"2024-09-15T00:14:31.142568+0000","last_became_active":"2024-09-15T00:14:16.040306+0000","last_became_peered":"2024-09-15T00:14:16.040306+0000","last_unstale":"2024-09-15T00:14:31.142568+0000","last_undegraded":"2024-09-15T00:14:31.142568+0000","last_fullsized":"2024-09-15T00:14:31.142568+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:54:01.189526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'731","reported_seq":9397,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763223+0000","last_change":"2024-09-15T00:14:08.081190+0000","last_active":"2024-09-15T00:14:31.763223+0000","last_peered":"2024-09-15T00:14:31.763223+0000","last_clean":"2024-09-15T00:14:31.763223+0000","last_became_active":"2024-09-15T00:13:49.218857+0000","last_became_peered":"2024-09-15T00:13:49.218857+0000","last_unstale":"2024-09-15T00:14:31.763223+0000","last_undegraded":"2024-09-15T00:14:31.763223+0000","last_fullsized":"2024-09-15T00:14:31.763223+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:04:29.276892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020637310000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":10066,"num_read_kb":57181,"num_write":720,"num_write_kb":17747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":15784649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":110,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502690+0000","last_change":"2024-09-15T00:14:15.038333+0000","last_active":"2024-09-15T00:14:31.502690+0000","last_peered":"2024-09-15T00:14:31.502690+0000","last_clean":"2024-09-15T00:14:31.502690+0000","last_became_active":"2024-09-15T00:14:15.037987+0000","last_became_peered":"2024-09-15T00:14:15.037987+0000","last_unstale":"2024-09-15T00:14:31.502690+0000","last_undegraded":"2024-09-15T00:14:31.502690+0000","last_fullsized":"2024-09-15T00:14:31.502690+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:27.770396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"177'774","reported_seq":12041,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142529+0000","last_change":"2024-09-15T00:14:20.036384+0000","last_active":"2024-09-15T00:14:31.142529+0000","last_peered":"2024-09-15T00:14:31.142529+0000","last_clean":"2024-09-15T00:14:31.142529+0000","last_became_active":"2024-09-15T00:14:20.033228+0000","last_became_peered":"2024-09-15T00:14:20.033228+0000","last_unstale":"2024-09-15T00:14:31.142529+0000","last_undegraded":"2024-09-15T00:14:31.142529+0000","last_fullsized":"2024-09-15T00:14:31.142529+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:36:37.663215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017650229999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12816,"num_read_kb":60614,"num_write":771,"num_write_kb":25261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1083,"num_bytes_recovered":41768720,"num_keys_recovered":205,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142653+0000","last_change":"2024-09-15T00:14:16.034807+0000","last_active":"2024-09-15T00:14:31.142653+0000","last_peered":"2024-09-15T00:14:31.142653+0000","last_clean":"2024-09-15T00:14:31.142653+0000","last_became_active":"2024-09-15T00:14:16.034489+0000","last_became_peered":"2024-09-15T00:14:16.034489+0000","last_unstale":"2024-09-15T00:14:31.142653+0000","last_undegraded":"2024-09-15T00:14:31.142653+0000","last_fullsized":"2024-09-15T00:14:31.142653+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:38:39.946046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"179'552","reported_seq":3247,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762929+0000","last_change":"2024-09-15T00:14:08.081714+0000","last_active":"2024-09-15T00:14:31.762929+0000","last_peered":"2024-09-15T00:14:31.762929+0000","last_clean":"2024-09-15T00:14:31.762929+0000","last_became_active":"2024-09-15T00:11:19.595633+0000","last_became_peered":"2024-09-15T00:11:19.595633+0000","last_unstale":"2024-09-15T00:14:31.762929+0000","last_undegraded":"2024-09-15T00:14:31.762929+0000","last_fullsized":"2024-09-15T00:14:31.762929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:24:30.596801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026087110000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3741,"num_read_kb":17273,"num_write":546,"num_write_kb":8984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"182'819","reported_seq":8159,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142046+0000","last_change":"2024-09-15T00:14:24.096290+0000","last_active":"2024-09-15T00:14:31.142046+0000","last_peered":"2024-09-15T00:14:31.142046+0000","last_clean":"2024-09-15T00:14:31.142046+0000","last_became_active":"2024-09-15T00:14:24.094819+0000","last_became_peered":"2024-09-15T00:14:24.094819+0000","last_unstale":"2024-09-15T00:14:31.142046+0000","last_undegraded":"2024-09-15T00:14:31.142046+0000","last_fullsized":"2024-09-15T00:14:31.142046+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:32:14.972007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090289999999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":7230,"num_read_kb":43219,"num_write":761,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":9273484,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":420,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762759+0000","last_change":"2024-09-15T00:14:16.050758+0000","last_active":"2024-09-15T00:14:31.762759+0000","last_peered":"2024-09-15T00:14:31.762759+0000","last_clean":"2024-09-15T00:14:31.762759+0000","last_became_active":"2024-09-15T00:14:16.046906+0000","last_became_peered":"2024-09-15T00:14:16.046906+0000","last_unstale":"2024-09-15T00:14:31.762759+0000","last_undegraded":"2024-09-15T00:14:31.762759+0000","last_fullsized":"2024-09-15T00:14:31.762759+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:49:39.277929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"179'696","reported_seq":7804,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142817+0000","last_change":"2024-09-15T00:14:19.050801+0000","last_active":"2024-09-15T00:14:31.142817+0000","last_peered":"2024-09-15T00:14:31.142817+0000","last_clean":"2024-09-15T00:14:31.142817+0000","last_became_active":"2024-09-15T00:14:19.048768+0000","last_became_peered":"2024-09-15T00:14:19.048768+0000","last_unstale":"2024-09-15T00:14:31.142817+0000","last_undegraded":"2024-09-15T00:14:31.142817+0000","last_fullsized":"2024-09-15T00:14:31.142817+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:42:10.166497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092066200000000004,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7368,"num_read_kb":33386,"num_write":726,"num_write_kb":14440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":31813799,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762886+0000","last_change":"2024-09-15T00:13:52.026234+0000","last_active":"2024-09-15T00:14:31.762886+0000","last_peered":"2024-09-15T00:14:31.762886+0000","last_clean":"2024-09-15T00:14:31.762886+0000","last_became_active":"2024-09-15T00:13:52.025864+0000","last_became_peered":"2024-09-15T00:13:52.025864+0000","last_unstale":"2024-09-15T00:14:31.762886+0000","last_undegraded":"2024-09-15T00:14:31.762886+0000","last_fullsized":"2024-09-15T00:14:31.762886+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:23:52.397810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"182'708","reported_seq":6729,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142024+0000","last_change":"2024-09-15T00:14:21.036152+0000","last_active":"2024-09-15T00:14:31.142024+0000","last_peered":"2024-09-15T00:14:31.142024+0000","last_clean":"2024-09-15T00:14:31.142024+0000","last_became_active":"2024-09-15T00:14:21.034477+0000","last_became_peered":"2024-09-15T00:14:21.034477+0000","last_unstale":"2024-09-15T00:14:31.142024+0000","last_undegraded":"2024-09-15T00:14:31.142024+0000","last_fullsized":"2024-09-15T00:14:31.142024+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:50.622032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075419699999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6724,"num_read_kb":49050,"num_write":759,"num_write_kb":22019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":6058433,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":114,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502620+0000","last_change":"2024-09-15T00:14:16.041987+0000","last_active":"2024-09-15T00:14:31.502620+0000","last_peered":"2024-09-15T00:14:31.502620+0000","last_clean":"2024-09-15T00:14:31.502620+0000","last_became_active":"2024-09-15T00:14:16.041672+0000","last_became_peered":"2024-09-15T00:14:16.041672+0000","last_unstale":"2024-09-15T00:14:31.502620+0000","last_undegraded":"2024-09-15T00:14:31.502620+0000","last_fullsized":"2024-09-15T00:14:31.502620+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:04:25.406297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"179'619","reported_seq":5394,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763692+0000","last_change":"2024-09-15T00:14:08.080584+0000","last_active":"2024-09-15T00:14:31.763692+0000","last_peered":"2024-09-15T00:14:31.763692+0000","last_clean":"2024-09-15T00:14:31.763692+0000","last_became_active":"2024-09-15T00:11:19.594528+0000","last_became_peered":"2024-09-15T00:11:19.594528+0000","last_unstale":"2024-09-15T00:14:31.763692+0000","last_undegraded":"2024-09-15T00:14:31.763692+0000","last_fullsized":"2024-09-15T00:14:31.763692+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:56:44.230264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001727704,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5349,"num_read_kb":42137,"num_write":591,"num_write_kb":17763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":437,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762777+0000","last_change":"2024-09-15T00:13:52.028546+0000","last_active":"2024-09-15T00:14:31.762777+0000","last_peered":"2024-09-15T00:14:31.762777+0000","last_clean":"2024-09-15T00:14:31.762777+0000","last_became_active":"2024-09-15T00:13:52.026457+0000","last_became_peered":"2024-09-15T00:13:52.026457+0000","last_unstale":"2024-09-15T00:14:31.762777+0000","last_undegraded":"2024-09-15T00:14:31.762777+0000","last_fullsized":"2024-09-15T00:14:31.762777+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:58:54.983326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":348,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502648+0000","last_change":"2024-09-15T00:14:18.046986+0000","last_active":"2024-09-15T00:14:31.502648+0000","last_peered":"2024-09-15T00:14:31.502648+0000","last_clean":"2024-09-15T00:14:31.502648+0000","last_became_active":"2024-09-15T00:14:18.046593+0000","last_became_peered":"2024-09-15T00:14:18.046593+0000","last_unstale":"2024-09-15T00:14:31.502648+0000","last_undegraded":"2024-09-15T00:14:31.502648+0000","last_fullsized":"2024-09-15T00:14:31.502648+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_clean_scrub_stamp":"2024-09-15T00:11:04.453672+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:59:10.044950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"179'844","reported_seq":8284,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.764021+0000","last_change":"2024-09-15T00:14:22.040149+0000","last_active":"2024-09-15T00:14:31.764021+0000","last_peered":"2024-09-15T00:14:31.764021+0000","last_clean":"2024-09-15T00:14:31.764021+0000","last_became_active":"2024-09-15T00:14:22.038746+0000","last_became_peered":"2024-09-15T00:14:22.038746+0000","last_unstale":"2024-09-15T00:14:31.764021+0000","last_undegraded":"2024-09-15T00:14:31.764021+0000","last_fullsized":"2024-09-15T00:14:31.764021+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:36:14.331079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012708680000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8015,"num_read_kb":57970,"num_write":928,"num_write_kb":24350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":14074270,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"179'1526","reported_seq":9116,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763808+0000","last_change":"2024-09-15T00:14:19.051081+0000","last_active":"2024-09-15T00:14:31.763808+0000","last_peered":"2024-09-15T00:14:31.763808+0000","last_clean":"2024-09-15T00:14:31.763808+0000","last_became_active":"2024-09-15T00:14:19.048942+0000","last_became_peered":"2024-09-15T00:14:19.048942+0000","last_unstale":"2024-09-15T00:14:31.763808+0000","last_undegraded":"2024-09-15T00:14:31.763808+0000","last_fullsized":"2024-09-15T00:14:31.763808+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1526,"log_dups_size":0,"ondisk_log_size":1526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:54:02.659520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061724499999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":14886,"num_read_kb":35520,"num_write":2067,"num_write_kb":12930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":409,"num_bytes_recovered":10512832,"num_keys_recovered":83,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"179'733","reported_seq":6627,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470085+0000","last_change":"2024-09-15T00:14:08.082753+0000","last_active":"2024-09-15T00:14:31.470085+0000","last_peered":"2024-09-15T00:14:31.470085+0000","last_clean":"2024-09-15T00:14:31.470085+0000","last_became_active":"2024-09-15T00:11:19.602625+0000","last_became_peered":"2024-09-15T00:11:19.602625+0000","last_unstale":"2024-09-15T00:14:31.470085+0000","last_undegraded":"2024-09-15T00:14:31.470085+0000","last_fullsized":"2024-09-15T00:14:31.470085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:24:41.939171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012377130000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6965,"num_read_kb":22637,"num_write":697,"num_write_kb":11622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"177'426","reported_seq":5524,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763453+0000","last_change":"2024-09-15T00:14:20.037949+0000","last_active":"2024-09-15T00:14:31.763453+0000","last_peered":"2024-09-15T00:14:31.763453+0000","last_clean":"2024-09-15T00:14:31.763453+0000","last_became_active":"2024-09-15T00:14:20.036368+0000","last_became_peered":"2024-09-15T00:14:20.036368+0000","last_unstale":"2024-09-15T00:14:31.763453+0000","last_undegraded":"2024-09-15T00:14:31.763453+0000","last_fullsized":"2024-09-15T00:14:31.763453+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:10:24.358861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087100100000000005,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3886,"num_read_kb":33280,"num_write":476,"num_write_kb":22691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":10092575,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'919","reported_seq":4934,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762892+0000","last_change":"2024-09-15T00:14:08.081241+0000","last_active":"2024-09-15T00:14:31.762892+0000","last_peered":"2024-09-15T00:14:31.762892+0000","last_clean":"2024-09-15T00:14:31.762892+0000","last_became_active":"2024-09-15T00:11:19.595088+0000","last_became_peered":"2024-09-15T00:11:19.595088+0000","last_unstale":"2024-09-15T00:14:31.762892+0000","last_undegraded":"2024-09-15T00:14:31.762892+0000","last_fullsized":"2024-09-15T00:14:31.762892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:47:04.219653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024006000000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7681,"num_read_kb":43201,"num_write":1093,"num_write_kb":15373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"177'985","reported_seq":6046,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502672+0000","last_change":"2024-09-15T00:14:08.088640+0000","last_active":"2024-09-15T00:14:31.502672+0000","last_peered":"2024-09-15T00:14:31.502672+0000","last_clean":"2024-09-15T00:14:31.502672+0000","last_became_active":"2024-09-15T00:13:51.028205+0000","last_became_peered":"2024-09-15T00:13:51.028205+0000","last_unstale":"2024-09-15T00:14:31.502672+0000","last_undegraded":"2024-09-15T00:14:31.502672+0000","last_fullsized":"2024-09-15T00:14:31.502672+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:13:15.922674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012164389999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7667,"num_read_kb":20584,"num_write":1269,"num_write_kb":5933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":1861870,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"168'1421","reported_seq":6068,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502758+0000","last_change":"2024-09-15T00:14:08.088029+0000","last_active":"2024-09-15T00:14:31.502758+0000","last_peered":"2024-09-15T00:14:31.502758+0000","last_clean":"2024-09-15T00:14:31.502758+0000","last_became_active":"2024-09-15T00:13:50.023348+0000","last_became_peered":"2024-09-15T00:13:50.023348+0000","last_unstale":"2024-09-15T00:14:31.502758+0000","last_undegraded":"2024-09-15T00:14:31.502758+0000","last_fullsized":"2024-09-15T00:14:31.502758+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1421,"log_dups_size":0,"ondisk_log_size":1421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:46:55.906750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001473739,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8919,"num_read_kb":45201,"num_write":2133,"num_write_kb":24448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":18482007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"176'516","reported_seq":6414,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502850+0000","last_change":"2024-09-15T00:14:08.086840+0000","last_active":"2024-09-15T00:14:31.502850+0000","last_peered":"2024-09-15T00:14:31.502850+0000","last_clean":"2024-09-15T00:14:31.502850+0000","last_became_active":"2024-09-15T00:13:49.218512+0000","last_became_peered":"2024-09-15T00:13:49.218512+0000","last_unstale":"2024-09-15T00:14:31.502850+0000","last_undegraded":"2024-09-15T00:14:31.502850+0000","last_fullsized":"2024-09-15T00:14:31.502850+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:27:37.063625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014419529999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5794,"num_read_kb":31109,"num_write":499,"num_write_kb":15000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5158884,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"182'731","reported_seq":7089,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470085+0000","last_change":"2024-09-15T00:14:19.055694+0000","last_active":"2024-09-15T00:14:31.470085+0000","last_peered":"2024-09-15T00:14:31.470085+0000","last_clean":"2024-09-15T00:14:31.470085+0000","last_became_active":"2024-09-15T00:14:19.054103+0000","last_became_peered":"2024-09-15T00:14:19.054103+0000","last_unstale":"2024-09-15T00:14:31.470085+0000","last_undegraded":"2024-09-15T00:14:31.470085+0000","last_fullsized":"2024-09-15T00:14:31.470085+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:41:29.549456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086791500000000003,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7073,"num_read_kb":31603,"num_write":836,"num_write_kb":14934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":8339863,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"176'596","reported_seq":4626,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502629+0000","last_change":"2024-09-15T00:14:08.086089+0000","last_active":"2024-09-15T00:14:31.502629+0000","last_peered":"2024-09-15T00:14:31.502629+0000","last_clean":"2024-09-15T00:14:31.502629+0000","last_became_active":"2024-09-15T00:13:49.218581+0000","last_became_peered":"2024-09-15T00:13:49.218581+0000","last_unstale":"2024-09-15T00:14:31.502629+0000","last_undegraded":"2024-09-15T00:14:31.502629+0000","last_fullsized":"2024-09-15T00:14:31.502629+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:17.388753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010536860000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4205,"num_read_kb":15404,"num_write":601,"num_write_kb":11097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":1517056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"173'674","reported_seq":7804,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.141963+0000","last_change":"2024-09-15T00:14:21.034205+0000","last_active":"2024-09-15T00:14:31.141963+0000","last_peered":"2024-09-15T00:14:31.141963+0000","last_clean":"2024-09-15T00:14:31.141963+0000","last_became_active":"2024-09-15T00:14:21.032233+0000","last_became_peered":"2024-09-15T00:14:21.032233+0000","last_unstale":"2024-09-15T00:14:31.141963+0000","last_undegraded":"2024-09-15T00:14:31.141963+0000","last_fullsized":"2024-09-15T00:14:31.141963+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:46:53.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089427500000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6790,"num_read_kb":50206,"num_write":784,"num_write_kb":25712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":5797342,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'1107","reported_seq":8225,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502722+0000","last_change":"2024-09-15T00:14:08.087888+0000","last_active":"2024-09-15T00:14:31.502722+0000","last_peered":"2024-09-15T00:14:31.502722+0000","last_clean":"2024-09-15T00:14:31.502722+0000","last_became_active":"2024-09-15T00:13:51.025771+0000","last_became_peered":"2024-09-15T00:13:51.025771+0000","last_unstale":"2024-09-15T00:14:31.502722+0000","last_undegraded":"2024-09-15T00:14:31.502722+0000","last_fullsized":"2024-09-15T00:14:31.502722+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:28:00.228817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001944035,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":10923,"num_read_kb":34402,"num_write":1383,"num_write_kb":19662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":27769124,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":521,"num_object_clones":0,"num_object_copies":1042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":521,"num_whiteouts":517,"num_read":240279,"num_read_kb":1281261,"num_write":28650,"num_write_kb":585223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10651,"num_bytes_recovered":390333624,"num_keys_recovered":726,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5367,"internal_metadata":0},"log_size":25207,"ondisk_log_size":25207,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":2361472,"data_compressed":25472,"data_compressed_allocated":147456,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87736,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284104,"statfs":{"total":96636764160,"available":96546922496,"internally_reserved":0,"allocated":2453504,"data_stored":1194007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51620,"kb_used_data":2380,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320220,"statfs":{"total":96636764160,"available":96583905280,"internally_reserved":0,"allocated":2437120,"data_stored":1188146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55836,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316004,"statfs":{"total":96636764160,"available":96579588096,"internally_reserved":0,"allocated":2490368,"data_stored":1784356,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542191,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77996,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293844,"statfs":{"total":96636764160,"available":96556896256,"internally_reserved":0,"allocated":2449408,"data_stored":1193988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47476,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324364,"statfs":{"total":96636764160,"available":96588148736,"internally_reserved":0,"allocated":2453504,"data_stored":1194007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":108,"seq":463856467990,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42588,"kb_used_data":2368,"kb_used_omap":6,"kb_used_meta":40185,"kb_avail":94329252,"statfs":{"total":96636764160,"available":96593154048,"internally_reserved":0,"allocated":2424832,"data_stored":1182304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6962,"internal_metadata":41149646},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55784,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316056,"statfs":{"total":96636764160,"available":96579641344,"internally_reserved":0,"allocated":2433024,"data_stored":1182342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78048,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293792,"statfs":{"total":96636764160,"available":96556843008,"internally_reserved":0,"allocated":2490368,"data_stored":1784356,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5367,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:14:36.083 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T00:14:36.362 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:14:36.362 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":203,"fsid":"bad2533c-9c09-4af4-ab93-147afbacb705","created":"2024-09-15T00:10:50.203030+0000","modified":"2024-09-15T00:14:31.135695+0000","last_up_change":"2024-09-15T00:12:55.092989+0000","last_in_change":"2024-09-15T00:14:15.021696+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T00:11:04.326356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T00:11:08.005967+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-15T00:11:17.676102+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":53,"snap_epoch":180,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7337111f-0e02-490d-976b-3814a03371f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6825","nonce":2906515165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6827","nonce":2906515165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6831","nonce":2906515165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2906515165},{"type":"v1","addr":"172.21.15.171:6829","nonce":2906515165}]},"public_addr":"172.21.15.171:6825/2906515165","cluster_addr":"172.21.15.171:6827/2906515165","heartbeat_back_addr":"172.21.15.171:6831/2906515165","heartbeat_front_addr":"172.21.15.171:6829/2906515165","state":["exists","up"]},{"osd":1,"uuid":"64412633-4dd4-4b8d-b3f1-ed3c664ddf48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6817","nonce":3039831960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6819","nonce":3039831960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6823","nonce":3039831960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":3039831960},{"type":"v1","addr":"172.21.15.171:6821","nonce":3039831960}]},"public_addr":"172.21.15.171:6817/3039831960","cluster_addr":"172.21.15.171:6819/3039831960","heartbeat_back_addr":"172.21.15.171:6823/3039831960","heartbeat_front_addr":"172.21.15.171:6821/3039831960","state":["exists","up"]},{"osd":2,"uuid":"0300adee-5ce3-43ce-997b-2c0e970edbc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":79,"up_from":108,"up_thru":190,"down_at":81,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":213053077},{"type":"v1","addr":"172.21.15.171:6809","nonce":213053077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":213053077},{"type":"v1","addr":"172.21.15.171:6811","nonce":213053077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":213053077},{"type":"v1","addr":"172.21.15.171:6815","nonce":213053077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":213053077},{"type":"v1","addr":"172.21.15.171:6813","nonce":213053077}]},"public_addr":"172.21.15.171:6809/213053077","cluster_addr":"172.21.15.171:6811/213053077","heartbeat_back_addr":"172.21.15.171:6815/213053077","heartbeat_front_addr":"172.21.15.171:6813/213053077","state":["exists","up"]},{"osd":3,"uuid":"ec668801-9b8f-401a-829c-0b1f8d8f08fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6801","nonce":1819098683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6803","nonce":1819098683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6807","nonce":1819098683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1819098683},{"type":"v1","addr":"172.21.15.171:6805","nonce":1819098683}]},"public_addr":"172.21.15.171:6801/1819098683","cluster_addr":"172.21.15.171:6803/1819098683","heartbeat_back_addr":"172.21.15.171:6807/1819098683","heartbeat_front_addr":"172.21.15.171:6805/1819098683","state":["exists","up"]},{"osd":4,"uuid":"9cc48936-5644-44c6-84bb-06510e933e8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6801","nonce":2801626173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6803","nonce":2801626173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6807","nonce":2801626173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2801626173},{"type":"v1","addr":"172.21.15.184:6805","nonce":2801626173}]},"public_addr":"172.21.15.184:6801/2801626173","cluster_addr":"172.21.15.184:6803/2801626173","heartbeat_back_addr":"172.21.15.184:6807/2801626173","heartbeat_front_addr":"172.21.15.184:6805/2801626173","state":["exists","up"]},{"osd":5,"uuid":"f6c53297-9b00-476f-9be8-e76ff762b044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6809","nonce":3271066302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6811","nonce":3271066302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6815","nonce":3271066302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3271066302},{"type":"v1","addr":"172.21.15.184:6813","nonce":3271066302}]},"public_addr":"172.21.15.184:6809/3271066302","cluster_addr":"172.21.15.184:6811/3271066302","heartbeat_back_addr":"172.21.15.184:6815/3271066302","heartbeat_front_addr":"172.21.15.184:6813/3271066302","state":["exists","up"]},{"osd":6,"uuid":"737f08a6-2223-46b3-b9fe-6ae18b067ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6817","nonce":1903994256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6819","nonce":1903994256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6823","nonce":1903994256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1903994256},{"type":"v1","addr":"172.21.15.184:6821","nonce":1903994256}]},"public_addr":"172.21.15.184:6817/1903994256","cluster_addr":"172.21.15.184:6819/1903994256","heartbeat_back_addr":"172.21.15.184:6823/1903994256","heartbeat_front_addr":"172.21.15.184:6821/1903994256","state":["exists","up"]},{"osd":7,"uuid":"5a5fba64-9249-426d-814b-fd251e8aa910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6825","nonce":978565004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6827","nonce":978565004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6831","nonce":978565004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":978565004},{"type":"v1","addr":"172.21.15.184:6829","nonce":978565004}]},"public_addr":"172.21.15.184:6825/978565004","cluster_addr":"172.21.15.184:6827/978565004","heartbeat_back_addr":"172.21.15.184:6831/978565004","heartbeat_front_addr":"172.21.15.184:6829/978565004","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:59.070337+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.884683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.998882+0000","dead_epoch":81},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.656630+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.802744+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.567865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:58.739964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:10:59.565310+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.9","osds":[3,0]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:14:37.376 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T00:14:37.377 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T00:14:37.513 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:37.513 INFO:teuthology.orchestra.run.smithi171.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T00:14:37.513 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:37.524 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T00:14:37.802 INFO:teuthology.orchestra.run.smithi171.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T00:14:37.816 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T00:14:37.816 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T00:14:37.939 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:37.939 INFO:teuthology.orchestra.run.smithi171.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T00:14:37.939 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:37.950 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T00:14:38.229 INFO:teuthology.orchestra.run.smithi171.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T00:14:38.242 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T00:14:38.242 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T00:14:38.365 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:38.366 INFO:teuthology.orchestra.run.smithi171.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T00:14:38.366 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:38.376 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T00:14:38.653 INFO:teuthology.orchestra.run.smithi171.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T00:14:38.666 INFO:tasks.ceph:Scrubbing osd.3 2024-09-15T00:14:38.666 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-15T00:14:38.788 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:38.788 INFO:teuthology.orchestra.run.smithi171.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T00:14:38.788 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:38.799 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-15T00:14:39.080 INFO:teuthology.orchestra.run.smithi171.stderr:instructed osd(s) 3 to deep-scrub 2024-09-15T00:14:39.093 INFO:tasks.ceph:Scrubbing osd.4 2024-09-15T00:14:39.094 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-15T00:14:39.224 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:39.225 INFO:teuthology.orchestra.run.smithi171.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T00:14:39.225 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:39.235 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-15T00:14:39.512 INFO:teuthology.orchestra.run.smithi171.stderr:instructed osd(s) 4 to deep-scrub 2024-09-15T00:14:39.525 INFO:tasks.ceph:Scrubbing osd.5 2024-09-15T00:14:39.526 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-15T00:14:39.650 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:39.650 INFO:teuthology.orchestra.run.smithi171.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T00:14:39.650 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:39.660 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-15T00:14:39.939 INFO:teuthology.orchestra.run.smithi171.stderr:instructed osd(s) 5 to deep-scrub 2024-09-15T00:14:39.951 INFO:tasks.ceph:Scrubbing osd.6 2024-09-15T00:14:39.952 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-15T00:14:40.076 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:40.076 INFO:teuthology.orchestra.run.smithi171.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T00:14:40.076 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:40.087 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-15T00:14:40.374 INFO:teuthology.orchestra.run.smithi171.stderr:instructed osd(s) 6 to deep-scrub 2024-09-15T00:14:40.387 INFO:tasks.ceph:Scrubbing osd.7 2024-09-15T00:14:40.388 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-15T00:14:40.511 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-15T00:14:40.511 INFO:teuthology.orchestra.run.smithi171.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T00:14:40.512 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-15T00:14:40.522 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-15T00:14:40.802 INFO:teuthology.orchestra.run.smithi171.stderr:instructed osd(s) 7 to deep-scrub 2024-09-15T00:14:40.815 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:14:41.100 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:14:41.100 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:14:41.112 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-15T00:14:40.328604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":525,"num_object_clones":0,"num_object_copies":1050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":525,"num_whiteouts":517,"num_read":240395,"num_read_kb":1281361,"num_write":28811,"num_write_kb":587641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10669,"num_bytes_recovered":392695172,"num_keys_recovered":726,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25380,"ondisk_log_size":25380,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":497088,"kb_used_data":19232,"kb_used_omap":17,"kb_used_meta":477678,"kb_avail":754477632,"statfs":{"total":773094113280,"available":772585095168,"internally_reserved":0,"allocated":19693568,"data_stored":10732716,"data_compressed":12736,"data_compressed_allocated":73728,"data_compressed_original":1179648,"omap_allocated":18117,"internal_metadata":489142587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001848"},"pg_stats":[{"pgid":"3.1f","version":"203'662","reported_seq":6847,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:37.821737+0000","last_change":"2024-09-15T00:14:37.821827+0000","last_active":"2024-09-15T00:14:37.821737+0000","last_peered":"2024-09-15T00:14:37.821737+0000","last_clean":"2024-09-15T00:14:37.821737+0000","last_became_active":"2024-09-15T00:14:18.042449+0000","last_became_peered":"2024-09-15T00:14:18.042449+0000","last_unstale":"2024-09-15T00:14:37.821737+0000","last_undegraded":"2024-09-15T00:14:37.821737+0000","last_fullsized":"2024-09-15T00:14:37.821737+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'662","last_scrub_stamp":"2024-09-15T00:14:37.821645+0000","last_deep_scrub":"203'662","last_deep_scrub_stamp":"2024-09-15T00:14:37.821645+0000","last_clean_scrub_stamp":"2024-09-15T00:14:37.821645+0000","objects_scrubbed":19,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:18:15.963427+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0011752869999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6942,"num_read_kb":26340,"num_write":697,"num_write_kb":18376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":5475412,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'632","reported_seq":7634,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763589+0000","last_change":"2024-09-15T00:14:08.080286+0000","last_active":"2024-09-15T00:14:31.763589+0000","last_peered":"2024-09-15T00:14:31.763589+0000","last_clean":"2024-09-15T00:14:31.763589+0000","last_became_active":"2024-09-15T00:11:19.598640+0000","last_became_peered":"2024-09-15T00:11:19.598640+0000","last_unstale":"2024-09-15T00:14:31.763589+0000","last_undegraded":"2024-09-15T00:14:31.763589+0000","last_fullsized":"2024-09-15T00:14:31.763589+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016674789999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7565,"num_read_kb":39401,"num_write":737,"num_write_kb":10312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'698","reported_seq":4933,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762732+0000","last_change":"2024-09-15T00:14:08.080849+0000","last_active":"2024-09-15T00:14:31.762732+0000","last_peered":"2024-09-15T00:14:31.762732+0000","last_clean":"2024-09-15T00:14:31.762732+0000","last_became_active":"2024-09-15T00:11:19.595954+0000","last_became_peered":"2024-09-15T00:11:19.595954+0000","last_unstale":"2024-09-15T00:14:31.762732+0000","last_undegraded":"2024-09-15T00:14:31.762732+0000","last_fullsized":"2024-09-15T00:14:31.762732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:28.144497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020206820000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5583,"num_read_kb":45090,"num_write":773,"num_write_kb":31484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"182'674","reported_seq":5880,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762701+0000","last_change":"2024-09-15T00:14:08.080354+0000","last_active":"2024-09-15T00:14:31.762701+0000","last_peered":"2024-09-15T00:14:31.762701+0000","last_clean":"2024-09-15T00:14:31.762701+0000","last_became_active":"2024-09-15T00:11:19.602583+0000","last_became_peered":"2024-09-15T00:11:19.602583+0000","last_unstale":"2024-09-15T00:14:31.762701+0000","last_undegraded":"2024-09-15T00:14:31.762701+0000","last_fullsized":"2024-09-15T00:14:31.762701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:33:11.479982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00168294,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":5888,"num_read_kb":35219,"num_write":807,"num_write_kb":10221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'844","reported_seq":6972,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142152+0000","last_change":"2024-09-15T00:14:20.035839+0000","last_active":"2024-09-15T00:14:31.142152+0000","last_peered":"2024-09-15T00:14:31.142152+0000","last_clean":"2024-09-15T00:14:31.142152+0000","last_became_active":"2024-09-15T00:14:20.032662+0000","last_became_peered":"2024-09-15T00:14:20.032662+0000","last_unstale":"2024-09-15T00:14:31.142152+0000","last_undegraded":"2024-09-15T00:14:31.142152+0000","last_fullsized":"2024-09-15T00:14:31.142152+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001922899,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7801,"num_read_kb":34537,"num_write":909,"num_write_kb":14651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":10954706,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"179'783","reported_seq":7558,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.141955+0000","last_change":"2024-09-15T00:14:24.098641+0000","last_active":"2024-09-15T00:14:31.141955+0000","last_peered":"2024-09-15T00:14:31.141955+0000","last_clean":"2024-09-15T00:14:31.141955+0000","last_became_active":"2024-09-15T00:14:24.097321+0000","last_became_peered":"2024-09-15T00:14:24.097321+0000","last_unstale":"2024-09-15T00:14:31.141955+0000","last_undegraded":"2024-09-15T00:14:31.141955+0000","last_fullsized":"2024-09-15T00:14:31.141955+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:40:15.897865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012814980000000001,"stat_sum":{"num_bytes":19,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":8900,"num_read_kb":62621,"num_write":837,"num_write_kb":33377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":66497974,"num_keys_recovered":149,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"177'678","reported_seq":5627,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142405+0000","last_change":"2024-09-15T00:14:22.038154+0000","last_active":"2024-09-15T00:14:31.142405+0000","last_peered":"2024-09-15T00:14:31.142405+0000","last_clean":"2024-09-15T00:14:31.142405+0000","last_became_active":"2024-09-15T00:14:22.036152+0000","last_became_peered":"2024-09-15T00:14:22.036152+0000","last_unstale":"2024-09-15T00:14:31.142405+0000","last_undegraded":"2024-09-15T00:14:31.142405+0000","last_fullsized":"2024-09-15T00:14:31.142405+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005256073,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4711,"num_read_kb":42462,"num_write":705,"num_write_kb":9831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":558,"num_bytes_recovered":6445817,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"179'706","reported_seq":8765,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.764074+0000","last_change":"2024-09-15T00:14:18.044374+0000","last_active":"2024-09-15T00:14:31.764074+0000","last_peered":"2024-09-15T00:14:31.764074+0000","last_clean":"2024-09-15T00:14:31.764074+0000","last_became_active":"2024-09-15T00:14:18.042710+0000","last_became_peered":"2024-09-15T00:14:18.042710+0000","last_unstale":"2024-09-15T00:14:31.764074+0000","last_undegraded":"2024-09-15T00:14:31.764074+0000","last_fullsized":"2024-09-15T00:14:31.764074+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014749780000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8102,"num_read_kb":40115,"num_write":776,"num_write_kb":15981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":4986328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'679","reported_seq":5550,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142368+0000","last_change":"2024-09-15T00:14:20.037042+0000","last_active":"2024-09-15T00:14:31.142368+0000","last_peered":"2024-09-15T00:14:31.142368+0000","last_clean":"2024-09-15T00:14:31.142368+0000","last_became_active":"2024-09-15T00:14:20.034657+0000","last_became_peered":"2024-09-15T00:14:20.034657+0000","last_unstale":"2024-09-15T00:14:31.142368+0000","last_undegraded":"2024-09-15T00:14:31.142368+0000","last_fullsized":"2024-09-15T00:14:31.142368+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002708182,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5548,"num_read_kb":45663,"num_write":634,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":12460957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"176'1032","reported_seq":8430,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762889+0000","last_change":"2024-09-15T00:14:08.079912+0000","last_active":"2024-09-15T00:14:31.762889+0000","last_peered":"2024-09-15T00:14:31.762889+0000","last_clean":"2024-09-15T00:14:31.762889+0000","last_became_active":"2024-09-15T00:13:50.031768+0000","last_became_peered":"2024-09-15T00:13:50.031768+0000","last_unstale":"2024-09-15T00:14:31.762889+0000","last_undegraded":"2024-09-15T00:14:31.762889+0000","last_fullsized":"2024-09-15T00:14:31.762889+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1032,"log_dups_size":0,"ondisk_log_size":1032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:21.539927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001158467,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8890,"num_read_kb":39257,"num_write":1264,"num_write_kb":20613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":23803494,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"179'667","reported_seq":6954,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502804+0000","last_change":"2024-09-15T00:14:08.088641+0000","last_active":"2024-09-15T00:14:31.502804+0000","last_peered":"2024-09-15T00:14:31.502804+0000","last_clean":"2024-09-15T00:14:31.502804+0000","last_became_active":"2024-09-15T00:13:50.025342+0000","last_became_peered":"2024-09-15T00:13:50.025342+0000","last_unstale":"2024-09-15T00:14:31.502804+0000","last_undegraded":"2024-09-15T00:14:31.502804+0000","last_fullsized":"2024-09-15T00:14:31.502804+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:50:45.634693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071033100000000003,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7596,"num_read_kb":45634,"num_write":737,"num_write_kb":14836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":19127384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'1079","reported_seq":7470,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142064+0000","last_change":"2024-09-15T00:14:24.095829+0000","last_active":"2024-09-15T00:14:31.142064+0000","last_peered":"2024-09-15T00:14:31.142064+0000","last_clean":"2024-09-15T00:14:31.142064+0000","last_became_active":"2024-09-15T00:14:24.093937+0000","last_became_peered":"2024-09-15T00:14:24.093937+0000","last_unstale":"2024-09-15T00:14:31.142064+0000","last_undegraded":"2024-09-15T00:14:31.142064+0000","last_fullsized":"2024-09-15T00:14:31.142064+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1079,"log_dups_size":0,"ondisk_log_size":1079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:10:03.817976+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.016604621,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":9874,"num_read_kb":58461,"num_write":1482,"num_write_kb":25008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":6083388,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":403,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470173+0000","last_change":"2024-09-15T00:13:46.030592+0000","last_active":"2024-09-15T00:14:31.470173+0000","last_peered":"2024-09-15T00:14:31.470173+0000","last_clean":"2024-09-15T00:14:31.470173+0000","last_became_active":"2024-09-15T00:13:46.030230+0000","last_became_peered":"2024-09-15T00:13:46.030230+0000","last_unstale":"2024-09-15T00:14:31.470173+0000","last_undegraded":"2024-09-15T00:14:31.470173+0000","last_fullsized":"2024-09-15T00:14:31.470173+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:26:48.293426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"203'737","reported_seq":8003,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:38.812526+0000","last_change":"2024-09-15T00:14:38.812587+0000","last_active":"2024-09-15T00:14:38.812526+0000","last_peered":"2024-09-15T00:14:38.812526+0000","last_clean":"2024-09-15T00:14:38.812526+0000","last_became_active":"2024-09-15T00:14:18.043887+0000","last_became_peered":"2024-09-15T00:14:18.043887+0000","last_unstale":"2024-09-15T00:14:38.812526+0000","last_undegraded":"2024-09-15T00:14:38.812526+0000","last_fullsized":"2024-09-15T00:14:38.812526+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'737","last_scrub_stamp":"2024-09-15T00:14:38.812452+0000","last_deep_scrub":"203'737","last_deep_scrub_stamp":"2024-09-15T00:14:38.812452+0000","last_clean_scrub_stamp":"2024-09-15T00:14:38.812452+0000","objects_scrubbed":22,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:03:49.386711+0000","scrub_duration":16,"objects_trimmed":2,"snaptrim_duration":0.00086129200000000005,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6781,"num_read_kb":42484,"num_write":652,"num_write_kb":25730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":26193256,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142602+0000","last_change":"2024-09-15T00:14:16.040721+0000","last_active":"2024-09-15T00:14:31.142602+0000","last_peered":"2024-09-15T00:14:31.142602+0000","last_clean":"2024-09-15T00:14:31.142602+0000","last_became_active":"2024-09-15T00:14:16.040306+0000","last_became_peered":"2024-09-15T00:14:16.040306+0000","last_unstale":"2024-09-15T00:14:31.142602+0000","last_undegraded":"2024-09-15T00:14:31.142602+0000","last_fullsized":"2024-09-15T00:14:31.142602+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'731","reported_seq":9398,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763272+0000","last_change":"2024-09-15T00:14:08.081190+0000","last_active":"2024-09-15T00:14:31.763272+0000","last_peered":"2024-09-15T00:14:31.763272+0000","last_clean":"2024-09-15T00:14:31.763272+0000","last_became_active":"2024-09-15T00:13:49.218857+0000","last_became_peered":"2024-09-15T00:13:49.218857+0000","last_unstale":"2024-09-15T00:14:31.763272+0000","last_undegraded":"2024-09-15T00:14:31.763272+0000","last_fullsized":"2024-09-15T00:14:31.763272+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020637310000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":10066,"num_read_kb":57181,"num_write":720,"num_write_kb":17747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":15784649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":110,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502690+0000","last_change":"2024-09-15T00:14:15.038333+0000","last_active":"2024-09-15T00:14:31.502690+0000","last_peered":"2024-09-15T00:14:31.502690+0000","last_clean":"2024-09-15T00:14:31.502690+0000","last_became_active":"2024-09-15T00:14:15.037987+0000","last_became_peered":"2024-09-15T00:14:15.037987+0000","last_unstale":"2024-09-15T00:14:31.502690+0000","last_undegraded":"2024-09-15T00:14:31.502690+0000","last_fullsized":"2024-09-15T00:14:31.502690+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:27.770396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"177'774","reported_seq":12042,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142573+0000","last_change":"2024-09-15T00:14:20.036384+0000","last_active":"2024-09-15T00:14:31.142573+0000","last_peered":"2024-09-15T00:14:31.142573+0000","last_clean":"2024-09-15T00:14:31.142573+0000","last_became_active":"2024-09-15T00:14:20.033228+0000","last_became_peered":"2024-09-15T00:14:20.033228+0000","last_unstale":"2024-09-15T00:14:31.142573+0000","last_undegraded":"2024-09-15T00:14:31.142573+0000","last_fullsized":"2024-09-15T00:14:31.142573+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017650229999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12816,"num_read_kb":60614,"num_write":771,"num_write_kb":25261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1083,"num_bytes_recovered":41768720,"num_keys_recovered":205,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:39.033225+0000","last_change":"2024-09-15T00:14:39.033245+0000","last_active":"2024-09-15T00:14:39.033225+0000","last_peered":"2024-09-15T00:14:39.033225+0000","last_clean":"2024-09-15T00:14:39.033225+0000","last_became_active":"2024-09-15T00:14:16.034489+0000","last_became_peered":"2024-09-15T00:14:16.034489+0000","last_unstale":"2024-09-15T00:14:39.033225+0000","last_undegraded":"2024-09-15T00:14:39.033225+0000","last_fullsized":"2024-09-15T00:14:39.033225+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:14:39.033147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:14:39.033147+0000","last_clean_scrub_stamp":"2024-09-15T00:14:39.033147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:55:04.706104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"179'552","reported_seq":3247,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762929+0000","last_change":"2024-09-15T00:14:08.081714+0000","last_active":"2024-09-15T00:14:31.762929+0000","last_peered":"2024-09-15T00:14:31.762929+0000","last_clean":"2024-09-15T00:14:31.762929+0000","last_became_active":"2024-09-15T00:11:19.595633+0000","last_became_peered":"2024-09-15T00:11:19.595633+0000","last_unstale":"2024-09-15T00:14:31.762929+0000","last_undegraded":"2024-09-15T00:14:31.762929+0000","last_fullsized":"2024-09-15T00:14:31.762929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:24:30.596801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026087110000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3741,"num_read_kb":17273,"num_write":546,"num_write_kb":8984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"182'819","reported_seq":8159,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142046+0000","last_change":"2024-09-15T00:14:24.096290+0000","last_active":"2024-09-15T00:14:31.142046+0000","last_peered":"2024-09-15T00:14:31.142046+0000","last_clean":"2024-09-15T00:14:31.142046+0000","last_became_active":"2024-09-15T00:14:24.094819+0000","last_became_peered":"2024-09-15T00:14:24.094819+0000","last_unstale":"2024-09-15T00:14:31.142046+0000","last_undegraded":"2024-09-15T00:14:31.142046+0000","last_fullsized":"2024-09-15T00:14:31.142046+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:32:14.972007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090289999999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":7230,"num_read_kb":43219,"num_write":761,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":9273484,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":420,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762759+0000","last_change":"2024-09-15T00:14:16.050758+0000","last_active":"2024-09-15T00:14:31.762759+0000","last_peered":"2024-09-15T00:14:31.762759+0000","last_clean":"2024-09-15T00:14:31.762759+0000","last_became_active":"2024-09-15T00:14:16.046906+0000","last_became_peered":"2024-09-15T00:14:16.046906+0000","last_unstale":"2024-09-15T00:14:31.762759+0000","last_undegraded":"2024-09-15T00:14:31.762759+0000","last_fullsized":"2024-09-15T00:14:31.762759+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:49:39.277929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"179'696","reported_seq":7805,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142857+0000","last_change":"2024-09-15T00:14:19.050801+0000","last_active":"2024-09-15T00:14:31.142857+0000","last_peered":"2024-09-15T00:14:31.142857+0000","last_clean":"2024-09-15T00:14:31.142857+0000","last_became_active":"2024-09-15T00:14:19.048768+0000","last_became_peered":"2024-09-15T00:14:19.048768+0000","last_unstale":"2024-09-15T00:14:31.142857+0000","last_undegraded":"2024-09-15T00:14:31.142857+0000","last_fullsized":"2024-09-15T00:14:31.142857+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092066200000000004,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7368,"num_read_kb":33386,"num_write":726,"num_write_kb":14440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":31813799,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:39.019989+0000","last_change":"2024-09-15T00:14:39.020009+0000","last_active":"2024-09-15T00:14:39.019989+0000","last_peered":"2024-09-15T00:14:39.019989+0000","last_clean":"2024-09-15T00:14:39.019989+0000","last_became_active":"2024-09-15T00:13:52.025864+0000","last_became_peered":"2024-09-15T00:13:52.025864+0000","last_unstale":"2024-09-15T00:14:39.019989+0000","last_undegraded":"2024-09-15T00:14:39.019989+0000","last_fullsized":"2024-09-15T00:14:39.019989+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:14:39.019904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:14:39.019904+0000","last_clean_scrub_stamp":"2024-09-15T00:14:39.019904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:47:08.654282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"182'708","reported_seq":6730,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142063+0000","last_change":"2024-09-15T00:14:21.036152+0000","last_active":"2024-09-15T00:14:31.142063+0000","last_peered":"2024-09-15T00:14:31.142063+0000","last_clean":"2024-09-15T00:14:31.142063+0000","last_became_active":"2024-09-15T00:14:21.034477+0000","last_became_peered":"2024-09-15T00:14:21.034477+0000","last_unstale":"2024-09-15T00:14:31.142063+0000","last_undegraded":"2024-09-15T00:14:31.142063+0000","last_fullsized":"2024-09-15T00:14:31.142063+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075419699999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6724,"num_read_kb":49050,"num_write":759,"num_write_kb":22019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":6058433,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":114,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502620+0000","last_change":"2024-09-15T00:14:16.041987+0000","last_active":"2024-09-15T00:14:31.502620+0000","last_peered":"2024-09-15T00:14:31.502620+0000","last_clean":"2024-09-15T00:14:31.502620+0000","last_became_active":"2024-09-15T00:14:16.041672+0000","last_became_peered":"2024-09-15T00:14:16.041672+0000","last_unstale":"2024-09-15T00:14:31.502620+0000","last_undegraded":"2024-09-15T00:14:31.502620+0000","last_fullsized":"2024-09-15T00:14:31.502620+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:04:25.406297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"203'631","reported_seq":5415,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:39.223059+0000","last_change":"2024-09-15T00:14:39.223099+0000","last_active":"2024-09-15T00:14:39.223059+0000","last_peered":"2024-09-15T00:14:39.223059+0000","last_clean":"2024-09-15T00:14:39.223059+0000","last_became_active":"2024-09-15T00:11:19.594528+0000","last_became_peered":"2024-09-15T00:11:19.594528+0000","last_unstale":"2024-09-15T00:14:39.223059+0000","last_undegraded":"2024-09-15T00:14:39.223059+0000","last_fullsized":"2024-09-15T00:14:39.223059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'631","last_scrub_stamp":"2024-09-15T00:14:39.222988+0000","last_deep_scrub":"203'631","last_deep_scrub_stamp":"2024-09-15T00:14:39.222988+0000","last_clean_scrub_stamp":"2024-09-15T00:14:39.222988+0000","objects_scrubbed":12,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:50:29.558130+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001727704,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5349,"num_read_kb":42137,"num_write":591,"num_write_kb":17763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":437,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762777+0000","last_change":"2024-09-15T00:13:52.028546+0000","last_active":"2024-09-15T00:14:31.762777+0000","last_peered":"2024-09-15T00:14:31.762777+0000","last_clean":"2024-09-15T00:14:31.762777+0000","last_became_active":"2024-09-15T00:13:52.026457+0000","last_became_peered":"2024-09-15T00:13:52.026457+0000","last_unstale":"2024-09-15T00:14:31.762777+0000","last_undegraded":"2024-09-15T00:14:31.762777+0000","last_fullsized":"2024-09-15T00:14:31.762777+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:08.518989+0000","last_clean_scrub_stamp":"2024-09-15T00:11:08.518989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:58:54.983326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":348,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502648+0000","last_change":"2024-09-15T00:14:18.046986+0000","last_active":"2024-09-15T00:14:31.502648+0000","last_peered":"2024-09-15T00:14:31.502648+0000","last_clean":"2024-09-15T00:14:31.502648+0000","last_became_active":"2024-09-15T00:14:18.046593+0000","last_became_peered":"2024-09-15T00:14:18.046593+0000","last_unstale":"2024-09-15T00:14:31.502648+0000","last_undegraded":"2024-09-15T00:14:31.502648+0000","last_fullsized":"2024-09-15T00:14:31.502648+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:04.453672+0000","last_clean_scrub_stamp":"2024-09-15T00:11:04.453672+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:59:10.044950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"179'844","reported_seq":8285,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.764065+0000","last_change":"2024-09-15T00:14:22.040149+0000","last_active":"2024-09-15T00:14:31.764065+0000","last_peered":"2024-09-15T00:14:31.764065+0000","last_clean":"2024-09-15T00:14:31.764065+0000","last_became_active":"2024-09-15T00:14:22.038746+0000","last_became_peered":"2024-09-15T00:14:22.038746+0000","last_unstale":"2024-09-15T00:14:31.764065+0000","last_undegraded":"2024-09-15T00:14:31.764065+0000","last_fullsized":"2024-09-15T00:14:31.764065+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012708680000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8015,"num_read_kb":57970,"num_write":928,"num_write_kb":24350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":14074270,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"179'1526","reported_seq":9117,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763850+0000","last_change":"2024-09-15T00:14:19.051081+0000","last_active":"2024-09-15T00:14:31.763850+0000","last_peered":"2024-09-15T00:14:31.763850+0000","last_clean":"2024-09-15T00:14:31.763850+0000","last_became_active":"2024-09-15T00:14:19.048942+0000","last_became_peered":"2024-09-15T00:14:19.048942+0000","last_unstale":"2024-09-15T00:14:31.763850+0000","last_undegraded":"2024-09-15T00:14:31.763850+0000","last_fullsized":"2024-09-15T00:14:31.763850+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1526,"log_dups_size":0,"ondisk_log_size":1526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061724499999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":14886,"num_read_kb":35520,"num_write":2067,"num_write_kb":12930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":409,"num_bytes_recovered":10512832,"num_keys_recovered":83,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"179'733","reported_seq":6627,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470085+0000","last_change":"2024-09-15T00:14:08.082753+0000","last_active":"2024-09-15T00:14:31.470085+0000","last_peered":"2024-09-15T00:14:31.470085+0000","last_clean":"2024-09-15T00:14:31.470085+0000","last_became_active":"2024-09-15T00:11:19.602625+0000","last_became_peered":"2024-09-15T00:11:19.602625+0000","last_unstale":"2024-09-15T00:14:31.470085+0000","last_undegraded":"2024-09-15T00:14:31.470085+0000","last_fullsized":"2024-09-15T00:14:31.470085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:24:41.939171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012377130000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6965,"num_read_kb":22637,"num_write":697,"num_write_kb":11622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"177'426","reported_seq":5525,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.763494+0000","last_change":"2024-09-15T00:14:20.037949+0000","last_active":"2024-09-15T00:14:31.763494+0000","last_peered":"2024-09-15T00:14:31.763494+0000","last_clean":"2024-09-15T00:14:31.763494+0000","last_became_active":"2024-09-15T00:14:20.036368+0000","last_became_peered":"2024-09-15T00:14:20.036368+0000","last_unstale":"2024-09-15T00:14:31.763494+0000","last_undegraded":"2024-09-15T00:14:31.763494+0000","last_fullsized":"2024-09-15T00:14:31.763494+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087100100000000005,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3886,"num_read_kb":33280,"num_write":476,"num_write_kb":22691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":10092575,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'919","reported_seq":4934,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.762892+0000","last_change":"2024-09-15T00:14:08.081241+0000","last_active":"2024-09-15T00:14:31.762892+0000","last_peered":"2024-09-15T00:14:31.762892+0000","last_clean":"2024-09-15T00:14:31.762892+0000","last_became_active":"2024-09-15T00:11:19.595088+0000","last_became_peered":"2024-09-15T00:11:19.595088+0000","last_unstale":"2024-09-15T00:14:31.762892+0000","last_undegraded":"2024-09-15T00:14:31.762892+0000","last_fullsized":"2024-09-15T00:14:31.762892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:47:04.219653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024006000000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7681,"num_read_kb":43201,"num_write":1093,"num_write_kb":15373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"177'985","reported_seq":6046,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502672+0000","last_change":"2024-09-15T00:14:08.088640+0000","last_active":"2024-09-15T00:14:31.502672+0000","last_peered":"2024-09-15T00:14:31.502672+0000","last_clean":"2024-09-15T00:14:31.502672+0000","last_became_active":"2024-09-15T00:13:51.028205+0000","last_became_peered":"2024-09-15T00:13:51.028205+0000","last_unstale":"2024-09-15T00:14:31.502672+0000","last_undegraded":"2024-09-15T00:14:31.502672+0000","last_fullsized":"2024-09-15T00:14:31.502672+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:13:15.922674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012164389999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7667,"num_read_kb":20584,"num_write":1269,"num_write_kb":5933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":1861870,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"168'1421","reported_seq":6068,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502758+0000","last_change":"2024-09-15T00:14:08.088029+0000","last_active":"2024-09-15T00:14:31.502758+0000","last_peered":"2024-09-15T00:14:31.502758+0000","last_clean":"2024-09-15T00:14:31.502758+0000","last_became_active":"2024-09-15T00:13:50.023348+0000","last_became_peered":"2024-09-15T00:13:50.023348+0000","last_unstale":"2024-09-15T00:14:31.502758+0000","last_undegraded":"2024-09-15T00:14:31.502758+0000","last_fullsized":"2024-09-15T00:14:31.502758+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1421,"log_dups_size":0,"ondisk_log_size":1421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:46:55.906750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001473739,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8919,"num_read_kb":45201,"num_write":2133,"num_write_kb":24448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":18482007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"176'516","reported_seq":6414,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502850+0000","last_change":"2024-09-15T00:14:08.086840+0000","last_active":"2024-09-15T00:14:31.502850+0000","last_peered":"2024-09-15T00:14:31.502850+0000","last_clean":"2024-09-15T00:14:31.502850+0000","last_became_active":"2024-09-15T00:13:49.218512+0000","last_became_peered":"2024-09-15T00:13:49.218512+0000","last_unstale":"2024-09-15T00:14:31.502850+0000","last_undegraded":"2024-09-15T00:14:31.502850+0000","last_fullsized":"2024-09-15T00:14:31.502850+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:27:37.063625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014419529999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5794,"num_read_kb":31109,"num_write":499,"num_write_kb":15000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5158884,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"182'731","reported_seq":7089,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.470085+0000","last_change":"2024-09-15T00:14:19.055694+0000","last_active":"2024-09-15T00:14:31.470085+0000","last_peered":"2024-09-15T00:14:31.470085+0000","last_clean":"2024-09-15T00:14:31.470085+0000","last_became_active":"2024-09-15T00:14:19.054103+0000","last_became_peered":"2024-09-15T00:14:19.054103+0000","last_unstale":"2024-09-15T00:14:31.470085+0000","last_undegraded":"2024-09-15T00:14:31.470085+0000","last_fullsized":"2024-09-15T00:14:31.470085+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:41:29.549456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086791500000000003,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7073,"num_read_kb":31603,"num_write":836,"num_write_kb":14934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":8339863,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"176'596","reported_seq":4626,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502629+0000","last_change":"2024-09-15T00:14:08.086089+0000","last_active":"2024-09-15T00:14:31.502629+0000","last_peered":"2024-09-15T00:14:31.502629+0000","last_clean":"2024-09-15T00:14:31.502629+0000","last_became_active":"2024-09-15T00:13:49.218581+0000","last_became_peered":"2024-09-15T00:13:49.218581+0000","last_unstale":"2024-09-15T00:14:31.502629+0000","last_undegraded":"2024-09-15T00:14:31.502629+0000","last_fullsized":"2024-09-15T00:14:31.502629+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:17.388753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010536860000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4205,"num_read_kb":15404,"num_write":601,"num_write_kb":11097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":1517056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"173'674","reported_seq":7805,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.142020+0000","last_change":"2024-09-15T00:14:21.034205+0000","last_active":"2024-09-15T00:14:31.142020+0000","last_peered":"2024-09-15T00:14:31.142020+0000","last_clean":"2024-09-15T00:14:31.142020+0000","last_became_active":"2024-09-15T00:14:21.032233+0000","last_became_peered":"2024-09-15T00:14:21.032233+0000","last_unstale":"2024-09-15T00:14:31.142020+0000","last_undegraded":"2024-09-15T00:14:31.142020+0000","last_fullsized":"2024-09-15T00:14:31.142020+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089427500000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6790,"num_read_kb":50206,"num_write":784,"num_write_kb":25712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":5797342,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'1107","reported_seq":8225,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-15T00:14:31.502722+0000","last_change":"2024-09-15T00:14:08.087888+0000","last_active":"2024-09-15T00:14:31.502722+0000","last_peered":"2024-09-15T00:14:31.502722+0000","last_clean":"2024-09-15T00:14:31.502722+0000","last_became_active":"2024-09-15T00:13:51.025771+0000","last_became_peered":"2024-09-15T00:13:51.025771+0000","last_unstale":"2024-09-15T00:14:31.502722+0000","last_undegraded":"2024-09-15T00:14:31.502722+0000","last_fullsized":"2024-09-15T00:14:31.502722+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:18.578266+0000","last_clean_scrub_stamp":"2024-09-15T00:11:18.578266+0000","objects_scrubbed":0,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:28:00.228817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001944035,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":10923,"num_read_kb":34402,"num_write":1383,"num_write_kb":19662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":27769124,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":521,"num_object_clones":0,"num_object_copies":1042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":521,"num_whiteouts":517,"num_read":240279,"num_read_kb":1281261,"num_write":28650,"num_write_kb":585223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10651,"num_bytes_recovered":390333624,"num_keys_recovered":726,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5367,"internal_metadata":0},"log_size":25260,"ondisk_log_size":25260,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":2361472,"data_compressed":25472,"data_compressed_allocated":147456,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542194,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87736,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284104,"statfs":{"total":96636764160,"available":96546922496,"internally_reserved":0,"allocated":2453504,"data_stored":1194007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51628,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320212,"statfs":{"total":96636764160,"available":96583897088,"internally_reserved":0,"allocated":2449408,"data_stored":1193988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55836,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316004,"statfs":{"total":96636764160,"available":96579588096,"internally_reserved":0,"allocated":2490368,"data_stored":1784356,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542193,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77996,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293844,"statfs":{"total":96636764160,"available":96556896256,"internally_reserved":0,"allocated":2449408,"data_stored":1193988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542190,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47468,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324372,"statfs":{"total":96636764160,"available":96588156928,"internally_reserved":0,"allocated":2453504,"data_stored":1194007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":108,"seq":463856467992,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42600,"kb_used_data":2392,"kb_used_omap":6,"kb_used_meta":40185,"kb_avail":94329240,"statfs":{"total":96636764160,"available":96593141760,"internally_reserved":0,"allocated":2449408,"data_stored":1193988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6962,"internal_metadata":41149646},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55792,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316048,"statfs":{"total":96636764160,"available":96579633152,"internally_reserved":0,"allocated":2457600,"data_stored":1194026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542192,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78032,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293808,"statfs":{"total":96636764160,"available":96556859392,"internally_reserved":0,"allocated":2490368,"data_stored":1784356,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5367,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:14:41.118 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.118 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.118 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.118 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.118 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.118 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.118 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.118 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.118 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.119 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.119 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.119 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-15T00:11:08.518989+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=8, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.119 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-15T00:11:08.518989+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=8, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.119 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.119 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-15T00:11:08.518989+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=8, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.119 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.119 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.119 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.120 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-15T00:11:08.518989+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=8, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.120 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.120 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.120 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-15T00:11:08.518989+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=8, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.120 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-15T00:11:08.518989+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=8, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.120 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T00:11:04.453672+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=4, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.120 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.120 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.120 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.121 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.121 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.121 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.121 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.121 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.121 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.121 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.121 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.121 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-15T00:11:18.578266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=14, tm_sec=36, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:14:41.122 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T00:15:01.123 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:15:01.414 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-15T00:15:01.414 INFO:teuthology.orchestra.run.smithi171.stderr:dumped all 2024-09-15T00:15:01.426 INFO:teuthology.orchestra.run.smithi171.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-15T00:15:00.333139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":525,"num_object_clones":0,"num_object_copies":1050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":525,"num_whiteouts":517,"num_read":240395,"num_read_kb":1281361,"num_write":28811,"num_write_kb":587641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10807,"num_bytes_recovered":392695191,"num_keys_recovered":726,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25852,"ondisk_log_size":25852,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":501628,"kb_used_data":19584,"kb_used_omap":17,"kb_used_meta":481838,"kb_avail":754473092,"statfs":{"total":773094113280,"available":772580446208,"internally_reserved":0,"allocated":20054016,"data_stored":10903813,"data_compressed":12736,"data_compressed_allocated":73728,"data_compressed_original":1179648,"omap_allocated":18117,"internal_metadata":493402427},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.100100"},"pg_stats":[{"pgid":"3.1f","version":"203'662","reported_seq":6857,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.244341+0000","last_change":"2024-09-15T00:14:37.821827+0000","last_active":"2024-09-15T00:14:59.244341+0000","last_peered":"2024-09-15T00:14:59.244341+0000","last_clean":"2024-09-15T00:14:59.244341+0000","last_became_active":"2024-09-15T00:14:18.042449+0000","last_became_peered":"2024-09-15T00:14:18.042449+0000","last_unstale":"2024-09-15T00:14:59.244341+0000","last_undegraded":"2024-09-15T00:14:59.244341+0000","last_fullsized":"2024-09-15T00:14:59.244341+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'662","last_scrub_stamp":"2024-09-15T00:14:37.821645+0000","last_deep_scrub":"203'662","last_deep_scrub_stamp":"2024-09-15T00:14:37.821645+0000","last_clean_scrub_stamp":"2024-09-15T00:14:37.821645+0000","objects_scrubbed":19,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:18:15.963427+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0011752869999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6942,"num_read_kb":26340,"num_write":697,"num_write_kb":18376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":5475412,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"203'646","reported_seq":7664,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.465652+0000","last_change":"2024-09-15T00:14:40.219643+0000","last_active":"2024-09-15T00:14:58.465652+0000","last_peered":"2024-09-15T00:14:58.465652+0000","last_clean":"2024-09-15T00:14:58.465652+0000","last_became_active":"2024-09-15T00:11:19.598640+0000","last_became_peered":"2024-09-15T00:11:19.598640+0000","last_unstale":"2024-09-15T00:14:58.465652+0000","last_undegraded":"2024-09-15T00:14:58.465652+0000","last_fullsized":"2024-09-15T00:14:58.465652+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'646","last_scrub_stamp":"2024-09-15T00:14:40.219464+0000","last_deep_scrub":"203'646","last_deep_scrub_stamp":"2024-09-15T00:14:40.219464+0000","last_clean_scrub_stamp":"2024-09-15T00:14:40.219464+0000","objects_scrubbed":14,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:08:20.649202+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0016674789999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7565,"num_read_kb":39401,"num_write":737,"num_write_kb":10312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"203'714","reported_seq":4968,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.241834+0000","last_change":"2024-09-15T00:14:42.836548+0000","last_active":"2024-09-15T00:14:59.241834+0000","last_peered":"2024-09-15T00:14:59.241834+0000","last_clean":"2024-09-15T00:14:59.241834+0000","last_became_active":"2024-09-15T00:11:19.595954+0000","last_became_peered":"2024-09-15T00:11:19.595954+0000","last_unstale":"2024-09-15T00:14:59.241834+0000","last_undegraded":"2024-09-15T00:14:59.241834+0000","last_fullsized":"2024-09-15T00:14:59.241834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'714","last_scrub_stamp":"2024-09-15T00:14:42.836365+0000","last_deep_scrub":"203'714","last_deep_scrub_stamp":"2024-09-15T00:14:42.836365+0000","last_clean_scrub_stamp":"2024-09-15T00:14:42.836365+0000","objects_scrubbed":16,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:32:11.371341+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0020206820000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5583,"num_read_kb":45090,"num_write":773,"num_write_kb":31484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"203'686","reported_seq":5911,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.241712+0000","last_change":"2024-09-15T00:14:43.809843+0000","last_active":"2024-09-15T00:14:59.241712+0000","last_peered":"2024-09-15T00:14:59.241712+0000","last_clean":"2024-09-15T00:14:59.241712+0000","last_became_active":"2024-09-15T00:11:19.602583+0000","last_became_peered":"2024-09-15T00:11:19.602583+0000","last_unstale":"2024-09-15T00:14:59.241712+0000","last_undegraded":"2024-09-15T00:14:59.241712+0000","last_fullsized":"2024-09-15T00:14:59.241712+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'686","last_scrub_stamp":"2024-09-15T00:14:43.809702+0000","last_deep_scrub":"203'686","last_deep_scrub_stamp":"2024-09-15T00:14:43.809702+0000","last_clean_scrub_stamp":"2024-09-15T00:14:43.809702+0000","objects_scrubbed":12,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:32:40.034550+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00168294,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":5888,"num_read_kb":35219,"num_write":807,"num_write_kb":10221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"203'859","reported_seq":7005,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.244419+0000","last_change":"2024-09-15T00:14:39.833045+0000","last_active":"2024-09-15T00:14:59.244419+0000","last_peered":"2024-09-15T00:14:59.244419+0000","last_clean":"2024-09-15T00:14:59.244419+0000","last_became_active":"2024-09-15T00:14:20.032662+0000","last_became_peered":"2024-09-15T00:14:20.032662+0000","last_unstale":"2024-09-15T00:14:59.244419+0000","last_undegraded":"2024-09-15T00:14:59.244419+0000","last_fullsized":"2024-09-15T00:14:59.244419+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'859","last_scrub_stamp":"2024-09-15T00:14:39.832938+0000","last_deep_scrub":"203'859","last_deep_scrub_stamp":"2024-09-15T00:14:39.832938+0000","last_clean_scrub_stamp":"2024-09-15T00:14:39.832938+0000","objects_scrubbed":15,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:19:31.466259+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.001922899,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7801,"num_read_kb":34537,"num_write":909,"num_write_kb":14651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":10954706,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"203'796","reported_seq":7576,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.257843+0000","last_change":"2024-09-15T00:14:59.253936+0000","last_active":"2024-09-15T00:14:59.257843+0000","last_peered":"2024-09-15T00:14:59.257843+0000","last_clean":"2024-09-15T00:14:59.257843+0000","last_became_active":"2024-09-15T00:14:59.251773+0000","last_became_peered":"2024-09-15T00:14:59.251773+0000","last_unstale":"2024-09-15T00:14:59.257843+0000","last_undegraded":"2024-09-15T00:14:59.257843+0000","last_fullsized":"2024-09-15T00:14:59.257843+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'796","last_scrub_stamp":"2024-09-15T00:14:42.515395+0000","last_deep_scrub":"203'796","last_deep_scrub_stamp":"2024-09-15T00:14:42.515395+0000","last_clean_scrub_stamp":"2024-09-15T00:14:42.515395+0000","objects_scrubbed":13,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:01:09.718801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012814980000000001,"stat_sum":{"num_bytes":19,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":8900,"num_read_kb":62621,"num_write":837,"num_write_kb":33377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":655,"num_bytes_recovered":66497993,"num_keys_recovered":149,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"203'692","reported_seq":5659,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.250660+0000","last_change":"2024-09-15T00:14:43.080275+0000","last_active":"2024-09-15T00:14:59.250660+0000","last_peered":"2024-09-15T00:14:59.250660+0000","last_clean":"2024-09-15T00:14:59.250660+0000","last_became_active":"2024-09-15T00:14:22.036152+0000","last_became_peered":"2024-09-15T00:14:22.036152+0000","last_unstale":"2024-09-15T00:14:59.250660+0000","last_undegraded":"2024-09-15T00:14:59.250660+0000","last_fullsized":"2024-09-15T00:14:59.250660+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'692","last_scrub_stamp":"2024-09-15T00:14:43.080092+0000","last_deep_scrub":"203'692","last_deep_scrub_stamp":"2024-09-15T00:14:43.080092+0000","last_clean_scrub_stamp":"2024-09-15T00:14:43.080092+0000","objects_scrubbed":14,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:16:20.545750+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.005256073,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4711,"num_read_kb":42462,"num_write":705,"num_write_kb":9831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":558,"num_bytes_recovered":6445817,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"203'721","reported_seq":8796,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.465563+0000","last_change":"2024-09-15T00:14:42.241518+0000","last_active":"2024-09-15T00:14:58.465563+0000","last_peered":"2024-09-15T00:14:58.465563+0000","last_clean":"2024-09-15T00:14:58.465563+0000","last_became_active":"2024-09-15T00:14:18.042710+0000","last_became_peered":"2024-09-15T00:14:18.042710+0000","last_unstale":"2024-09-15T00:14:58.465563+0000","last_undegraded":"2024-09-15T00:14:58.465563+0000","last_fullsized":"2024-09-15T00:14:58.465563+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'721","last_scrub_stamp":"2024-09-15T00:14:42.241278+0000","last_deep_scrub":"203'721","last_deep_scrub_stamp":"2024-09-15T00:14:42.241278+0000","last_clean_scrub_stamp":"2024-09-15T00:14:42.241278+0000","objects_scrubbed":15,"log_size":721,"log_dups_size":0,"ondisk_log_size":721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:20:45.982972+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0014749780000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8102,"num_read_kb":40115,"num_write":776,"num_write_kb":15981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":4986328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"203'700","reported_seq":5589,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.244331+0000","last_change":"2024-09-15T00:14:40.813365+0000","last_active":"2024-09-15T00:14:59.244331+0000","last_peered":"2024-09-15T00:14:59.244331+0000","last_clean":"2024-09-15T00:14:59.244331+0000","last_became_active":"2024-09-15T00:14:20.034657+0000","last_became_peered":"2024-09-15T00:14:20.034657+0000","last_unstale":"2024-09-15T00:14:59.244331+0000","last_undegraded":"2024-09-15T00:14:59.244331+0000","last_fullsized":"2024-09-15T00:14:59.244331+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'700","last_scrub_stamp":"2024-09-15T00:14:40.813159+0000","last_deep_scrub":"203'700","last_deep_scrub_stamp":"2024-09-15T00:14:40.813159+0000","last_clean_scrub_stamp":"2024-09-15T00:14:40.813159+0000","objects_scrubbed":21,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:22:32.635082+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.002708182,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5548,"num_read_kb":45663,"num_write":634,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":12460957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"203'1048","reported_seq":8465,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.241806+0000","last_change":"2024-09-15T00:14:44.850282+0000","last_active":"2024-09-15T00:14:59.241806+0000","last_peered":"2024-09-15T00:14:59.241806+0000","last_clean":"2024-09-15T00:14:59.241806+0000","last_became_active":"2024-09-15T00:13:50.031768+0000","last_became_peered":"2024-09-15T00:13:50.031768+0000","last_unstale":"2024-09-15T00:14:59.241806+0000","last_undegraded":"2024-09-15T00:14:59.241806+0000","last_fullsized":"2024-09-15T00:14:59.241806+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'1048","last_scrub_stamp":"2024-09-15T00:14:44.850151+0000","last_deep_scrub":"203'1048","last_deep_scrub_stamp":"2024-09-15T00:14:44.850151+0000","last_clean_scrub_stamp":"2024-09-15T00:14:44.850151+0000","objects_scrubbed":16,"log_size":1048,"log_dups_size":0,"ondisk_log_size":1048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:50:43.794465+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0.001158467,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8890,"num_read_kb":39257,"num_write":1264,"num_write_kb":20613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":23803494,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"203'685","reported_seq":6988,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.466177+0000","last_change":"2024-09-15T00:14:43.876409+0000","last_active":"2024-09-15T00:14:58.466177+0000","last_peered":"2024-09-15T00:14:58.466177+0000","last_clean":"2024-09-15T00:14:58.466177+0000","last_became_active":"2024-09-15T00:13:50.025342+0000","last_became_peered":"2024-09-15T00:13:50.025342+0000","last_unstale":"2024-09-15T00:14:58.466177+0000","last_undegraded":"2024-09-15T00:14:58.466177+0000","last_fullsized":"2024-09-15T00:14:58.466177+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'685","last_scrub_stamp":"2024-09-15T00:14:43.876272+0000","last_deep_scrub":"203'685","last_deep_scrub_stamp":"2024-09-15T00:14:43.876272+0000","last_clean_scrub_stamp":"2024-09-15T00:14:43.876272+0000","objects_scrubbed":18,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:08:47.704567+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.00071033100000000003,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7596,"num_read_kb":45634,"num_write":737,"num_write_kb":14836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":19127384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"203'1099","reported_seq":7501,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-15T00:14:55.228169+0000","last_change":"2024-09-15T00:14:40.460057+0000","last_active":"2024-09-15T00:14:55.228169+0000","last_peered":"2024-09-15T00:14:55.228169+0000","last_clean":"2024-09-15T00:14:55.228169+0000","last_became_active":"2024-09-15T00:14:24.093937+0000","last_became_peered":"2024-09-15T00:14:24.093937+0000","last_unstale":"2024-09-15T00:14:55.228169+0000","last_undegraded":"2024-09-15T00:14:55.228169+0000","last_fullsized":"2024-09-15T00:14:55.228169+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'1099","last_scrub_stamp":"2024-09-15T00:14:40.459916+0000","last_deep_scrub":"203'1099","last_deep_scrub_stamp":"2024-09-15T00:14:40.459916+0000","last_clean_scrub_stamp":"2024-09-15T00:14:40.459916+0000","objects_scrubbed":20,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:56:29.111651+0000","scrub_duration":17,"objects_trimmed":3,"snaptrim_duration":0.016604621,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":9874,"num_read_kb":58461,"num_write":1482,"num_write_kb":25008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":6083388,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":422,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.243648+0000","last_change":"2024-09-15T00:14:41.898253+0000","last_active":"2024-09-15T00:14:59.243648+0000","last_peered":"2024-09-15T00:14:59.243648+0000","last_clean":"2024-09-15T00:14:59.243648+0000","last_became_active":"2024-09-15T00:13:46.030230+0000","last_became_peered":"2024-09-15T00:13:46.030230+0000","last_unstale":"2024-09-15T00:14:59.243648+0000","last_undegraded":"2024-09-15T00:14:59.243648+0000","last_fullsized":"2024-09-15T00:14:59.243648+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:14:41.898140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:14:41.898140+0000","last_clean_scrub_stamp":"2024-09-15T00:14:41.898140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:53:58.359439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048856699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"203'737","reported_seq":8013,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.244222+0000","last_change":"2024-09-15T00:14:38.812587+0000","last_active":"2024-09-15T00:14:59.244222+0000","last_peered":"2024-09-15T00:14:59.244222+0000","last_clean":"2024-09-15T00:14:59.244222+0000","last_became_active":"2024-09-15T00:14:18.043887+0000","last_became_peered":"2024-09-15T00:14:18.043887+0000","last_unstale":"2024-09-15T00:14:59.244222+0000","last_undegraded":"2024-09-15T00:14:59.244222+0000","last_fullsized":"2024-09-15T00:14:59.244222+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'737","last_scrub_stamp":"2024-09-15T00:14:38.812452+0000","last_deep_scrub":"203'737","last_deep_scrub_stamp":"2024-09-15T00:14:38.812452+0000","last_clean_scrub_stamp":"2024-09-15T00:14:38.812452+0000","objects_scrubbed":22,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:03:49.386711+0000","scrub_duration":16,"objects_trimmed":2,"snaptrim_duration":0.00086129200000000005,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6781,"num_read_kb":42484,"num_write":652,"num_write_kb":25730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":26193256,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":58,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.250990+0000","last_change":"2024-09-15T00:14:40.020492+0000","last_active":"2024-09-15T00:14:59.250990+0000","last_peered":"2024-09-15T00:14:59.250990+0000","last_clean":"2024-09-15T00:14:59.250990+0000","last_became_active":"2024-09-15T00:14:16.040306+0000","last_became_peered":"2024-09-15T00:14:16.040306+0000","last_unstale":"2024-09-15T00:14:59.250990+0000","last_undegraded":"2024-09-15T00:14:59.250990+0000","last_fullsized":"2024-09-15T00:14:59.250990+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:14:40.020368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:14:40.020368+0000","last_clean_scrub_stamp":"2024-09-15T00:14:40.020368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:55:17.416264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"203'750","reported_seq":9433,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.465707+0000","last_change":"2024-09-15T00:14:41.208317+0000","last_active":"2024-09-15T00:14:58.465707+0000","last_peered":"2024-09-15T00:14:58.465707+0000","last_clean":"2024-09-15T00:14:58.465707+0000","last_became_active":"2024-09-15T00:13:49.218857+0000","last_became_peered":"2024-09-15T00:13:49.218857+0000","last_unstale":"2024-09-15T00:14:58.465707+0000","last_undegraded":"2024-09-15T00:14:58.465707+0000","last_fullsized":"2024-09-15T00:14:58.465707+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'750","last_scrub_stamp":"2024-09-15T00:14:41.208210+0000","last_deep_scrub":"203'750","last_deep_scrub_stamp":"2024-09-15T00:14:41.208210+0000","last_clean_scrub_stamp":"2024-09-15T00:14:41.208210+0000","objects_scrubbed":19,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:08:14.542811+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0020637310000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":10066,"num_read_kb":57181,"num_write":720,"num_write_kb":17747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":15784649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":126,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.465954+0000","last_change":"2024-09-15T00:14:46.838074+0000","last_active":"2024-09-15T00:14:58.465954+0000","last_peered":"2024-09-15T00:14:58.465954+0000","last_clean":"2024-09-15T00:14:58.465954+0000","last_became_active":"2024-09-15T00:14:15.037987+0000","last_became_peered":"2024-09-15T00:14:15.037987+0000","last_unstale":"2024-09-15T00:14:58.465954+0000","last_undegraded":"2024-09-15T00:14:58.465954+0000","last_fullsized":"2024-09-15T00:14:58.465954+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:14:46.837905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:14:46.837905+0000","last_clean_scrub_stamp":"2024-09-15T00:14:46.837905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:43:46.963730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047616800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"203'790","reported_seq":12076,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.250703+0000","last_change":"2024-09-15T00:14:42.045912+0000","last_active":"2024-09-15T00:14:59.250703+0000","last_peered":"2024-09-15T00:14:59.250703+0000","last_clean":"2024-09-15T00:14:59.250703+0000","last_became_active":"2024-09-15T00:14:20.033228+0000","last_became_peered":"2024-09-15T00:14:20.033228+0000","last_unstale":"2024-09-15T00:14:59.250703+0000","last_undegraded":"2024-09-15T00:14:59.250703+0000","last_fullsized":"2024-09-15T00:14:59.250703+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'790","last_scrub_stamp":"2024-09-15T00:14:42.045786+0000","last_deep_scrub":"203'790","last_deep_scrub_stamp":"2024-09-15T00:14:42.045786+0000","last_clean_scrub_stamp":"2024-09-15T00:14:42.045786+0000","objects_scrubbed":16,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:20:05.438411+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0017650229999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12816,"num_read_kb":60614,"num_write":771,"num_write_kb":25261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1083,"num_bytes_recovered":41768720,"num_keys_recovered":205,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":58,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.250825+0000","last_change":"2024-09-15T00:14:39.033245+0000","last_active":"2024-09-15T00:14:59.250825+0000","last_peered":"2024-09-15T00:14:59.250825+0000","last_clean":"2024-09-15T00:14:59.250825+0000","last_became_active":"2024-09-15T00:14:16.034489+0000","last_became_peered":"2024-09-15T00:14:16.034489+0000","last_unstale":"2024-09-15T00:14:59.250825+0000","last_undegraded":"2024-09-15T00:14:59.250825+0000","last_fullsized":"2024-09-15T00:14:59.250825+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:14:39.033147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:14:39.033147+0000","last_clean_scrub_stamp":"2024-09-15T00:14:39.033147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:55:04.706104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039516200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"203'565","reported_seq":3279,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.241895+0000","last_change":"2024-09-15T00:14:41.789330+0000","last_active":"2024-09-15T00:14:59.241895+0000","last_peered":"2024-09-15T00:14:59.241895+0000","last_clean":"2024-09-15T00:14:59.241895+0000","last_became_active":"2024-09-15T00:11:19.595633+0000","last_became_peered":"2024-09-15T00:11:19.595633+0000","last_unstale":"2024-09-15T00:14:59.241895+0000","last_undegraded":"2024-09-15T00:14:59.241895+0000","last_fullsized":"2024-09-15T00:14:59.241895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'565","last_scrub_stamp":"2024-09-15T00:14:41.789199+0000","last_deep_scrub":"203'565","last_deep_scrub_stamp":"2024-09-15T00:14:41.789199+0000","last_clean_scrub_stamp":"2024-09-15T00:14:41.789199+0000","objects_scrubbed":13,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:15:14.756480+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0026087110000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3741,"num_read_kb":17273,"num_write":546,"num_write_kb":8984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"203'835","reported_seq":8186,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-15T00:14:55.228199+0000","last_change":"2024-09-15T00:14:41.474302+0000","last_active":"2024-09-15T00:14:55.228199+0000","last_peered":"2024-09-15T00:14:55.228199+0000","last_clean":"2024-09-15T00:14:55.228199+0000","last_became_active":"2024-09-15T00:14:24.094819+0000","last_became_peered":"2024-09-15T00:14:24.094819+0000","last_unstale":"2024-09-15T00:14:55.228199+0000","last_undegraded":"2024-09-15T00:14:55.228199+0000","last_fullsized":"2024-09-15T00:14:55.228199+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'835","last_scrub_stamp":"2024-09-15T00:14:41.474122+0000","last_deep_scrub":"203'835","last_deep_scrub_stamp":"2024-09-15T00:14:41.474122+0000","last_clean_scrub_stamp":"2024-09-15T00:14:41.474122+0000","objects_scrubbed":16,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:34:06.942732+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00090289999999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":7230,"num_read_kb":43219,"num_write":761,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":9273484,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"203'3","reported_seq":440,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.241670+0000","last_change":"2024-09-15T00:14:46.800289+0000","last_active":"2024-09-15T00:14:59.241670+0000","last_peered":"2024-09-15T00:14:59.241670+0000","last_clean":"2024-09-15T00:14:59.241670+0000","last_became_active":"2024-09-15T00:14:16.046906+0000","last_became_peered":"2024-09-15T00:14:16.046906+0000","last_unstale":"2024-09-15T00:14:59.241670+0000","last_undegraded":"2024-09-15T00:14:59.241670+0000","last_fullsized":"2024-09-15T00:14:59.241670+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'3","last_scrub_stamp":"2024-09-15T00:14:46.800168+0000","last_deep_scrub":"203'3","last_deep_scrub_stamp":"2024-09-15T00:14:46.800168+0000","last_clean_scrub_stamp":"2024-09-15T00:14:46.800168+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:58:50.020859+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0011300780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"203'714","reported_seq":7841,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.250684+0000","last_change":"2024-09-15T00:14:41.084359+0000","last_active":"2024-09-15T00:14:59.250684+0000","last_peered":"2024-09-15T00:14:59.250684+0000","last_clean":"2024-09-15T00:14:59.250684+0000","last_became_active":"2024-09-15T00:14:19.048768+0000","last_became_peered":"2024-09-15T00:14:19.048768+0000","last_unstale":"2024-09-15T00:14:59.250684+0000","last_undegraded":"2024-09-15T00:14:59.250684+0000","last_fullsized":"2024-09-15T00:14:59.250684+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'714","last_scrub_stamp":"2024-09-15T00:14:41.084192+0000","last_deep_scrub":"203'714","last_deep_scrub_stamp":"2024-09-15T00:14:41.084192+0000","last_clean_scrub_stamp":"2024-09-15T00:14:41.084192+0000","objects_scrubbed":18,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:33:42.615061+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.00092066200000000004,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7368,"num_read_kb":33386,"num_write":726,"num_write_kb":14440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":894,"num_bytes_recovered":31813799,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.246853+0000","last_change":"2024-09-15T00:14:39.020009+0000","last_active":"2024-09-15T00:14:59.246853+0000","last_peered":"2024-09-15T00:14:59.246853+0000","last_clean":"2024-09-15T00:14:59.246853+0000","last_became_active":"2024-09-15T00:13:52.025864+0000","last_became_peered":"2024-09-15T00:13:52.025864+0000","last_unstale":"2024-09-15T00:14:59.246853+0000","last_undegraded":"2024-09-15T00:14:59.246853+0000","last_fullsized":"2024-09-15T00:14:59.246853+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:14:39.019904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:14:39.019904+0000","last_clean_scrub_stamp":"2024-09-15T00:14:39.019904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:47:08.654282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"203'719","reported_seq":6759,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.244055+0000","last_change":"2024-09-15T00:14:42.846997+0000","last_active":"2024-09-15T00:14:59.244055+0000","last_peered":"2024-09-15T00:14:59.244055+0000","last_clean":"2024-09-15T00:14:59.244055+0000","last_became_active":"2024-09-15T00:14:21.034477+0000","last_became_peered":"2024-09-15T00:14:21.034477+0000","last_unstale":"2024-09-15T00:14:59.244055+0000","last_undegraded":"2024-09-15T00:14:59.244055+0000","last_fullsized":"2024-09-15T00:14:59.244055+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'719","last_scrub_stamp":"2024-09-15T00:14:42.846870+0000","last_deep_scrub":"203'719","last_deep_scrub_stamp":"2024-09-15T00:14:42.846870+0000","last_clean_scrub_stamp":"2024-09-15T00:14:42.846870+0000","objects_scrubbed":11,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:34.040446+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00075419699999999996,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6724,"num_read_kb":49050,"num_write":759,"num_write_kb":22019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":6058433,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":130,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.465788+0000","last_change":"2024-09-15T00:14:47.815927+0000","last_active":"2024-09-15T00:14:58.465788+0000","last_peered":"2024-09-15T00:14:58.465788+0000","last_clean":"2024-09-15T00:14:58.465788+0000","last_became_active":"2024-09-15T00:14:16.041672+0000","last_became_peered":"2024-09-15T00:14:16.041672+0000","last_unstale":"2024-09-15T00:14:58.465788+0000","last_undegraded":"2024-09-15T00:14:58.465788+0000","last_fullsized":"2024-09-15T00:14:58.465788+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:14:47.815792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:14:47.815792+0000","last_clean_scrub_stamp":"2024-09-15T00:14:47.815792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:39:22.677409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"203'631","reported_seq":5423,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.465539+0000","last_change":"2024-09-15T00:14:39.223099+0000","last_active":"2024-09-15T00:14:58.465539+0000","last_peered":"2024-09-15T00:14:58.465539+0000","last_clean":"2024-09-15T00:14:58.465539+0000","last_became_active":"2024-09-15T00:11:19.594528+0000","last_became_peered":"2024-09-15T00:11:19.594528+0000","last_unstale":"2024-09-15T00:14:58.465539+0000","last_undegraded":"2024-09-15T00:14:58.465539+0000","last_fullsized":"2024-09-15T00:14:58.465539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'631","last_scrub_stamp":"2024-09-15T00:14:39.222988+0000","last_deep_scrub":"203'631","last_deep_scrub_stamp":"2024-09-15T00:14:39.222988+0000","last_clean_scrub_stamp":"2024-09-15T00:14:39.222988+0000","objects_scrubbed":12,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:50:29.558130+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001727704,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5349,"num_read_kb":42137,"num_write":591,"num_write_kb":17763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"203'2","reported_seq":457,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.241678+0000","last_change":"2024-09-15T00:14:45.821308+0000","last_active":"2024-09-15T00:14:59.241678+0000","last_peered":"2024-09-15T00:14:59.241678+0000","last_clean":"2024-09-15T00:14:59.241678+0000","last_became_active":"2024-09-15T00:13:52.026457+0000","last_became_peered":"2024-09-15T00:13:52.026457+0000","last_unstale":"2024-09-15T00:14:59.241678+0000","last_undegraded":"2024-09-15T00:14:59.241678+0000","last_fullsized":"2024-09-15T00:14:59.241678+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'2","last_scrub_stamp":"2024-09-15T00:14:45.821164+0000","last_deep_scrub":"203'2","last_deep_scrub_stamp":"2024-09-15T00:14:45.821164+0000","last_clean_scrub_stamp":"2024-09-15T00:14:45.821164+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:01:13.703070+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00037807299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"203'119","reported_seq":366,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.465919+0000","last_change":"2024-09-15T00:14:48.854455+0000","last_active":"2024-09-15T00:14:58.465919+0000","last_peered":"2024-09-15T00:14:58.465919+0000","last_clean":"2024-09-15T00:14:58.465919+0000","last_became_active":"2024-09-15T00:14:18.046593+0000","last_became_peered":"2024-09-15T00:14:18.046593+0000","last_unstale":"2024-09-15T00:14:58.465919+0000","last_undegraded":"2024-09-15T00:14:58.465919+0000","last_fullsized":"2024-09-15T00:14:58.465919+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'119","last_scrub_stamp":"2024-09-15T00:14:48.854309+0000","last_deep_scrub":"203'119","last_deep_scrub_stamp":"2024-09-15T00:14:48.854309+0000","last_clean_scrub_stamp":"2024-09-15T00:14:48.854309+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:31:28.173364+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"203'862","reported_seq":8319,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.465535+0000","last_change":"2024-09-15T00:14:45.177913+0000","last_active":"2024-09-15T00:14:58.465535+0000","last_peered":"2024-09-15T00:14:58.465535+0000","last_clean":"2024-09-15T00:14:58.465535+0000","last_became_active":"2024-09-15T00:14:22.038746+0000","last_became_peered":"2024-09-15T00:14:22.038746+0000","last_unstale":"2024-09-15T00:14:58.465535+0000","last_undegraded":"2024-09-15T00:14:58.465535+0000","last_fullsized":"2024-09-15T00:14:58.465535+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'862","last_scrub_stamp":"2024-09-15T00:14:45.177743+0000","last_deep_scrub":"203'862","last_deep_scrub_stamp":"2024-09-15T00:14:45.177743+0000","last_clean_scrub_stamp":"2024-09-15T00:14:45.177743+0000","objects_scrubbed":18,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:46:50.105091+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0012708680000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8015,"num_read_kb":57970,"num_write":928,"num_write_kb":24350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":14074270,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"203'1541","reported_seq":9148,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.465535+0000","last_change":"2024-09-15T00:14:43.220064+0000","last_active":"2024-09-15T00:14:58.465535+0000","last_peered":"2024-09-15T00:14:58.465535+0000","last_clean":"2024-09-15T00:14:58.465535+0000","last_became_active":"2024-09-15T00:14:19.048942+0000","last_became_peered":"2024-09-15T00:14:19.048942+0000","last_unstale":"2024-09-15T00:14:58.465535+0000","last_undegraded":"2024-09-15T00:14:58.465535+0000","last_fullsized":"2024-09-15T00:14:58.465535+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'1541","last_scrub_stamp":"2024-09-15T00:14:43.219894+0000","last_deep_scrub":"203'1541","last_deep_scrub_stamp":"2024-09-15T00:14:43.219894+0000","last_clean_scrub_stamp":"2024-09-15T00:14:43.219894+0000","objects_scrubbed":15,"log_size":1541,"log_dups_size":0,"ondisk_log_size":1541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:42:55.811202+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.00061724499999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":14886,"num_read_kb":35520,"num_write":2067,"num_write_kb":12930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":409,"num_bytes_recovered":10512832,"num_keys_recovered":83,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"203'747","reported_seq":6660,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.243621+0000","last_change":"2024-09-15T00:14:40.904651+0000","last_active":"2024-09-15T00:14:59.243621+0000","last_peered":"2024-09-15T00:14:59.243621+0000","last_clean":"2024-09-15T00:14:59.243621+0000","last_became_active":"2024-09-15T00:11:19.602625+0000","last_became_peered":"2024-09-15T00:11:19.602625+0000","last_unstale":"2024-09-15T00:14:59.243621+0000","last_undegraded":"2024-09-15T00:14:59.243621+0000","last_fullsized":"2024-09-15T00:14:59.243621+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'747","last_scrub_stamp":"2024-09-15T00:14:40.904496+0000","last_deep_scrub":"203'747","last_deep_scrub_stamp":"2024-09-15T00:14:40.904496+0000","last_clean_scrub_stamp":"2024-09-15T00:14:40.904496+0000","objects_scrubbed":14,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:55.721617+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0012377130000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6965,"num_read_kb":22637,"num_write":697,"num_write_kb":11622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"203'441","reported_seq":5556,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.465585+0000","last_change":"2024-09-15T00:14:44.172365+0000","last_active":"2024-09-15T00:14:58.465585+0000","last_peered":"2024-09-15T00:14:58.465585+0000","last_clean":"2024-09-15T00:14:58.465585+0000","last_became_active":"2024-09-15T00:14:20.036368+0000","last_became_peered":"2024-09-15T00:14:20.036368+0000","last_unstale":"2024-09-15T00:14:58.465585+0000","last_undegraded":"2024-09-15T00:14:58.465585+0000","last_fullsized":"2024-09-15T00:14:58.465585+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'441","last_scrub_stamp":"2024-09-15T00:14:44.172217+0000","last_deep_scrub":"203'441","last_deep_scrub_stamp":"2024-09-15T00:14:44.172217+0000","last_clean_scrub_stamp":"2024-09-15T00:14:44.172217+0000","objects_scrubbed":15,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:50:47.528702+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.00087100100000000005,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3886,"num_read_kb":33280,"num_write":476,"num_write_kb":22691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":10092575,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"203'935","reported_seq":4969,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.241724+0000","last_change":"2024-09-15T00:14:40.777630+0000","last_active":"2024-09-15T00:14:59.241724+0000","last_peered":"2024-09-15T00:14:59.241724+0000","last_clean":"2024-09-15T00:14:59.241724+0000","last_became_active":"2024-09-15T00:11:19.595088+0000","last_became_peered":"2024-09-15T00:11:19.595088+0000","last_unstale":"2024-09-15T00:14:59.241724+0000","last_undegraded":"2024-09-15T00:14:59.241724+0000","last_fullsized":"2024-09-15T00:14:59.241724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'935","last_scrub_stamp":"2024-09-15T00:14:40.777458+0000","last_deep_scrub":"203'935","last_deep_scrub_stamp":"2024-09-15T00:14:40.777458+0000","last_clean_scrub_stamp":"2024-09-15T00:14:40.777458+0000","objects_scrubbed":16,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:55:56.485358+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0024006000000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7681,"num_read_kb":43201,"num_write":1093,"num_write_kb":15373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"203'1001","reported_seq":6078,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.465831+0000","last_change":"2024-09-15T00:14:45.808988+0000","last_active":"2024-09-15T00:14:58.465831+0000","last_peered":"2024-09-15T00:14:58.465831+0000","last_clean":"2024-09-15T00:14:58.465831+0000","last_became_active":"2024-09-15T00:13:51.028205+0000","last_became_peered":"2024-09-15T00:13:51.028205+0000","last_unstale":"2024-09-15T00:14:58.465831+0000","last_undegraded":"2024-09-15T00:14:58.465831+0000","last_fullsized":"2024-09-15T00:14:58.465831+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'1001","last_scrub_stamp":"2024-09-15T00:14:45.808829+0000","last_deep_scrub":"203'1001","last_deep_scrub_stamp":"2024-09-15T00:14:45.808829+0000","last_clean_scrub_stamp":"2024-09-15T00:14:45.808829+0000","objects_scrubbed":16,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:13:24.907974+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.0012164389999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7667,"num_read_kb":20584,"num_write":1269,"num_write_kb":5933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":1861870,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"203'1434","reported_seq":6097,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.465790+0000","last_change":"2024-09-15T00:14:42.858404+0000","last_active":"2024-09-15T00:14:58.465790+0000","last_peered":"2024-09-15T00:14:58.465790+0000","last_clean":"2024-09-15T00:14:58.465790+0000","last_became_active":"2024-09-15T00:13:50.023348+0000","last_became_peered":"2024-09-15T00:13:50.023348+0000","last_unstale":"2024-09-15T00:14:58.465790+0000","last_undegraded":"2024-09-15T00:14:58.465790+0000","last_fullsized":"2024-09-15T00:14:58.465790+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'1434","last_scrub_stamp":"2024-09-15T00:14:42.858253+0000","last_deep_scrub":"203'1434","last_deep_scrub_stamp":"2024-09-15T00:14:42.858253+0000","last_clean_scrub_stamp":"2024-09-15T00:14:42.858253+0000","objects_scrubbed":13,"log_size":1434,"log_dups_size":0,"ondisk_log_size":1434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:38:49.056413+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.001473739,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8919,"num_read_kb":45201,"num_write":2133,"num_write_kb":24448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":18482007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"203'535","reported_seq":6449,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.466105+0000","last_change":"2024-09-15T00:14:41.847291+0000","last_active":"2024-09-15T00:14:58.466105+0000","last_peered":"2024-09-15T00:14:58.466105+0000","last_clean":"2024-09-15T00:14:58.466105+0000","last_became_active":"2024-09-15T00:13:49.218512+0000","last_became_peered":"2024-09-15T00:13:49.218512+0000","last_unstale":"2024-09-15T00:14:58.466105+0000","last_undegraded":"2024-09-15T00:14:58.466105+0000","last_fullsized":"2024-09-15T00:14:58.466105+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'535","last_scrub_stamp":"2024-09-15T00:14:41.847151+0000","last_deep_scrub":"203'535","last_deep_scrub_stamp":"2024-09-15T00:14:41.847151+0000","last_clean_scrub_stamp":"2024-09-15T00:14:41.847151+0000","objects_scrubbed":19,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:45:09.632713+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.0014419529999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5794,"num_read_kb":31109,"num_write":499,"num_write_kb":15000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5158884,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"203'747","reported_seq":7124,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.243618+0000","last_change":"2024-09-15T00:14:42.921495+0000","last_active":"2024-09-15T00:14:59.243618+0000","last_peered":"2024-09-15T00:14:59.243618+0000","last_clean":"2024-09-15T00:14:59.243618+0000","last_became_active":"2024-09-15T00:14:19.054103+0000","last_became_peered":"2024-09-15T00:14:19.054103+0000","last_unstale":"2024-09-15T00:14:59.243618+0000","last_undegraded":"2024-09-15T00:14:59.243618+0000","last_fullsized":"2024-09-15T00:14:59.243618+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'747","last_scrub_stamp":"2024-09-15T00:14:42.921356+0000","last_deep_scrub":"203'747","last_deep_scrub_stamp":"2024-09-15T00:14:42.921356+0000","last_clean_scrub_stamp":"2024-09-15T00:14:42.921356+0000","objects_scrubbed":16,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:32.592373+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.00086791500000000003,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7073,"num_read_kb":31603,"num_write":836,"num_write_kb":14934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":8339863,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"203'614","reported_seq":4660,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.465788+0000","last_change":"2024-09-15T00:14:40.830193+0000","last_active":"2024-09-15T00:14:58.465788+0000","last_peered":"2024-09-15T00:14:58.465788+0000","last_clean":"2024-09-15T00:14:58.465788+0000","last_became_active":"2024-09-15T00:13:49.218581+0000","last_became_peered":"2024-09-15T00:13:49.218581+0000","last_unstale":"2024-09-15T00:14:58.465788+0000","last_undegraded":"2024-09-15T00:14:58.465788+0000","last_fullsized":"2024-09-15T00:14:58.465788+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'614","last_scrub_stamp":"2024-09-15T00:14:40.830006+0000","last_deep_scrub":"203'614","last_deep_scrub_stamp":"2024-09-15T00:14:40.830006+0000","last_clean_scrub_stamp":"2024-09-15T00:14:40.830006+0000","objects_scrubbed":18,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:34:57.273115+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0010536860000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4205,"num_read_kb":15404,"num_write":601,"num_write_kb":11097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":1517056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"203'697","reported_seq":7846,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-15T00:14:59.244067+0000","last_change":"2024-09-15T00:14:41.836017+0000","last_active":"2024-09-15T00:14:59.244067+0000","last_peered":"2024-09-15T00:14:59.244067+0000","last_clean":"2024-09-15T00:14:59.244067+0000","last_became_active":"2024-09-15T00:14:21.032233+0000","last_became_peered":"2024-09-15T00:14:21.032233+0000","last_unstale":"2024-09-15T00:14:59.244067+0000","last_undegraded":"2024-09-15T00:14:59.244067+0000","last_fullsized":"2024-09-15T00:14:59.244067+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'697","last_scrub_stamp":"2024-09-15T00:14:41.835894+0000","last_deep_scrub":"203'697","last_deep_scrub_stamp":"2024-09-15T00:14:41.835894+0000","last_clean_scrub_stamp":"2024-09-15T00:14:41.835894+0000","objects_scrubbed":23,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:34:40.469556+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.00089427500000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6790,"num_read_kb":50206,"num_write":784,"num_write_kb":25712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":5797342,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"203'1125","reported_seq":8259,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-15T00:14:58.466117+0000","last_change":"2024-09-15T00:14:44.839537+0000","last_active":"2024-09-15T00:14:58.466117+0000","last_peered":"2024-09-15T00:14:58.466117+0000","last_clean":"2024-09-15T00:14:58.466117+0000","last_became_active":"2024-09-15T00:13:51.025771+0000","last_became_peered":"2024-09-15T00:13:51.025771+0000","last_unstale":"2024-09-15T00:14:58.466117+0000","last_undegraded":"2024-09-15T00:14:58.466117+0000","last_fullsized":"2024-09-15T00:14:58.466117+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'1125","last_scrub_stamp":"2024-09-15T00:14:44.839314+0000","last_deep_scrub":"203'1125","last_deep_scrub_stamp":"2024-09-15T00:14:44.839314+0000","last_clean_scrub_stamp":"2024-09-15T00:14:44.839314+0000","objects_scrubbed":18,"log_size":1125,"log_dups_size":0,"ondisk_log_size":1125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:35:30.367514+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.001944035,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":10923,"num_read_kb":34402,"num_write":1383,"num_write_kb":19662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":27769124,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":521,"num_object_clones":0,"num_object_copies":1042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":521,"num_whiteouts":517,"num_read":240279,"num_read_kb":1281261,"num_write":28650,"num_write_kb":585223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10789,"num_bytes_recovered":390333643,"num_keys_recovered":726,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5367,"internal_metadata":0},"log_size":25728,"ondisk_log_size":25728,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":2361472,"data_compressed":25472,"data_compressed_allocated":147456,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542198,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87724,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284116,"statfs":{"total":96636764160,"available":96546934784,"internally_reserved":0,"allocated":2453504,"data_stored":1194007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542197,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55836,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316004,"statfs":{"total":96636764160,"available":96579588096,"internally_reserved":0,"allocated":2498560,"data_stored":1217589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542196,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55884,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315956,"statfs":{"total":96636764160,"available":96579538944,"internally_reserved":0,"allocated":2539520,"data_stored":1807957,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542197,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78044,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293796,"statfs":{"total":96636764160,"available":96556847104,"internally_reserved":0,"allocated":2498560,"data_stored":1217589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542194,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47524,"kb_used_data":2444,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324316,"statfs":{"total":96636764160,"available":96588099584,"internally_reserved":0,"allocated":2502656,"data_stored":1217608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":108,"seq":463856467996,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42672,"kb_used_data":2456,"kb_used_omap":6,"kb_used_meta":40185,"kb_avail":94329168,"statfs":{"total":96636764160,"available":96593068032,"internally_reserved":0,"allocated":2514944,"data_stored":1223479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6962,"internal_metadata":41149646},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542195,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55848,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315992,"statfs":{"total":96636764160,"available":96579575808,"internally_reserved":0,"allocated":2506752,"data_stored":1217627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542196,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78096,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293744,"statfs":{"total":96636764160,"available":96556793856,"internally_reserved":0,"allocated":2539520,"data_stored":1807957,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6368,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5367,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:15:01.429 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T00:15:01.724 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T00:15:01.725 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T00:15:01.725 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T00:15:01.725 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:15:01.765 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T00:15:01.765 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T00:15:01.765 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:15:01.835 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T00:15:01.835 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T00:15:01.835 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:15:01.906 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T00:15:01.906 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T00:15:01.907 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:15:01.981 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T00:15:01.981 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-15T00:15:01.981 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:15:02.025 INFO:tasks.ceph.osd.4:Stopped 2024-09-15T00:15:02.025 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-15T00:15:02.026 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:15:02.067 INFO:tasks.ceph.osd.5:Stopped 2024-09-15T00:15:02.067 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-15T00:15:02.067 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:15:02.112 INFO:tasks.ceph.osd.6:Stopped 2024-09-15T00:15:02.113 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-15T00:15:02.113 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:15:02.154 INFO:tasks.ceph.osd.7:Stopped 2024-09-15T00:15:02.154 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T00:15:02.155 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-15T00:15:02.155 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:15:02.180 INFO:tasks.ceph.mgr.y:Stopped 2024-09-15T00:15:02.180 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T00:15:02.181 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:15:02.207 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T00:15:02.207 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T00:15:02.207 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T00:15:02.207 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:15:02.232 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T00:15:02.232 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T00:15:02.233 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:15:02.284 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T00:15:02.284 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T00:15:02.284 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:15:02.297 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T00:15:02.297 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T00:15:02.297 DEBUG:teuthology.orchestra.run.smithi171:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-15T00:15:02.357 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-15T00:15:02.357 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T00:15:02.681 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-15T00:15:02.681 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T00:15:02.755 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-15T00:15:02.756 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T00:15:02.801 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-15T00:15:02.801 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-15T00:15:02.895 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-15T00:15:02.895 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-15T00:15:03.014 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-15T00:15:03.014 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-15T00:15:03.116 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-15T00:15:03.116 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-15T00:15:03.165 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-15T00:15:03.165 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-15T00:15:03.222 INFO:tasks.ceph:Archiving mon data... 2024-09-15T00:15:03.223 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904819/data/mon.a.tgz 2024-09-15T00:15:03.223 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-09-15T00:15:03.239 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.3wFg5Tn6nA 2024-09-15T00:15:03.239 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.3wFg5Tn6nA 2024-09-15T00:15:03.405 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.3wFg5Tn6nA 2024-09-15T00:15:03.488 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.3wFg5Tn6nA is 441KB 2024-09-15T00:15:03.584 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.3wFg5Tn6nA 2024-09-15T00:15:03.599 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904819/data/mon.c.tgz 2024-09-15T00:15:03.599 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-09-15T00:15:03.653 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.Es5h31NYlX 2024-09-15T00:15:03.653 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Es5h31NYlX 2024-09-15T00:15:03.839 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.Es5h31NYlX 2024-09-15T00:15:03.921 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.Es5h31NYlX is 589KB 2024-09-15T00:15:04.024 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.Es5h31NYlX 2024-09-15T00:15:04.040 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904819/data/mon.b.tgz 2024-09-15T00:15:04.040 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-09-15T00:15:04.059 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.KOg4MUBjBl 2024-09-15T00:15:04.059 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.KOg4MUBjBl 2024-09-15T00:15:04.262 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.KOg4MUBjBl 2024-09-15T00:15:04.345 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.KOg4MUBjBl is 589KB 2024-09-15T00:15:04.443 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.KOg4MUBjBl 2024-09-15T00:15:04.458 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T00:15:04.458 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T00:15:04.461 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T00:15:04.540 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T00:15:04.578 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T00:15:04.578 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T00:15:04.580 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904819/remote/smithi171/crash 2024-09-15T00:15:04.581 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T00:15:04.609 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904819/remote/smithi184/crash 2024-09-15T00:15:04.610 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T00:15:04.643 INFO:tasks.ceph:Compressing logs... 2024-09-15T00:15:04.643 DEBUG:teuthology.orchestra.run.smithi171:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T00:15:04.653 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T00:15:04.674 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T00:15:04.675 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41700.log 2024-09-15T00:15:04.675 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52875.log 2024-09-15T00:15:04.727 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33473.log 2024-09-15T00:15:04.727 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33262.log 2024-09-15T00:15:04.727 INFO:teuthology.orchestra.run.smithi184.stderr:gzip/var/log/ceph/ceph-client.admin.33473.log: -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-15T00:15:04.727 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33473.log.gz 2024-09-15T00:15:04.727 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.33262.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-15T00:15:04.727 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33262.log.gz 2024-09-15T00:15:04.727 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T00:15:04.727 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33382.log 2024-09-15T00:15:04.728 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-15T00:15:04.728 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.33382.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-15T00:15:04.728 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33382.log.gz 2024-09-15T00:15:04.728 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T00:15:04.728 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33302.log 2024-09-15T00:15:04.728 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33222.log 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T00:15:04.728 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-09-15T00:15:04.728 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.33302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33302.log.gz 2024-09-15T00:15:04.728 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33513.log 2024-09-15T00:15:04.728 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.33222.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33222.log.gz 2024-09-15T00:15:04.728 INFO:teuthology.orchestra.run.smithi184.stderr: -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T00:15:04.728 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.33513.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33426.log 2024-09-15T00:15:04.728 INFO:teuthology.orchestra.run.smithi184.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33513.log.gz 2024-09-15T00:15:04.729 INFO:teuthology.orchestra.run.smithi184.stderr: 85.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T00:15:04.729 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T00:15:04.729 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.33426.log: -- /var/log/ceph/ceph-client.admin.33342.log 2024-09-15T00:15:04.729 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33426.log.gz 2024-09-15T00:15:04.729 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.33342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33342.log.gz 2024-09-15T00:15:04.729 INFO:teuthology.orchestra.run.smithi184.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T00:15:04.770 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.41700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40639.log 2024-09-15T00:15:04.771 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41700.log.gz 2024-09-15T00:15:04.771 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.52875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56032.log 2024-09-15T00:15:04.771 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52875.log.gz 2024-09-15T00:15:04.771 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.40639.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40639.log.gz 2024-09-15T00:15:04.771 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43745.log 2024-09-15T00:15:04.771 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60750.log 2024-09-15T00:15:04.771 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.56032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56032.log.gz 2024-09-15T00:15:04.771 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57114.log 2024-09-15T00:15:04.771 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.43745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43745.log.gz 2024-09-15T00:15:04.771 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61329.log 2024-09-15T00:15:04.771 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60750.log.gz 2024-09-15T00:15:04.771 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.57114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40979.log 2024-09-15T00:15:04.772 INFO:teuthology.orchestra.run.smithi171.stderr: 55.8% -- replaced with /var/log/ceph/ceph-client.admin.57114.log.gz 2024-09-15T00:15:04.772 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58889.log 2024-09-15T00:15:04.772 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61329.log.gz 2024-09-15T00:15:04.772 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.40979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59688.log 2024-09-15T00:15:04.772 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40979.log.gz 2024-09-15T00:15:04.772 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.58889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52616.log 2024-09-15T00:15:04.772 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58889.log.gz 2024-09-15T00:15:04.772 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.59688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41680.log 2024-09-15T00:15:04.772 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59688.log.gz 2024-09-15T00:15:04.772 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.52616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42926.log 2024-09-15T00:15:04.772 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52616.log.gz 2024-09-15T00:15:04.772 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.41680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56752.log 2024-09-15T00:15:04.772 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41680.log.gz 2024-09-15T00:15:04.773 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.42926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45002.log 2024-09-15T00:15:04.773 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42926.log.gz 2024-09-15T00:15:04.773 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.56752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62549.log 2024-09-15T00:15:04.773 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56752.log.gz 2024-09-15T00:15:04.773 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.45002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51081.log 2024-09-15T00:15:04.773 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45002.log.gz 2024-09-15T00:15:04.773 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60755.log 2024-09-15T00:15:04.773 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62549.log.gz 2024-09-15T00:15:04.773 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.51081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37994.log 2024-09-15T00:15:04.773 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51081.log.gz 2024-09-15T00:15:04.773 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43424.log 2024-09-15T00:15:04.773 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60755.log.gz 2024-09-15T00:15:04.773 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38401.log 2024-09-15T00:15:04.773 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37994.log.gz 2024-09-15T00:15:04.774 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.43424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59945.log 2024-09-15T00:15:04.774 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43424.log.gz 2024-09-15T00:15:04.774 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.38401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56264.log 2024-09-15T00:15:04.774 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38401.log.gz 2024-09-15T00:15:04.774 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.59945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37654.log 2024-09-15T00:15:04.774 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59945.log.gz 2024-09-15T00:15:04.774 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.56264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60973.log 2024-09-15T00:15:04.774 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56264.log.gz 2024-09-15T00:15:04.774 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38443.log 2024-09-15T00:15:04.774 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37654.log.gz 2024-09-15T00:15:04.774 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60970.log 2024-09-15T00:15:04.774 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60973.log.gz 2024-09-15T00:15:04.774 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.38443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57811.log 2024-09-15T00:15:04.774 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38443.log.gz 2024-09-15T00:15:04.774 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37316.log 2024-09-15T00:15:04.775 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60970.log.gz 2024-09-15T00:15:04.775 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.57811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53672.log 2024-09-15T00:15:04.775 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57811.log.gz 2024-09-15T00:15:04.775 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60029.log 2024-09-15T00:15:04.775 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37316.log.gz 2024-09-15T00:15:04.775 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.53672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39265.log 2024-09-15T00:15:04.775 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53672.log.gz 2024-09-15T00:15:04.775 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40202.log 2024-09-15T00:15:04.775 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60029.log.gz 2024-09-15T00:15:04.775 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.39265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61877.log 2024-09-15T00:15:04.775 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39265.log.gz 2024-09-15T00:15:04.775 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.40202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37657.log 2024-09-15T00:15:04.775 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40202.log.gz 2024-09-15T00:15:04.775 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61877.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61877.log.gz 2024-09-15T00:15:04.775 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53631.log 2024-09-15T00:15:04.776 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37657.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T00:15:04.776 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37657.log.gz 2024-09-15T00:15:04.776 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.53631.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.37658.log 2024-09-15T00:15:04.776 INFO:teuthology.orchestra.run.smithi171.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53631.log.gz 2024-09-15T00:15:04.776 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36743.log 2024-09-15T00:15:04.776 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59772.log 2024-09-15T00:15:04.776 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37658.log.gz 2024-09-15T00:15:04.776 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.36743.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36743.log.gz 2024-09-15T00:15:04.776 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55731.log 2024-09-15T00:15:04.776 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.59772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44285.log 2024-09-15T00:15:04.776 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59772.log.gz 2024-09-15T00:15:04.776 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.55731.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55731.log.gz 2024-09-15T00:15:04.776 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43180.log 2024-09-15T00:15:04.776 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.44285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62129.log 2024-09-15T00:15:04.776 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44285.log.gz 2024-09-15T00:15:04.777 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.43180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43180.log.gz 2024-09-15T00:15:04.777 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47631.log 2024-09-15T00:15:04.777 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56676.log 2024-09-15T00:15:04.777 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62129.log.gz 2024-09-15T00:15:04.777 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.47631.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47631.log.gz 2024-09-15T00:15:04.777 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41501.log 2024-09-15T00:15:04.777 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.56676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41541.log 2024-09-15T00:15:04.777 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56676.log.gz 2024-09-15T00:15:04.777 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.41501.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41501.log.gz 2024-09-15T00:15:04.777 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41346.log 2024-09-15T00:15:04.777 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.41541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46809.log 2024-09-15T00:15:04.777 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41541.log.gz 2024-09-15T00:15:04.777 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.41346.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41346.log.gz 2024-09-15T00:15:04.777 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33255.log 2024-09-15T00:15:04.778 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.46809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53373.log 2024-09-15T00:15:04.778 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46809.log.gz 2024-09-15T00:15:04.778 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.33255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37855.log 2024-09-15T00:15:04.778 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33255.log.gz 2024-09-15T00:15:04.778 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.53373.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53373.log.gz 2024-09-15T00:15:04.778 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49210.log 2024-09-15T00:15:04.778 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37315.log 2024-09-15T00:15:04.778 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37855.log.gz 2024-09-15T00:15:04.778 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.49210.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49210.log.gz 2024-09-15T00:15:04.778 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37648.log 2024-09-15T00:15:04.778 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62087.log 2024-09-15T00:15:04.778 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37315.log.gz 2024-09-15T00:15:04.778 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37648.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37648.log.gz 2024-09-15T00:15:04.778 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37314.log 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42570.log 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62087.log.gz 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37314.log.gz 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54786.log 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.42570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58767.log 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42570.log.gz 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.54786.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54786.log.gz 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43802.log 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.58767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45583.log 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58767.log.gz 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.43802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38643.log 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43802.log.gz 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.45583.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.46315.log 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45583.log.gz 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.38643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38275.log 2024-09-15T00:15:04.779 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38643.log.gz 2024-09-15T00:15:04.780 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/objectstore_tool.46315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58050.log 2024-09-15T00:15:04.780 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.38275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38275.log.gz 2024-09-15T00:15:04.780 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62171.log 2024-09-15T00:15:04.780 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58710.log 2024-09-15T00:15:04.780 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.58050.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58050.log.gz 2024-09-15T00:15:04.780 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62171.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62171.log.gz 2024-09-15T00:15:04.780 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42023.log 2024-09-15T00:15:04.780 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54468.log 2024-09-15T00:15:04.780 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.58710.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58710.log.gz 2024-09-15T00:15:04.780 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.42023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40003.log 2024-09-15T00:15:04.780 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42023.log.gz 2024-09-15T00:15:04.780 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40722.log 2024-09-15T00:15:04.780 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.54468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54468.log.gz 2024-09-15T00:15:04.780 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.40003.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40003.log.gz 2024-09-15T00:15:04.781 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45863.log 2024-09-15T00:15:04.781 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61408.log 2024-09-15T00:15:04.781 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.40722.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40722.log.gz 2024-09-15T00:15:04.781 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.45863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53888.log 2024-09-15T00:15:04.781 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45863.log.gz 2024-09-15T00:15:04.781 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61408.log.gz 2024-09-15T00:15:04.781 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51339.log 2024-09-15T00:15:04.781 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49654.log 2024-09-15T00:15:04.781 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.53888.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53888.log.gz 2024-09-15T00:15:04.781 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.51339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48240.log 2024-09-15T00:15:04.781 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51339.log.gz 2024-09-15T00:15:04.781 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.49654.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49654.log.gz 2024-09-15T00:15:04.781 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48715.log 2024-09-15T00:15:04.781 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38505.log 2024-09-15T00:15:04.782 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.48240.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48240.log.gz 2024-09-15T00:15:04.782 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.48715.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.60281.log 2024-09-15T00:15:04.782 INFO:teuthology.orchestra.run.smithi171.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48715.log.gz 2024-09-15T00:15:04.782 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.38505.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38505.log.gz 2024-09-15T00:15:04.782 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44044.log 2024-09-15T00:15:04.782 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39764.log 2024-09-15T00:15:04.782 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60281.log.gz 2024-09-15T00:15:04.782 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.44044.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44044.log.gz 2024-09-15T00:15:04.782 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48357.log 2024-09-15T00:15:04.782 INFO:teuthology.orchestra.run.smithi171.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.46315.log.gz 2024-09-15T00:15:04.782 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37317.log 2024-09-15T00:15:04.782 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.39764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39764.log.gz 2024-09-15T00:15:04.782 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.48357.log: -- /var/log/ceph/ceph-client.admin.60533.log 2024-09-15T00:15:04.782 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48357.log.gz 2024-09-15T00:15:04.783 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52215.log 2024-09-15T00:15:04.783 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37317.log.gz 2024-09-15T00:15:04.783 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60533.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60533.log.gz 2024-09-15T00:15:04.783 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62045.log 2024-09-15T00:15:04.783 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.52215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55772.log 2024-09-15T00:15:04.783 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52215.log.gz 2024-09-15T00:15:04.783 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62045.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.62045.log.gz -- 2024-09-15T00:15:04.783 INFO:teuthology.orchestra.run.smithi171.stderr: /var/log/ceph/ceph-client.admin.38645.log 2024-09-15T00:15:04.783 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37992.log 2024-09-15T00:15:04.783 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.55772.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55772.log.gz 2024-09-15T00:15:04.783 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.38645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39403.log 2024-09-15T00:15:04.783 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38645.log.gz 2024-09-15T00:15:04.783 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62003.log 2024-09-15T00:15:04.784 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37992.log.gz 2024-09-15T00:15:04.784 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.39403.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39403.log.gz 2024-09-15T00:15:04.784 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51936.log 2024-09-15T00:15:04.784 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62255.log 2024-09-15T00:15:04.784 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62003.log.gz 2024-09-15T00:15:04.784 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.51936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51936.log.gz 2024-09-15T00:15:04.784 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33343.log 2024-09-15T00:15:04.784 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37987.log 2024-09-15T00:15:04.784 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62255.log.gz 2024-09-15T00:15:04.784 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.33343.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33343.log.gz 2024-09-15T00:15:04.784 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53811.log 2024-09-15T00:15:04.784 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37987.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T00:15:04.784 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37987.log.gz 2024-09-15T00:15:04.785 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.53811.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53811.log.gz 2024-09-15T00:15:04.785 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62297.log 2024-09-15T00:15:04.785 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60323.log 2024-09-15T00:15:04.785 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62297.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62297.log.gz 2024-09-15T00:15:04.785 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38359.log 2024-09-15T00:15:04.785 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54368.log 2024-09-15T00:15:04.785 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60323.log.gz 2024-09-15T00:15:04.785 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.38359.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38359.log.gz 2024-09-15T00:15:04.785 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44141.log 2024-09-15T00:15:04.785 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58371.log 2024-09-15T00:15:04.785 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.54368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54368.log.gz 2024-09-15T00:15:04.786 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.44141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43541.log 2024-09-15T00:15:04.786 INFO:teuthology.orchestra.run.smithi171.stderr: 27.2% -- replaced with /var/log/ceph/ceph-client.admin.44141.log.gz 2024-09-15T00:15:04.786 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.58371.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58371.log.gz 2024-09-15T00:15:04.786 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62465.log 2024-09-15T00:15:04.786 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47983.log 2024-09-15T00:15:04.786 INFO:teuthology.orchestra.run.smithi171.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T00:15:04.786 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.43541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43541.log.gz 2024-09-15T00:15:04.786 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62465.log.gz 2024-09-15T00:15:04.786 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45417.log 2024-09-15T00:15:04.786 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.47983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62423.log 2024-09-15T00:15:04.786 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47983.log.gz 2024-09-15T00:15:04.786 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.45417.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.45417.log.gz /var/log/ceph/ceph-client.admin.60756.log 2024-09-15T00:15:04.787 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-09-15T00:15:04.787 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36981.log 2024-09-15T00:15:04.787 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62423.log.gz 2024-09-15T00:15:04.787 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60756.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.60756.log.gz /var/log/ceph/ceph-client.admin.61289.log 2024-09-15T00:15:04.787 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-09-15T00:15:04.787 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.36981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37057.log 2024-09-15T00:15:04.787 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36981.log.gz 2024-09-15T00:15:04.787 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61083.log 2024-09-15T00:15:04.787 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61289.log.gz 2024-09-15T00:15:04.787 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37645.log 2024-09-15T00:15:04.787 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37057.log.gz 2024-09-15T00:15:04.787 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49609.log 2024-09-15T00:15:04.787 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61083.log.gz 2024-09-15T00:15:04.787 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42968.log 2024-09-15T00:15:04.787 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37645.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37645.log.gz 2024-09-15T00:15:04.788 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.49609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62591.log 2024-09-15T00:15:04.788 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49609.log.gz 2024-09-15T00:15:04.788 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.42968.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.42968.log.gz -5 2024-09-15T00:15:04.788 INFO:teuthology.orchestra.run.smithi171.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57413.log 2024-09-15T00:15:04.788 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57519.log 2024-09-15T00:15:04.788 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62591.log.gz 2024-09-15T00:15:04.788 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.57413.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.57413.log.gz -- 2024-09-15T00:15:04.788 INFO:teuthology.orchestra.run.smithi171.stderr: /var/log/ceph/ceph-client.admin.61085.log 2024-09-15T00:15:04.788 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.57519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61328.log 2024-09-15T00:15:04.788 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57519.log.gz 2024-09-15T00:15:04.788 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51380.log 2024-09-15T00:15:04.788 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61085.log.gz 2024-09-15T00:15:04.788 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61328.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61328.log.gz 2024-09-15T00:15:04.789 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54287.log 2024-09-15T00:15:04.789 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.51380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38233.log 2024-09-15T00:15:04.789 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51380.log.gz 2024-09-15T00:15:04.789 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.54287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54287.log.gz 2024-09-15T00:15:04.789 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37991.log 2024-09-15T00:15:04.789 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.38233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37641.log 2024-09-15T00:15:04.789 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38233.log.gz 2024-09-15T00:15:04.789 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37991.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.37991.log.gz -- 2024-09-15T00:15:04.789 INFO:teuthology.orchestra.run.smithi171.stderr: /var/log/ceph/ceph-client.admin.59447.log 2024-09-15T00:15:04.789 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60239.log 2024-09-15T00:15:04.789 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37641.log.gz 2024-09-15T00:15:04.789 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.59447.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59447.log.gz -5 2024-09-15T00:15:04.789 INFO:teuthology.orchestra.run.smithi171.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57433.log 2024-09-15T00:15:04.789 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61961.log 2024-09-15T00:15:04.790 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60239.log.gz 2024-09-15T00:15:04.790 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.57433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32953.log 2024-09-15T00:15:04.790 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57433.log.gz 2024-09-15T00:15:04.790 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61961.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.61961.log.gz 2024-09-15T00:15:04.790 INFO:teuthology.orchestra.run.smithi171.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36925.log 2024-09-15T00:15:04.790 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.32953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44522.log 2024-09-15T00:15:04.790 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32953.log.gz 2024-09-15T00:15:04.790 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.36925.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.36925.log.gz 2024-09-15T00:15:04.790 INFO:teuthology.orchestra.run.smithi171.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61411.log 2024-09-15T00:15:04.790 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.44522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61667.log 2024-09-15T00:15:04.790 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44522.log.gz 2024-09-15T00:15:04.790 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61411.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.61411.log.gz 2024-09-15T00:15:04.790 INFO:teuthology.orchestra.run.smithi171.stderr: /var/log/ceph/ceph-client.admin.61709.log 2024-09-15T00:15:04.790 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51659.log 2024-09-15T00:15:04.791 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61667.log.gz 2024-09-15T00:15:04.791 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48987.log 2024-09-15T00:15:04.791 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61709.log.gz 2024-09-15T00:15:04.791 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.51659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51659.log.gz 2024-09-15T00:15:04.791 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59861.log 2024-09-15T00:15:04.791 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.48987.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-15T00:15:04.791 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48987.log.gz 2024-09-15T00:15:04.791 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.59861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56753.log 2024-09-15T00:15:04.791 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59861.log.gz 2024-09-15T00:15:04.791 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56337.log 2024-09-15T00:15:04.791 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.56753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56753.log.gz 2024-09-15T00:15:04.791 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61835.log 2024-09-15T00:15:04.791 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40062.log 2024-09-15T00:15:04.791 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.56337.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56337.log.gz 2024-09-15T00:15:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37006.log 2024-09-15T00:15:04.792 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61835.log.gz 2024-09-15T00:15:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.40062.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40062.log.gz 2024-09-15T00:15:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42530.log 2024-09-15T00:15:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54827.log 2024-09-15T00:15:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37006.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37006.log.gz 2024-09-15T00:15:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.42530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59903.log 2024-09-15T00:15:04.792 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42530.log.gz 2024-09-15T00:15:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.54827.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.54827.log.gz --verbose 2024-09-15T00:15:04.792 INFO:teuthology.orchestra.run.smithi171.stderr: -- /var/log/ceph/ceph-client.admin.60491.log 2024-09-15T00:15:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42270.log 2024-09-15T00:15:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.59903.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59903.log.gz 2024-09-15T00:15:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37659.log 2024-09-15T00:15:04.792 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60491.log.gz 2024-09-15T00:15:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.42270.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42270.log.gz 2024-09-15T00:15:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45098.log 2024-09-15T00:15:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61086.log 2024-09-15T00:15:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37659.log.gz 2024-09-15T00:15:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.45098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45098.log.gz 2024-09-15T00:15:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42085.log 2024-09-15T00:15:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51039.log 2024-09-15T00:15:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61086.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61086.log.gz 2024-09-15T00:15:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.42085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39804.log 2024-09-15T00:15:04.793 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42085.log.gz 2024-09-15T00:15:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.51039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51039.log.gz 2024-09-15T00:15:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40541.log 2024-09-15T00:15:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43307.log 2024-09-15T00:15:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.39804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39804.log.gz 2024-09-15T00:15:04.794 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.40541.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.40541.log.gz 2024-09-15T00:15:04.794 INFO:teuthology.orchestra.run.smithi171.stderr: -- /var/log/ceph/ceph-client.admin.46182.log 2024-09-15T00:15:04.794 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61625.log 2024-09-15T00:15:04.794 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.43307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43307.log.gz 2024-09-15T00:15:04.794 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.46182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46182.log.gz 2024-09-15T00:15:04.794 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41459.log 2024-09-15T00:15:04.794 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54227.log 2024-09-15T00:15:04.794 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61625.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61625.log.gz 2024-09-15T00:15:04.794 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.41459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60113.log 2024-09-15T00:15:04.794 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41459.log.gz 2024-09-15T00:15:04.794 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.54227.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54227.log.gz 2024-09-15T00:15:04.794 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52596.log 2024-09-15T00:15:04.794 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62674.log 2024-09-15T00:15:04.794 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60113.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60113.log.gz 2024-09-15T00:15:04.795 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.52596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59814.log 2024-09-15T00:15:04.795 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52596.log.gz 2024-09-15T00:15:04.795 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62674.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62674.log.gz 2024-09-15T00:15:04.795 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61409.log 2024-09-15T00:15:04.795 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43483.log 2024-09-15T00:15:04.795 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.59814.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59814.log.gz 2024-09-15T00:15:04.795 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59305.log 2024-09-15T00:15:04.795 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61409.log.gz 2024-09-15T00:15:04.795 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.43483.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43483.log.gz 2024-09-15T00:15:04.795 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54150.log 2024-09-15T00:15:04.795 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54327.log 2024-09-15T00:15:04.795 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.59305.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59305.log.gz 2024-09-15T00:15:04.795 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.54150.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54150.log.gz 2024-09-15T00:15:04.795 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38984.log 2024-09-15T00:15:04.796 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.54327.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54327.log.gz 2024-09-15T00:15:04.796 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48281.log 2024-09-15T00:15:04.796 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60407.log 2024-09-15T00:15:04.796 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.38984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38984.log.gz 2024-09-15T00:15:04.796 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.48281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48281.log.gz 2024-09-15T00:15:04.796 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27129.log 2024-09-15T00:15:04.796 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45418.log 2024-09-15T00:15:04.796 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60407.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60407.log.gz 2024-09-15T00:15:04.796 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph.tmp-client.admin.27129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55830.log 2024-09-15T00:15:04.796 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27129.log.gz 2024-09-15T00:15:04.796 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.45418.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.45418.log.gz 2024-09-15T00:15:04.796 INFO:teuthology.orchestra.run.smithi171.stderr: -- /var/log/ceph/ceph-client.admin.52315.log 2024-09-15T00:15:04.796 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48614.log 2024-09-15T00:15:04.797 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.55830.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55830.log.gz 2024-09-15T00:15:04.797 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.52315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52315.log.gz 2024-09-15T00:15:04.797 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T00:15:04.797 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44181.log 2024-09-15T00:15:04.797 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.48614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48614.log.gz 2024-09-15T00:15:04.797 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45500.log 2024-09-15T00:15:04.797 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.44181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58252.log 2024-09-15T00:15:04.797 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44181.log.gz 2024-09-15T00:15:04.797 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.45500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37995.log 2024-09-15T00:15:04.797 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45500.log.gz 2024-09-15T00:15:04.797 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.58252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48559.log 2024-09-15T00:15:04.797 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58252.log.gz 2024-09-15T00:15:04.797 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37141.log 2024-09-15T00:15:04.797 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37995.log.gz 2024-09-15T00:15:04.798 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.48559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38559.log 2024-09-15T00:15:04.798 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48559.log.gz 2024-09-15T00:15:04.798 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56195.log 2024-09-15T00:15:04.798 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37141.log.gz 2024-09-15T00:15:04.798 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.0.38559.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T00:15:04.798 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.56195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56195.log.gz 2024-09-15T00:15:04.798 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59987.log 2024-09-15T00:15:04.798 INFO:teuthology.orchestra.run.smithi171.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.0.38559.log.gz 2024-09-15T00:15:04.798 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42065.log 2024-09-15T00:15:04.798 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.59987.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59987.log.gz 2024-09-15T00:15:04.798 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55388.log 2024-09-15T00:15:04.798 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.42065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42065.log.gz 2024-09-15T00:15:04.798 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41965.log 2024-09-15T00:15:04.798 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.55388.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55388.log.gz 2024-09-15T00:15:04.799 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47786.log 2024-09-15T00:15:04.799 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.41965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41965.log.gz 2024-09-15T00:15:04.799 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45904.log 2024-09-15T00:15:04.799 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.47786.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47786.log.gz 2024-09-15T00:15:04.799 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62507.log 2024-09-15T00:15:04.799 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.45904.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45904.log.gz 2024-09-15T00:15:04.799 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62213.log 2024-09-15T00:15:04.799 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62507.log.gz 2024-09-15T00:15:04.799 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-15T00:15:04.799 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62213.log.gz 2024-09-15T00:15:04.799 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37989.log 2024-09-15T00:15:04.799 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51598.log 2024-09-15T00:15:04.799 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37989.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59106.log 2024-09-15T00:15:04.799 INFO:teuthology.orchestra.run.smithi171.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37989.log.gz 2024-09-15T00:15:04.800 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.51598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51598.log.gz 2024-09-15T00:15:04.800 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46161.log 2024-09-15T00:15:04.800 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37322.log 2024-09-15T00:15:04.800 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.59106.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59106.log.gz 2024-09-15T00:15:04.800 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.46161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46161.log.gz 2024-09-15T00:15:04.800 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51777.log 2024-09-15T00:15:04.800 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37322.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37322.log.gz 2024-09-15T00:15:04.800 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44872.log 2024-09-15T00:15:04.800 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38191.log 2024-09-15T00:15:04.800 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.51777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51777.log.gz 2024-09-15T00:15:04.800 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.44872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44872.log.gz 2024-09-15T00:15:04.800 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55368.log 2024-09-15T00:15:04.800 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.38191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38191.log.gz 2024-09-15T00:15:04.800 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62381.log 2024-09-15T00:15:04.801 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T00:15:04.801 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.55368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55368.log.gz 2024-09-15T00:15:04.801 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62381.log.gz 2024-09-15T00:15:04.801 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60071.log 2024-09-15T00:15:04.801 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph.log: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T00:15:04.801 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44863.log 2024-09-15T00:15:04.801 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37993.log 2024-09-15T00:15:04.801 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60071.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60071.log.gz 2024-09-15T00:15:04.801 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.44863.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44863.log.gz 2024-09-15T00:15:04.801 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55567.log 2024-09-15T00:15:04.801 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37993.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37993.log.gz 2024-09-15T00:15:04.801 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45341.log 2024-09-15T00:15:04.801 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59528.log 2024-09-15T00:15:04.802 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.55567.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55567.log.gz 2024-09-15T00:15:04.802 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.45341.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45341.log.gz 2024-09-15T00:15:04.802 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37323.log 2024-09-15T00:15:04.802 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.59528.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59528.log.gz 2024-09-15T00:15:04.802 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56135.log 2024-09-15T00:15:04.802 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37099.log 2024-09-15T00:15:04.802 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37323.log.gz 2024-09-15T00:15:04.802 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.56135.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56135.log.gz 2024-09-15T00:15:04.802 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62634.log 2024-09-15T00:15:04.802 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37099.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37099.log.gz 2024-09-15T00:15:04.802 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48097.log 2024-09-15T00:15:04.802 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57155.log 2024-09-15T00:15:04.802 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62634.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62634.log.gz 2024-09-15T00:15:04.802 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.48097.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54207.log 2024-09-15T00:15:04.803 INFO:teuthology.orchestra.run.smithi171.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48097.log.gz 2024-09-15T00:15:04.803 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.57155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57155.log.gz 2024-09-15T00:15:04.803 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38317.log 2024-09-15T00:15:04.803 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56413.log 2024-09-15T00:15:04.803 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.54207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54207.log.gz 2024-09-15T00:15:04.803 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.38317.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38317.log.gz 2024-09-15T00:15:04.803 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56031.log 2024-09-15T00:15:04.803 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.56413.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56413.log.gz 2024-09-15T00:15:04.803 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40598.log 2024-09-15T00:15:04.803 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56936.log 2024-09-15T00:15:04.803 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.56031.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56031.log.gz 2024-09-15T00:15:04.803 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.40598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40598.log.gz 2024-09-15T00:15:04.803 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47312.log 2024-09-15T00:15:04.803 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.56936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56936.log.gz 2024-09-15T00:15:04.804 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49265.log 2024-09-15T00:15:04.804 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37650.log 2024-09-15T00:15:04.804 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.47312.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47312.log.gz 2024-09-15T00:15:04.804 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.49265.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49265.log.gz 2024-09-15T00:15:04.804 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52992.log 2024-09-15T00:15:04.804 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62339.log 2024-09-15T00:15:04.804 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37650.log.gz 2024-09-15T00:15:04.804 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.52992.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52992.log.gz 2024-09-15T00:15:04.804 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42849.log 2024-09-15T00:15:04.804 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.62339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62339.log.gz 2024-09-15T00:15:04.804 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47354.log 2024-09-15T00:15:04.804 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39383.log 2024-09-15T00:15:04.804 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.42849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42849.log.gz 2024-09-15T00:15:04.805 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.47354.log: 30.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55266.log 2024-09-15T00:15:04.805 INFO:teuthology.orchestra.run.smithi171.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47354.log.gz 2024-09-15T00:15:04.805 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.39383.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39383.log.gz 2024-09-15T00:15:04.805 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60748.log 2024-09-15T00:15:04.805 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61793.log 2024-09-15T00:15:04.805 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.55266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55266.log.gz 2024-09-15T00:15:04.805 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60748.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60748.log.gz 2024-09-15T00:15:04.805 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41883.log 2024-09-15T00:15:04.805 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61793.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61793.log.gz 2024-09-15T00:15:04.805 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55085.log 2024-09-15T00:15:04.805 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61091.log 2024-09-15T00:15:04.805 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.41883.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41883.log.gz 2024-09-15T00:15:04.805 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.55085.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55085.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39504.log 2024-09-15T00:15:04.806 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-09-15T00:15:04.806 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61091.log.gz 2024-09-15T00:15:04.806 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53074.log 2024-09-15T00:15:04.806 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37321.log 2024-09-15T00:15:04.806 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.39504.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39504.log.gz 2024-09-15T00:15:04.806 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.53074.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53074.log.gz 2024-09-15T00:15:04.806 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48871.log 2024-09-15T00:15:04.806 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37321.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37321.log.gz 2024-09-15T00:15:04.806 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58411.log 2024-09-15T00:15:04.806 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51819.log 2024-09-15T00:15:04.806 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.48871.log: /var/log/ceph/ceph-client.admin.58411.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48871.log.gz 2024-09-15T00:15:04.806 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58411.log.gz 2024-09-15T00:15:04.806 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42183.log 2024-09-15T00:15:04.806 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60365.log 2024-09-15T00:15:04.806 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.51819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51819.log.gz 2024-09-15T00:15:04.807 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.42183.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42183.log.gz 2024-09-15T00:15:04.807 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55176.log 2024-09-15T00:15:04.807 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60365.log.gz 2024-09-15T00:15:04.807 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61451.log 2024-09-15T00:15:04.807 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61092.log 2024-09-15T00:15:04.807 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.55176.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55176.log.gz 2024-09-15T00:15:04.807 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61451.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61451.log.gz 2024-09-15T00:15:04.807 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60449.log 2024-09-15T00:15:04.807 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61092.log.gz 2024-09-15T00:15:04.807 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47549.log 2024-09-15T00:15:04.807 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44781.log 2024-09-15T00:15:04.807 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60449.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60449.log.gz 2024-09-15T00:15:04.807 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.47549.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47549.log.gz 2024-09-15T00:15:04.807 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39041.log 2024-09-15T00:15:04.822 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.44781.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44781.log.gz 2024-09-15T00:15:04.822 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60197.log 2024-09-15T00:15:04.823 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55627.log 2024-09-15T00:15:04.823 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.39041.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39041.log.gz 2024-09-15T00:15:04.823 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60197.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60197.log.gz 2024-09-15T00:15:04.823 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41277.log 2024-09-15T00:15:04.823 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.55627.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55627.log.gz 2024-09-15T00:15:04.823 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45644.log 2024-09-15T00:15:04.823 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53550.log 2024-09-15T00:15:04.823 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.41277.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41277.log.gz 2024-09-15T00:15:04.823 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.45644.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45644.log.gz 2024-09-15T00:15:04.823 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38644.log 2024-09-15T00:15:04.823 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.53550.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53550.log.gz 2024-09-15T00:15:04.823 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61093.log 2024-09-15T00:15:04.824 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47866.log 2024-09-15T00:15:04.824 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.38644.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38644.log.gz 2024-09-15T00:15:04.824 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61093.log.gz 2024-09-15T00:15:04.824 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60754.log 2024-09-15T00:15:04.824 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.47866.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47866.log.gz 2024-09-15T00:15:04.824 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41298.log 2024-09-15T00:15:04.824 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55150.log 2024-09-15T00:15:04.824 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60754.log.gz 2024-09-15T00:15:04.824 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.41298.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41298.log.gz 2024-09-15T00:15:04.824 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52158.log 2024-09-15T00:15:04.824 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.55150.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55150.log.gz 2024-09-15T00:15:04.824 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40255.log 2024-09-15T00:15:04.824 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61410.log 2024-09-15T00:15:04.824 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.52158.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52158.log.gz 2024-09-15T00:15:04.825 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.40255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40255.log.gz 2024-09-15T00:15:04.825 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54568.log 2024-09-15T00:15:04.825 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61410.log.gz 2024-09-15T00:15:04.825 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50095.log 2024-09-15T00:15:04.825 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60757.log 2024-09-15T00:15:04.825 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.54568.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54568.log.gz 2024-09-15T00:15:04.825 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.50095.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50095.log.gz 2024-09-15T00:15:04.825 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41020.log 2024-09-15T00:15:04.825 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60757.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60757.log.gz 2024-09-15T00:15:04.825 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60155.log 2024-09-15T00:15:04.825 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49498.log 2024-09-15T00:15:04.825 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.41020.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41020.log.gz 2024-09-15T00:15:04.825 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60155.log.gz 2024-09-15T00:15:04.826 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48933.log 2024-09-15T00:15:04.826 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59646.log 2024-09-15T00:15:04.826 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.49498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49498.log.gz 2024-09-15T00:15:04.826 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.48933.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48933.log.gz 2024-09-15T00:15:04.826 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59730.log 2024-09-15T00:15:04.826 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57674.log 2024-09-15T00:15:04.826 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.59646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59646.log.gz 2024-09-15T00:15:04.826 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.59730.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59730.log.gz 2024-09-15T00:15:04.826 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-15T00:15:04.826 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60575.log 2024-09-15T00:15:04.826 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.57674.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57674.log.gz 2024-09-15T00:15:04.826 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61919.log 2024-09-15T00:15:04.826 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60575.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60575.log.gz 2024-09-15T00:15:04.827 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54408.log 2024-09-15T00:15:04.827 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61919.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61919.log.gz 2024-09-15T00:15:04.827 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61751.log 2024-09-15T00:15:04.827 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.54408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54408.log.gz 2024-09-15T00:15:04.827 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44563.log 2024-09-15T00:15:04.827 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.61751.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61751.log.gz 2024-09-15T00:15:04.827 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58049.log 2024-09-15T00:15:04.827 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.44563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44563.log.gz 2024-09-15T00:15:04.827 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42330.log 2024-09-15T00:15:04.827 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.58049.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58049.log.gz 2024-09-15T00:15:04.827 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50325.log 2024-09-15T00:15:04.827 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.42330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42330.log.gz 2024-09-15T00:15:04.827 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50901.log 2024-09-15T00:15:04.828 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.50325.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50325.log.gz 2024-09-15T00:15:04.828 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53271.log 2024-09-15T00:15:04.828 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.50901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50901.log.gz 2024-09-15T00:15:04.828 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37320.log 2024-09-15T00:15:04.828 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.53271.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53271.log.gz 2024-09-15T00:15:04.828 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52394.log 2024-09-15T00:15:04.828 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.37320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37320.log.gz 2024-09-15T00:15:04.828 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57933.log 2024-09-15T00:15:04.828 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.52394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52394.log.gz 2024-09-15T00:15:04.828 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59148.log 2024-09-15T00:15:04.828 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.57933.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57933.log.gz 2024-09-15T00:15:04.828 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60752.log 2024-09-15T00:15:04.828 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.59148.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59148.log.gz 2024-09-15T00:15:04.829 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60749.log 2024-09-15T00:15:04.829 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60752.log.gz 2024-09-15T00:15:04.829 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.60749.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60749.log.gz 2024-09-15T00:15:04.829 INFO:teuthology.orchestra.run.smithi171.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-15T00:15:04.844 INFO:teuthology.orchestra.run.smithi184.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T00:15:04.936 INFO:teuthology.orchestra.run.smithi171.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T00:15:05.020 INFO:teuthology.orchestra.run.smithi171.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T00:15:05.507 INFO:teuthology.orchestra.run.smithi171.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T00:15:06.320 INFO:teuthology.orchestra.run.smithi184.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-15T00:15:06.887 INFO:teuthology.orchestra.run.smithi171.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T00:15:07.017 INFO:teuthology.orchestra.run.smithi184.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-15T00:15:07.929 INFO:teuthology.orchestra.run.smithi171.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T00:15:09.602 INFO:teuthology.orchestra.run.smithi184.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-15T00:15:10.936 INFO:teuthology.orchestra.run.smithi184.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-15T00:15:10.938 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-09-15T00:15:10.938 INFO:teuthology.orchestra.run.smithi184.stderr:real 0m6.236s 2024-09-15T00:15:10.938 INFO:teuthology.orchestra.run.smithi184.stderr:user 0m14.558s 2024-09-15T00:15:10.938 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m0.603s 2024-09-15T00:15:11.065 INFO:teuthology.orchestra.run.smithi171.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-15T00:15:11.068 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-09-15T00:15:11.068 INFO:teuthology.orchestra.run.smithi171.stderr:real 0m6.403s 2024-09-15T00:15:11.068 INFO:teuthology.orchestra.run.smithi171.stderr:user 0m12.658s 2024-09-15T00:15:11.068 INFO:teuthology.orchestra.run.smithi171.stderr:sys 0m0.676s 2024-09-15T00:15:11.068 INFO:tasks.ceph:Archiving logs... 2024-09-15T00:15:11.068 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904819/remote/smithi171/log 2024-09-15T00:15:11.069 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T00:15:13.066 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904819/remote/smithi184/log 2024-09-15T00:15:13.067 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T00:15:14.991 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T00:15:15.001 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T00:15:15.001 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T00:15:15.004 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T00:15:15.089 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T00:15:15.098 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T00:15:15.098 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T00:15:15.101 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T00:15:15.115 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-09-15T00:15:15.118 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-09-15T00:15:15.120 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T00:15:15.120 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-09-15T00:15:15.121 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv01.front.sepia.ceph.com 3 6 377 36 +344us[ +341us] +/- 55ms 2024-09-15T00:15:15.121 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 101 +560us[ +446us] +/- 58ms 2024-09-15T00:15:15.121 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -1031us[-1031us] +/- 61ms 2024-09-15T00:15:15.121 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:15:15.123 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T00:15:15.123 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-09-15T00:15:15.123 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv01.front.sepia.ceph.com 3 6 377 36 +349us[ +234us] +/- 55ms 2024-09-15T00:15:15.123 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 35 +528us[ +528us] +/- 59ms 2024-09-15T00:15:15.123 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -1042us[-1157us] +/- 61ms 2024-09-15T00:15:15.123 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:15:15.124 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T00:15:15.133 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T00:15:15.133 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T00:15:15.144 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T00:15:15.245 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T00:15:15.327 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T00:15:15.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:15:15.357 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T00:15:15.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:15:15.389 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T00:15:15.398 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T00:15:15.407 INFO:teuthology.task.internal:Duration was 654.818018 seconds 2024-09-15T00:15:15.408 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T00:15:15.418 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T00:15:15.419 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T00:15:15.421 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T00:15:15.458 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T00:15:15.471 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T00:15:15.835 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T00:15:15.836 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-09-15T00:15:15.836 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T00:15:15.896 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2024-09-15T00:15:15.896 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T00:15:15.920 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T00:15:15.920 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T00:15:15.939 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T00:15:15.989 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T00:15:15.989 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T00:15:16.005 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T00:15:16.149 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T00:15:16.159 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T00:15:16.159 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T00:15:16.184 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T00:15:16.209 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T00:15:16.218 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T00:15:16.227 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T00:15:16.248 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-09-15T00:15:16.273 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2024-09-15T00:15:16.295 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T00:15:16.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:15:16.329 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T00:15:16.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:15:16.350 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T00:15:16.360 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T00:15:16.361 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904819/remote/smithi171 2024-09-15T00:15:16.361 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T00:15:16.409 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904819/remote/smithi184 2024-09-15T00:15:16.409 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T00:15:16.442 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T00:15:16.442 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T00:15:16.444 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T00:15:16.492 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T00:15:16.504 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T00:15:16.504 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T00:15:16.515 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T00:15:16.515 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T00:15:16.517 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T00:15:16.530 INFO:teuthology.orchestra.run.smithi171.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 00:15 /home/ubuntu/cephtest 2024-09-15T00:15:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 00:15 /home/ubuntu/cephtest 2024-09-15T00:15:16.549 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T00:15:16.560 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T00:15:16.594 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit} duration: 654.8180177211761 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-15T00:15:16.594 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T00:15:16.679 INFO:teuthology.run:pass